People Power
You are one of only two experts in your field, but you're teaching a large class on the topic of your latest research and you want to run a project.
What is someone in a good place to use crowd-assisted coding?
Design and validate a new coding frame for legal precedent analysis on a tricky subject. Accuracy is paramount, few experts exist, and any errors could have disastrous consequences.
What is a picture-perfect opportunity for hybrid (expert/automated) coding?
200 new bug reports on a product set to go live in two weeks.
What is a case for automated sorting with expert response/oversight?
An investigation of changes in writing styles over the course of a semester-long Creative Writing class?
What is a good opportunity for crowd-assisted coding?
What narrative frames and discursive patterns exist with respect to Autism Spectrum Disorders in US Federal law?
What is something that was done with expert research when it happened, but would be better served by automation now?
You work in a well-established field with vicious divides between theoretical camps, each of whose positions are well known. You and your coworkers have been challenged to defend a recent decision using company records, which are unfortunately not all digital.
What is someone who needs to use expert coding?
Collect large amounts of longitudinal data from a random population about common, neutral-to-enjoyable activities.
What is something remote crowd-assisted coding is pretty awesome at? (Mellis & Bickel, 2020)
End of Semester feedback from students in a new pilot program.
What is a good case for expert coding?
A study tracking the emergence and change in use of a particular narrative frame in the 1800s, including all of the primary and secondary sources found in one geographical area.
What is something that will probably be hybrid unless the area is very small or the items involved difficult to work with?
How does this loose pile of tweets and news articles fall across the following categories: Relevance topic A, Relevance topic B, Stance on legislation (keep, repeal, neutral), topic identification (6 classes), Framing A (problem, solution, neutral) Framing B (14 classes)?
What is something best accomplished via automation, specifically ChatGPT in today's reading?
You're a graduate student writing a dissertation on a social media phenomenon.
What is someone who could benefit from automated coding?
Validate an instrument measuring how likely people are to take novel analgesics based on varying levels of pain relief and probability of addiction.
What is something that crowd-assisted coding works well for? (Mellis & Bickle, 2020)
Seven fragile handwritten manuscripts in an obscure dialect.
What is a task only experts should even contemplate?
A study on the treatment and progression of COVID-19 symptoms across a set of hospitals using the same interlinked records system.
What is something good for automation?
Are incoming posts adhering to our content guidelines?
What is something that benefits from some automation, but should be hybridized with either expert or crowd assistance?
You're representing an organization researching a sensitive healthcare matter and you've been asked to give testimony before Congress about its scope and history.
What is someone best served by hybrid coding (and deep pockets)?
Build a dataset of microinteractions between doctors and patients during routine check-ups.
What is something requiring expert analysis of data that is usually collected by crowd-assisted services (and, increasingly, AI)?
Multiple copies of the same document in Esperanto, Toki Pana, Golic, tlhlngon Hol, Sindarin and Dothraki, which need to be checked against each other for translation accuracy.
What is a perfect job for automated coding?
(Natural language translation and conlang translation have opposite profiles.)
A deep dive into intercultural variations in politeness markers used by Japanese, Indonesian and English-speaking V-Tubers during Telephone games.
What is a project requiring expert coding?
What kinds of misinformation tactics appear in short-form videos about a new health policy, and how frequently do they occur?
What is something that needs expert direction but could benefit from hybridization?
You’re a citizen-science organizer trying to mobilize thousands of volunteers to label images of endangered species.
What is someone running a crowd-assisted research project by design?
Collect and classify news articles about extreme weather events to track changes in framing over time.
What is a fairly straightforward case for automated coding?
Recorded voice notes from doctors covering routine visits, to be transcribed and added to charts.
What is something done by crowd-sourcing as industry standard? (Though automation is creeping in there too.)
A study on emerging language trends on TikTok with an eye toward identifying/preempting fascist or antisocial phenomena.
What is a project that almost has to be hybrid?
Does user-generated content about a disaster event comply with emerging ethical guidelines for victim privacy?
What is something that needs expert oversight even if other labor is involved?