Bias Detective: Question Wording
A survey asks “How satisfied do you think your classmates are with Brown’s dining halls?” What problem does this question have?
What is asking respondents to report another person's opinions or feelings?
A survey asks: “A survey asks whether students support a new AI policy with options “Yes” or “No.” What problem can this create?
What is forced choice / insufficient response categories?
This is the term for each individual question asked in a survey.
What is an item?
You’re testing whether dark mode affects reading speed. What type of variable is dark mode versus light mode?
What is an independent variable?
A survey asks: “Do you agree that Brown should switch to digital or tap ID access for dorms?” What problem does this question have?
What is a double-barrelled question / two questions in one?
A survey asks: “How often does your computer crash?” with the response options “Less than once per month,” “Once per month,” “Once per week,” and “More than once per week.” What problem do these response options have?
What are missing intervals? (no twice per month option)
This is the term for what researchers are trying to measure from a survey, such as satisfaction or trust.
What is a construct, domain, or variable?
You want to compare two versions of an app by having one group of participants test Version A and a different group test Version B. What type of experimental design is this?
What is a between-subjects / between-group design?
A survey asks: “How satisfied are you with the new mayor’s policies?” What problem does this question have?
What is a leading question?
A survey asks: “How often do you exercise?” with response options “Regularly,” “Occasionally,” and “Never.” What problem do these response options have?
What are vague wording / vague response categories?
You want to learn the exact sequence of clicks students use when registering for classes. What research method would be most appropriate?
What is observational usability testing?
You conclude that a new AI tutoring tool improves exam performance, even though it actually has no effect. What type of statistical error have you made?
What is a Type I error / false positive?
A survey asks “How many visits to a primacy care provider have you had in a lifetime?”
What is recall bias?
A survey asks: “How many times a week do you walk your dog?” with response options “0 times,” “1-2 times,” “3-4 times,” and “5 or more times.” What problem does this question have?
What is assumption about the respondent? (no way to indicate that you don't have a dog)
You want your survey sample to proportionally represent Brown students from each year (first-year, sophomore, junior, and senior). What probability-based sampling method should you use?
What is stratified sampling?
A company compares productivity between its New York office, which adopts a four-day workweek, and its Boston office, which keeps a five-day schedule. What type of experiment is this?
What is quasi-experiment? (not randomly assigned to two conditions)
A survey asks participants to select an age range (18-25, 26-35, 36-45, 46+), but researchers later need to analyze how outcomes change with each year of age. What problem does this create?
What is data degradation?
A survey asks students to pick their favorite restaurant among a long list. The first three get picked far more often than the rest. What effect might explain this?
What is the primacy effect?
You survey students across all four years at Brown. While 82% of sampled first-years respond, only 7% of sampled seniors respond, leaving seniors underrepresented. What potential source of bias does this create?
What is nonresponse bias?
Researchers ask participants to complete the same task twice using two different interfaces. Participants perform better the second time. What is this effect called?
What is the learning effect?