Experimental Designs
_______________ is a type of single-factor design where participants are only exposed to one level of the independent variable.
Between-Subjects Design.
When are the mean, median, and mode of a data set the same?
In a perfectly normal distribution
If you had to pick one measure of variability to summarize a dataset, what would it be? Explain your answer.
Standard Deviation - accounts for all data points, describes distribution of all data around the mean.
What separates independent samples t-tests from one-way ANOVAs?
Alex finds that a mindfulness program reduces anxiety for low-social-support participants but not high-social-support participants. Name and define the type of effect being described here.
Interaction Effect - where the effect of one IV on the DV differs depending on the level of another IV.
(AKA: Moderator)
Sara has a database of U.S. gold, silver, and bronze medal winners from the most recent Summer and Winter Olympics.
Her friend asks her to provide one data point that tells her how well the U.S. athletes performed. Which measure of central tendency would you recommend?
Will accept Mode or Median with the right explanation!
Which measure of variability helps you identify outliers? What is the threshold used?
Z-score (>3.0 / <-3.0)
What distinguishes inferential statistics from descriptive statistics?
Inferential statistics use sample data to make inferences about whether results generalize to the population. Descriptive statistics only summarize the data in hand.
Name and define the three disadvantages of within-subjects designs and how you would address each of them?
Practice effects (improvement from repeated DV exposure), fatigue effects (decline from repeated exposure), and contrast effects (response to one condition is influenced by comparison to another).
Counterbalancing addresses all 3.
Andy is evaluating his sales team over the past month and the data is as follows:
Gerry: 8, 0, 4, 12
Paul: 6, 6, 6, 6
Lisa: 0, 12, 0, 12.
Management want to promote the best performer. Who would you recommend and which measure of central tendency would you use to justify your decision?
Mode
Jenny scored a 75 on an exam where the class mean was 50 and SD was 5. What is Jenny's z-score, and how would you describe her performance?
z = (75 − 50) / 5 = 5.0. Jenny's score is 5 standard deviations above the class mean — exceptional performance.
What two factors influence the p-value in null-hypothesis significance testing? Explain the effects and their rationale.
(1) Sample size — larger samples reduce sampling error and produce smaller p-values. The larger the sample the more of the population you are capturing and the less likely any relationship you find will exist if the null were also true. (2) Relationship strength — stronger effects produce smaller p-values. The stronger the relationship the less likely the finding is by chance / the null is true.
Dr. Okafor is studying whether a new cognitive-behavioral therapy program helps adults manage anxiety. She recruits participants by political affiliation (liberal, centrist, conservativel) and randomly assigns half of each group to receive the therapy, while the other half receives no treatment. After eight weeks, she measures anxiety reduction scores. Her results surprise her: liberals in the therapy condition showed an average 12-point drop in anxiety, conservatives in the therapy condition showed only a 1-point drop, and centrists respondents showed no improvement — despite being in the same program.
(1) Is this a within or between subjects design?
(2) Write the factorial notation for this design?
(3) Name and define the specific type of effect this pattern illustrates.
1. Between
2. 3 X 2
3. Interaction Effect / Moderation
Which measure of central tendency is best to describe the below data set? Explain your answer.
DATA: 1, 0, 5, 2, 31, 6, 4
Median
Mode: No Mode
Mean: Big Outlier
Three researchers each report their the following regarding their dataset exploring similar topics:
A: Range = 40, Mean=30, SD = 2
B: Range = 40, Mean = 45, SD = 12
C: Range = 40, Mean = 45, SD = 19
(1) For Researcher A, what does a very low SD relative to the range suggest about the data? What other data point supports this conclusion?
(2) For Researchers B and C, what do the differences in SD tell you about the two data sets?
(1) Scores are tightly clustered near the mean; the wide range is likely driven by one or two extreme outliers. The lower mean.
(2) Scores are more widely spread throughout the distribution.
Marcus is a graduate student evaluating whether study format affects exam performance. He tests three groups: students who studied alone, in pairs, or in groups of five. After the exam, he runs three separate t-tests — alone vs. pairs, alone vs. groups, pairs vs. groups — and finds that the alone vs. groups comparison yields p = .04. He concludes that study format significantly affects performance and rejects the null hypothesis. His advisor looks at the analysis and immediately spots a problem.
(1) What is the fundamental flaw in Marcus's analytic approach, and what should he have done instead?
(2) Should Marcus reject or retain his null hypothesis?
(3) If he were to publish these results, this would have likely been an example of what type of error?
Marcus runs a study where participants all participants complete a memory test after 0mg, 200mg, and 400mg of caffeine across three separate sessions.
(1) What type of design is this?
(2) Name two advantages of this type of design?
(4) Which statistical test is most appropriate for analyzing these data?
(1) Single-factor within-subjects design. (2) Reduces individual differences as a confound; requires fewer participants. (3) Repeated measures ANOVA.
A researcher reports annual income for a random sample of households in Gary, Indiana. He reports Mode = $30K, Median = $45K, Mean = $101K for the sample measures of central tendency.
Answer the following based on the information provided:
(1) What does this data set illustrate about the median and mode?
(2) Which measure best represents the 'typical' earner and why?
(3) If the researcher removed the top 5% of earners, predict what would happen to each of the three measures of central tendency.
1. Less sensitive to outliers.
2. Median - Mean is likely skewed by outliers and we can't know how many people endorsed the mode.
3. Mean should drop a lot. Median may drop a little. Mode shouldn't change.
Answer the following based on scores of a standardized test that has M=500 and SD=100.
(1) Mia scores 250 — what is her z-score?
(2) Omar has z=−1.5 — what is his raw score?
(3) A third student has z=3.2 — should this be classified as an outlier?
(4) A fourth student has z=0 — what does this tell us about her score?
(1) -2.5
(2) 350
(3) Yes
(4) Score of 500 / Z=0=M
Dr. Vasquez has spent years arguing with her colleagues about the best way to teach introductory statistics. She recruits 80 students — none of whom have taken statistics before — and randomly assigns them to one of four course formats: a traditional lecture section, a flipped classroom where students watch videos before class, a Socratic seminar driven entirely by discussion, or an online-only section with no in-person meetings. At the end of the semester, all students take the same standardized exam. When she runs her analysis, she gets p = .03 and immediately drafts an email to her department chair declaring victory.
(1) Assuming she made the correct choice, what statistical test did Dr. Vasquez use for this analysis, and why is it the appropriate choice given her design?
(2) Her p = .03 — what does this result actually tell her, and what important question does it not answer?
(3) What should she do before sending that email to her chair, and what is the purpose of this next step?
(1) One-way anova
(2) that the groups are significant different one another - different from each other to such a degree that it is unlikely to have occurred by chance if the null hypothesis were true. Doesn't tell you WHICH are different and how.
(3) Post-Hoc Testing