Internal Validity
External Validity
Matching Research Question to Study Design
Matching Study Design to Analytic Approach
Data Analysis
100
Internal validity refers to the extent to which a study design can rule out _______ _______ for findings.
alternative explanations
100
This recruitment strategy will result in a sample that generalizes only to people you know.
Recruiting from one's personal social media networks
100
A research question should include all of the main ________ explored in the study.
constructs/variables
100
Adam plans to run a linear correlation to examine the relationship between self-esteem and academic performance. What is his research question?
Is there a linear association between self-esteem and academic performance?
100
To run a linear correlation, you need two variables that are not categorical but ____________
Continuous
200
Dr. Dutra designs a Research Methods tutoring program for students in her class. She takes the lowest performing students early in the semester and enrolls them in the program. From the beginning to end of the program, students' grades improve by 15%! She concludes that her program is a success. Name one relevant threat to internal validity.
Regression to the mean; history; maturation; others
200
This type of research does not concern itself with external validity, because it is not intended to generalize beyond the participants studied.
Qualitative Research
200
Longitudinal designs are considered superior to cross-sectional designs for establishing cause and effect relationships. Why?
They can establish the temporal relationship between cause and effect (that the presumed cause happens before the presumed effect).
200
Ernesto plans to compare individuals with Major Depressive Disorder to a healthy control group on sleep quality, using a continuous index of sleep quality from high to low. Name one appropriate statistical test.
ANOVA or t-test
200
What is the appropriate statistical test to examine one or more variables as possible predictors of an outcome?
Regression
300
Dr. Bad-Researcher wants to know whether Zoom-based teaching differs from classroom-based teaching in its effectiveness. He administers measures of teaching effectiveness pre-Covid, then administers the same measures to the same classes after the transition to remote learning during Covid. Name and explain the most salient threat to internal validity.
History effects - teaching effectiveness may have changed during this time for a variety of reasons related to a major historical event - the pandemic. These are not necessarily limited to the change in teaching modality (classroom vs. Zoom).
300
Ethan hypothesizes that being in a positive mood will improve performance on a cognitive test. He randomly assigns participants to a positive mood induction (viewing pictures of puppies) vs. a neutral mood induction (viewing a blank screen), then asks them to complete a cognitive test. Consistent with his hypothesis, participants in the positive mood condition perform better than those in the neutral mood condition. Name the most salient threat to external validity for this study.
Narrow stimulus sampling - unclear whether results would generalize to other positive mood induction stimuli and/or other types of cognitive tasks. (Generality across measures is also acceptable here)
300
A researcher hypothesizes that individuals with Major Depressive Disorder will show lower levels of self-compassion compared to a healthy control group. Name an appropriate assessment tool for determining which participants are appropriate for the Major Depressive Disorder group.
The SCID Depression Module (or another structured, empirically validated diagnostic tool). Continuous measures of symptom severity (e.g. BDI) cannot diagnose!
300
A researcher would like to know whether men and women in their sample differ on whether or not they have completed a college degree. What is the appropriate statistical test to evaluate this?
Chi square test (compares groups on another categorical variable)
300
A factorial ANOVA is one that includies multiple ________ variables.
Indepdendent
400
For her DP, Larissa wants to know whether students' first year coursework in the PsyD program improves their critical thinking. She assesses first year students' critical thinking in September, then again in June. She finds that critical thinking has significantly improved, and concludes that the academic coursework is responsible.
Name and explain the most salient threat to internal validity.
Maturation effects (though history effects, practice effects, and others may also be relevant)
400
Historically, most research participants in academic psychology have been drawn from a particular type of demographic - name the acronym that describes this demographic and specify what each letter stands for.
WEIRD participants - Western, Educated, and from Industrialized, Rich, and Democratic societies
400
This is a visual representation of what kind of model?
Mediation Model
400
Kim's research question is the following: "Does age predict wisdom via accumulated knowledge?"
What is the appropriate statistical test/analytic approach for this research question?
Mediation analysis
400
A MANOVA is an ANOVA that includes multiple _______ variables.
Dependent
500
Drs. Ott and Dutra want to compare the effectiveness of two teaching strategies. Dr. Dutra implements Strategy 1 in her sections, while Dr. Ott implements Strategy 2 in his sections. At the end of the year, learning outcome data is compared between students who received Strategy 1 vs. Strategy 2. Name the most salient threat to internal validity.
Selection bias - sections may differ in important ways having nothing to do with teaching strategies. For example, Dr. Dutra's sections may be held at different days/times than Dr. Ott's sections, and this may affect learning outcomes.
500
Participants drawn from online samples have been found to generate similar results compared to their in-person, laboratory counterparts. However, they are different in one way that may enhance generalizability of results. How are online samples different from laboratory samples?
Online samples are more demographically diverse - online recruitment allows individuals who could not come to a lab physically to participate.
500
This is a visual representation of what kind of model?
Moderation model
500
A researcher plans to run a one-way ANOVA comparing dog owners with pet-free individuals on life satisfaction. In order to run the analysis as planned, should the researcher select a measure of life satisfaction that sorts individuals into groups (high, moderate, and low), OR one that generates a life satisfaction score for each individual, ranging from 0 to 100?
Continuous Measure (life satisfaction score for each individual, ranging 0-100). ANOVAs require continuous dependent variables.
500
Multicollinearity occurs when predictors in a regression are too highly __________
Correlated