Name That Design
Researchers interview 20 Grade 6 students about how they experience group work.
Qualitative research
Students who report greater school belonging also report greater classroom engagement. What can we conclude?
School belonging and classroom engagement are associated/related.
Finding: Students who regularly eat breakfast tend to have higher grades. What else could explain the relationship?
Examples: family resources; sleep quality; household routines; school attendance; health; stress.
An intervention is studied with Grade 5 students. A teacher wants to use the findings when considering the same intervention with another group of Grade 5 students. Should we automatically assume it will work exactly the same way?
No
Someone says, “Research shows homework works.” What’s ONE question you should ask before accepting the claim?
Examples: Works for what? What kind of homework? Which students? What outcome? Under what conditions? What research?
Researchers measure students’ sense of school belonging and classroom engagement to see whether the two are related.
Correlational research
Students who sleep more tend to earn higher grades. A researcher concludes, “Getting more sleep improves students’ grades.” What’s wrong with that conclusion?
The evidence shows an association, not causation.
Finding: Students who participate in extracurricular activities report greater school belonging. What else could explain it?
Examples: pre-existing social connections; motivation; family resources; personality; opportunities/access; prior sense of belonging.
A strategy improves an outcome in a study of Grade 2 students. Someone says, “Great! This proves it will work with Grade 12 students.” What’s the problem?
The populations differ substantially in age/developmental context.
Someone says, “Research proves smaller classes improve learning.” What information would you want before accepting that conclusion?
Any relevant questions about the claim, evidence, context, alternatives, or application.
Researchers observe students during independent work and record how often they ask teachers or peers for help.
Descriptive research
Students are randomly assigned to Strategy A or Strategy B. Students receiving Strategy A subsequently perform better on the assessment. What can we reasonably conclude?
The experimental evidence supports the conclusion that Strategy A contributed to the difference in outcomes.
Finding: Students who read more books tend to have larger vocabularies. What else could explain it?
Examples: prior language ability; home literacy environment; access to books; family resources; interest in reading; parental education/support.
A study finds that a strategy supports learning during Grade 7 mathematics. A school decides, “We should use it in every subject.” What’s the caution?
Evidence from one subject does not automatically generalise to every subject/context.
A single well-designed study finds positive results for a new teaching strategy. A teacher says, “It’s research-based, so everyone should use it.” What’s the problem?
One study does not settle an educational question or automatically justify universal implementation.
Students are randomly assigned to either use retrieval practice or reread their notes. Researchers then compare their test performance.
Experimental research
Researchers interview students who used a new collaborative-learning strategy. Many describe feeling anxious and excluded. A teacher concludes, “Collaborative learning causes anxiety.” What can the study actually tell us?
It provides evidence about how these students experienced and understood the strategy.
Finding: Students who attend optional exam-review sessions earn higher exam grades. A professor concludes, “The review sessions improve achievement.” Give TWO alternative explanations.
Examples: more motivated or higher-achieving students may attend; conscientiousness; available time; study habits; prior preparation.
Researchers find positive results with students from four urban schools in one Canadian city. A headline says, “New strategy works for all Canadian students.” What’s wrong with the claim?
It generalises beyond the population and contexts actually studied.
Three studies conducted in different educational contexts reach similar conclusions using different research methods. What happens to our confidence in the findings?
Our confidence generally increases.
Researchers survey 300 students about school belonging and then interview 25 students to better understand experiences that contribute to feeling included or excluded.
Mixed methods
FIX THE HEADLINE: “STUDY PROVES PHONES DESTROY STUDENT LEARNING.” Actual finding: Students reporting greater daily phone use tended to have lower average grades. Rewrite the headline so it matches the evidence.
Higher reported phone use was associated with lower grades.
Finding: Schools using more educational technology have higher standardised test scores. Generate THREE plausible alternative explanations.
Examples: school funding/resources; socioeconomic context; teacher training; staffing; educational supports; school leadership; student demographics.
FIND THE BOUNDARIES. Study: 75 Grade 7 students; one private school; mathematics; three weeks; outcome = self-reported engagement. A principal says, “This proves we should implement the strategy K-12, in every subject, because it improves student learning.” Identify THREE reasons for caution.
WHO: Grade 7 only. WHERE: one private school. WHAT: mathematics only. WHEN: three weeks. HOW: self-reported engagement. CLAIM: “student learning” goes beyond what was measured.
“Research proves technology increases student engagement.” Your team has 60 seconds to generate as many questions as possible that you would need answered before accepting this claim.
technology; use; definition/measurement of engagement; students/age; setting; comparison; duration; research design; manipulation/random assignment; alternative explanations; body of evidence; replication; applicability