Study Design Basics
What was the independent variable in Nicholas et al. (2024), and was it actively manipulated by the researcher?
The independent variable was syntax variability in verb-focused intervention sessions. It was actively manipulated by the researchers by systematically alternating transitive and intransitive syntactic frames during treatment.
What was the dependent variable, and how was it measured systematically over time?
The dependent variable was receptive target verb knowledge, measured repeatedly through action demonstration probes where toddlers performed an desired action when given an object and prompt
- This was measured multiple times per week across 17 weeks
Why do we need a stable baseline before starting an intervention?
To ensure that later change is due to the intervention itself (internal validity).
What type of design did the authors use?
Concurrent Multiple Baseline Design
How many total phases are needed to meet WWCH design standards?
6
How many data points were collected per phase, and did this meet WWCH criteria?
≥5 per phase for each participant; meets WWCH minimum threshold.
Does visual analysis support presence of a quality baseline?
why and/or why not?
Mixed evidence
- participants showed variability in baseline 1 (poor)
- no presence of a trend in the baseline phase (good)
- 5 or more points per baseline phase (good)
How did the researchers demonstrate replication of effect?
- By staggering intervention onset across three participants in a multiple-baseline design and withdrawal of intervention in maintenance phase
Were responses coded independently by more than one assessor?
Yes, sessions were coded by trained research assistants masked to session order, with interrater reliability of 91.8%–97.2%
- disagreements between research clinician and undergraduate research clinician encoders were resolved by a blinded 3rd coder
What level of inter-assessor agreement meets minimal WWCH threshold, and did the article meet this standard?
80–90% required; article exceeded with 91.8–97.2%.
Does visual analysis support the presence of an effect? Why and/or why not?
Mixed evidence
- delayed latency of change for participant 3 (not till treatment 8) (delayed latency of change) (poor)
- no trend in the desired direction during baseline phase (good)
- Participant 1 showed high variability in baseline phase (highly variable data with overlap between baseline and intervention). (poor) NOTE SIGNIFICANCE IN EFFECT SIZES ANALYSIS FOR BASELINE-MAINTENANCE COMPARISON SAYS LARGE EFFECT OBSERVED
- Participants 2 and 3 showed variable treatment performance (possibly due to disengagement). (poor)
Does the study meet WWC standards for demonstrating an effect across cases?
Yes, meets minimum 3 cases and minimum 6 phase standards.
Did the study meet the WWCH phase standards for multiple baseline designs?
Yes, 3 participants × 2 phases = 6 phases; meets minimum WWCH standard.
How many times was the intervention effect reliably demonstrated? How do we know?
3 replications across participants in at least 6 phases; interrater reliability 91.8–97.2% confirmed consistency.
Overall how strong is the visual evidence?
Moderately strong
Based on visual and replication evidence, how strong is the overall evidence?
Moderate — improvements across participants but variable of gains (3–4 points each) latency.