Sampling Error, CLT, & LLN
This term describes the natural variation in a statistic from sample to sample within a population.
What is sampling error?
In NHST, this hypothesis says any difference found is just sampling error.
What is the null hypothesis (H₀)?
If a 95% CI excludes zero, the result is this.
What is statistically significant?
Cohen’s d, Pearson’s r, and η² are all examples of this.
What are effect sizes?
Rejecting a null hypothesis that is actually true.
What is a Type I error (α)?
Larger samples reduce sampling error, according to this law.
What is the Law of Large Numbers?
A p-value below alpha (usually .05) leads to this decision about H₀.
What is reject the null hypothesis?
A narrow confidence interval signals this, usually thanks to a larger n.
What is precision (a precise estimate)?
Statistical significance tells you the effect is likely real. Effect size tells you this.
What is how much it matters (whether it’s meaningful in practice)?
A Type II error means failing to reject a null that is this.
What is false?
With large enough samples, the sampling distribution approximates this shape, even if the population variable isn’t shaped that way.
What is a normal curve (Central Limit Theorem)?
The p-value is <.001, so you should make this decision.
What is the reject the null hypothesis?
A 95% CI for a mean difference is [−1.2, 3.4]. What is your decision about H₀, and why?
What is fail to reject, because the interval includes zero (not statistically significant)?
A “small” effect is described as this.
What is there, but not meaningful (or noticeable)?
Power equals this expression using β.
What is 1 − β?
This measures how much a sample statistic typically varies from sample to sample just by chance.
What is the standard error?
In the courtroom analogy, failing to reject the null is equivalent to this verdict.
What is “not guilty”? (We never prove H₀ true.)
Technically, if we repeated a study 100 times, about this many 95% CIs would contain the true population parameter.
What is 95?
Researchers use an expected effect size in a power analysis to plan this before collecting data.
What is sample size?
The conventional target level of power.
What is 80% (.80)?
Name the two things that make the standard error smaller.
What are larger samples and less variability in scores?
The p-value is .06, so your decision about the null hypothesis would lead you to this conclusion about statistical significance.
What is not statistically significant?
The “textbook” definition says there’s a 95% chance the true parameter lies in your interval. The more technical definition instead describes this.
What is the long-run behavior across repeated studies (95% of the intervals from repeated samples would contain the true parameter)?
A result has p = .001 but a tiny effect size. It is statistically significant, but this is questionable.
What is its practical meaningfulness (does it matter)?
Power analysis needs 3 of these 4 to calculate the 4th, and is done a priori. Name all four.
What are power, sample size, alpha (significance level), and minimally interesting effect size?