Ch. 5: Sampling & Probability
This is the colloquial name for a Type I error
What is false positive?
Augustus has received back a confusingly marked exam which indicates his grade is "z = -2". This is a description of Augustus' performance compared to his peers.
What is Augustus' score is 2 standard deviations below the mean score for the class?
Or: Augustus scored equal to or better than 50-(95/2) = ~2.5% of students in the class
This is the definition of a critical value.
What is the p-value threshold for determining whether a result is statistically significant?
This is the appropriate statistic for assessing variability when you are estimating a population-level confidence interval from a sample.
What is standard error? AKA standard error of the mean (SEM)
This is the "correct" numeric format (for this class) on any problem which requires answering as a "proportion" or "probability".
What is a decimal representation?
A researcher has performed a z test to assess the difference in mean scores between two groups. They find their results are significant at α = 0.05 but not at α = 0.01. Which of the following cannot be the researcher's p-value? (Choose all that apply)
a) p = 0.005
b) p = 0.05
c) p = 0.035
d) p = 0.95
What is (a) (p = 0.005) and (d) p = 0.95?
Amanda has received back a confusingly marked exam which indicates her grade is "z = 1". This is Amanda's percentile rank.
What is ~84th percentile?
(68% of data within 1 standard deviation from the mean, so 34% between z=0 and z=1, plus 50% for z < 0)
A researcher selects 100 people from her local library to examine the effects of reading on attentional processing. She randomly assigns 50 subjects to read an excerpt from a novel and 50 to listen to a clip of the corresponding audiobook before giving them a test of attention. When she looks at her data, she is dismayed to see that the results are not normally distributed! Should she proceed with a z-test? Justify your answer.
What is ¯\_(ツ)_/¯ ?
For yes: non-normally distributed data don't necessarily mean that you can't run a statistical test -- just that you should be very careful in your interpretation!
For no: There are legitimate concerns about normality and external validity/generalizeability given the convenience sample described. Anoter test (such as a nonparametric test) may be more appropriate in this case.
This is a list of values one needs in order to compute a confidence interval.
What are mean, standard deviation, sample size, and the z values for the interval?
I collect a sample from 100 grad students about their experiences at BU and compute a confidence interval on their reported levels of satisfaction. (It's low!)
Afterwards, I add another 100 grads to my sample*. Assuming no major differences between the groups, this is the effect on my confidence interval.
What is the confidence interval shrinks?
Interval == mean +/- standard error == mean +/- (standard deviation/sqrt(sample size)) -- so increasing sample size decreases standard error, causing the interval to get smaller.
A researcher has performed a z test to assess the difference in mean scores between two groups. They find their results are significant at α = 0.05 but not at α = 0.01. These are at least two ways the researcher may "improve" their results?
What are increasing sample size, using a one-tailed instead of a two-tailed hypothesis, adjusting sample to be more representative, ...
Note: this is not good practice!
Hazel has received back a confusingly marked exam which indicates their grade is "z = -2". The professor posts that the mean grade was an 86%, and the standard deviation is 12%. This is the best estimate of Hazel's score.
[Daily double]: Can you spot the issue with this question?What is 62%? (86% - 2 * 12%)
[DD]: The data can't be perfectly normally distributed as described! (2 standard deviations above the mean would be above the maximum score, so this data must be skewed!
A researcher is testing the relationship between eating peanut butter and happiness. They collect a simple random sample of 100 people and separate them into groups; one group eats peanut butter, while the other eats broccoli. Then, the researcher asks participants to fill out a survey indicating their overall level of satisfaction in life. These are the one-tailed and two-tailed hypotheses in this experiment.
(One-tailed): H0: (Multiple ways to say this)
a) There is no difference in mean happiness scores between peanut butter and broccoli eaters
b) The happiness levels of peanut butter eaters are not higher than those of broccoli eaters
HA: The mean happiness of peanut butter eaters is greater than the happiness of broccoli eaters (other direction also accepted)
(Two-tailed): H0: Same as (a) above
HA: The mean happiness scores of peanut butter and broccoli eaters differ
A researcher samples 50 subjects from the Boston Public Library and 50 subjects from the Cambridge Public Library on daily time spent reading. They find that BPL readers only read for 35 minutes a day on average, while CPL readers read for 65 minutes. The researcher calculated the pooled standard deviation to be 7 minutes. This is the effect size (name of statistic + value)
What is 4.29 (very strong)
(mean of group 1) - (mean of group 2) / standard deviation = (65 - 35)/7 = 30/7
This is the difference (conceptually and mathematically) between standard deviation and standard error
What is that standard error scales your estimate of variability by sample size
Formula: Standard error = Standard deviation / sqrt(sample size) (Equivalently, SD = SE * sqrt(n))
A researcher conducts five separate experiments using five distinct independent variables that are thought to represent information about participant motivation. All experiments are performed with a set of 50 subjects (the same 50 in all cases). The experimenter is trying to address questions about this type of research validity.
What is internal validity?
These are the properties of a normal distribution that make it such a useful tool in statistics
What is that it is symmetric, unimodal, and always follows something close to the 68%-95%-99.7% rule? (68% of data within 1 sd from mean, 95% within 2 sd, 99.7% within 2)
These are the three assumptions/requirements one must meet to run a proper z test
What are:
1) Dependent variable is scalar (i.e., interval or ratio value)
2) Participants are randomly sampled from the population
3) Data are approximately normally distributed
** Note: You can still "run a z test" if one or more of these is violated -- just be very cautious interpreting your results!
These are at least 3 of 4 ways that a researcher can manipulate statistical power
What is...
a) Adjust alpha
b) Choose a different hypothesis (i.e. one-tailed vs two-tailed)
c) Change sample size/ choose a different sample
d) Manipulating the range of values your dependent variable can be (change in means or change in standard deviation)
(I will) Draw two overlapping normal distributions on the board and (you should) label α and β.
(Visual!)
Edges of H0: α
Edges of HA (below α cutoff): β
A researcher examining the efficacy of Covid-19 rapid tests finds that one in ten tests on Covid-positive patients shows no indication of the illness. They develop a new indicator formula which is 5x more effective at detecting Covid-19 when it is present in the sample. This is the effect that this change has on the false positive rate.
What is the false positive rate increases?
(False positive and false negative are inversely related! If you improve one, you Jeopardize the other. Pun intended :))
This is a description of the central limit theorem.
What is that building a distribution of means from repeated sampling will approximate a normal distribution, even when the underlying score distribution is non-normal?
[Daily double] A researcher finds a statistically significant result with p = 0.0162. They successfully recorded the mean of group 1 (2.5) and the standard deviation (.75) but misplaced their notebook containing the mean for group 2. What are the two possible options for the missing data?
What is 0.90 or 4.11?
p = 0.0162 => Look in z table => z = +/- 2.14 =>
mean - 2.14 * sd = 2.5 - 2.14 * 0.75 = 0.90 OR
mean + 2.14 * sd = 2.5 + 2.14 * 0.75 = 4.11
A researcher records the weight from a random sample of 100 college students and finds a mean of 165 lb and a standard deviation of 35 lbs. What is the 95% confidence interval for the population?
What is... [158.14, 171.86] lbs?
mean = 165
standard deviation = 35
standard error = 35/sqrt(100) = 35/10 = 3.5
95% confidence interval -> z = +/-1.96
mean +/- 1.96*standard error = 165 +/- 6.86 = [158.14, 171.86] lbs
A professor for a stats class finishes grading the first exam and decides to glance at the score distribution. To her surprise, the data are perfectly normal with a mean of 80% and a standard deviation of 5%. This is the percent of scores that fall between 72% and 94%.
What is 94.26%?
(72 - 80) / 5 => z = -1.6
(94 - 80) / 5 => z = 2.8
z=-1.6 => 5.48% of data "in tail" (below this)
z=2.8 => 0.26% of data "in tail" (above this) => 100% - 0.26% = 99.74% of data fall below this
% of data between 72% and 94% ==
% of data between z==-1.6 and z==2.8 == 99.74% - 5.48% = 94.26%