Measurement
Explain internal validity
internal validity means the study allows for the isolation of a causal effect from independent variable (X) to dependent variable (Y). Properly maps onto to concept of interest
Define and provide an example of a spurious correlation
Spurious correlation is a third (Z) variable that is causally related to both X and Y that makes it appear as though X and Y are causally related. (e.g. Prejudice produces spuriousness in the relationship between core political values and preferred level of immigration)
Define and provide an example of an observational non-experimental study
Observational studies lack researcher control over treatment and random assignment. Approach to theory-testing is similar because observational studies evaluate a causal claim by comparing different values of X on outcome of Y
e.g. He Earns, She Earns (Less)
What are some strengths of non-experimental studies?
Allows researchers to better understand phenomena that cannot be experimented on. Correlation between gender and income. Correlation between country of immigration and visa denial.
Explain external validity
External validity means that the result obtained in the data generalizes to the broader population of interest.
If I advance a hypothesis that economic inequality increases the likelihood of political violence, what is the null hypothesis
H0: Economic inequality is unrelated to the likelihood of political violence. The null hypothesis is quite simply that no relationship exists between the two variables
Define and provide an example of a natural experiment
In a natural experiment values of the independent variable X are assigned on a random basis by nature (not by the researcher!)
e.g. Cholera outbreak (1849)
How should we think about slope coefficients when explaining the direction of a correlation?
Imagine that the slope coefficient is 1.32 and that the correlation is positive (Doesn't matter what the variables are)
A one unit increase in the independent variable is associated with a 1.32 point increase in the dependent variable
Explain measurement reliability
Reliability means that you should get the same (or close to) the same result even if repeating the measurement multiple times.
Before inferring that a correlation is causal what are some things that you need to think about?
Reverse causation, mistaken causal mechanisms (ie. the why?), spurious correlation
Define and provide an example of an experiment
Treatment and control group randomly assigned by researcher. Researcher has control over treatment and manipulated the values that the independent variable (X) takes on
What is the difference between probability sampling and quota sampling?
Probability sampling is when each member of the population of interest has an equal and non-zero chance of being included in the sample. This should give you a sample that is representative of the entire population (in expectation).
Quota sampling is where the researcher divides the population into subgroups based on certain group characteristics (gender, race, education level etc, rural vs. urban...) and then sets quotas for each subgroup.
Using an example explain the difference between sampling bias and measurement bias
Sampling bias occurs when the individuals selected for the study are selected in such a way that systematically over represents a certain group relative to the entire population. (e.g. only using university students in your sample)
Measurement bias occurs when the method of measurement systematically produces an upwards or downwards bias of the true value of a variable (e.g. how racist are you on a scale from 0 to 100. Those who actually hold racist attitudes will be discouraged from revealing them because being a racist is not socially desirable. Won't give you an accurate measure of prejudice)
In Rectanglia and Parallelistan both countries had revolutions. Rectanglia has low economic development, a free press, extensive ethnic diversity, and a discontented middle class. Parallelistan has high economic development, extensive press restrictions, is ethnically homogenous, and has a discontented middle class.
Using Mill's method of agreement was is the causal factor of the revolution?
The only causal factor that is the same in both countries is the discontented middle class. Therefore the IV middle class discontent is causing the DV revolution.
How do experiments allow researchers to rule out reverse causation
Researcher control over the design of the treatment and control questions allows us to rule out the possibility of reverse causation. For example, if a researcher wants to know whether manipulating awareness of functional and humanitarian goals of immigration policies increases support for immigration, they might say something like:
Treatment - A non-partisan commission has recommended increasing the number of legal immigrants allowed to come to the Canada from Mexico each year by 100,000 to protect innocent people whose lives are threatened by violent drug cartels. Do you support or oppose this proposal?
First awareness of humanitarian goals are primed, then participant is asked their preferred level of immigration.
What is the danger of selecting on the dependent variable?
If we select on the outcome we might miss a case that shares the outcome but not the common cause, or we might miss a case that shares a common cause but had a different outcome
What does statistical significance mean?
P value must be less than 0.05. Statistical significance means that there is a less than 5% chance that in a world where the null hypothesis is true we observed the results we did purely by chance
The country of Rectanglia had a revolution but Hexagonium did not!
Rectanglia has low economic development, a free press, extensive ethnic diversity, and a discontented middle class. In Hexagonium, there is low economic development, a free press, and extensive ethnic diversity, but the middle class is content.
Using Mill's method of difference explain why Rectanglia had a revolution but Hexagonium did not
In Rectanglia the middle class is discontent must be the causal factor for revolution because there was no revolution is Hexagonium AND no middle class discontent.
How do experiments allow researchers to rule out spurious correlations
Random assignment to control and treatment groups rules out possibility of spuriousness. Randomness cannot be correlated to anything. Any spurious variables that might be influencing the causal relationship between X and Y are equal across control and treatment groups and would therefore cancel each other out. Only thing that is different is the treatment itself.
Imagine that an observational study finds a positive correlation between UFO sightings and vote share for the Republican party.
How would you make this observational study into an experimental one? You're not restricted to surveys.
- randomly assign some piece of media dealing with UFOs, see if it makes them more supportive of Republican candidates
- randomly assign some fake UFO (ie drones or something) to areas, see if it makes them more supportive of Republican candidates