Experiments I
What is the #1 strength of experiments? What is the #1 weakness?
Strength-internal validity; weakness-external validity
What are the two groups in the classical experiment?
The experimental/treatment group and the control group
An example of the one-shot case study is that a man who exercises is observed to be in good shape. Why is this not proof that exercising causes one to be in good shape?
Lack of time order: Does he have a fast metabolism and is already in good shape, so that makes it easier to exercise?
Could be spurious: Is there something else that could be causing him to be in good shape, such as eating healthy?
No control group for reference
Validity talks about the soundness of the research design and methods. While carrying out the experiment, the researcher has two objectives, relating to the validity, i.e. draw conclusions about the impact of an independent variable on the group under study and make inferences about the population as a whole. The first objective stresses on _______ validity, whereas the second focuses on __________validity.
Internal; External
What are the three criteria for causality?
correlation , causal direction(time order), and non-spuriousness
Are experiments better for inductive or deductive research? Why?
Deductive research – testing hypotheses & demonstrating causal effects
Experiments are good for focusing attention on a few variables and holding all others _______
constant
An example of the one group pre-test/post-test design is that an overweight man who exercises is later observed to be in good shape. Why is this not proof that exercising causes one to be in good shape?
Could be spurious: Is there something else that could be causing him to be in good shape, such as eating healthy?
No control group for reference.
_________ validity refers to the degree to which cause and effect relationship based on the experiment is warranted, ascertained by the extent to which the experiment avoids systematic errors. High ________validity allows the researcher to choose one explanation over the other with enough confidence, as it ignores confounds. The less the confounding in an experiment, the higher is its ________validity.
Internal
Why is “testing”(being part of an experiment) a potential threat to internal validity?
the effect of being measured could change your responses
Professor Causality is running an experiment. He wants to estimate the effect of watching a film about climate change on attitudes towards environmental regulation. He has randomly divided his study population into two groups – C and T. Group T will watch the film about climate change and Group C will watch a film about art history. Before administering the treatment, he administers a survey to members of both groups to gauge their attitudes towards environmental regulation. He administers the survey again after the treatment. If the randomization for assignment and the treatment are both successful, then we would expect that:
a. Group T and C were slightly different before, but much more different after the treatment
b. Group T and C were the same before, and the same after the treatment
c. Group T and C were different before, and different after the treatment
d. Group T and C were the same before, but different after the treatment
d
__________ give us a basis for comparing what happens with and without the stimulus.
Control groups
An example of the static group comparison is that a man who exercises is observed to be in good shape while one who doesn’t is observed to be overweight. Why is this not proof that exercising causes one to be in good shape?
Lack of time order: Does the one who exercises have a faster metabolism and is already in good shape, so that makes it easier to exercise?
The term ________ validity implies the determination of whether the casual relationship observed in the study can be generalized or not. It ascertains, can the results obtained through the experiment be generalized to other situations and if so, to what settings, groups of people, times it can be extrapolated? Threats to _______ validity take place when the specific set of research conditions does not practically consider the interactions of other variables of the real world.
external
We compare the grades of students who received tutoring to those who do not and find that tutored students have higher grades. Assess the internal validity of this claim.
Internal validity –the issue of whether we have shown that our IV causes our DV – is questionable because we can think of reasonable alternatives. Tutored students are more focused on school and less on partying, getting a job, work, sports, etc., so they study harder. Also, tutored students don’t stay out as late and get more sleep, which improves their recall.
Why would you study an organization or community in a laboratory? It allows researchers to create (and thus study) processes or outcomes that are difficult to find in _____________.
Naturally occurring settings/ the real world
A _____________ experiment is when neither the participant nor the researcher knows who is in the experimental/treatment group and who is in the control group.
double-blind
This one doesn't belong in this category(sorry)...
The measurements taken before and after the IV occurs are referred to as what?
pre-tests and post-tests
Why is attrition(people dropping out of the experiment) a potential threat to internal validity?
there may be something different about the people who drop out that will skew your results
Name one benefit and one downside of doing a field study over a lab experiment.
Downsides:
Generally lower internal validity than a lab experiment
Fewer variables are controlled, so inferences are often difficult.
Benefits:
Generally higher external validity than a lab experiment
Generally higher construct validity than a lab experiment
Less artificial than laboratory experiments
What are the two ways to assign subjects to treatment groups? Describe them.
Matching(choosing who to assign to treatment and control) and randomization(use probability sampling to randomly assign people to groups)
When we can’t eliminate external causal factors, what can we do instead?
Measure the alternative causal factors and control for their effects statistically, using multiple regression.
EVERYBODY DOES THIS ONE:
How would you design an experiment to support the hypothesis that lifting weights causes one to be in good shape?
- Consider control and treatment groups.
- Spuriousness.
- How to define "being in good shape"
(Your answer does not have to be perfect. I just want you to hit some of the main points of experimental design)
Have a treatment and a control group. Randomly assign participants to each group. The people in both groups would have to eat the exact same food or you would have to figure out how to control for diet statistically. Metabolism could be a spurious variable so you would want to control for its effects statistically. You might also want to think about the types of activities people do that they would not consider exercise(walking, walking up stairs, carrying things, etc) and control for those. Also, you would have to think about how to define “being in good shape”? (How fast someone’s heart-rate comes down? Their fat-to-muscle ratio?)
Treatment group: does 1 hour of the same weight-lifting exercises 5 days/week
Control group: does 1 hour of walking 5 days/ week
*Why are we having the control group walk instead of doing nothing? We don’t want to know the effects of exercising on good health? We want to know specifically the effects of weight-lifting? Also, what if just doing physical activity allows people to feel more positive and then that positivity is affecting their health rather than the exercise itself?
Why is it a potential problem that college undergrads are the most common experimental subject pool?
Could lower the external validity of your experiment because:
Not representative of the typical American
Typically come from better off families
Modal college student is female and Asian-American
Age (younger)
(correct if you name at least 2 of these)
Explain how “selection” is a potential threat to internal validity. Give an example.
Subjects in the 2 groups are not equal due to selection into the groups. Groups should be the same before the treatment is administered.
Example: An experiment on whether or not tutoring improves grades where you let the people choose what group they want to be in