Experiment Basics
What is the difference between RANDOM ASSIGNMENT and RANDOM SAMPLING? (i.e., which one increases internal validity, and which one increases external validity?)
*for 50 bonus pts, define MATCHED RANDOM ASSIGNMENT.
RANDOM ASSIGNMENT- each participant has an equal chance to be in any condition (once you gather WHO will be in the study). It increases INTERNAL validity, and has NO effect on external validity.
RANDOM SAMPLING- each population member has an equal chance of being sampled for the study. It increases EXTERNAL validity and has NO effect on internal validity.
MATCHED RANDOM ASSIGNMENT- cluster participants based on how similar they are, then randomly assign each member of the cluster to a condition.
We test the _________ hypothesis, not the _________ hypothesis.
We test the null hypothesis, not the alternative hypothesis.
If I'm conducting a paired-samples t-test, and my experimental hypothesis is:
"People will cry more during a sad movie when their dog is around."
What would be the null hypothesis?
People will not cry more when their dog is around.
This type of test, tests for how much means differ from one another (between-groups variance), compared to how much they expected to differ BY CHANCE (within-groups variance). It also can assess experiments with an IV with more than 2 levels and experiments with more than 1 IV.
ANOVA (observed variance in means/expected variance in means)
Name an issue with informed consent.
-Can compromise validity
-Some cannot give informed consent (ex. children/minors, people with intellectual disabilities)
There are 2 types of WITHIN-SUBJECTS designs. Name these for 100 pts each.
________ measures design -all levels of the IV are presented at the same time, then DV is measured.
________ measures design -participant is exposed to each level of the IV separately, but DV is measured after each exposure (instead of at the end).
CONCURRENT measures design - all levels of the IV are presented at the same time, then DV is measured.
REPEATED MEASURES measures design - participant is exposed to each level of the IV separately, but DV is measured after each exposure (instead of at the end).
Fill in the blank for 100 pts each.
a) ______________ allows us to estimate the "expected" difference (error variance) and defines the shape and location of sampling distribution.
b) When you draw random samples from the population and plot the means, you end up with a normal distribution centered around the _________.
***for 50 bonus pts, "you also end up with a smaller standard error proportional to the size of _________."
a) CENTRAL LIMIT THEOREM allows us to estimate the "expected" difference (error variance) and defines the shape and location of sampling distribution.
b) POPULATION MEAN
****the sample you took
What is the difference between a Posttest-only and a Pretest-Posttest one-way design?
POSTTEST-only: DV measured ONLY AFTER exposure to IV
POSTTEST: DV measured before AND after exposure to IV
For ANOVA tests, if my experimental hypothesis is: "Student satisfaction will differ by class schedule." What would be my null hypothesis?
Student satisfaction will not differ by class schedule.
Regarding the right to privacy, what is the difference between confidentiality and anonymity?
CONFIDENTIALITY- identifying info is collected but kept private/secret. "confidential information is not released."
ANONYMITY- identifying info is NOT collected (or it's simply impossible to link data to someone)
REPEATED MEASURES designs (a type of within-subjects design) can become vulnerable to ORDER EFFECTS due to a participant responding to later IV levels after being exposed to previous IV levels. As we know, this can affect measurements of the DV.
What are the 3 types of ORDER EFFECTS that can occur? (100 pts each)
*for 50 bonus pts, how can order effects be prevented/minimized?
PRACTICE EFFECTS- (ex. participants get better over time)
FATIGUE EFFECTS- (ex. they get worse over time due to being tired or bored)
CARRYOVER EFFECTS-something about one level carries over into how a participant performs/responds to another level (weights, tastes)
**partial/full COUNTERBALANCING -participants randomly assigned to combination of order of levels.
Type I and Type II errors cannot be minimized simultaneously but are both related to power. For 150 pts each, Describe how we can decrease each type of error.
***For 50 bonus pts describe at least one solution to having too much variance within groups.
To decrease TYPE I ERROR, we can decrease alpha.
To decrease TYPE II ERROR, we can increase power.
***Solutions: get more participants (increase N), decrease measurement error by standardizing procedures or using reliable measures, account for individual differences
Paired-samples t-tests are similar to independent-groups t-test, but paired-samples t-tests are for matched-subjects or ___________ designs.
WITHIN-SUBJECTS
There are multiple types of factorial designs. Which is which?(Randomized groups, repeated measures, mixed factorial) For 100 pts each.
1) ________: All IVs are manipulated within-subjects in that everyone experiences every cell (i.e., ever level of every factor)
2)________: At least one IV is manipulated between-subjects, and at least one is within-subjects in that each person experiences all levels of only 1 factor/IV.
3) _______: all IVs are manipulated between-subjects in that each participant only gets 1 level (i.e., each person experiences only 1 cell).
1) REPEATED MEASURES: All IVs are manipulated within-subjects in that everyone experiences every cell (i.e., ever level of every factor)
2) MIXED FACTORIAL: At least one IV is manipulated between-subjects, and at least one is within-subjects in that each person experiences all levels of only 1 factor/IV.
3) RANDOMIZED GROUPS: all IVs are manipulated between-subjects in that each participant only gets 1 level (i.e., each person experiences only 1 cell)
What are the 3 main ethical principles of the Belmont Report (1979)? 100 pts per principle named, BUT TRY TO DESCRIBE THEM!
I. RESPECT FOR PERSONS-individuals must be treated as independent, autonomous agents free to make decisions. Includes the concept of informed consent in that participants must understand costs and benefits & cannot be coerced into participating (must also consider deception and vulnerable populations)
II. BENEFICENCE- participants must be protected from undue harm. In doing so, researchers must weigh costs and benefits of study, provide the right to privacy, and debrief participants!
III. JUSTICE- participants must represent those who will benefit from the study by having just/fair sampling practices.
Name 2 strengths of INDEPENDENT-GROUPS designs and 2 strengths of WITHIN-GROUPS designs for 100 pts each.
INDEPENDENT-GROUPS: fewer order effects, useful when you can't measure multiple levels of IV w/ 1 participant (WEAKNESS -lower power: so need more participants)
WITHIN-GROUPS: higher power: need fewer participants, individual differences between conditions are eliminated because it's same person being compared with themself (WEAKNESS -possible order effects)
For 100 pts each, identify the error being describe (ie., Type I or Type II)
1) failed to reject the null when it was false (aka "miss") *THOUGHT ERROR VARIANCE BUT THERE WAS A REAL EFFECT!
2) rejected the null when it was true (aka false alarm) *THERE WAS AN EFFECT BUT JUST ERROR VARIANCE!
3) uses alpha
4) uses Beta
****FOR 50 BONUS pts, the ability to reject the null hypothesis when it's false is called _______.
1) failed to reject the null when it was false (aka "miss) *THOUGHT ERROR VARIANCE BUT THERE WAS A REAL EFFECT! ---TYPE II error--
2) rejected the null when it was true (aka false alarm) *THERE WAS AN EFFECT BUT JUST ERROR VARIANCE! --TYPE I error--
3) uses alpha --TYPE I error--
4) uses Beta ---TYPE II error--
****the ability to reject the null hypothesis when it's false is called POWER!!
For 100 pts each, name 2 benefits to using Posttest-only design and 2 benefits to using a Pretest-Posttest design.
***for 50 bonus pts explain how testing the DV multiple times can threaten internal validity!
POSTTEST-ONLY BENEFITS: helpful for minimizing demand characteristics, order effects (such as practice effects), and maturation effects and theoretically takes less time and is easier.
PRETEST-POSTTEST BENEFITS: higher power, can confirm if groups are similar from the start, & can evaluate degree of change from baseline .
***carryover, people can remember what they previously stated, did, etc., can have maturation/practice effects.
In ANOVA, how would you determine which groups differ using Tukey's HSD?
You would calculate the difference between means of all the groups and compare the absolute value of each difference to the Tukey's HSD value. If the difference is larger than Tukey's HSD, it's significant.
Define the 3 forms of replication that were discussed in class.
(100pts) DIRECT-
(150 pts) CONCEPTUAL-
(150 pts) REPLICATION-PLUS EXTENSION-
DIRECT- simply repeating the study without modification with identical operational definitions and manipulations.
CONCEPTUAL- using different operational definitions to test original claim (theoretical work done) by changing how a variable is manipulated/measured.
REPLICATION-PLUS EXTENSION- repeating the study but ADDING new variable(s) to test in order to assess clarifying explanations (ex. potential moderators, mediators, additional groups).
For 100 pts each, describe at least 2 types of basic threats and 3 types of time-related threats from "The Dirty Dozen."
BASIC THREATS
Design Confounds -systematic differences between/ conditions based on design of study
Selection Effects -Differences between conditions (aside from IV) due to participants (ex. when participants choose condition)
Observer Bias -Observers' scores based on what they expect/want to see
Demand Characteristics -Participants change behavior based on what they think experiment is testing
Placebo Effects -results that occur based on participants belief (ex. treatment)
TIME-RELATED THREATS
Maturation effects -response change due to natural development over time
History effects -an event that occurs during study that affects DV
Regression effects -Scores become less extreme over time
Attrition effects -participants drop out of study in NON-random way
Instrumentation effects -scores change over time because of change in instruments used
Order effects -practice, fatigue, and carryover, testing effects (don't need to know)
How do you know when to reject the null hypothesis?
When the statistic of interest (obtained) falls beyond the threshold (critical).
Describe the calculation steps of t-test! (For 100pts per step, which means there are 5 steps :D)
1) Calculate means/mean difference.
2) Calculate standard error.
3) Calculate obtained t-statistic.
4) Find critical value(s) or calculate exact p-value.
5) Reject or Fail to reject the null hypothesis.
What is the formula for effect size in ANOVA (eta squared)?
η2 = SSbg / SStotal
How did the Tuskegee Syphilis Study (1932-1972) violate the principle of beneficence?
Participants were not cured of the illness and were subject to painful and dangerous tests during experimentation.