Will's Rules to Remember and Live By
An arbitrary coding scheme for numerically representing categorical predictors. This term has the condition that the values of the contrast-coded variable sum to zero across the categories, not across the individual observations that belong to those categories.
What are contrast codes?
Let's be creative:
Create your own orthogonal contrast code for an experiment that has four conditions based on your research or something you know in the field (hint: may be best to structure it based on theory!)
In this study, the four conditions involve different types of outcome feedback provided to participants during a decision-making task. According to reinforcement learning theories, the nature of the feedback (its presence, valence, and clarity) is critical for updating action values and guiding future behavior. The four conditions are defined as follows:
Contrast 1: Presence vs. Absence of Feedback
Contrast 2: Valence of Clear Feedback
Contrast 3: Clarity of Feedback
In a hypothetical experiment, participants were administered a memory recall test after receiving either a caffeine pill or a placebo. Participants were randomly assigned to one of two groups. Those in the caffeine group received a 200 mg dose of caffeine 30 minutes before the test, while those in the placebo group received an inert pill. The dependent variable is the number of words correctly recalled from a list of 20 words. Suppose five subjects were randomly assigned to each condition, with the data shown below. Did caffeine administration affect recall performance?
1. Construct a contrast code for the categorical IV
2. What are the models A and C being compared?
3. What are the corresponding null and alternative hypothesis?
What is (-1) for placebo group and (1) for caffeine group (can be any two numbers that sum to 0)
What is...
Model C: Words recall = B0 + ERROR
Model A: Words recall = B0 + B1 (-1 for placebo, and 1 for caffeine group) + ERROR
What is...
H0: the average number of words recall is the same for both groups (B1 = 0)
H1: the average number of words recalled differs between the caffeine and placebo groups (B1 is not equal to 0)
In a hypothetical experiment, participants were administered a memory recall test after receiving either a caffeine pill or a placebo. Participants were randomly assigned to one of two groups. Those in the caffeine group received a 200 mg dose of caffeine 30 minutes before the test, while those in the placebo group received an inert pill. The dependent variable is the number of words correctly recalled from a list of 20 words. Suppose five subjects were randomly assigned to each condition, with the data shown below. Did caffeine administration affect recall performance?
Answer the following two questions using the appropriate R output:
What type of test did the researchers use to run this set of data, and compute the following R code?
What conclusion do you make about the researcher's question (be sure to write the following results using appropriate APA format)?
What is the researchers used a two-group independent samples t-test?
What is there is a significant difference in recall performance as the numbers of words recalled was significantly higher in the caffeine group (M = 7.6) than the placebo group (M = 4.4), t(8) = -4.44, p = .002?
Dr. Alex Frost, a cognitive psychologist, is conducting a study on how different types of background music influence cognitive performance. Participants are randomly assigned to one of five groups, each exposed to a different type of music while solving complex puzzles. Dr. Frost, however, did not plan any specific contrasts before running the experiment and now realizes that he wants to compare all possible group differences. He has no predefined hypotheses and wants to explore every potential comparison after the fact. What test is best for Dr. Frost to use and what limitation is he accepting by using this test?
What is Scheffe test and he is accepting a greater risk of a type II error by keeping the Type I error as small as possible seeing that it is a conservative test?
In multiple comparison with non-orthogonal comparisons, this evaluates each contrast as if it were the only one, ignoring the rest of the contrasts
What is the traditional approach?
Researchers in a recent study by Lee and Patel (2023) sought to determine whether watching a motivational video before attempting a problem-solving task would enhance performance. Participants were asked to complete a series of puzzles (with a maximum score of 15) after being randomly assigned to one of four conditions:
The study’s primary hypothesis was that participants who viewed the motivational video before attempting the puzzles (the experimental condition) would solve more puzzles correctly than participants in any of the other conditions (the control groups).
The researchers created the first two orthogonal contrast codes for the conditions but got lazy and forgot to code the last one.
Fill in the last row of the table above to complete the set of orthogonal contrasts codes and describe what the three contrasts are testing.
What is X3: -2, 0, 1, 1
X1: This contrast compares the experimental condition (motivational video before) against the average of the three groups.
X2: This contrast compares the timing and content in seeing if the motivational context itself has any effect on performance by testing the motivational video after the task against the netural video before the task
X3: This contrast tests whether watching a video regardless of the motivation content or timing affects problem-solving performance when compared to no video at all
Let’s assume we are testing preferences for five different ice cream flavors, and we want to see how each flavor compares step by step to the average of the next set of flavors.
Experimental Groups:
The researchers generated this set of orthogonal contrast code.
What type of code is shown in the following table?
Why would you use this set of code?
Helmert contrast codes
Set of orthogonal codes that can be used to compare each group sequentially to the remaining groups instead of comparing everything to one reference group.
A group of researchers is investigating how different levels of caffeine intake affect productivity in a simulated work environment. Participants are assigned to one of three groups based on their daily coffee consumption:
The researchers measure task productivity (number of completed tasks in a 3-hour work session). They want to analyze whether caffeine intake improves productivity and whether heavy coffee drinkers perform differently from moderate drinkers.
Using Helmert contrasts, the researchers set up the following comparisons:
The following results are in the R code. What are the results of the omnibus test? How much variance does this model explain in productivity? *report these findings in APA format
We can conclude there is a significant difference among the group means, F(2, 12) = 42.08, p < .001. The model explains 87.5% of the variance in productivity (R^2 = 0.875).
There is an inequality between two tests in the a prior multiple comparisons for orthogonal contrasts.
This leads to the () alpha per comparison being less than or equal to () alpha per comparison. This suggests that one of the tests is more conservative / less powerful than the other correction.
Fill in the blank of the two tests!
What is Bonferroni and Sidak corrections?
These two types of comparisons are used for multiple comparisons when a researcher does not have a plan for what they want to find. The other two types of comparisons are used for multiple comparisons when a researcher does not have set predictions for the contrasts. *these answers are selected around the ones we discussed in class
What is Sidak and Bonferroni correction?
What is Scheffe and Turkey test?
A research team at Sweet Science Labs is investigating which ice cream flavors are the most popular. Participants in a taste test are randomly assigned to one of five groups, each tasting a different flavor:
Questions the researchers are interested in asking:
Create the appropriate code for the following research questions.
In the table :)
A cognitive science research team is investigating whether caffeine enhances memory recall beyond the effects of sleep and number of hours studying.
After the intervention, participants take a memory recall test, where they attempt to recall as many words as possible from a list of 25 words.
The researchers are interested in testing whether the mean in the caffeine condition differed from the average of the means of sleep and number of hours studying.
Create the following Model C and Model A to test this and the appropriate null / alternative hypothesis.
Model C: Yi = B0 + B1 (sleep) + B2 (studying) + ERROR
Model A: Yi = B0 + B1 (sleep) + B2 (studying) + B3 (caffeine) + ERROR
H0: B3 = 0
H1: B3 does not equal 0
A group of researchers is investigating how different levels of caffeine intake affect productivity in a simulated work environment. Participants are assigned to one of three groups based on their daily coffee consumption:
The researchers measure task productivity (number of completed tasks in a 3-hour work session). They want to analyze whether caffeine intake improves productivity and whether heavy coffee drinkers perform differently from moderate drinkers.
Using Helmert contrasts, the researchers set up the following comparisons:
What are the results of the estimated coefficients (write in APA format)? Why are you able to examine the coefficients in this manner?
The test for the first contrast was significant, t(12)=−7.845, p<.001, ηp^2=0.84. On average, participants in the Moderate Coffee Group (M = 15.2) and Heavy Coffee Group (M = 21.2) completed significantly more tasks than those in the Control Group (M = 10.0). These results suggest that drinking coffee increases productivity compared to consuming no coffee.
The test for the second contrast was significant, t(12)=−4.756, p=.0005, ηp2=0.65.Heavy coffee drinkers (M = 21.2) completed more tasks than moderate coffee drinkers (M = 15.2). This suggests that caffeine has a dose-dependent effect on task performance, where consuming more coffee results in greater productivity.
With a complete set of orthogonal contrasts, the tests of the model coefficients correspond to tests of the individual contrasts.
The following R output is showing an approach to correct for multiple comparisons.
What approach is it and which set of codes do you use it for?
If we were to conduct the other approach, what would that look like in R and why would we use it?
The R output is showing the traditional approach and it is used for the ignoring the other contrasts when testing any single contrast. You use it for non-orthogonal comparisons.
The other approach for non-orthogonal comparisons with multiple comparisons is modern approach that would use a regression-based approach to test the unique contribution of each contrast over and above the other contrasts.
A university research team is investigating how different study techniques affect final exam performance. Students are randomly assigned to one of three groups based on their primary study strategy:
The researchers coded the contrasts with:
X1 (difference in passive learning versus active recall): (-1, 0, -1)
X2 (difference in guided practice to the average of the other two conditions): (-1, 2, -1)
What would happen if we used a different set of orthogonal contrast codes? What would change?
If we use a different set of orthogonal contrast codes, the model is equivalent to the model with the original set of contrast codes. It makes the same predictions for all observations (the group means) and has the same sum of squared error. The omnibus test results would be the same and the estimated intercept is still the mean of the three condition means. However, the estimated regression coefficients would change as we are making different comparisons among the group mean.
Researchers are interested in how the length of a research methods workshop influences graduate students’ research skills. A total of 80 graduate students are randomly assigned to one of four workshop durations:
Each group consists of 20 subjects. The dependent variable is a composite research skills score (ranging from 0 to 100).
What kind of contrast codes can the researchers use based on this design?
Based on this set of codes, explain what each of the three contrasts tests / or its purpose in the context of the study?
What is orthogonal polynomials?
What is ...
X1 (Linear trend) research skills will improve as the workshop duration increases
X2 (Quadratic trend) research skills will be optimal at the intermediate durations (2 & 3) compared to the extreme durations (1 & 4)
X3 (Cubic trend) is there an inconsistent pattern amongst the days that the two other trends are not able to capture *simply there to complete the set of contrast codes
A behavioral neuroscientist is investigating the effects of a new drug on cognitive performance. The study includes one treatment group and six different control groups. The groups are as follows:
However, the researcher has concerns about his research design for purposes of power. What should the researcher be concerned about? What is the solution to this concern (hint: it has to do with structuring a contrast)
The researcher should be concerned as when the number of groups is large in a design and only a few mean differences are expected, the power for the omnibus test is reduced. The researchers omnibus test may be misleading as he may be making a type one error.
The researcher should instead focus on the contrast between the treatment group and the average of all the control groups.
What type of contrast code is being tested in this R code?
Why do we conduct these constants separately instead of testing them simultaneously?
Non-orthogonal contrast codes
We are testing them separately because they are correlated / not independent
What test should you put where the # is in the R code?
What are the researchers testing in this multiple comparison method?
What is Turkey?
What is researchers are testing the pairwise comparisons between the levels?
Define the differences between all the codes we discussed (orthogonal, non-orthogonal, dummy codes, and orthogonal polynomial codes).
Orthogonal Contrast Codes:
Non-orthogonal Codes:
Dummy Codes:
Polynomial Codes:
A research team is investigating the effects of different treatment levels on an outcome variable. Participants are divided into five groups based on increasing dosage levels of a new medication:
The team has three RA's that generate three SEPARATE sets of contrast codes to analyze their data:
Each contrast code set is designed to answer a specific question. Identify what kind each set is from the R output and what questions the researchers are attempting to answer.
A & B: orthogonal set of codes
A: does treatment vs no treatment matter
B: are low and very high doses different from medium and high doses
C & D: non-orthogonal set of codes
C: do low doses differ from high doses?
D: does the lowest dose differ from all other doses
E & F: polynomial set of codes
E: does the effect increase/decrease in a straight line with dose (linear)
F: will the effects be better for the middle levels than for the extreme levels (quadratic)
A research team investigated how different study habits impact academic performance. Students were divided into three groups based on their preferred study methods:
Below is the summary source table for the regression analysis. Using this table, answer the following questions:
How do you calculate PRE in the omnibus test? How do you calculate PRE in the first and second contrasts?
Omnibus Test: SSE C (33.744), SSE A (23.910) and SSR (9.834). PRE = SSR / SSR + SSE(C)
X1: you do not know SSE C, SSE A (23.910) and SSR (7.230). PRE = SSR / SSR + SSE (A)
X2: you do not know SSE C, SSE A (23.910) and SSR (2.604). PRE = SSR / SSR + SSE (A)
What does this function calculate in R? What is this?
What do you put in the first and second section that consists of hashtags?
It calculates partial eta squared and tells us the proportion of the variance in the DV is explained by that specific contrast.
You put the specific t value of that contrast and the degrees of freedom which is calculated by (n - PA).
Remember how Will says he hates diagrams to classify/organize stats! Let's make one for him.
Create a diagram for what tests to use for planned orthogonal comparisons, planned non-orthogonal comparisons, and post hoc comparisons! Write in your diagram when you use the method.
Pull up flow diagram :)