Scientific Reasoning
Research Foundations for Any Claim
Tools for Evaluating Frequency Claims
Tools for Evaluating Association Claims
Tools for Evaluating Causal Claims
Balancing Research Priorities
100

The Loch Ness monster is real, but you cannot see her because she disappears when observed by the eyes of humans (even in video).  Is this theory falsifiable?

No

100

I ask participants to complete a likert scale survey and I add up the items to give them a score at the end.  Is this variable categorical or continuous?

Continuous

100

For which type of claim is construct validity not a priority?

None of them.  It is always a priority. 

100

Is the relationship between height and shoe size positive or negative?

Positive

100

I randomly assign people to either use hand sanitizer or wash their hands with soap. I measure which group is more likely to get coronavirus.  What type of study design is this?

Ranomized groups post test only 

100

If a study is conducted in a real life setting, what type of validity is it very high in?

External validity (ecological validity)

200

I want to know how the coronavirus started.  One theory is that the virus evolved and was transmitted from animals to humans.  Another theory is that it was developed in a laboratory by China and released upon the world as a bio-weapon.  Which theory has parsimony?

Animals to humans. 

200

Participants in the Tuskegee Syphilius study were not provided a cure when one was made available. Which of the three principles of the Belmont report does this violate?

Beneficence. 

200

For which type of claim is having a truly random sample the most important?

Frequency claim

200

What is the key feature of a study examining an association claim?

The variables are measured, not manipulated (and there are at least 2). 

200

What is absolutely necessary in order to be able to say that one thing causes another?

Random assignment to conditions. 

200

I go to Costco and measure the degree to which people are wearing masks, washing their hands and social distancing.  I am wearing a lab coat with an NIH badge.  Because the participants see me observing their behavior, they step up their social distancing, mask wearing, and hand washing. This is called. 

Demand Characteristics.

300
'Medically complex' children most at risk for severe COVID-19. What claim is this headline making?
Association Claim
300

One's satisfaction with online education could be classified on the following scale: Extremely dissatisfied, moderately dissatisfied, mildly dissatisfied, mildly satisfied, moderately satisfied, extremely satisfied.

What level of measurement is this variable?

Ordinal

300

When asked about a politically charged issue, some people will answer with neutral responses to avoid looking extreme.  This is called:

Fence Sitting

300

I am doing a longitudinal study examining the relationship between smoking and lung cancer.  I look at smoking rates and indicators of lung cancer in both 2008 and 2012 in the same group of participants.  If I measure the correlation between smoking in 2008 and smoking in 2012, what is this kind of correlation called?

Autocorrelation

300

I found that those with migraines are more likely to report lower academic achievement.  Do I have temporal precedence?

No

300

George found that people who wear purple are more likely to graduate from college.  I redo the study exactly as he did, but I also test if people who simply SEE a lot of purple are more likely to graduate from college.  What kind of replication is this?

Replication plus extension

400

An operational definition of depression could be . . . 

diagnosis, score on a survey, self-report, etc. 
400

I want to know if the coronavirus test is a good one. So, for some patients, I give them the test twice to see if it produces the same result each time.  This is called: 

test-retest reliability

400

On the following scales, rate your experience with online education: 

Unproductive 1 2 3 4 5 Productive

Uneducational 1 2 3 4 5 Educational

Frustrating 1 2 3 4 5 Rejuvenating

What types of questions are these?

Semantic Differential

400

True/False: If you do not have external validity, your study is no good. 

False

400

True or false: Between subjects designs are not subject to order effects.  

True

400

How does a meta analysis differ from a review article?

Meta analysis uses data to calculate an effect size across studies. 

500

I am terrified to go in the ocean because of sharks.  Shark attacks happen all the time, I saw it on TV.  What type of bias is this?

Faulty Thinking: Availability Heuristic

500

I developed an antibodies test for coronavirus so we can test to see who has had it and who has not.  However, I want to make sure that it is actually measuring COVID-19, not other coronaviruses that the patient might have been exposed to.  What type of validity am I aiming for? Face, Content, Criterion, Convergent, Divergent. 

Divergent validity. 

500

I want to see how many people in the US have ACTUALLY been infected with the coronavirus, but I can't test everyone.  So, I randomly select 50 cities in the country and then I randomly test 500 people in each of those 50 cities.  What type of sampling is this?

Multi-stage sampling

500

True/False: Using regression allows us to make a causal claim. 

False

500

How do we evaluate the statistical validity of a study?

P value, correct tests, correct interpretation, and effect size.

500

I recruit for a depression study by finding people who have score high on the Perceived Stress Scale. However, the next time I test them, I find their scores have gone down.  What is this confound called?

Regression to the mean