test-retest, alternate/parallel forms, split-half or internal reliability, inter-rater reliability.
100
What are the different types of validity?
face, content, criterion, and construct validity
100
What reliability is not always designed to be high?
test-retest reliability
100
What is the most common type of item format in psychology tests?
Likert scales
100
What are some examples of "bad" administration behavior?
leading questions, distracting behaviors, too much / too little encouraging behaviors
200
What is the Classical Test-Score Theory?
True Score = Observed Score - Error
200
What is face validity? Is it useful?
"Common sense" validity, not useful because just because it looks like something doesn't mean it is valid.
200
What is the minimum reliability coefficient for a scale to be considered internal consistency reliable?
alpha > .70
200
What is the formula for correcting for guessing?
For every wrong answer, subtract 1/(m-1) points
200
What are expectancy effects?
tendency for a person to find evidence that supports a pre-existing hypothesis
300
What are examples of sources of error?
test itself, test-taker, environment, how the test was scored
300
How do you establish content validity?
Conduct literature review on topic to make sure you are covering all facets of it.
300
What is the standard error of measurement and how do you calculate it?
SEM = Standard Deviation * Sqrt(1 - r)
It measures how close an observed score is to the true score
300
What is Item Difficulty?
% of people who get the item correct
300
What is drift and reactivity during behavioral assessment?
Drift = performance/ratings tends to change slowly over time. Reactivity = more reliable when observed.
400
What does r = .60 mean in terms of variation in test scores attributed to true / error variance?
60% of the variation in test scores is due to variation in the "true" score.
400
What is the difference between norm-referenced test and criterion-referenced test?
Norm-referenced = Hire the top 5% test scores. Criterion-referenced = Hire those who can do "specific skill."
400
What are the characteristics of a "good" criterion?
relevant (pertinent to the construct) and valid (evidence must exist that criterion is already valid)
400
What are three elements of a good multiple choice questions?
1. Has grammatically parallel distractors. 2. no "cute" distractors. 3. has one correct target. 4. not too many distractors. 5. no double-negatives wording.
What is the z-score to use for building a 95% confidence interval?
Z = 1.96
500
What is convergent and divergent evidence/validity?
Convergent = data from multiple sources all tend to point to the same conclusion. Divergent = constructs do not correlate with scores on other measures it should theoretically not relate to.
500
What is item discriminability? What does it mean if the item characteristic curve is flat? What if it means if it steadily increases?
People who answers correctly. Flat = bad questions and people are guessing. Positive = good question and people who do well get it correct more often than ppl who do poorly.
500
What is malingering and how do we detect it?
intentionally faking bad/good scores. Rey's fifteen item test and Hiscock forced-choice test are examples of malingering tests.