Reliability
What is the term used to describe the consistency of test scores?
A. validity
B. reliability/precision
C. distribution
D. standard deviation
B. reliability/precision
Which of the following would provide evidence of a test’s validity based on content?
A. the extent to which test questions are representative of the knowledge domain measured
B. the extent to which a person’s test scores can predict his or her current and future behavior
C. the extent to which the test taker recognizes the purpose of each test question
D. a numerical analysis of the test’s construct dimensions
A. the extent to which test questions are representative of the knowledge domain measured
What do we call the measure of performance that we expect to correlate with test scores?
A. the predictor
B. the criterion
C. a standardized test
D. an intercept
B. the criterion
In 1954, the American Psychological Association (APA) published recommendations that suggested a new method for establishing validity in which the researcher provides evidence that the test measures behavior predicted by a psychological theory. What did the APA call this evidence?
A. construct validity
B. factor analysis
C. multitrait, multimethod analysis
D. generalizability
A. construct validity
As the interval between administrations lengthens, test–retest reliability will most likely _______________.
A. increase
B. decrease
C. vary unpredictably
D. remain unchanged
B. decrease
When test scores used to select employees are systematically related to a score on a job performance appraisal instrument, the test is demonstrating evidence of validity based on its _______________.
A. content
B. relationship with another construct
C. reliability
D. relationship with a criterion
D. relationship with a criterion
Having a restricted range of test scores means that the observed validity coefficient is likely to be _______________.
A. higher
B. lower
C. unaffected
D. unpredictable
B. lower
Defining or explaining a psychological construct is referred to as _______________.
A. construct validity
B. research that bears out a theory’s predictions
C. calculating convergent validity
D. construct explication
D. construct explication
What do we mean when we say that a test is internally consistent?
A. a test taker only takes the test once
B. a test taker’s scores will remain similar over time
C. the test questions are measuring a similar concept
D. the scores of a group of test takers will be very similar
C. the test questions are measuring a similar concept
_______________ is/are difficult to describe because people disagree on what behaviors represent, whereas _______________ is/are clearly described in observable and measurable behaviors.
A. Concrete attributes; abstract attributes
B. Abstract attributes; concrete attributes
C. Concurrent validity; construct validity
D. Construct validity; concurrent validity
B. Abstract attributes; concrete attributes
Which method of demonstrating evidence of validity do test developers use when the test scores and criterion scores are collected at approximately the same time?
A. predictive method
B. content method
C. concurrent method
D. convergent method
C. concurrent method
If test scores correlate with measures of constructs that an underlying theory says are related, then we say the test has _______________.
A. convergent evidence of validity
B. discriminant evidence of validity
C. evidence of validity based on content
D. concurrent evidence of validity
A. convergent evidence of validity
Which one of the following methods of estimating reliability/precision requires dividing the test into halves and then correlating the set of individual test scores on the first half with the set of individual test scores on the second half?
A. test–retest method
B. coefficient alpha
C. split-half method
D. correlation method
C. split-half method
Evidence of validity based on content is most appropriate for what types of tests?
A. achievement tests that measure concrete attributes
B. personality tests that measure absolute attributes
C. concrete tests that measure aptitude and achievement
D. comparative tests that measure personality characteristics
A. achievement tests that measure concrete attributes
Which of the following is TRUE about objective criteria?
A. Their scope is often quite narrow.
B. They are often based on personal experience.
C. They are often based on a person’s judgment.
D. They are often expressed as ratings.
A. Their scope is often quite narrow.
When test scores do not correlate with unrelated constructs, there is _______________.
A. convergent evidence of validity
B. discriminant evidence of validity
C. evidence of validity based on relations with external criteria
D. evidence of face validity
B. discriminant evidence of validity
What is the formula that Cronbach proposed for calculating internal consistency for questions that have more than two possible responses called?
A. coefficient alpha
B. KR-20
C. product moment correlation
D. Spearman Brown
A. coefficient alpha
Which one of the following types of evidence of validity can be gathered without using correlations?
A. evidence based on relations with a construct
B. evidence based on test content
C. predictive evidence of validity
D. concurrent evidence of validity
B. evidence based on test content
A p value < .01 tells the test user that the likelihood that the relationship being measured was found by chance was _______________.
A. less than 5 chances of 100
B. less than 1 chance of 100
C. not significant
D. significant
B. less than 1 chance of 100
Which one of the following coefficients is NOT found in a multitrait–multimethod correlation matrix?
A. reliability coefficient
B. correlation coefficient between the test and a criterion
C. convergent coefficient
D. discriminant coefficient
B. correlation coefficient between the test and a criterion