Psych Tests
Psychological tests require test-takers to perform ________ that are used to draw ________ about a _________.
Psychological tests require test-takers to perform behaviors that are used to draw inferences about a construct.
What are the two types of biographical inventories?
Rational – Ask information that is job-relevant and score it according to the perceived importance of each piece of information
Empirical – Ask a bunch of information, and let computers tell you what is most important and how to score it
What are unstructured interviews (aka organic interviews), semi-structured interviews, and structured interviews?
•Unstructured interviews (aka organic interviews) – interviewer(s) asks whatever questions come to mind; very conversational
•Semi-structured interviews – series of pre-planned questions but freedom to deviate and ask unplanned follow-up questions
•Structured interviews – preplanned set of questions that are asked of everyone interviewed in the same order; no follow-up or probing questions
What is a work sample?
An assessment device that requires a person to demonstrate how well they can perform a task involved in a job under standardized conditions
Examples: medical simulation, computer science project, draft an email, physical driving test, teach a class
What is an assessment center?
•An assessment center uses a series of simulation exercises to measure some job-relevant abilities and skills that a person has.
•Most assessment centers are designed to assess leadership/management skills, but they are used for nonmanagement jobs as well.
•The entire center can take several days to complete and are often administered to several individuals at a time.
What are 3 characteristics of good psychological tests?
Representatively sample behaviors thought to measure a KSAO
Have standardized conditions
Have clear rules for scoring (e.g., rubric, correct/incorrect)
Give 3 examples of biodata.
Examples: GPA, degree attainment, previous work experience, extracurricular activities (pretty much any data you would find on a job application)
What are some advantages of structured interviews over unstructured interviews?
•Better job of assessing job-relevant KSAOs
•Easier to compare candidates when everyone is asked the same questions
•Reduces bias
•Higher criterion-related validity (i.e., better job of predicting applicants’ job performance)
What are some pros of work samples?
Clearly job-relevant
Good predictor of job performance
May have fewer racial differences than cognitive ability tests
Perceived as fair by applicants
What are common assessment center exercises?
•In-basket – Folks pretend it is their first day at work, and they are presented with various things in their “in-basket” (e.g., emails, letters, memos, texts). They have to prioritize their various tasks and explain how they would tackle them.
•Interview
•Leaderless group discussion – Folk are presented with a scenario. The scenario may involve working as a team to solve a problem (collaborative scenario), or handling a scenario in which different people have opposing needs/goals (competitive scenario).
•Role play
Inferences should directly align with the _______ being assessed. Provide an example of a rational inference and an irrational inference.
psychological construct
What are pros and cons of the 2 types of biographical inventories?
Rational
Pros – measures KSAOs that are job-relevant
Cons – often lower validity than empirical
Empirical
Pros – good job of predicting job performance
Cons – not always clearly job-relevant
What is the major disadvantage of structured interviews compared to unstructured interviews?
applicants may not like them because they may come across as cold or robotic (not personalized); managers may dislike them for the same reason
What are some cons of work samples?
Time consuming for applicants
If unpaid, a company may be getting free labor (unethical?)
Complex jobs may have MANY tasks, and a work sample only covers a small portion of the performance domain
What are common assessment center dimensions?
problem-solving, organizing and planning, communication, influencing others, consideration/awareness of others, drive
What are 4 assumptions of psychological tests?
Assumption #1: Individuals’ scores on ability and personality tests will remain stable over time
Assumption #2: Individuals understand items the same way. Example: What does it mean to “make a mess of things”
Assumption #3: Individuals will report accurately about themselves. Example: I am the life of the party.
Assumption #4: Individuals will report honestly about themselves. Example: How often have you stolen something from work?
Summarize the purpose, methods, results, and conclusion of the article below.
Mount, M. K., Witt, L. A., & Barrick, M. R. (2000). Incremental validity of empirically keyed biodata scales over GMA and the five factor personality constructs. Personnel Psychology, 53(2), 299-323.
Purpose: To determine if empirically keyed cross-validated biodata scales accounted for incremental validity over the Big 5 personality factors and GMA
Method: A concurrent validation study was conducted using 376 clerical employees. Participants completed the Wonderlic Personnel Test to measure GMA, the Personal Characteristics Inventory to measure the Big 5, and a 196-item biodata scale. Measures of their job performance were also collected.
Results: The empirically keyed biodata accounted for significant incremental variance over personality and GMA for 3 criterion measures.
Conclusion: Rigorously created empirically keyed biodata can be useful when used along with measures of personality and GMA.
What are "better" types of questions in interviews?
•Situational questions
•Past behavior questions
•Background questions
•Job knowledge questions
What does a work sample have in common with an assessment center? What differentiates them?
Similarities:
•Involves simulation
•Very job-relevant (doing what would be done on the job)
•Performance should be evaluated using BARS
•Structure is good
Differences:
•ACs involve multiple exercises whereas a work sample involves one task
•ACs are evaluated live whereas a work sample is often completed remotely
•ACs often have group activities whereas work samples are usually done independently
•ACs are often used to select folks for leadership positions. Work samples are used in selection for a wider variety of positions.
•ACs are particularly costly; often involves assessees and assessors spending multiple full days on the AC.
What is the OAR in an assessment center?
overall assessment rating - After candidates leave, assessors get together to determine an overall assessment rating (OAR) for each candidate (i.e., overall score)
Summarize the purpose, methods, results, and conclusion of the article below.
Hogan, J., Barrett, P., & Hogan, R. (2007). Personality measurement, faking, and employment selection. Journal of Applied Psychology, 92(5), 1270.
Purpose: To evaluate the extent to which faking occurs on personality measures in real-world selection settings.
Methods: Real job applicants completed a 5-factor model personality measure (Hogan) as part of the job application process. They were rejected; 6 months later they (5,266) reapplied for the same job and completed the same personality measure.
Results: Study 1 showed no meaningful changes between personality scores between the application to the reapplication. Study 2 showed that personality scale score changes as did occur were systematic and predictable using measures of social skill, social desirability, and integrity. Study 3
found no meaningful score differences between participants taking the test for research purposes versus a selection setting.
Conclusion: Results suggest that faking on personality measures is not a significant problem in real-world selection settings.
Empirically keyed (empirical) biodata scales should be cross-validated. What does that mean?
The effectiveness of biodata scales to predict job performance is tested across multiple samples to ensure that the findings derived from the first sample were not a fluke.
Name at least 10 different methods of structuring interviews.
•base questions on a job analysis
•ask the exact same questions of candidates
•limit prompting
•use better types of questions
•use longer interviews/more questions
•control ancillary information
•do not allow candidate questions until the end
•rate each answer
•use detailed anchored rating scales
•take detailed notes
•use multiple interviewers
•use the same interviewers across candidates
•do not discuss candidates/answers between interviews
•provide interviewing training
Summarize the purpose, methods, results, and conclusion of the article below.
Robertson, I. T., & Kandola, R. S. (1982). Work sample tests: Validity, adverse impact and applicant reaction. Journal of Occupational Psychology, 55(3), 171-183.
Purpose: review the research literature on work sample tests with a focus on the validity, adverse impact, and applicant reactions
Method: literature review – reviewed literature on four types of work sample tests (i.e., psychomotor, individual situational decision making, job-related information, and group discussions/decision making)
Results/conclusion: Work samples can have high criterion-related validity, reduce adverse impact (racial disparities), and produce favorable applicant reactions. Psychomotor work samples have particularly high criterion-related validity.
List at least 6 guidelines for performing high-quality assessment centers?
ACs should involve a Systematic Analysis to Determine Job-Relevant Behavioral Constructs, Behavioral Classification, Multiple Assessment Center Components, Linkages Between Behavioral Constructs and Assessment Center Components, Simulation Exercises, Assessors, Assessor Training, Recording and Scoring of Behaviors, Data Integration, and Standardization