Show:
Benchmarking
Developing Surveys
Focus Groups
Rubrics
Odds and Ends
100
Name two reasons why we benchmark
• Put our data in context with national or peer data • Compare programs or services • Track trends • Identify areas of success • Identify areas of challenges • Set goals or targets for strategic planning
100
Participants are not individually identified and identity cannot be connected to responses- is this confidential or anonymous?
• Anonymous
100
Is a focus group a quantitative or qualitative means of assessment?
• Qualitative
100
Is a rubric a direct or indirect measure of learning?
• Direct
100
State the difference between a direct and indirect method
• D- any process employed to gather data which requires subjects to display their knowledge, behavior, or thought process • I- any process employed to gather data which asks subjects to reflect upon their knowledge, behaviors, or thought processes
200
Name two business reasons for using benchmarking data
• Strategic or other planning (setting targets/baselines) • Audience needs data in context • Show improvements • Resource justifications • Determine priorities
200
A scale that is either positive or negative and can be with or without a midpoint is a unipolar or a bipolar scale?
• Bipolar
200
How many “rounds” of focus groups should you anticipate needing?
• 3-5 (until redundancy is reached)
200
Is a rubric a quantitative or qualitative means of assessment?
• Both! Mixed method that provides qualitative descriptions of learning and quantitative results
200
A scale that contains no negative is a unipolar and a bipolar scale?
• Unipolar
300
Name two ways or resources to establish peer groups
• Aspirant schools • Current comparisons • IPEDS (Integrated Postsecondary Education Data System) • Carnegie classifications • Region of Country • Competitor Institutions • Institutional characteristics • Athletic conference • Ourselves
300
“Why would it be a good idea to eliminate smoking on the RIT campus?” is an example of a ____ question
• Leading
300
Name three cons of a focus group
• Facilitation requires skill/practice • Results are not generalizable • Time needed for training and analysis • Lack of control over discussion • Groups can influence individual responses • Challenge to get people to attend
300
**Image Question** What is missing from this example rubric describing the top row?
• Scale
300
**Image Question** What step is missing from the assessment cycle?
• Use findings to plan/results for improvement
400
**Image Question** What 2 steps are missing from the planning for benchmarking chart?
• Determine goals/purpose • Establish peer groups
400
“Please use the space below to articulate what you learned from this training program” is an example of what type of question?
• Open ended • Free response • Text response
400
State four of the necessary or highly recommended materials that you need for a focus group
• Audio or video recorder • Note-taking materials • Nametags • Consent and non-disclosure form • Incentives
400
What are four things you want to avoid in an effective rubric?
• Multiple learning outcomes in one rubric • Too many dimensions • Not enough difference between dimensions • Wordy and complicated descriptions (takes too long to understand and read) • Not specific enough • Not defining words like “adequate”, “sufficient”, “little”, “some”, etc.
400
Name three responsibilities of a focus group facilitator.
• Remain impartial to topic • Remain impartial to participants • Manage time • No power position • Listening/active listening • Encourage participation/discussion • Keep participants on track/focus • Probe for further information/clear statements • Have FG facilitating practice or training
500
Name three sources of benchmarking data
• Locally developed instrument • Campus information systems • Commercially developed instrument • National organizations (usually operational data) Or • Data Systems • Tracking/usage • Surveys • Rubrics
500
State three strategies you can implement to increase survey response
• Make it relevant (specific student population sample) • Help participants understand the value of their response (invitation text, request for help) • Decrease/limit the number of times you ask a student to complete a survey • Use incentives • Send notifications at appropriate times • Keep the survey length down (about 22 questions, 13 minutes) • Establish credibility (pre-announcement email, endorsements, confidentiality statements)
500
Challenge: You have asked a question, and no one responds. What can you do?
• Rephrase the question • Call out a specific person to probe answer • Sit in silence for 30-60 seconds before asking a different way • Come back to the question later
500
**Image Question** What 2 steps are missing from the rubric implementation chart?
• Test rubric • Analyze and report
500
This describes how important or relevant a survey topic is to the survey recipient
• salience