ACQUIRING ORGANIZATIONAL EVIDENCE
Data becomes “big data” when it gains:
Volume, Variety, Velocity
A _______ explains why/when a problem occurs and how this leads to a particular outcome, and not having one can lead to “fishing expeditions” where organizations blindly analyze a large number of variables until they find something that is significant.
logic model
One of the defining characteristics of stakeholder evidence is that it concerns ____.
subjective feelings and perceptions
A hospital wants to evaluate patient satisfaction with a new scheduling system. To gather feedback, they invite their 40 most frequent patients to complete an online survey. What factors might negatively affect the trustworthiness of the results?
The 40 most frequent patients may not be representative of the hospital's total population of patients.
When we tie a metric to a target, goal, or norm critical to the organization’s success, we refer to it as
A key performance indicator (KPI)
Given the data below, what should the company conclude regarding the turnover in R&D?
*** Special
It may be the result of the "small number problem"
You run a regression analysis with customer service trainees' scores on a role play exercise as the predictor variable, and customer satisfaction ratings (10pt scale) six months later as the outcome variable. You find an unstandardized regression coefficient of b = 3.2. What does this mean?
For every one unit higher score on the role play exercise, on average the
subsequent customer satisfaction ratings are 3.2 scale points higher.
When it's not feasible to get information from all relevant stakeholders, one may need to rely on ______.
"key" stakeholders (or a representative sample)
Critical appraisal of stakeholder evidence is primarily focused on the ____ of the stakeholder sample/data.
representativeness
______ are metrics that are tied to standards or best practices within an industry.
Benchmarks
Your company has employees complete an engagement survey every year, to track/compare this metric at different stores, time points, etc. In order to determine the extent to which measurement error is a problem, the company should
Analyze data to estimate the reliability of the survey
You run a multiple regression analysis with monthly sales as the outcome, and the number of advertisements, customer foot traffic, and average product price as predictors. Your results indicate a multiple of .65. What does this mean?
The set of three predictors explain 65% of the variance in monthly sales
Acquiring information from stakeholders most often relies on methods, such as focus groups or interviews.
qualitative
If the stakeholders that we interview are unique/different in key respects from the total group of stakeholders, we would call this ______.
selection or sampling bias
________ evidence is used to monitor organizational data relative to past facts or current environmental conditions or standards, whereas evidence transforms this through analytic techniques into indicators of association or predictive models.
Descriptive; inferential
The average employee experience in our company is 7 years, and the standard deviation is 3 years. Assuming experience is normally distributed, 95% of employees have between _____ and _____ years of experience.
1 - 13 years (i.e., ± 2 SDs; 7-6=1, 7+6=13)
A company finds that scores on a survey of customer satisfaction correlate with sales performance at r = .50. This would be considered a ______ effect, and indicates that the customer satisfaction scores explain of the variability in sales performance.
large; 25%
Stakeholder relevance is determined by ______, which is the extent to which the stakeholder can affect the decision or outcomes.
power to influence
What could you do to try to minimize selection/sampling bias?
Take a random sample or select randomly from subgroups within the larger stakeholder group so that the sample matches the population.
According to CEBMa, ________ is consistently cited as the main reason data analytics projects fail.
lack of executive sponsorship (i.e., organizational politics)
The average customer satisfaction last year was 9.5, this year it is 5.5, the standard deviation is 5. How would you judge the size of the change?
Large decrease (i.e., 9.5 - 5.5 = 4; 4/5 = .80)
In the general population the correlation between cognitive ability and salary is approximately .25. If you re-calculated the correlation using a sample of 500 graduates from Harvard University, what effect would you expect this to have on the correlation?
The correlation would likely be weaker
Stakeholder relevance is determined by ______, which is the extent to which the stakeholders' interests are affected by a decision.
harms and benefits
Assuming your stakeholder sample is representative of the larger stakeholder group, what is another factor that might impact the trustworthiness of the evidence you gather?
The extent to which your stakeholder sample had the opportunity to freely express their perceptions/feelings.