Critique of Forensic Focus Group

Methodology

100

Are there any issues with the "key questions" that the researchers asked the participants?

We don't know, they don't list them. This is an issue because we cannot replicate the methodology, or assess how well-constructed the questions were.

200

What did the facilitator do that risked "leading" the participants to give certain responses?

The facilitator "reflected on the service user's contributions for clarity." Without further information, this seems like it poses a big risk for experimenter bias.

300

How did the researchers finally decide on the 5 themes? Is this a methodological concern?

Yes, they used a "trial-and-error process until all themes worked and appropriately represented the data." Obviously there are many issues with this, one being subjectivity. The themes could have looked completely different if another person were sorting the data.

400

Describe an extraneous variable that could have influenced the results of the study, and how.

1) Experience with FMMGs or other programs, 2) Medication, 3) Gap between focus groups, etc.

500

In this setting, why might recording the focus groups be an issue?

The service users are used to being monitored and evaluated daily. Do you think they would ever say something negative about their recovery on a recording that could be used against them? Of course they're going to say everything is working great and that they're brand new model citizens.

Validity

100

In this study, sampling bias is a threat to which type of validity and why?

External validity. The 7 service users who volunteered to participate are likely much different from the others in the same hospital who chose not to volunteer. (More social? Most recovered?)

200

Social desirability is a threat to which type of validity, and why?

Internal validity. We can't be sure that these are the service users' true feelings. They are speaking in front of other service users, in the presence of an authority figure, and there's actually motivation to appear more recovered.

300

The same researchers conducted the focus groups AND interpreted the data for themes, all while knowing the purpose of the study. This is a threat to which type of validity and why?

Descriptive validity. DV refers to the accuracy and objectivity in how the results document what actually happened in the focus groups. The accuracy is questionable because they never mentioned if/how they used their written notes to code non-verbal things, and the construction of the themes was highly subjective and influenced by their own expectations.

400

The researchers use self-reported experiences with the FMMG to conclude whether or not it "promotes recovery." This is a threat to which type of validity and why?

Construct validity. It is questionable whether "recovery" can be self-reported by service users. If so, they would all be "recovered" and released by now.

500

The researchers came up with names for themes, and defined what these themes meant using their own words. This a threat to which type of validity and why?

Interpretive validity. We can't be sure that these themes or definitions actually represent what the service users were trying to say. Just because the researcher sees "confidence" in a service user comment, this doesn't mean that they would ever use that word themselves, or agree to the researchers' explanations about what their contributions mean.