In a previous blog I talked about the issue of smaller sample sizes in qualitative research*. As you may remember, one of the most common responses I get when raising this issue is, “but it’s HARD to conduct all those interviews! Plus, who’s going to transcribe all that data and code it?! [insert sad face here].” Aside from the fact that such responses seems to suggest that qualitative research is the lazy person’s refuge – a characterization that makes me gnash my tiny fangs. The reality is that you can achieve decent sample sizes, reduce some of your coding issues and generate quick descriptive statistics from your qualitative data, all with only a little ingenuity. What is this wonder of nature, you ask? It’s a little tool called the structured interview guide (aka ‘checklist’).
There is a fair amount of variety in interview approaches. Some people go in with a list of set questions they read, others have general topics they wish to hit upon but otherwise leave their interviews unstructured. Any and all approaches are fine depending on what you're trying to accomplish. For example, an unstructured approach is handy if you’re doing exploratory research and aren’t sure what you might find. If generating a large N (or number or sample size) is your goal, then creating a checklist is a great way to do this. It also has a significant added advantage, especially if more than one person is conducting interviews: it helps to increase the level of consistency across interviews, helping you to make sure that you ask the same questions of each participant.
The example above is drawn from one of my own studies of victimization**. As you can see, categories of questions are broken down into clusters (demographics, victimization (pre and post-homelessness, symptoms of trauma, etc.). The checklist doesn’t need to make perfect sense to everyone, however it does have to make sense to anyone who’s using it, so training is key here.
As you might guess, one of the biggest advantages is that filling out the checklist as you’re conducting the interview allows you, as the researcher, to immediately start coding respondent answers (which you can then check against transcripts of your recorded interviews later on to increase reliability).
Now let's address one of the major criticisms typically leveled against questionnaires and administered surveys: they don’t feel like natural conversation and therefore can be more than a bit off-putting to research participants. Trust me, I get it. There’s nothing worse than an awkward conversation in which one person feels like they’re under a microscope. That said, with some diligent practice, interviewers can quickly develop the skills necessary to run through the checklist while maintaining the semblance of a ‘real conversation.’ I’m not just saying this. I’ve had two different teams use a version of the same checklist above to generate over 200 interviews and, having either supervised the interview, conducted it and/or reviewed the interview transcripts, I can honestly say that, as interviewers got more comfortable, the process generally became much easier for both them and the participants. As a result, we learned a lot about how women process trauma post-victimization and the supports (or lack thereof) available to them. Given what we were trying to accomplish, the larger sample size was necessary and the checklist approach made our life muuuuuuch easier (without sacrificing any methodological rigor).
*I am not opposed to smaller sample sizes if they are justified in light of both the research questions and the conclusions. However, if you're giving me a sample size of 10 and telling me, based on this, that we need to overhaul the criminal justice system, you're getting some side eye. Just sayin'.
**The fact we were conducting this work in the U.S. explains the use of the term 'native' (as in Native American) in the demographic section of the checklist. If we were working in Canada, it would've read 'fn' or something similar for 'First Nations.'
