It’s time to revisit one of my favourite topics: improving rigor in qualitative research. What is this, you say?
Anyone who has ever taken a class or training session with me has heard me rip a new one – metaphorically speaking – into examples of junky research. I’ve picked on lots of quantitative studies (the famous study that purports to show that yogic chanting reduces crime), more than a few experiments (getting monkeys high on the wrong illicit drug to explain why rave drugs are bad), and a few horrible qualitative pieces (a media analysis of ‘hobocops’ that concludes, from pretty much nothing, that cops like dressing up as homeless people to humiliate homeless people).
As a qualitative researcher, I am pained by poor qualitative research. Like a skilled police officer, who has learned her craft after years of applied work, I too believe that what I do is a highly important type of work and I want others to see it that way too. And when I say ‘others’, I mainly mean other qualitative researchers – particularly the ones who think they can crank out any piece garbage and get it published to accolades. You see, I respect something called ‘rigor.’ The non-academic term for this is ‘quality’.
In the interests of offering my tiny contribution to improving quality in qualitative, I want to sketch out some old and new ways in which to potentially increase the rigor of one’s qualitative work.
Idea #1: Triangulation
This is a big word for using 3 different methods to study the same phenomenon. I’m not sure why 3 became a popular number for using multiple methods, likely because quadrangulate doesn’t have the same ring. Regardless, the principle is simple: the more methods you use to explore a phenomenon, the more you increase the likelihood you will arrive at a fuller, more complete – dare I say it? More accurate – picture of what it is that you are looking at.
For my PhD on ‘skid row policing,’ I used the following:
You don’t need to use these exact 3 methods, of course. You have to use whatever methods, in whatever combination, you think will increase the strength of your work.
Idea #2: Charting results
I stole this one from Janne Gaub, Natalie Todak and Michael White (thanks Janne!) and their paper on police officers in specialty units.
One of the issues with qualitative research is that some researchers fail to obey one of the central tenets of good research: documentation. That is, they do not provide a detailed discussion of either what they did or how they came to reach their conclusions from the data they collected and analyzed. Thus, you get to the end and they’ve slammed you with a conclusion that you’re left wondering where the hell it came from.
What Janne, Natalie and Mike do differently is they actually present a chart of the major themes that emerged in their focus groups with police officers, thus showing us visually (and numerically) how much agreement there was among officers on different themes, concepts, ideas, issues, and so on. As a result, you get a better sense of how much overlap there was among officers.
You should steal this idea*!
Idea #3: Sampling frame
If you are working with a larger population, or segment of a population, use a sampling frame.
What is this?
A sampling frame is a way of conceptualizing the entire population you are interested in and how you will sample a portion of it. Let me give an example:
Let’s say there are 350 Sergeants in all of the Vancouver Police Department and your study focuses on their work experiences (this is your population). Now list all of the departments/units that have one or more Sergeants in them. That is your sampling frame.
If you want your study to be representative of the experiences of all of the Sergeants in your sample population, then you should try to ensure your sample is reflective of the different roles and tasks across the organization by using your sampling frame to construct your sample. Or, in English, don’t just focus on 14 guys/girls from Patrol if you think you’re going to want to say something meaningful about what it’s like to be a Sergeant in Ident.
There you have it: three easy ways to increase the rigor of qualitative research. I’ll have more to say on this in future installments!
*From: One Size Doesn’t Fit All: The Deployment of Police Body-Worn Cameras to Specialty Units by Gaub, Janne E; Todak, Natalie; White, Michael D International Criminal Justice Review, 07/2018
