In response to last week’s post about the research (or lack thereof) surrounding street checks, one poster on LinkedIn asked, “Would it be worthwhile to look at crime numbers in Toronto as compared to the number of street checks being conducted.” This is a great question. In research terms he was asking can we do a pre-test/post-test study using crime and street check stats to see if Ontario’s regulations, which curtailed street check volume, also had an effect on violent crime levels. The short answer is: No.
This research design (pre-test/post-test) won’t answer the basic question of whether street checks deter crime because there is the possibility of a little phenomenon called “confounding variables.”*Now I realize that pretty much everyone hates it when academics get all academic-y, but you need to bear with me on this one because it is important.
Confounding variables are: factors that the researcher may not have accounted for, that could be influencing a study’s findings.
You know the old saying “correlation does not equal causation”? “Spurious correlation” is the reason why people say that. What is it? It’s when researchers see two things as related and suggest that one causes the other when there are other likely factors involved (confounding variables!!). When I teach this is in policing workshops, I always use my favourite example:
Murder rates and ice cream.
It is a well-established fact in criminology that when murder rates go up during the year, so does ice cream consumption. We could suggest that this relationship is a causal one and argue that:
Brain freeze from ice cream causes people to become violent
People who commit murder like frozen sugary treats
People murder to get increased access to ice cream
Or, there is NO direct relationship between homicides and ice cream and these two phenomena can be explained by a third factor: weather conditions. In North America, SUMMER produces both more ice cream consumption AND is when we see higher rates of violent crime.
Let’s go back to the street check problem. It has been said that criminal shootings increased after the Ontario regulations went into effect in January 2017. If we look at Toronto's numbers, as an example, we can see what appears to be evidence of this:
Source: TPS online.
The actual number of shooting victims went up from 2016 to 2018, as did the number of occurrences. Small problem, though: look at the massive jump from 2016 to 2017. If lack of street checks emboldened criminals in 2017 and 2018, what was going on in 2016? The single largest increase in shootings in Toronto over the five year period occurred when street checks were still in use. Now you could tell me that increased public criticism of their use caused some police officers to stop conducing them, but I'd need to see some good evidence on that to be convinced (and, trust me, anti-street check advocates would not be likely to take that assertion on faith either).
So, you need some way to test just street checks independently of any other factors. Making things more difficult: there's a BIG factor that could potentially derail any tests: visible police presence.
It is very well established in the research literature that visible police presence alone can produce crime deterrent effects – in fact, the entire body of research known as ‘hot spots policing’ attests to this. So, we come to the chicken and egg problem of street checks. Let’s say for argument’s sake there IS a crime deterrent effect, what produced it: the street check or the visible police presence? To know whether it’s the act of performing a street check, you would have to rule out visible police presence as a factor and the only way to do that is with a randomized controlled trial* – that is, an experiment in which you assign carefully selected and matched geographical areas to one of three ‘treatment’ conditions:
Locations that get street checks (which necessarily includes visible police presence)
Locations that get only visible police presence and no street checks
Sites that get neither street checks or visible police presence (just regular call response)
Then, you sit back and see what happens. Wash, rinse and repeat (in as many cities/site as possible until you start to develop a reliable evidence base).
*I have a paper exploring this in greater detail that will be out shortly and be publicly available.
**Before anyone accuses me of setting myself up with a new, nifty research project, this type of research demands technical skills I do not possess. You need to call David Weisburd.
