#AlternativeFacts in the Land of Criminal Justice Research
Updated: Aug 24, 2019
A year or so ago I was asked to participate in a conference the aim of which was ostensibly to begin the process of generating rigorous research in a hitherto unexplored area of public safety. The talk was well-attended by police, government, community representatives and a healthy smattering of consultants who work in the criminal justice field. My part of the proceedings was simple: to argue for the need to generate quality research and explain what scientific principles make for rigorous research*.
The talk generated some interest, as evidenced by the questions afterwards. But what stuck out in my mind – and really, I’m pretty naïve not to have seen this coming a mile off – were the comments by one of the consultants. In essence, this individual challenged me on methodology, arguing that the types of rigorous research I was demanding were not reasonable given the limitations researchers can face in the field. While I disagree, and did so then, what struck me dumb was the assertion put forward by this person that two sets of facts can be simultaneously true at the same time – in this case, that a program can be both proven (that is by very weak evidence and anecdote), and not proven (as far as rigorous research standards go). If this was the only time this claim was advanced, I might’ve dismissed it as an off-the-cuff remark or something I misheard, but in subsequent correspondence the claim was reiterated in the exact same context. The second time I was told: “Of course we value pure and objective research … but more than one thing can be true at the same time.”
In essence, this whole faux debate – because, really, it’s not much of a debate unless you also ascribe to #alternativefacts and #fakenews - is a bit like one side arguing the efficacy of witch doctors can be both true and untrue at the same time. From a scientific perspective this is crazy talk and the response should be, "Unh … no." Fortunately, science has a concept that also helps us to understand why people might want to bend themselves into believing multiple truths at the same time, such as believing that placing a diamond in the bottom of a vat of water and chanting over it will infuse it with "love" and healing energies that can cure you: it’s called placebo. That you benefited from a placebo effect does not mean that science is wrong about the claim that diamonds can change the molecular structure of water, or that there’s more than one truth out there on the topic. It means that you were so persuaded by your beliefs that you see things how you see them. When challenged to back up one's position, the water fan then attempts to justify their belief with some post hoc rationalizing, relying heavily on anecdotes and weak data. Stories of miracle cures may make for interesting tales at sales pitches, conferences and workshops, but that's not how science works and it's anathema to evidence based policing.
Having posted this, I guess this means I'm not going to be invited to any more of these conferences. And anyone who's seen me (or, more aptly, not seen me) at a conference can only imagine how I feel about that.
** To be clear: these are not my personal standards for research we are discussing. I did not make up the principles of reliability, validity, transparency and the other foundational values which form the bedrock of modern science. I wish I had. That would be pretty cool!