We’ve all seen it: the spectacle of the ‘talking head’ on television or in print, discoursing forcefully on some subject matter in which they claim, or have received sans proof, the label of ‘expert.’ One of the reasons I avoid the daily news, is because, I, like many others whose work gets represented in the media, find it a source of mild annoyance to see some colleague, whose claim to fame is studying hip hop culture or research into family demography, waxing sociologically on serial homicide. Probably the single worst, and most telling example, of this I have personally witnessed was whilst chatting to a very notable ‘expert on terrorism’, while I was waiting to present the results of a study my team had been diligently slaving away on on young women’s Islamic State affiliated twitter activities. When I asked this fellow, who was then routinely featured as an expert panelist by various Canadian media outlets, what type of research he works on, he replied, “I do secondary data studies.”
The TL/DR version: I read what other people research.
Momentarily confused, I attempted to clarify the response, “you mean, you don’t collect your own data?” “No”, he said, “I don’t want to end up on a No-Fly List.” And here was sucker me actively following Islamic State figures on Twitter and being visited by CSIS*, when all I had to do was ‘read what other people write’ to get some expert cred.
I see a similar phenomenon in the world of policing and policing research, whereby individuals miraculously appear and become the go-to ‘experts’ on different topics. Usually my introduction to these ‘experts’ goes like this: “Do you know Dr. So-and-So.” “Um … no.” “Oh, that’s funny. He’s our big expert on [insert topic du jour].’ The world of policing research is fairly small, particularly in Canada. When I started the predecessor to CAN-SEBP – the Canadian Policing Research Network – I reached out to every single academic in Canada who had ever published any research on policing, including people who had only ever done 1 or 2 studies. The total was 80. Of these, less than half were full-time policing researchers. Most were critical criminologists; only a smaller handful were applied researchers. Conservative estimate: maybe 20-25 academic researchers (who only or frequently conduct direct research with police agencies.
What we do have, however, is a number of:
Junior scholars (graduate students and assistant professors) who want to break into policing research or be cited frequently as an expert in the media because universities now reward this behaviour;
Experienced policing scholars who do not do applied work, but can bamboozle with the best of them in order to gain access to police data;
Consultants with a doctorate (my famous friends, Drs. So-and-So) who parlay expertise in one area into something potentially more lucrative;
Consultants without a doctorate who claim research expertise but have, at best, some graduate level training in research methods and a feel for where emerging lucrative markets might exist.
Whenever possible, I privately advise police agencies as to the backgrounds of researchers to whom they might potentially provide access. However, there is no better substitute for doing one’s own due diligence. So, what should police agencies do?
Ask to receive a copy of the CV of any potential researcher and, where possible, compare it to online versions to see if anything's missing or added (most of us have our CVs available on department websites);
Ask for the names and contact info of any police agencies the researcher previously worked with;
Actually contact those agencies given as referrals (it's shocking how often this isn't done)!
If someone lists publications on their CV, try to track down the full version or at least an online abstract to see what type of work they actually do. One police service wasted considerable time and energy supporting the work of a well-known policing researcher who produces dense, critical works that are of zero use to the agencies studied. Had they looked at his publishing history, they would’ve seen this.
Look at the nature of the person's funding and the awards they’ve received (if any). Just as is the case with publications, awards and funding are typically peer-reviewed and/or reviewed by external funders and indicate whether an individual’s work is valued by others. Conversely, if they are a tenured professor, and they’re still listing awards from their undergraduate days, you might want to ask, ‘what have you done lately?’
Verify through their CV they actually have the expertise they claim. Recently, I saw both a terrorism expert and an urban sociologist claiming expertise on gun violence – neither with any significant training or having produced a body of research to back up a claim of expertise.
Do not base a decision on whether you ‘like that guy’ or think, ‘she seems alright’ or if they seem to be saying ‘all the right things.’ Yes, being able to work well with someone is important, but should not be a substitute for true expertise in the field. Remember: people thought Ted Bundy was a great guy.
Do not be taken in by degrees. Having a degree is an accomplishment, but lots of idiots are able to accomplish the feat of getting one. It’s not the prestige of having a degree that counts, it’s what someone is able to do with what they have presumably learned.
If you’re on social media, check out their posts and their social networks. My friend Jerry Ratcliffe has a saying that I think fits here: ‘expertise is not self-endowed; someone is only an expert when others think they are.’ Part of learning whether others think your ‘expert’ is really an expert is to peruse their social networks and who follows them. I often find this to be a pretty good indicator of how valued or respected someone’s opinion or work is by others in their field.
And lastly, even if they do seem to have some prestige attached to them, still listen carefully to what they say and how well they support knowledge claims made. From Andrew Leigh's Randomistas:
"Strong advocates of evidence-based medicine, such as Alvan Feinstein and David Sackett, argued that the public should pay less attention to the prestige of an expert and more to the quality of their evidence."
This sentiment also applies to evidence based policing.
Why do I even care about this? Because, as any researcher in the field will tell you, every time someone goes into an agency and misrepresents themselves, their work, their intent, their goals and/or their deliverables, it makes it just that much harder for the next person to be let in.
* Not to worry, it was a friendly chat. We had coffee and I got some great advice on re-sanding my original Edwardian hardwood floors.
