Assessing what works research: What is good enough?

Assessing what works research: What is good enough? Jennifer Brown Mannheim Centre LSE [email protected] Paper for Canterbury Christchurch University Conference Evidence based policing: Beyond the RTC 22nd -23rd June 2016 Starting position It is difficult to imagine anyone arguing that policy should be based on anything but the best available evidence (Marston and Watts, 2003:144), rooting policy in evidence has all the appeal of motherhood and apple pie. (Tilley and Laycock 2000:3) Why does it matter Exclusion of much relevant evidence Justice and human rights Narrowness of questions

Definition of EBP Strict definition research findings resulting from a medical model of randomised control trials and excludes those from non-scientific studies not achieving the gold standard of excellence of the RCT Broader definition drawing on a wider rand of inputs including practitioner experience and user consultations Purposes Decision making-Sherman Change-Stanko Legitimation-Weisburd and Neyroud Tool-kit-Hough Dramatis Personae

Strong experimentalists-police scientists Quasi experimentalists Crime scientists Constructionists Pragmatists Maryland Scientific Scale Level one - simple correlational analysis and represents the lowest level as there is no means to establish cause and effect (reconviction measured for intervention group only) Level two - before and after study but which cannot rule out possible confounding effects (comparison of actual and predicted reconviction for intervention group only) Level three- compares an experimental and comparable non experimental area at the same point in time (comparison of reconviction rates from treatment and unmatched controls) Level four -multiple experimental and comparable control areas ( comparison of

reconviction rates from treatment and controls matched on theoretically relevant factors) Level five - RCT random assignments to the experimental and control conditions. (comparison of the reconviction rates from treatment and control groups with randomisation to groups) Holy Grail? There is "no gold standard methodology" (Laycock, 2005:8) There is no toolbox to be had containing "a set of experimentally validated policy spanners (Hough 2011:3). Wicked Problem Quest Who are the stakeholders? What are the issues? What are the constraints? What are our assumptions? What are the key decisions you have to make? How will you make them? More questions?

How does it work? Why does it work? On whom does it work or not work? Are the changes sustainable? Is it fair? Is it transferable? How do participants feel about it? Are there any unintended consequences? What do the general public think? Accounting for differences Epistemological bi-furcation Within discipline fragmentation Competition for ascendency - a kind of disciplinary imperialism. The advent of the policy entrepreneur Contingent repertoires Pawson and Tilley Quasi experimental paradigm has resulted in moribund evaluation being itself a contributory factor to the nothing works lament (p91)

Bennett Their claims that we have learnt nothing from experiments ..seem unnecessarily nihilist and dismissive (p570) Policy entrepreneur Policing is political (Reiner, 1980) Scientists are political (Bissett, 1972) Both have policy entrepreneurs (Kingdon, 2003) Someone willing to invest resources, energy and reputation to promote a particular approach Politics of police research Consensus Controversy Conflict Contradictory Crime control Collaboration Combative

} extensions to the Reiner classification Politics of Criminal/Forensic Psychology Research Early accord Parting of the ways Not on speaking terms Return to cordiality Business as usual Timeline Events 1950s Police evidence

Government evidence Soc/crim research evidence Fabian (1950) Home Office Research Group Royal Commission 6062 Consensus Pych research evidence Other evidence Accord Journalistic accounts Investigative Journalistts 60s

Corruption allegations St Johnstone Simpson 70s Red Lion 74 PF Law/Order campaign 75-9 Mark (Operation Countryman) 7882 Newman 85 Plus 89 Scarman (Red Lion Square inquiry) 74/75 Conflict Parting of the ways Scarman 81 HO Circs114/83

Contradictory Not on speaking Wolfe Olins 88 terms Saatchi and Saatchi Royal Commission CJ 91-93 Sheehy 93 Posen 95 White paper 98 MacPherson 99 Confrait 77 Policing initiatives Unit Beat Policing Controversy 80s Brixton 81 Miners' Strike 84/5

90s Quashed terrorist convictions 89-91 Discrimination cases 90-91 Lawrence murder 93 OPR 90 QOS 90-92 00s London bombings 05 De Menezes 05 G20 demo 09 ILP 00 MPS /STAU 03 Morris Inq 04 Flanagan Ind Review 2008 White paper 01 White paper 04

Collaboration Present Summer riots 11 Hacking 14 Rotherham 15 Hillsborough appeal 16 Angelini review 2014 Comprehensive Spending Review 10 Winsor 11/12 Leveson 15 CoP what works centre Combative Audit Commission papers Crime control Return to cordiality

Business as usual Problem Oriented Policing Policing by Objectives Performance Matrix QOS Total Geographic Policing Zero Tolerance Policing KPMG Quest 2008 Reform policing papers NIM National Reassurance Policing Workforce Modernisation Hillsborough IP 12 IPC 13 Jay 14

Drew 16 PCC Total Policing Evidence Based Policing PROBLEMS Unit Beat Policing Workforce issues Recruitment and wastage problems Changing demands Rise in crime Changing patterns (largely due to increased car ownership Changing expectations

a) Police morale b) Pay and working conditions (Edmund Davies) Effectiveness of public police interactions Some evidence from Royal Commission Banton criticisms Solutions offered by UBP would do more for less and be "more interesting" thus help recruitment and minimise wastage Motorising the

police would help respond to new demands offered some excitement to the role and also facilitated changes in working hours Evidence-based policing Reduction in numbers through austerity cuts Solutions offered by EBP Contributing to efficiencies enabling doing more for less and more "intelligent" approach strategically Fall in crime Focus on Changing pattern prevention

through cyber crime Tackling new forms of crime a) Police morale b) Working conditions (Winsor) offered Stop and Search opportunities for problems better police-public contacts Organisational procedural justice approaches Work on unconscious bias DRIVERS Unit Beat Policing Solutions offered by UBP

Evidence based policing Solutions offered by EBP Antidote to scandal Arrival of technology as modernising agenda Demonstrable use of new technology (radios) and increased speed of response demonstrable modernising of the police Arrival of science as integral to the professionalisation agenda CoP systematic

reviews and good practice guidance Chiefs of Lancashire and the Met actively promoted and introduced "experiments in their forces Larry Sherman Gloria Laycock Mike Hough Betsy Stanko Active promoters of particular approaches Policy St Johnston/ entrepreneur Simpson Zeitgeist White heat of Enthusiasm for the Fit with what works- Norming research technology

police to be seen at the measurement quality to scientific forefront standards DISILLUSIONMENT Unit Beat Policing Evidence Based Policing "Evaluations" Police culture Special course critiques De-legitimation thro more coercive style of policing Anti experimental academics Anti-intellectualism reaction Critique of domination of single role Fire-brigade reactive policing Crime fighting

Undermining of community role Shift from foot patrols Diversion of community police officers Loss of "quality" contact with the public Fast cars reactive patrol no time to get out and "talk" Bureaucracy and paperwork attached to PIs Promotion of internet contact and virtual file exchange Compromising intelligencedirected patrol cycle Problems with collators role

Disconnect between national security and local policingHigh vs low policing Conclusion "Like many wars, the methodological paradigm wars in criminology (and throughout the social sciences) may be based primarily on misunderstandings of what each side is really about. Quantitative researchers are often dismissive and condescending about qualitative research. Likewise qualitative researchers can spend a great deal of time criticizing quantitative research when the generation that fought the qual v quant paradigm wars of the late twentieth century passes into retirement, I imagine that few of their successors will likely remember or understand what the fighting was all about." (Maruna, 2010:137) Take home Messages New wine old bottles Problems with the Medical model The dialogue of the deaf Research imperialism

Some alternatives Solving the wicked problem The alternatives A medical model using scientific experiments, the Maryland Scale and accepting only evidence approved by the Campbell Collaboration A multidisciplinary model offered by Crime Science A social science model comprising either/or both psychologists and criminologists offering a range of methods to suit the question A mixed methods integration of quals and quants A legal model resolving particular cases and developing a set of standards and principles Other models such as pragmatic psychology which proposes the creation of a peer reviewed data base of systematic, rigorous, solution focused case studies. A futile task?

Recently Viewed Presentations

  • Practical RSS for Collecting and Disseminating New Content

    Practical RSS for Collecting and Disseminating New Content

    What is needed so I can use it ? You need to locate content that is syndicated. You need to use an RSS Aggregator (aka RSS Reader) to view the syndicated content. The RSS Aggregator Newspaper analogy Another aggregator (Newsgator)...
  • Présentation du réseau de neurone RBF

    Présentation du réseau de neurone RBF

    Cette solution permettra de d'obtenir la meilleure répartition des centroïdes possible. Beaucoup proposent aussi d'appliquer une carte auto- organisatrice de Kohonen afin de choisir les centres. * Le réseau RBF Paramètres (7) * Une fois tous les centres Cj choisis,...
  • Academic Vocabulary - Mrs. Baker: English 9 and Advanced ELA 9

    Academic Vocabulary - Mrs. Baker: English 9 and Advanced ELA 9

    Academic Vocabulary Practice In your seating groups, write one detailed sentence for each of the academic vocabulary words (6 total) The sentences should be about TMDG Each person should share in the writing of and creation of the sentences (Work...
  • The Initiatives of the International Aerospace Quality Group

    The Initiatives of the International Aerospace Quality Group

    Supplier training pack in preparation. NPI framework & training needs resolving before this process can be implemented. Engr support to indentify all CCFs created to date. P Gesanidis. 3/29 audits completed and reviewed per new GQP template. 14 NARs outstanding,...
  • Company Analysis - Beedie School of Business

    Company Analysis - Beedie School of Business

    PC Makers Presented by: Doey Au-Yeung Pierrick Chamois Liang Min Huy Le Overview Industry Analysis Company Analysis HP Apple Dell What is a PC Multipurpose computer system Assembled from standardized components Components perform identical functions Minimal compatibility issues >90% use...
  • Homeopathy - Weebly

    Homeopathy - Weebly

    1835 - 1st homeopathic medical college - Allentown, Pennsylvania. By the turn of the 20th century, 8 % of all . American medical practitioners were homeopaths, and there were 20 homeopathic medical colleges and more than 100 homeopathic hospitals in...
  • COEN 252 Computer Forensics - Computer Engineering

    COEN 252 Computer Forensics - Computer Engineering

    Hard Drive Geometry. Cylinder. Cylinder is formed by the tracks on all the platters with fixed actuator. (Due to different temperatures and hence different arm length, it is impossible to read and write in parallel.)
  • Toulmin Model PPT - Mrs. Garcia's Class

    Toulmin Model PPT - Mrs. Garcia's Class

    Now, I would like each group to use the prompt we've been discussing and create a first paragraph/introduction. Synthesis Essay Handout / Scoring Rubric. HOMEWORK: AP Style Reading Questions (due Tuesday)