The killing of George Floyd made vividly clear what experts in criminal justice have long known. Interactions between officers and civilians that should have been routine too often escalate to arrests or violence, and officers use their discretion in ways that systematically treat white and black civilians differently. These problems are persistent and pervasive, and have created a legitimacy crisis for the police so severe that it has led to calls for them to be dismantled. 

The tragic, unavoidable fact is that we simply do not know nearly enough about how to actually reduce abusive, racially motivated policing. As an expert on criminal justice policy, I have several ideas, as do the activists crying out for action. Given the state of social scientist knowledge, however, there are few ideas for improving policing outcomes in America that are much more than untested hypotheses. 

Justice demands that we devote extraordinary resources to testing those hypotheses, keeping what works and discarding what doesn’t. All the good will and political force in the world will be for naught if the ideas we implement in the heat of the moment don’t work in practice. We need to be prepared to fail, again and again, until we finally get it right. 

There are, thankfully, a handful of well-designed studies that point in the direction of changes that could reduce the lethality and racialized disproportion of police-citizen encounters. These studies provide some useful evidence that diversity makes police departments more effective, procedural justice training improves police behavior, and accountability for bad behavior is important–but that the type of oversight matters. Change on the scale that the protests have called for demands that we learn from what little high-quality research we have, but even more importantly, invest whatever it takes to learn more. 

Diversity

Beginning in the 1970s, courts ordered police departments to implement affirmative action plans to increase the share of officers who are black. Researchers have used those court orders as a natural experiment, to test the effect of increasing the share of black officers on various outcomes.

A study by Anna Harvey and Taylor Mattia found that affirmative action for black officers not only had the intended effect of increasing the share of black officers, but also reduced racial disparities in crime victimization. This appears to be because black victims were more likely to report crime to police, once they trusted officers to help.

Does this mean that increasing the number of black officers on police forces today could have similar benefits? Perhaps. The world has changed a lot since the 1970s, and police forces are already much less white than they used to be. But too many police departments are still far less diverse than the communities they’re serving. 

We should test the hypothesis that hiring more black and Hispanic cops would improve outcomes, along with other ideas related to recruiting and eligibility requirements. Does having a college degree make you a better cop? Or does a degree requirement simply screen out high-quality applicants who didn’t have the financial resources to pursue higher education? Unclear. The same goes for other common rules like bans on marijuana use and tattoos, and requirements that police live in the city they work for. It’s not yet clear which eligibility requirements are helpful, and which are unnecessary.

Changing who becomes a police officer in the first place could improve how police officers interact with their communities, and improve mutual trust. We need much more experimentation in this area to figure out how best to build a police force that is fair and effective. 

Training

Once police are on the force, departments have many opportunities to train them. Can new training programs improve officer behavior? There is remarkably little evidence on the efficacy of common training programs. It was thus a pleasant surprise to find not one, but two, rigorous studies of procedural justice training for police officers.

A study by Emily Owens and colleagues tested a program based on procedural justice principles in Seattle’s police department. The goal was to push officers to slow down their decision-making and take civilians’ views into consideration. The researchers found that officers who were randomly assigned to this new training were less likely to resolve incidents with arrests or use force. 

Although promising, the study used a small sample. A subsequent research evaluation by George Wood and colleagues tested another procedural justice training program in Chicago. All Chicago police were required to complete the one-day program, but the department couldn’t train all 8,000 officers at once. The training was delivered to 25 officers per month, over several years. This allowed researchers to measure its effects by comparing officers who received the training earlier to those who received it later. They found that the training dramatically reduced citizen complaints against officers, as well as the use of force. Many benefits of such behavioral changes are difficult to quantify, but one is not: researchers estimated that this one-day training program reduced payouts for settled complaints by $4.2 million over the two-year follow-up period.

Will this program produce similar outcomes in other contexts? We don’t yet know. Other cities may face different challenges that this program is less-well-suited to address. And scaling any program to cover more participants is always challenging. Often training programs are run by people with unusual talent and charisma, such that asking someone else to lead the same program inevitably reduces that program’s quality. The question is how much quality will fall as the program is scaled. In this case, the trainings were conducted by a group of ten local police officers in rotating groups of three, not by someone with highly specialized skills. (Those officers still need to be trained by an expert, but that is easier than having the expert train the entire police department.) Scaling up thus seems more feasible than it does in many cases.

Regardless, the way this training was implemented–using a “staggered rollout”–should be a model for other police departments, because it allows researchers to measure whether those programs are effective. I hope to see this and other programs tested this way, so that we can learn more about which trainings work and which are a waste of time. 

Accountability

There should be consequences for officers who, despite their training, behave badly on the job. Many police departments now require body-worn cameras, in response to earlier calls for police accountability. The hope was that if officers knew their behavior was being recorded, they would behave better. Several departments randomly assigned cameras across officers or shifts, to test the effects of the technology on citizen complaints and officer behavior. Results are mixed. In some places the cameras appear helpful, but in others they seem to make things worse. A large study in Washington, DC, found no significant effects. 

Why aren’t body-worn cameras more effective? It could be because officers already assume they are being recorded by cell phone cameras or local surveillance cameras all the time, so that wearing their own camera adds no additional benefit. It could also be that force is often used when an officer is genuinely afraid, rather than due to a conscious violation of his duties; in such cases, better training would be the answer, not accountability. On the other hand, cameras alone don’t imply greater accountability. Police officers also have to expect that there will be consequences for any bad behavior that is caught on video. And, based on past events, they have good reason to expect that they will not be punished.  

This leads to the question of how to ensure that officers face consequences for their bad behavior. Some reformers argue that strengthening external monitoring of police would reduce misconduct. But not all oversight is created equal.

A study by economist Bocar Ba shows that making it easy for citizens to file complaints has important benefits: a Chicago policy change that increased the cost of filing complaints both decreased the number of reports filed and increased police use of force, presumably due to the reduction in civilian oversight.

But another study in Chicago, by Roman Rivera and Ba, found that self-monitoring triggered by an internal police memo about recent misconduct was more effective than external monitoring by civilians once news of that misconduct became public. The internal memo reduced serious complaints from civilians without increasing crime or reducing police effort out on the job; once the public became aware of the incident, though, police reduced their effort and civilian complaints and crime rates increased.

These studies demonstrate the importance of testing the efficacy of police oversight efforts–some may do more harm than good, at least in the short run. Increasing oversight in some form, however, is clearly important. And reducing protections for officers who are abusing their power will be crucial. Police unions have typically accepted job security for their members over pay increases, but job security is not free. Reducing our ability to fire bad cops has important costs to the community, in the form of more violence and racial bias.

Next steps

Well before George Floyd’s killing, police departments across the country had begun working with academic researchers to test the effectiveness of their policies. We need more such collaborations, endeavors aimed specifically at improving the relationship between law enforcement and the communities they are sworn to protect and serve. 

But external researchers also have an important role to play in testing the efficacy of policy changes. They have more power to ask questions that might make police departments uncomfortable. 

A variety of police reforms are now underway across the country, but we don’t yet know which will be effective. Approaching this problem as scientists has already taught us important lessons about how to improve officer performance and behavior. But we have a lot more work to do. 

The good news is that, with 18,000 police departments in the United States, we have many opportunities to test new approaches, including major reforms that fundamentally change the role of policing. The bad news is that we can’t assume that all well-meaning reforms will be effective; some will surely fail, and some might even backfire. 

The urgent demands for justice that we have seen in the streets over the last few weeks do not mean that we actually know how to make citizens’ rights real where they matter, in the millions of ground-level interactions between police and civilians that occur every year. Police themselves need to develop a culture of experimentation, and citizens need to hold them accountable for trying over and over again until we know what works, and what doesn’t. When we know what works, citizens need to hold police forces’ feet to the fire to implement those reforms, even if they require uncomfortable changes in the way that cops do their work. 

Police, mobilized citizens, and social scientists all have a role to play in creating fair and effective police departments. The citizens in the street have done their part—it is time for police and social scientists to do theirs.  

Jennifer Doleac is an Associate Professor of Economics and Director of the Justice Tech Lab at Texas A&M University.