If you fill out the "Forgot Password" form but don't get an email to reset your password within 5-10 minutes, please email logistics@ire.org for assistance.
Rob Barry and Coulter Jones set out to analyze police killings, not poke holes in the system that tracks them.
But when their sources started questioning the way they’d looked at the numbers – questioning, really, the numbers themselves – the two decided there was a more fundamental issue to be explored.
Their December report for the Wall Street Journal found that the FBI’s national tally of police killings didn’t include hundreds of such incidents from the six-year period they examined. Barry and Jones received information from 105 of the largest police agencies in the country and found that more than 550 police killings recorded at the agencies from 2007 to 2012 didn’t make it to the FBI’s tally. Because the FBI’s numbers are believed to be the most accurate available, the findings show it’s next to impossible to get an accurate grasp on how many homicides, justified or otherwise, occur at the hands of police each year.
In the aftermath of Ferguson, Barry and Jones sought data that reflected how many times each year police kill someone. They’d heard that the FBI had the most accurate tally, so they started there. They got the info, ran some analyses, and took what they had to academics.
"Pretty much across the board, we were told, 'Well, you can’t use the data like that,'" Barry said. "'You can’t look at it like that. That’s just not right; it’s not going to work.'"
Hit with the realization that any story derived from these stats would be inherently flimsy, Barry and Jones made a decision: Instead of finding meaning in the data they had, they’d try to find out why the data had almost no meaning.
The findings show it’s next to impossible to get an accurate grasp on how many homicides, justified or otherwise, occur at the hands of police each year.
Barry and Jones turned their attention toward the inconsistencies in the reporting process.
The FBI receives data on justifiable homicides from the states, which collect tallies from individual police agencies. Barry and Jones first reached out to several agencies that they expected, through some lapse in process, weren’t accurately getting their numbers to the FBI. They asked for the number of times police had killed in the respective agencies from 2007 to 2012.
Through careful correspondence with the departments, Barry and Jones weeded out the rare non-justifiable homicides in order to produce an apples-to-apples comparison to the FBI’s numbers.
Their targets – 20 in the first round – were easy to pick out.
"An agency might have had five incidents two years in a row, and then suddenly there’s nothing,” said Jones, who is now a data reporter for MedPage Today. “So that means that either this agency has either really improved in that area of justified homicides, or they’re just not reporting.”
When Barry and Jones put their local numbers up against those from the FBI, problems with the federal statistics became clear. But they wanted to make the story more objective and the sample bigger, so they decided to expand their search to the 110 largest police agencies in the country. Their goal was to receive responses from 90 to 100 of them.
The process was taxing. Although some agencies posted the information online, Barry and Jones still had to send out more than 80 informal requests for information. They dealt with the gamut of responses. Some agencies shot the information back within the hour. Others asked for FOIAs, some claiming the info would cost thousands of dollars. Others said they didn’t have the info, which is typically compiled in year-end reports.
"I think when you request from 100 different agencies, you’re going to end up getting pretty much every excuse that every journalist has heard at some point," Jones said.
Many of the agencies said they had the information but were wary of giving it to the Wall Street Journal.
"So I told them: we’d asked for this information from a lot of agencies," Barry said. "We hadn’t singled them out. We were coming to them because they were one of the larger agencies in the country. And I think knowing that this information was in a broader context made a lot of agencies feel better."
As they became more familiar with the data, one city stood out as an interesting case study and ended up serving as the setting for the story’s lead.
In 1998, the Washington Post had reported that Washington D.C. had one of the highest rates of police killings in the country. The city didn’t report the data for the next decade.
And while plenty of incidents from the years Barry and Jones examined did make it to the FBI, the city didn’t report any police homicides in 2012. The reporters decided to lead the series with a 2012 incident in Washington D.C. that seemed to be a clear case of an agency failing to report to the FBI.
"I think when you request from 100 different agencies, you’re going to end up getting pretty much every excuse that every journalist has heard at some point."
- Coulter Jones
The two selected the case of 24-year-old Albert Jermaine Payton, who wielded a knife and was killed by several responding officers.
"They wouldn’t even tell me the number of officers involved in the case, which is information that would have been included in what’s reported to the FBI," Barry said. "So it was just this strange incident where two and a half years ago almost, this guy was killed by police and it’s still kind of this black hole of information that we just don’t know anything about."
Some of the biggest gaps in reporting came from Florida, New York and Illinois, three states that weren’t reporting any of their agencies’ justifiable homicides.
After the Journal’s story ran, New York said they’d start collecting and reporting the information in 2015. Florida hinted it would do the same.
And, probably more importantly: the Death in Custody Reporting Act of 2013, which mandates that agencies report incidents in which individuals die in police custody or during the process of arrest, was signed into law by President Obama on Dec. 18. It had passed the House before the story ran and cleared the Senate shortly after.
"There’s no question that it will include a lot of the cases that we looked at, though there is some question as to whether or not every case falls under its definition," Barry said.
For Barry and Jones, the story provided plenty of lessons. The two said they benefited from allowing the opinions of their expert sources to change the story entirely – a decision that, after already investing hours on an analysis of the FBI’s data, would have been easy to overlook.
"Certainly this data was perfectly fair game to use in a story, to say, 'Here’s the breakdown,' and I think that’s fine to do," Barry said. "But I think it’s also important for us to, when we’re told that it’s not right, that the information is wrong, that we take a hard look at that."
Shawn Shinneman is a graduate student at the Missouri School of Journalism and a student employee at IRE. Prior to graduate school, he spent two and a half years reporting daily news at a newspaper in the Chicago suburbs. You can follow him on Twitter here or email him at shawns@ire.org.
Listen to Barry and Jones discuss the story and learn more about reporting on police killings on the next episode of the IRE Radio Podcast, out Friday.
Looks like you haven't made a choice yet.
