Last year, a company called Neighborhood Scout put out a report that called an area of St. Louis near 14th Street and the Rev. Dr. Martin Luther King Drive the “14th most dangerous” neighborhood in the country.
To be sure, St. Louis has many pockets of poverty and despair. But this slice of western downtown isn’t one of them. If you were going to get “whacked” standing at the corner of what local residents call “14th and MLK, ” it would likely be from a volleyball flying out of Wave Taco, a local outdoor beach-themed bar, not a stray bullet.
In fact, it's not even a “neighborhood.” As someone who has covered crime for several years at the St. Louis Post-Dispatch, I can confirm that it does not appear as such on any of the crime data reports for police districts and city neighborhoods published regularly by the city. The area spotlighted by Neighborhood Scout would be better described as a census tract.
We ignored the report because we had better stories to write. The 2009 rankings, however –published first on AOL’s personal finance site WalletPop–got wide attention across the country. That may be one reason the site seems to be planning to make them an annual event. This month, WalletPop published the 2010 “figures” from Neighborhood Scout. St. Louis wasn't even on the list.
Presumably, 14th and MLK are now safe enough.
But don't expect an explanation. Neighborhood Scout doesn't bother to give one for anyone who might want to compare what happens year to year. (In 2010, a Chicago neighborhood–West Lake Street–earned Neighborhood Scout’s “most dangerous” distinction.) Again, that’s not even a real Chicago neighborhood. But according to Neighborhood Scout, if you live there you run a 1:4 chance of being a crime victim over the course of a year.
Rounding out the “worst” five, in order, were: Scovill Avenue in Cleveland, Balzar Avenue in Las Vegas, North 28th Street in Las Vegas, and Carter Street in Atlanta.
How does Neighborhood Scout arrive at its conclusions? It's an important question. Many news outlets, such as CNN, ABC News and The Chicago Tribune travel section, have quoted these rankings uncritically. And the site's readers may use them to decide crucial questions about things like where to live, where to visit, or where to buy homes–or where not to.
Neighborhood Scout holds itself out as a resource for people looking for a place to live, start a business or invest in real estate. Its site advises using the neighborhood crime rankings so you can “know before you buy or rent.”
Neighborhood Scout is based in Woonsocket, R.I., and directed by Andrew Schiller, who earned a Ph.D. in geography from Clark University in Worcester, Mass., according to the company’s web site. The site says he previously worked as a scientist at Oak Ridge National Laboratory and Atomic Energy Complex, and was director of science for The Nature Conservancy’s Tennessee chapter. The company also offers rankings of the safest and most dangerous cities, the best public schools, and most affordable neighborhoods. The company makes some reports available for free, and others premium services for subscribers.
Schiller, writing in 2008 on a blog about economic development in Rhode Island, said he had created the “biggest, most in-depth smart search engine for neighborhoods ever built” and bragged about the attention he was getting from national media including Forbes Magazine, Yahoo! and AOL.
The WalletPop story claims that the neighborhood crime rankings were compiled from “exclusive data” developed by Schiller’s team “based on FBI data from all 17,000 local law enforcement agencies.”
The story goes on to say, tellingly, that the rankings represent “the top 25 most dangerous neighborhoods with the highest predicted rates of violent crime in America”(emphasis added).
That offers a clue to what the rankings really mean: Neighborhood Scout doesn't measure crime; it predicts it. But we aren't told how it creates the predictive model that it uses to develop its “exclusive” data.
And we aren't likely to be. When contacted by e-mail, Schiller refused to explain the methodology beyond what was already described in the WalletPop site, adding its models were proprietary and “intellectual property.”
When it was pointed out to him that the findings were treated with considerable skepticism by journalists and police officials, he didn't respond.
WalletPop spokeswoman Mandy Albers said that “there is no doubt in our mind about the veracity” of the Neighborhood Scout data.
The exact relationship between Neighborhood Scout and WalletPop was unclear. The Philadelphia Daily News quoted Albers as saying the site provides “an outlet for [Neighborhood Scout] to showcase their research.”
I noticed one other discrepancy while scouring Neighborhood Scout for insight into its methodology: Schiller and his team are using FBI crime data from 2008. Those are some old numbers. Journalists and criminologists often gripe about how long it takes the FBI to release crime statistics; the bureau just released final data from 2009 a few weeks ago. And Schiller’s team is a full year behind that.
Neighborhood Scout isn't the only self-appointed expert in crime rankings. Another company, for instance, annually uses repackaged data from the FBI's annual crime report, Crime in the United States, to come up with “Most Dangerous City rankings.”
Such rankings are “bad science,” says Richard Rosenfeld, a criminologist at the University of Missouri-St. Louis and current president of the American Society of Criminology.
Rosenfeld, an expert on crime statistics, says the methodology for establishing rankings should be transparent and the fact that companies won't explain how they arrive at their conclusions should alert anyone who is tempted to take them seriously. Rosenfeld added that Neighborhood Scout researchers should have asked cities for real neighborhood crime data to test whether their predictive models were accurate — but said there is no evidence they did so. If they had, “I think they’d be in much less trouble with police departments.”
He added: “I would never base a decision to move anywhere, leave anywhere or do business anywhere on a model the construction of which I didn’t understand.”
Charles Wiley, the police chief of Galveston, Texas couldn't agree more.
After Neighborhood Scout ranked a neighborhood in his city as the country's 21st most dangerous this year, he was surprised—and angered.
“We have no idea where (they) got those numbers, what parameters were used in the research, or any other details by which we might offer comparisons,” says Wiley. “We don’t believe it and would be pleased to offer our own data for those interested.”
Atlanta cops made the same point in response to Neighborhood Scout’s decision to place four city neighborhoods among the nation's 25 most dangerous in 2010. While it conceded that some of those neighborhoods registered high crime rates, it disputed that they were bad enough to merit being part of a worst-of-the nation list.
Perhaps they were being defensive?
Not at all, insist the Atlanta police. “The report was not conducted by someone performing scholarly research, or sanctioned by an academic institution or think-tank,” the department said in a statement.. “It was conducted by the CEO of a … for-profit entity that sells real estate information. Our Tactical Crime Analysis Unit is attempting to make contact with the author of the report, in order to understand in detail what statistics and methodology he used in assessing Most Dangerous Neighborhoods.”
And, Atlanta police added pointedly, “as of yet, the unit has not been able to get the information requested of the author, in spite of numerous attempts.”
After publication of the 2010 rankings, crime journalists around the country decided to take a closer look.
Tony Rizzo of the Kansas City Star visited a neighborhood in his city that earned the 23rd most dangerous ranking in 2010 only to find the area “downright tranquil.” In his account, he described seeing “mothers pushing baby strollers, grocery shoppers, young couples, or men hitting the pavement looking for work. The threat of violent crime seemed far from their minds.”
Neighborhood Scout called the area around Charlotte’s North Tryon Street the nation’s 11th most dangerous neighborhood. Charlotte Observer reporters Fred Clasen-Kelly and Steve Lyttle quoted a city councilwoman as calling those results “nuts” and “wrong.”
The Observer said Neighborhood Scout officials did not respond to a request for an interview, but instead e-mailed an explanation of how the organizers used “mathematical algorithms” to statistically estimate incidence of crime for neighborhoods. The newspaper said Neighborhood Scout told its reporters that a 6-month study boasted “upwards of 87 percent accuracy in most cases.”
Randy Simes, of the blog UrbanCincy, compared Neighborhood Scout's analysis of Cincinnati's Over-The-Rhine neighborhood to actual crime data and found the report to be “wildly inaccurate.”
The report “has, once again, overstated the number of crimes in portions of Over-the-Rhine. Unless this site can refine its data, the mainstream media should be hesitant to give any credit to these reports,” Simes declared.
Christine Olley, with the Philadelphia Daily News, seemed incredulous that the area near Philly’s North 13th Street was labeled 6th most dangerous. “Not even close,” she wrote. The neighborhood was not in one of the nine high-crime districts targeted by Philadelphia Police Commissioner Charles Ramsey.
A story by the Houston Chronicle‘s Zain Shauk suggested errors in Schiller’s calculations of crime in Galveston.
The 2000 census counted 1,858 living in the tract that Neighborhood Scout called 21st most dangerous this year, Shauk noted.
The Census Bureau has not collected population data at the census tract level since the 2000 Census, he noted. But Neighborhood Scout reported just 954 people living in the tract. It was unclear where the number came from, Shauk reported. He reported that Schiller didn’t respond to repeated requests for an interview.
Michael A. Smith, of The Daily News in Galveston, was baffled, too.
“That's the 21st most dangerous neighborhood in the whole United States?” he wrote. “That's not even the 21st most dangerous neighborhood in Galveston. To buy Neighborhood Scout's assessment, you have to buy the notion that the trendy, lofty little neighborhood where I live is more dangerous than any neighborhood in New York City, Los Angeles, Detroit, Pittsburgh, East St. Louis, Baltimore, Washington, D.C., Dallas or Houston.”
He quipped: “It's a wonder anybody survives ArtWalk.”
Jeremy Kohler is an investigative reporter with the St. Louis Post-Dispatch.
Photo by John Cannenberg via Flickr.