While New Yorkers applaud the police for the drop in crime and are generally satisfied with police performance, many city residents firmly believe the police use race as a factor in deciding who to stop on the streets. In a survey taken in 2000, 62 percent of respondents said they believed the police racially profiled. This included 79 percent of blacks, 66 percent of Hispanics and 51 percent of whites.

But a recent study challenged such deeply held notions, largely exonerating the New York City Police Department of explicit racial bias in who officers stop and sometimes frisk. Despite its nuances and subtleties, the study – which received widespread press coverage, including a front page article in the New York Times –provides the police department with powerful statistical ammunition to back its claims that the high proportion of racial minorities stopped by the police in 2006 (89 percent of all stops) reflects the differing crime rates among races, not overt bias.

Civil libertarians and other law enforcement critics struck back, taking aim at the report's methodologies and conclusions. One critic called the report "full of disinformation" and "comprised of endless excuses [and] statistical justifications," according to the New York Daily News. Partisans on the right jumped into the debate as well, with conservative pundit Michele Malkin (among others) accusing the New York Times of playing down the study's pro-police conclusions.

Underlying the partisan debate are some legitimate questions about the statistical techniques used by the study's authors, as well as the police department's refusal to release key data to the public. Perhaps equally important, media accounts have tended to play down the modest concerns raised in the report about police tactics and its recommendations for change. Finally, the study was not designed to answer a central question that all New Yorkers must grapple with: How can we balance the kind of aggressive policing that promotes public safety with the need to maintain trust and confidence in the police, particularly among racial and ethnic minorities?

The Study's Conclusions

The raw statistics released by the police department in February 2007 were stark: Over 500,000 pedestrians were stopped by the police in 2006, an almost five-fold increase in the number of stops recorded five years previously. Equally troubling was the racial breakdown of the stops themselves. Non-whites represented 89 percent of individuals stopped, with blacks, who comprise 24 percent of the city population, accounting for 55 percent of stops. In addition, 45 percent of black and Hispanic people stopped were frisked, compared with 29 percent of white suspects -- even though the police were 70 percent more likely to recover guns from white suspects.

The police insisted that the racial breakdown of the stops was justified, pointing to statistics showing that blacks represented 68 percent of suspects identified by victims and witnesses. In the department's view, the racial breakdown of stops was driven by patterns of crime, not racial bias. The stops might also reflect the allocation of police – more police officers are deployed in high-crime neighborhoods that tend to have high proportions of non-whites.

Faced with widespread public criticism and threats of a class action lawsuit by the Rev. Al Sharpton, Police Commissioner Ray Kelly turned to a nationally respected think tank, the Rand Corp., to examine the stop and frisk reports – known as "UF250s" – filled out by police in 2006. The UF250s record detailed information about each stop, such as its location, the time of day it took place and the reason for the stop, as well as post-stop outcomes, such as whether the stop resulted in a search, the discovery of a weapon or an arrest. The challenge for Rand in looking at the data would be to separate stops driven by legitimate public safety concerns from stops driven by illegitimate racial bias.

Examining the Data

To analyze the stop data, the Rand researchers used a controversial, though generally accepted, statistical technique knows as benchmarking. To do this, they first analyzed the data to predict how the police would behave if they were not acting in a racially discriminatory model. Then, they compared what actually happened with the benchmarks to determine if there was a pattern of racial bias. Using this method, the researchers found that blacks were stopped in rates comparable to their share of arrests and 20 to 30 percent less often than descriptions of crime suspects by race would suggest. Interestingly, the report found that Hispanics were "overstopped" by between 5 and 10 percent relative to their share of arrestees and crime suspects.

Benchmarking, while commonly used, has limitations. "All benchmarks have problems," said the lead author of the Rand report, Greg Ridgeway. "They're really just ways at poking at the data."

For example, according to Ridgeway, relying on arrest data as a benchmark does not necessarily eliminate racial bias. If the police are biased in their arrest decisions then the benchmark will be fundamentally flawed. To avoid this, Ridgeway prefers to use the reports made by victims and witnesses of alleged crimes because those come from average citizens and not police officers. But that approach has a shortcoming too: Only 20 percent of stops are made in response to report made by a crime victim or witness.

In preparing his report, Ridgeway also examined stop outcomes – such as a search, an arrest or issuance of a summons, the use of force or the recovery of a weapon or drugs – to see if he could detect any pattern of racial discrimination there. As with the stop analysis, benchmarking would be critical. The technique he used was relatively sophisticated. Whites, on one hand, and blacks and Hispanics on the other tend to be stopped for different reasons, including at different times of day, in different precincts and for different types of crime. To correct for that, Ridgeway compared the post-stop outcomes of "similarly situated" stops for white citizens and for black and Hispanic ones. The goal was to compare apples to apples: Stop outcomes would be matched according to time of day, location, type of suspected crime and a host of other factors.

Here Ridgeway found small but significant differences. According to the analysis, non-whites were frisked in 33 percent of stops, compared to 29 percent of similarly situated whites. Police were somewhat more likely to use force on black pedestrians than white pedestrians, but white suspects were slightly more likely to be issued a summons than similarly situated non-whites (5.7 percent versus 5.2 percent), while arrest rates for white suspects were slightly lower than for non-whites (4.8 percent versus 5.1 percent).

While these differences are small in percentage terms, they translate to large numbers of people given the high volume of stops. A two percent difference in frisk rates for black pedestrians relative to whites translates to 5,350 "extra" frisks for blacks per year. The same is true for use of force. The report estimates that there were 2,000 more use of force incidents with black pedestrians than non-blacks. These differences in frisk and use of force rates were particularly pronounced in Staten Island. For example, police in Staten Island used force in 13.5 percent of stops of black suspects, but only with 10.1 percent of similarly situated white suspects.

To Ridgeway, many of the differences in post-stop outcomes, with the exception of those reported in Staten Island, while worrisome, do not indicate a system that is "out of whack and needs restructuring." "These types of differences, with the exception of Staten Island, don't call for a complete overhaul of procedures," he concluded.

Ridgeway's analysis of arrest data, broken down by precinct are, almost identical to the method used in a 1999 study performed by then Attorney General Eliot Spitzer. The Spitzer study, which found that blacks were stopped 23 percent more often that whites and Hispanics were stopped 39 percent more often than whites.

There were some important differences between the Rand and Spitzer reports, however. In the Spitzer study, researchers examined a sample of 5,000 UF250 forms that included a space for the officer to describe the reasons for the stop. Using the standard set by the United States Supreme Court for permissible stops, researchers determined that 61 percent of stops met the legal standard and15 percent did not. In the remaining 24 percent, they did not have enough information to make a determination. Using that information, they found that 1 in 7 legally justified stops led to an arrest, compared to only 1 in 29 stops where the legal standard was not met. Although the rate of unreasonable stops was roughly equal for whites and non-whites, this was a provocative finding, suggesting that the police would be more "efficient" if they only stopped pedestrians for a narrow set of reasons. Rand was unable to repeat this analysis because the newer UF250 eliminated the place to explain the stops.

The Press Reaction

The Rand report received widespread coverage in local newspapers. Despite the subtleties of the study, the headlines were unambiguous for the most part: "'Profiling' Study Gives NY All Clear" read the New York Post. Other newspaper left out the nuances as well. The Daily News: "Review says half-million NYPD stop-and-frisk cases not racial profiling." Newsday: "Study Downplays Bias in NYPD Stop-Frisk policy." Only the New York Times headline was more measured: "City Police Stop Whites Equally but Frisk Them Less, a Study Finds."

Civil libertarians were quick to dispute the study's findings. Both the Center for Constitutional Rights and the New York Civil Liberties Union criticized the report's methodology, although neither organization suggested an alternative other than relying on census data.

Beyond the report, critics fault the police for their apparent unwillingness to share the UF250 data more generally with the research community. Current city law requires the department to report stop-and-frisk data to the City Council on a quarterly basis. Citing technical difficulties, the police did not report data from 2004 and 2005. More galling to researchers and advocates is that the council had access to only a limited set of the 2006 data, preventing researchers from independently confirming the Rand analysis. "It would be wonderful if the data were a public resource," said Jeffery Fagan, a researcher at Columbia University who worked on the Spitzer report.

Recommendations for Change

For his part, Ridgeway expressed surprise at the media reaction. "I thought it tended to say that the report was giving the NYPD a clean bill of health, and I don't think that's what the report was saying at all" he said.

Ridgeway has had to fight his own battles with the media. An op-ed he prepared for the Daily News with colleague K. Jack Riley was initially given the headline "Police Stop Black New Yorkers More – and with Good Reason" before it was changed to "Police Need to Do a Better Job of Explaining Stop-and-Frisk."

Meanwhile the recommendations made in the report about improved police stop and frisk procedures seem to have gotten lost. Although many of Ridgeway's recommendations were procedural in nature, other recommendations, if implemented, would have a big impact on citizen interactions with the police. In addition to recommending a closer look at police operations in Staten Island, the report calls for the police to be required to clearly explain the reasons behind their action to anyone they stop. Ridgeway argues that police officers should also give each stopped pedestrian an information card with contact information in case they want to file a complaint. The recommendation is based on an emerging body of social science research that suggests that people care more about whether an interaction with law enforcement is fair than the result of the interaction itself. "Just giving information can help defuse things," said Ridgeway.

The report also discusses how to deal with individual officers with higher-than-normal stop and frisk rates. Ridgeway devoted an entire section of his report to a process called "internal benchmarking," which compares the stop rates and post-stop outcomes of individual officers against similarly situated officers. As with the other benchmarks, the goal here is to predict how a police officer, who is not racially biased, would act and compare that to the actual performance of individual officers. Using this technique, Ridgeway identified five officers who stopped more black suspects than expected and ten officers who stopped more Hispanic suspects. (The report also identified nine officers who stopped substantially fewer black suspects than expected and four officers who stopped substantially fewer Hispanic suspects.) In Cincinnati, where Rand pioneered the internal benchmarking technique, supervisors meet with identified officers to determine if there are legitimate reasons for the disparities.

Ridgeway says he verbally presented these recommendations to Kelly two weeks before the written report was submitted. Although it's not clear if the recommendations will be implemented, Kelly told the New York Times that he plans to "put in place what we think is appropriate as quickly as possible."

Critics of the NYPD generally have been cool to internal benchmarking and said little about the other recommendations in the report, largely because they believe its focus is misplaced. Stop and frisk policy "is set at the precinct level, not the individual police officer level," said Columbia law professor Jeffrey Fagan. The fear is that stop and frisk will be reduced to a problem of a "few bad apples, rather than a system-wide problem," said Kamau Franklin, a racial justice fellow with the Center for Constitutional Rights.

Balancing Public Safety and Public Trust

The report was not designed to address the effectiveness of stop and frisk as a policing strategy. Even the department's critics generally agree that aggressive stop and frisk strategies used to take illegal guns off the streets in the 1990s played a role in the city's historic crime drops. However, they wonder if the five-fold increase in recorded stops over the last five years – which could be partially attributable to improved recordkeeping – is justified in a city with relatively stable crime rates. "It's an astonishingly high number of interactions," said civil rights attorney Andrew Celli, who oversaw the 1999 Spitzer report as the chief of the Civil Rights Bureau in the attorney general's office. "How effective and how useful at combating crime is this kind of police activity?"

That's difficult to determine. Even the widely cited statistic that only 10 percent of stops result in a summons or arrest is open to interpretation, according to Ridgeway. He described riding along with police officers and a victim who had been assaulted by three individuals. The police stopped three groups of men before they found the assailants. Even though six of the nine individuals stopped were innocent, it's hard to criticize the police response, Ridgeway said.

One seemingly intractable problem involves the gulf between those who have experienced - or believe they have experienced -- a racially biased and intrusive police search and those who have not. No matter what the official rationale for a race-based stop, it is a traumatic experience. It causes "fourfold harm" in the words of William Stuntz, a legal theorist, because it raises the threat of physical violence, stigmatizes the individual by treating them like a criminal, violates their privacy and singles them out on the basis of their skin color. It is difficult to imagine that any statistical analysis, no matter how sophisticated, can convince a person who experienced an intrusive search that the police were acting in good faith. Scholar Michael Tonry has put it another way: Regardless of whether the criminal justice system is acting in a biased way (Tonry believes it is not), black men in particular have borne the brunt of police suspicion on the basis of race.

It seems clear, then, that the Rand report will not quiet the debate about the effectiveness of aggressive police strategies. Dueling stories from late November suggest the ongoing dynamic: The police release statistics showing that murders have been reduced to a 40-year low; at the same time, the number of suits filed against the police and grievances submitted to the Civilian Compliant Review Board, an independent police watchdog agency, continue to rise.

Aubrey Fox is project director of Bronx Community Solutions, aimed at changing the Bronx court system's approach to low-level crime.