Former Mayor Edward I. Koch was famous for asking, "How'm I doing?" He usually received an earful.
In the last two decades, governmental agencies in New York have gotten an earful too. Often, this is the result of reports generated internally, but, in the case of the subways, it is helped along by report of the New York Public Interest Research Group Straphangers Campaign on the State of the Subways. Straphangers' annual subway service report culls data from New York City Transit to evaluate and rank each of the subway system's lines. Scores are based on the frequency and regularity of service, rate of breakdowns, chance of getting a seat, cleanliness inside the cars and in-car announcements.
This year, the L train gained the #1 ranking, based in part on the arrival of new subway cars that reduced the frequency of breakdowns. Other high-ranked lines were the #3, D, E, #6 and #1/9. Ranked last was the #5 line, bedeviled by a high breakdown rate, irregular train arrivals and severe crowding. Other low scorers were the N, C, M and R.
The report also highlighted how service has improved in some respects and worsened in others since release of the 2000 report. (No report was done in the aftermath of the September 2001 terrorist attacks.) The good news is that subway cars are breaking down far less often than they did in 2000. In the second half of 2002, subway cars traveled an average of 151,300 miles without a mechanical problem causing a delay, compared with 110,600 in the second half of 2000. This improvement largely reflects the arrival of new high-tech subway cars.
The bad news is that subway cars were less clean in 2002 than 2000. Over this time period, the proportion of subway cars with clean seats and floors declined from 85 percent to 78 percent.
One of the strengths of the Straphangers report is the detail by subway line. The line-specific data show how varied service is across the system. Take subway car breakdowns, for example. While breakdowns lessened on 10 lines, the situation worsened on five lines (1/9, 4, 7, A and N). Conversely, while cleanliness worsened on 12 lines, it improved on three lines (2, 3 and A).
Looking across subway lines, breakdowns occur twice as often on the #5, #7, A, R and V as compared with the #3, D, L and Q.
Straphangers is not the only organization to make performance indicators readily available for public viewing. The Metropolitan Transportation Authority recently began to put system-wide performance statistics on its website. Currently showing data through May 2003, the MTA data are more up-to-date compared with the Straphangers report for the latter part of 2002. But the MTA does not break out subway lines or other geographic detail.
City agencies also make performance indicators available on the Internet. The Transportation Department, however, focuses on work measures such as traffic lights installed and permits issued, rather than performance measures. One performance measure concerning Staten Island ferry on-time performance is included.
What comes of the considerable effort put forth to measure agency performance? In some cases, such as the Department of Transportation, what is actually measured is not actually service performance and thus does not fulfill the basic objectives of these efforts.
In other cases, performance is measured but little heed is paid, or action taken, based on the results.
But when agency officials and the public pay attention and take these measures seriously, real improvements felt by everyday citizens can occur. Straphangers has found that complaints and advocacy based on their past reports improved service on certain subway lines. Within government agencies, managers' feet can be held to the fire to produce better results.
More broadly, these reports have two very positive effects. First, they focus attention on the right set of issues. The old saw at the Transit Authority went that the trains would run just fine if there were no passengers. On-time performance was once measured solely by when trains arrived at the end of the line, and dispatchers have been caught faking the numbers. New indicators such as the regularity of train arrivals reflect how passengers actually experience subway service. In government, what isn't measured generally didn't happen, at least officially, so having official and systematic statistics that reflect riders' experiences is very valuable.
Second, these reports underscore that budgetary decisions and choices about priorities have real consequences. The subway car breakdown rate on the L line improved because of millions of dollars spent to purchase them. As the city undergoes one round of budget cutting after another, it's worth watching how services are affected.
Like any good thing, however, performance measurement can have unintended consequences. High-stakes testing in schools has resulted in recently publicized scandals over underreported dropout rates and serious doubt about the appropriateness of Regents exams. Let's hope that performance measures in transit and transportation get the attention they deserve, but without these types of abuses.
Bruce Schaller is Principal of Schaller Consulting, which provides research and analysis to government, business and non-profit groups seeking to identify and meet customer needs in the transportation sector. He is also a Visiting Scholar at the Center for Transportation Policy and Management at New York University.
