The City Council’s Committee on Governmental Operations held an oversight hearing on Wednesday to evaluate the structure and content of two annual city reports that measure everything from the wait times on 311 calls and the conditions of city parks to the number of unsheltered homeless people and traffic crashes in the city each year. These key performance indicators help the City Council and the public keep city agencies and offices accountable. Those agencies, in turn, rely on these reports to improve their policies and functioning.

The administration releases two detailed reports each year - the Preliminary Mayor’s Management Report (PMMR), two weeks after the January financial plan, and the Mayor’s Management Report (MMR), in September. The reports include evaluations of annual goals set by 41 city agencies and 3 non-mayoral agencies that report directly to the Mayor, and whether these agencies are succeeding or failing to meet their objectives.

Wednesday’s hearing, the third one held by the governmental operations committee on this issue, gave Council members an opportunity to delve into the process of how agency goals are included in the reports and to reiterate their request for adding more data to future reports. Council Member Ben Kallos, chair of the committee, highlighted some of the key concerns the Council has about a lack of clarity in certain agency goals, agency indicators with no set targets, and the need to directly link agency performance with corresponding budget allocations.

The PMMR and MMR are produced over six to eight weeks by the Mayor’s Office of Operations (MOO). About 10 MOO staff members work with more than 150 senior staff across the agencies, including deputy mayors and agency commissioners, and coordinate efforts with City Hall to release the final products, which detail agency goals and indicators of whether those goals are being met year after year.

There are 1649 metrics by which agency performance is tracked in the two reports. Examples include quality-of-life summonses issued by the New York Police Department, the number of people on the city’s cash assistance rolls, the average daily attendance in city schools, and the number of families entering the city’s homeless shelter system.

The report tracked the NYPD’s goal to reduce the number of quality-of-life violations, a critical indicator without a set target. The four-month numbers (the timeframe for the PMMR) fell from 142,434 in Fiscal 2015 to 125, 685 in Fiscal 2016.

Under the Department of Education’s performance, the report looked at average daily attendance in city schools, which surpassed the 91.7 percent target in Fiscal 2016 (93.2 percent) and Fiscal 2015 (93.5 percent). Similarly, the report showed that the Department of Homeless Services had been successful in reducing the number of adult families entering the shelter system from 521 families to 466.

The data, which is available on an interactive website, includes comparisons with previous years, five-year trends and a look at desired directions for performance. This year’s PMMR data was also available on the city’s Open Data Portal, providing public access to all the information collected under the report. Kallos, an ardent supporter of transparent and accessible information and technology, praised MOO for this effort.

“We really take care not to take a one-size-fits-all approach to targets and indicators in the PMMR and MMR,” said Mindy Tarlow, MOO director, at one point during Wednesday’s hearing. “We try to see each indicator and each agency as a complex diverse case that’s in some ways reflective of the complex diverse city that we’re monitoring and reporting on.”

City Council members took turns questioning Tarlow and her colleague Tina Chiu, deputy director for performance management. The members usually stuck with issues germane to issue committees they chair or sit on, but the larger concerns voiced at the hearing were about the lack of targets for many critical performance indicators and the ways in which targets are set and the reports are improved.

Council Member Ben Kallos, who chairs the government operations committee, commended MOO for changes they had made to the PMMR based on recommendations from a December 2015 hearing about how targets were defined in the reports. This year’s PMMR included a more nuanced description of targets, which can be numeric or directional. Numeric targets set expected performance levels including a maximum that should not be exceeded or a minimum level that should be met. Directional targets indicate whether an indicator should go up or down. Many indicators had no target at all, which gave cause for concern to Kallos and Dick Dadey, executive director of government reform group Citizens Union, who testified at the hearing.

When Kallos asked why certain indicators had no targets, citing the number of lawsuits brought against the city as an example, Tarlow clarified that many of these indicators are “neutral” statistics out of their control, such as certain population numbers, for which it’s impossible to set directional targets.

“Although the committee is pleased that some improvements have been made,” Kallos said in his opening statement, “our review of the most recent PMMR suggests that further changes could make both of these publications more helpful tools for the Council, the public, as well as  the agencies.”

The City Council, in its Monday response to the mayor’s preliminary budget for fiscal year 2017, identified 57 indicators across 18 agencies that they want included in the MMR going forward. Kallos brought this up at the hearing, specifically pointing to indicators for the Board of Standards and Appeals, which falls under the Department of Citywide Administrative Services, which is under Kallos’ committee’s oversight. He also mentioned a City Council data study that found 107 indicators (70 critical ones) where there was a 28 percent discrepancy over last year’s PMMR and average performance over the prior three years.

Kallos (pictured*) also questioned Tarlow and Chiu about indicators relating to the Department of Health and Mental Hygiene, the 311 helpline system and the Department of Homeless Services, but his main concerns were about holding people accountable for changes to the PMMR and MMR, further clarifying how targets are defined, minimizing the number of indicators without targets, and linking performance indicators in the PMMR to the corresponding allocations in the preliminary budget, as the City Charter mandates for the PMMR.

“Who has ultimate responsibility for the changes that are being implemented?” Kallos asked Tarlow, frustrated that Council members’ requests for additional indicators in the past had been routinely agreed to by agencies but not followed through on. “Is it with the agency or Operations, and how do those changes actually end up happening?

Tarlow stressed that the creation and addition of indicators was a collaborative process with agencies. “We don’t dictate play to the agencies and the agencies don’t just unilaterally make changes,” she said, agreeing to consider improvements in communication with Council members. She also said that they had started to work with the Mayor’s Office of Management and Budget to cross-reference and link the PMMR with the preliminary budget.

Kallos’ concerns were echoed by Dadey, who reiterated recommendations that Citizens Union had made in the past, which shows, he said, “a challenge that we face that we’ve been back here several times making the same recommendations and while some of them get adopted we think that some of the more common sense ones are not being embraced and we’re at a loss to understand why.”

Dadey floated three specific proposals: setting targets for half the indicators in the report which had no metrics to judge them by (the number of indicators with no target increased from 948 to 961 this year from last); providing more budget detail alongside performance indicators; and expanded reporting on cross-agency initiatives to track transparency (for instance Freedom of Information Law requests) and voting registration programs at agencies.

He called the lack of targets for half the proposals “disconcerting” as “it indicates one of two troubling possibilities. Either the agency experienced difficulty in setting goals in coordination with the office of the mayor or that these goals have been established and are possibly being concealed from the public. Neither is satisfactory,” he said.

To the administration’s credit, MOO has made effort to add sections on multi-agency initiatives, such as IDNYC, which was eventually folded into the report on the Human Resources Administration. “That’s what we want,” Tarlow said in response to questions from Council Member Carlos Menchaca. “To really feature a multi-agency new initiative and then have it be so routine that it becomes baked into the fabric of what we do on a day-to-day basis.”

Menchaca wanted to know how the PMMR and MMR reports evolve and about efforts to make the report machine readable. His larger concern, as chair of the Council’s immigration committee, was that the data be accessible to New York’s immigrant communities. “What do you do to get the report out in other languages?,” he asked.

Tarlow’s answer was unimpressive. “I actually don’t know,” she said.

Lisa Neary, general counsel for the Independent Budget Office, also testified, although less about the content of the reports than the process. IBO had testified at the December 2015 hearing as well, recommending legislation that would require citizen surveys to be included in the reports. The idea has been championed for a long time by Baruch Professor Doug Muzzio and a relevant bill has been introduced by Council Member Corey Johnson, though it has not received a vote.

On Wednesday, Neary focused on the timing of the two reports. She said the PMMR being released early in the year was inadequate since it only accounts for the first four months of the financial year (July to October).

“With only this partial picture in hand,” Neary said, “the Council lacks crucial information that would allow you to link objectives to resources, and resources to outcomes.” She cited the current PMMR’s data on Department of Education programs, where at least a dozen critical indicators were listed as “not available.” She said the MMR’s release was “even more poorly timed,” and should be some time between January and June to have maximum influence on budget decisions.

When Kallos asked her for suggestions, she said perhaps the MMR should be released with the executive budget in May and then a full accounting of the year should come out in October or November. “The key thing is to have as much information as you can at the time that you’re making resource allocation decisions,” she said.

Council Member Mark Levine, chair of the parks committee, questioned Tarlow and Chiu about PMMR indicators on park maintenance and capital park projects, as well as on the lack of information on eviction prevention and legal services. Council Member Vincent Gentile, in keeping with his position as chair of the oversight and investigations committee, asked MOO to consider tracking the performance of individual Inspectors General under the Department of Investigation. Tarlow had ready answers for both Council members and promised to discuss their concerns with individual agencies.

*Photo of Ben Kallos by William Alatriste

Note: Gotham Gazette is an independent publication of Citizens Union Foundation, sister organization to Citizens Union.