Chirlane McCray (photo: Michael Appleton/Mayor's Office)

Following repeated criticism about spending hundreds of millions of dollars with unclear outcomes, ThriveNYC, First Lady Chirlane McCray’s signature mental health initiative, is now publishing nearly 100 metrics to track the performance of dozens of programs that have been launched in the last three years.

The metrics, provided to Gotham Gazette just before their public launch, will show outcomes of programs, not just their reach or the city’s investment into them as has been the case since the inception of ThriveNYC.

Over the last few months, ThriveNYC partnered with the CUNY Institute for State and Local Governance and various city agencies to create 94 performance indicators across the board, and will begin to publish that data on its website starting this fall, with varying timelines depending on individual initiatives.

ThriveNYC will publicly post a detailed description of each initiative, the agency handling the program, and the indicators associated with it. For instance, for NYC Well, the mental health phone helpline run by the Department of Health and Mental Hygiene, ThriveNYC will measure the percentage of callers who say they are satisfied by its services. To measure the Mental Health First Aid training program, the office will look at the percentage of people that say they shared or used the training they received. 

“Now that the programs are implemented and even the ones that are about to launch, we’re putting up the metrics that we’re going to be looking at to see what kind of effect they’re having,” said Susan Herman, ThriveNYC’s director since February, in a phone interview on Thursday. 

Inaugurated in 2015, ThriveNYC was meant as a paradigm shift in how the city handles mental health issues, seeking to tear down stigma around getting mental health care, targeting services at New Yorkers who cannot afford the cost of care or have been long underserved, and ensuring that people of all backgrounds are more attuned to their own mental health needs, as well as those of their friends, family, and colleagues.

With $850 million pledged over four years, the overarching initiative, called a mental health “roadmap,” sought to bring together a range of services, with more than 50 programs (including some that were already running) across a dozen-odd city agencies. McCray has been the face of the program, touting its gospel across the city and in appearances, both with and without Mayor Bill de Blasio, across the country.

But while many have praised the program’s goals, the accountability involved has been vague. In April 2018, McCray said in an interview on the Max & Murphy podcast from Gotham Gazette and City Limits that Thrive would begin a “resetting” phase, including by bringing on its first executive director and tracking more metrics for the various programs. Gotham Gazette reported soon after that many of the ThriveNYC goals were difficult to track and that the budget was opaque.

By February of this year, a second reset began. ThriveNYC became a standalone office housed under the mayor’s office and Susan Herman, a former NYPD deputy commissioner, was named its director and senior advisor to the mayor, with the task of better organizing the effort and ensuring that it had the right structures in place. But several news reports, first in Politico New York and then The New York Times, further pointed out that ThriveNYC was struggling to track and show results despite having spent nearly $600 million.

In March, the New York City Council convened an oversight hearing where they tried to hold McCray and Herman’s feet to the fire, demanding to know if the city was getting the bang for its buck. 

Those answers are now forthcoming, the city says. 

Herman’s first task when she took over ThriveNYC, she said in the interview, was to clarify to the public the scope of ThriveNYC and the services provided under its umbrella. She denied that the metrics were created in response to the criticism the program faced, saying that they couldn’t be established earlier since there was no data to assess and that the various initiatives had to first be up and running.

“First of all we’re measuring the right things at the right time. This is when you start measuring outcomes, when you are fully implemented,” she said. “And I think most researchers would tell you, ‘Yeah this is just exactly when you should be doing exactly what you’re doing.’ And secondly, it’s very important to know that we are also engaged in formal evaluations, many of which are up and running and have been up and running, some of them for over a year, some of them two years, some of them just launched, some of them still to come but we have worked with outside researchers for years to do formal evaluations of Thrive programs and we’ll continue to do that.” 

Michael Jacobson, executive director of the CUNY Institute for State and Local Governance, echoed Herman in a separate interview. “In this kind of work there are generally two kinds of measures for programs and program success: there’s outputs and outcomes,” he said. “Outputs tend to be process-y, like how many meetings did you do, how many contacts, and those are really useful for management. In the end, whether you’re the government or general public, it’s the outcomes that matter. And they’re both more important and usually the trickier thing to measure.”

While McCray and de Blasio have talked extensively about the importance and impact of ThriveNYC, the measures have almost all been outputs, as Jacobson put it, with McCray admitting at the recent Council oversight hearing that limited systems were in place to even try, difficult as it may be, to track outcomes for individuals serviced by Thrive programs.

With the next fiscal year, FY 2020, beginning on Monday, ThriveNYC will also launch new programs, with plans to spend nearly $243 million overall through the course of the year. These programs are expected to have metrics attached, with reports issued quarterly, semi-annually, and annually, depending on the data gathered.

Some are already evident and showing impact. For instance, according to ThriveNYC, 48% of young children receiving mental health treatment for early childhood trauma at seven clinics run by the Department of Health and Mental Hygiene have shown signs of improvement. At 25 Department for the Aging senior centers with on-site mental health clinicians, 56% of seniors receiving treatment have shown improvement in depression symptoms and 65% have shown improvement in anxiety symptoms.

Even the host of new metrics are an “interim” phase, Herman said, which will eventually be followed by broader evaluations of programmatic effects on specific populations -- school students, for example -- and then a holistic review of ThriveNYC’s effect on the entire city. 

Herman couldn’t explain why the city had failed to measure the impact of many of the programs involved that had existed before ThriveNYC, though she said these had not been repackaged but instead enhanced under the de Blasio administration.

“Some of these programs, could some of them have had outcome measures earlier? Perhaps,” she said, “but this is a way of looking at Thrive in total...It’s an important distinction. Because really what people are asking is what’s the impact of Thrive? And at this point what we should be looking at is the impact of each of the initiatives. It’s only later still that we can look at the impact of Thrive on larger populations and eventually on New York as a whole.” 

Jacobson said, “You can start to think about this for any program but you can’t do it until you get some historical data...It couldn’t have been on day one.” 

Part of the failure to show qualitative and quantitative results, Herman said, is because of the nature of ThriveNYC. Akin to other long-term public health programs, it’s hard to gauge in the span of a mayoralty.

“It’s comparable frankly to universal pre-K where you can see the immediate result, you can see that kids are in school, families are happier but, like Headstart, you won’t know the impact on those children’s lives for many years to come,” she said. “And then you see it and it’s very clear. Just like smoking bans, just like trans fats bans, just like universal pre-k. You’re gonna have an impact that you won’t see for years.”