ALBANY, N.Y. — Just a decade ago, the idea of open government in Albany would have been unthinkable.

Here was a town that seemed to exist for the express purpose of hiding the facts about how the state functions and for keeping the public at bay while elected officials made deals in back rooms. Information was gold and state officials were the dragons hoarding it.

Now digital technologies are pushing Albany to transform the way it does business.

“We are in a midst of a revolution now,” said Bob Freeman of The Committee on Open Government, which advises the state and journalists on Freedom of Information laws. “It is almost an inevitability because of technology. For years, this process involved people putting in Freedom of Information requests and demanding information. Now the government is pushing out information. It is a sea change.”

John Kaehny of Reinvent Albany, a group focused on improving data access, said that a lot of states are creating open data portals but, “They are very superficial. A lot of their open government portals and follow-money websites are very weak,” he said of other state’s open data platforms.

Kaehny thinks that New York is positioned to be able to lead the open data movement but it isn’t quite there yet. “I think New York is incredibly good in some areas and middling in other areas. But the trend is very good,” he said.

And there appears to be any number of high-profile state elected officials embracing open government through digital technologies.

Gov. Andrew Cuomo unveiled a major open data initiative today at the start of Sunshine Week, coming on the heels of the announcement earlier this year of OpenBudget.ny.gov, a portal that allows citizens to look up the state's finances. Cuomo's new site, Open.ny.gov will be a clearing house for information from all state agencies. Cuomo issued an executive order directing all executive agencies to catalogue their data and make it available via open.ny.gov. 

Later this year, the state Comptroller’s office will roll out a system that allows New Yorkers to see every check issued by the government and to follow exactly who is doing business with the state down to the penny. The system, tentatively titled “Checkbook,” is the next step in New York’s slow but steady embrace of open government and open data. It is modeled after city Comptroller John Liu's Checkbook 2.0 site that was introduced earlier this year and allows users to track city spending in a similar fashion.

Even with these improvements and a growing cadre of open government fans, much of the old system remains. Data in some sectors is still hard to come by, hidden by absurd interfaces or simply incomplete. For instance, the State Assembly still fails to meet the most basic standards of open government: It does not broadcast its sessions on the Internet and does not post the results of votes.

Bill Mahoney of the New York Public Interest Research Group doesn't think attitudes have changed sufficiently.

“There is certainly a movement toward open data,” said Mahoney. “But it’s not really the default mode for government. We often don’t see agencies put information online unless they face external pressure.”

Mahoney points to the state's Joint Commission on Public Ethics as an example. To access lobbyist data, users must contend with an unwieldy search engine and, even then, it is at times difficult to get precise results. Mahoney and others have requested JCOPE simplly make a spreadsheet database available on their website of all the lobbyists doing business in the state.

“It would be extremely easy to publish that kind of database,” Mahoney said.

The Governor's Efforts On Open Government

The recent Sage Commission (a group of prominent New Yorkers brought together by Gov. Cuomo to “modernize and rightsize state government) report discusses the development of NY Performs a website that will share key performance indicators for all state agencies with the public. Data like wait times at office windows at the Department of Motor Vehicles would be made available along with perhaps the agencies spending rates. But the Sage Report notes that there will be different levels of information shared with different people.

“The most important information about an agency, including certain background facts, key performance indicators and main strategic initiatives should be presented to the public, while more detailed operational information would be available to internal users to help manage State operations,” the Sage Report reads.

And this is concerning to Kaehny because he thinks the public should have access to all of the data. “The Sage Commission lays out that NY Performs will have three tiered management data where the first tier will be visible to the public. This will be like a real time mayor’s management report. The more people use it the more we can advocate seeing more of the layers of data.”

The Sage Report also briefly mentions the Open New York initiative that Cuomo announced during his latest state of the state speech. But it is less precise in detailing exactly what it will be. “Under the Open New York initiative, data will be presented in a common, downloadable, easy-to-access format, and will be searchable and mappable,” reads the report.

The Cuomo administration has already unveiled two transparency-minded websites. The first is CitizenConnects, which has been described by the administration as a “town hall.” But the attraction for the press has been the inclusion of Cuomo’s schedules. The schedules are updated slowly and reporters have discovered a number of omissions.

The other site, OpenBudget, is flush with spending data dating back to 1954, and allows viewers to sort through the proposed spending plans and actual budgets dating back to 1993. Kaehny says it is a step in the right direction and puts New York ahead of a lot of other states when it comes to budgeting and open data. “I give the governor a lot of credit for OpenBudget," he said.

He said other states' efforts at using technology to bring transparency to their budgets were "really lame." But he said the Legislature could make its budget proposals searchable online in easily downloadable formats.

One major step towards improving New York’s data management occurred in April last year when the state implemented a sophisticated financial management system that provides users with an impressive array of analytical tools. The comptroller's office and, presumably, governor’s office are both looking at ways to make more of this data available to the public. The implementation of the system was no simple task; it cost millions of dollars to implement.

The Assembly Lags

If you are looking for the antithesis of open government in New York you need look no further than the Assembly. Its website is just about as barebones as any modern government website can get. Not all sessions are broadcast, committee meetings aren’t broadcast and vote results are not published online. In comparison, the state Senate website is a modern marvel — despite some slow response times and the fact that not all committee meetings are broadcast.

“If you look at the archived version of the Assembly site today and compare it to a cached site from a decade ago, it hasn’t changed,” Mahoney said. “They promised a few years ago to expand New York’s CSPAN to include committee meetings and more background legislators when the awful elevator music play and nothing has happened since then. It wouldn’t have taken much just to place those videos online if they had been filmed for television.”

The Senate site was the only one updated with open data features after Democrats broke the Republicans' decades long hold on the chamber and revamped the Senate’s websites. To their credit, the Senate Republicans continued to use the system when they returned to power two years later. But the Assembly’s site has remained static along with its leadership. Speaker Sheldon Silver has enjoyed a Democratic supermajority in the Assembly for years now, and nothing has prompted change. That isn’t for lack of trying.

Democratic Assemblyman Andrew Hevesi and Senator Carl Marcellino have sponsored legislation that would require session and committee votes be reported online and that all session and committee hearings be broadcast online.

“With the passage of this bill, legislators will be made accountable to the public not just for their floor votes and statements, but also their committee votes and discussions that take place therein, which is of the utmost importance in shaping and guiding the legislative agendas of each house,” Hevesi said in an email to the Gotham Gazette. “These are simple and common sense steps that Albany can take to continue its pursuit of providing transparency within the New York State government."

Marcellino advanced the bill out of committee in the Senate but it isn’t clear what the bill’s fate will be in the Assembly.

The Assembly’s bill search mechanism is also outdated compared to the Senate’s open data platform. What is so striking about the disparity to people who use these systems like Mahoney and Kaehny is that a system exists that is far better than both tools: However, it is only available to the paying public for an annual subscription fee of $2,000 and made available to legislators for free.

The Legislative Retrieval Service reports bills faster than the other services. So, for example, subscribers were alerted about the introduction of Cuomo’s gun package before those who relied on the Assembly or Senate site.

“The LRS is similar to the Senate service but, in reality, it is better than either of the services. The fact that you know about bills being introduced before everyone else is a selling point for lobbyists,” said Mahoney, whose group subscribes to the service. The LRS also allows users to look back further in time at previous bills to see how they were voted on. The service is run by the Legislative Bill Drafting Commission, and funds received from the subscriptions go to the Legislature.

Open Data is Business-Friendly

Margaret Becker, deputy comptroller for contracts and state expenditures, says that while creating the comptroller’s current open government portal she was unsure exactly who would use the information provided by the site. The comptroller’s office held public meetings, consulted journalists and businesses about what would be useful but in the end there was no predicting who would get the most benefit.

As it turns out, businesses in the state are using a lot of the data. “From our inception in 2008, we’ve had 2.7 million hits and actually contract searches have the highest number of hits," she said. "You can search contract number, agency contract type, and time period.”

That kind of data not only keeps the state accountable but allows businesses to research what they could do to win bids, by seeing who has won them and how much they are paid.

“It is much easier for vendors to do business with state agencies because they can use the system to research what they can expect," she said.

Kaehny notes that real estate firms use lots of state provided data from the Department of Environmental Conservation, including details about permitting.

“Putting more data online is a way to make New York open for business as people claim it already is,” Mahoney said. “Businesses can use state provided data to help figure out the hurdles they face and how other companies are doing it. It makes life easier for them.” Instead of paying lawyers and consultants, a start-up business could access state databases and analyze what they would need to do to win a contract, or what permits they would need to build.

Freeman agrees. “My belief is creative people will make use of the data in new ways as soon as it becomes available," she said. "It is hard to predict who will use it and how it will be used. But I’m gonna guess the private sector and business community will make use of it in some way that boosts the economy.”

Freeman points to the Weather Channel as an example. “They use government open data on weather and they have turned it into a million dollar industry," she said.

The Next Step

Experts say that when it comes to open data, Albany needs to aspire to be more like New York City, where a now-year-old law calls for about 40 agencies to post any previously publicly available data to a portal. There are now hundreds of data sets on the site, including on everything from 311 complaints to restaurant inspections.

There is nothing like that on the state level.

Instead, agencies and branches of government have spearheaded their own efforts, with little coordination. For instance, Attorney General Eric Schneiderman has a site that tracks state contracts, campaign contributions and lobbying information. The State Board of Election has similar campaign data, while JCOPE has lobbying data. And, of course, their are the efforts coming out of the governor's office and the office of the state comptroller.

Advocates would like to see one NY State open data page that all agencies and segments of government report to automatically without prompting. “In Albany, the tendency has been for everyone to want transparency for everyone but themselves,” Kaehny said.

Mahoney said a portal is the next step. “The state should put all its data online in one spot," he said. "It seems to be the direction a lot of cities and states are taking. Gov. Cuomo is looking to make New York an example for other states and this would be one area where it would be cheap and easy to be the leader.”C

With the unveiling of open.gov.ny Cuomo's office hopes to do exactly that--put all of New York's open data in one place.