The Center for Public Integrity published a series of reports from 2012 to 2015 that assessed “accountability and transparency” across the 50 states. These reports used more than 200 indicators to evaluate public records laws, lobbying disclosure requirements, political spending and more. The results produced an “integrity rating” and measured a state’s “risk of corruption.”

These rankings received, and continue to receive, wide media attention. They are cited in thousands of articles in most mainstream outlets. Often, these rankings are misconstrued or misinterpreted. The Detroit Free Press declared that the rankings proved Michigan to have the worst “laws on ethics [and] transparency.” Fox Business, meanwhile, indicated that these rankings exposed “America’s Most Corrupt States.”

These headlines and interpretations are common, but they are gross exaggerations. This is partly understandable because the reports give this impression despite relying on inappropriate methods that do not actually measure corruption or integrity at all.

Here are the 13 areas the Center for Public Integrity analyzed to come up with an integrity rating:

• Public access to information

• Political finance

• Electoral oversight

• Executive accountability

• Legislative accountability

• Judicial accountability

• State budget processes

• State civil service management

• Procurement

• Internal auditing

• Lobbying disclosure

• Ethics enforcement agencies

• State pension fund management

For several indicators, there were no objective measures that could be compared among the states. In others, the assumptions built in are dubious.

For example, whether a state allows people to contribute $3,400 to a political candidate or caps contributions at $6,800 is not a meaningful measure of corruption. But the reports’ rankings assumed that higher caps on political donations automatically meant less integrity and more corruption. The ratings also assumed that states with certain types of redistricting commissions had more integrity than those with different redistricting procedures.

The Center for Public Integrity sent questions out to a list of reporters and researchers to gather information. These journalists and academics graded their respective states using a “blend of social science and journalism” in the areas listed above. It is unclear how many people were involved (it doesn’t look like very many) or if those surveyed used objective measures to rank a state or just their opinion. The opinions of just a handful of people is too small of a sample to develop a robust comparison of states.

And in some cases, the opinions of the people involved are contradicted by the most relevant data. For example, “state pension fund management” is a category where there are plenty of objective measures to compare states. New Jersey and Illinois have the most underfunded pension systems in the country, setting aside only 36% and 39% of what public employees have earned, respectively. I could not find any reports, from any source, that rate those two states highly for pension management — think tanks, universities, auditors and credit agencies rate them poorly. And yet, the Center for Public Integrity gives New Jersey an “A” and Illinois a “B+” in the category, ranking them first and fourth among the 50 states.

Anyone familiar with New Jersey politics would be shocked to learn that these reports name the Garden State as the nation’s top state for integrity. Or take Illinois again, with four out of its last 10 governors serving prison time, which the report also ranks among the least corrupt states. It’s possible those states have made major changes to their laws that will boost integrity and reduce corruption — but it hasn’t shown up in the data yet.

The Cato Institute produced its own report to analyze public corruption. It uses a simple measurement: Department of Justice criminal convictions of public officials. The main limitation, the authors note, is that some federal prosecutors may not care as much about prosecuting corruption as others. But at least this is an objective measure, and prosecutors have lots of incentive to pursue criminal charges against public officials.

Adjusted for population, this measurement finds that Washington, D.C. has the most public corruption in the United States. Following the nation's capital are several U.S. territories. Among the 50 states, the eastern district in Louisiana is the highest followed by Montana and Oklahoma. Illinois ranked highly for corrupt public officials, while New Jersey is in the top third of states. The least corrupt states are North Carolina, New Hampshire and Utah, according to Cato’s report.

There is no correlation between the scores on the Center for Public Integrity’s report and the number of criminal convictions of public officials. Michigan, Wyoming and Delaware are the lowest-rated states on the public integrity index, but all have a lower rate of public officials being convicted of crimes than the national average. Alaska, California and Connecticut are the highest-rated states on the public integrity index, but Alaska and Southern California have above average conviction, rates while Northern California and Connecticut fall below the national average.

Pressing the government to be more transparent and less corrupt is always a worthy endeavor. But it’s difficult to measure and compare integrity among states. And when measuring state corruption, researchers and journalists should use the most objective data possible, not simply opinions, perceptions and other weak measures of integrity.

Permission to reprint this blog post in whole or in part is hereby granted, provided that the author (or authors) and the Mackinac Center for Public Policy are properly cited.