Part I: Michigan Education’s Return on Investment

When evaluating whether Michigan’s public education system is producing acceptable outcomes, it is important to look at both levels of achievement and levels of spending. If Michigan were spending more than other states but achieving at a higher rate, it might be reasonable to argue that the education system is relatively efficient. One could claim that the state spends more and gets more. Another way to evaluate spending and achievement is to look at trends over time. If the state started at lower levels of student achievement and education spending than other states, but student achievement grew at a faster rate, one could reasonably conclude that the education system was receiving a reasonable return on its investment.

Unfortunately, spending has increased, but student achievement has either remained level or lost ground in national comparisons. Such comparisons are important because single or absolute measures of a state’s performance can be misleading. If we were to see that Michigan improved five points on a math exam, for example, we might think there was cause for celebration. However, if the national average gain were nine points, we would rethink that assessment. Alternatively, if Michigan were to lose two points on the reading assessment, we might think that there was cause for concern. However, if the national average dropped eight points, Michigan might not be doing poorly after all.

Additionally, rather than looking exclusively at proficiency levels on a state’s own exams, comparisons of student achievement across states should also be made using the National Assessment of Educational Progress exams. Although the federal No Child Left Behind Act mandates that every state test students annually in grades three through eight in math and language arts, states are not required to use any specific test.[*] As a result, states can adopt NCLB exams that are more challenging or less challenging based on their own individual state policies. Moreover, states can set proficiency cut-points at varying levels. These variations make comparisons across states based on state exams particularly difficult.

By contrast, the NAEP exams are nationally administered and uniform for all states. The National Center for Education Statistics in Washington, D.C., draws a representative sample of students in each state and tests them in grades four and eight in math and reading. NCES also periodically administers science and writing tests as a part of the NAEP battery.[1]

NCES processes NAEP test score data and reports student achievement levels on a standardized scale. Graphic 1 below shows a comparison between Michigan students and students nationwide on the fourth-grade NAEP math exam.[2] The figure shows that Michigan students were achieving above the national average in 1996. In subsequent years, average scores improved nationwide, including in Michigan. Unfortunately, Michigan students’ scores fell relative to the rest of the nation during the last few years and dropped below the national average in 2007.

Graphic 1 — click to enlarge

Graphic 2 shows the situation in eighth grade. At this grade level, the performance of Michigan students has remained relatively stable over time. In comparison, however, students nationwide, who performed at a lower level in 1996, have surpassed Michigan students.[†][3]

Graphic 2 — click to enlarge

This story is only marginally better for Michigan students in reading. Graphics 3 and 4 show that Michigan students, once slightly above the national average in fourth- and eighth-grade, are now at or below the national average.[§]

Graphic 3 — click to enlarge

Graphic 4 — click to enlarge

Some might argue that Michigan students’ performance, which is at or only slightly below the national average in math and reading, is acceptable.[**] Looking at achievement relative to expenditures, however, shifts the perspective.[‡] State education expenditures have increased dramatically over the last two decades. In school year 2004-2005, Michigan spent more than $19.3 billion[4] on primary and secondary public education. Using constant 2005 dollars, that amount reflected an increase of more than 50 percent since school year 1988-1989. Graphic 5 shows the long-term increase in the state’s real spending on public education in the years before and during the administration of the NAEP exams charted in Graphics 1 through 4 above.[***][5]

Graphic 5 — click to enlarge

Finally, it is instructive to see how the state ranks in various spending measures and how Michigan students rank in achievement levels. Stating that Michigan student performance falls at or near national averages does not reveal where state achievement ranks in comparison to other states. Additionally, the spending figures above do not show Michigan’s expenditures relative to other states.

Although Michigan students’ 2007 NAEP test scores are close to national averages, they actually rank Michigan students 32nd in eighth-grade reading, 36th in eighth-grade math, 30th in fourth-grade reading and 32nd in fourth-grade math.[6] These results are striking considering that in 2007, Michigan had the eighth-highest average elementary school teacher salaries.[§§][7] In addition, in 2006 Michigan had the ninth-highest total current spending per $1,000 of personal income, where current spending essentially represents operating expenditures.[††][8] Taxpayers are paying disproportionately high amounts for relatively low levels of student achievement (see Graphic 6). Michigan’s total current expenditures per pupil has also been relatively high, ranking 16th in the 2004-2005 school year.[9]

Graphic 6 — click to enlarge

[*] Strictly speaking, the mandates of the No Child Left Behind Act apply only to those states that accept the federal money tied to the act. Michigan currently accepts this money.

[†] Average fourth-grade math scores of about 240 and average eighth-grade reading scores of about 280 reflect only that the scales for fourth- and eighth-grade student tests are different, not that eighth-grade students are performing better than fourth-grade students compared to national standards.

[§] The observed differences between Michigan’s performance and the national average in Graphics 1-4 are statistically significant at the 95 percent confidence level in some cases. The comparisons in Graphic 1 (grade four math) and Graphic 2 (grade eight math) each contain one statistically significant difference at the 95 percent confidence level between Michigan’s performance (higher) and the national performance (lower) before 2007. In both cases, that advantage has been lost. In fact, in Graphic 2 (grade eight math), a lead that was once statistically significant for Michigan students is now a statistically significant disadvantage. In Graphic 3 (grade four reading), the difference between Michigan and the national average approaches statistical significance at the 95 percent confidence level and represents statistical significance at the 90 percent confidence level. The performance of Michigan students in grade four reading is now statistically indistinguishable from the national average. In Graphic 4 (grade eight reading), the difference between Michigan students and the national average has never been statistically significantly at the 95 percent confidence level, but the trend in this subject is consistent with the trend in the other subjects regarding comparisons to the national average. In all, it is safe to say that Michigan students do not compare as favorably to the national average as they once did. These graphics should also be interpreted with reference to the findings in Graphics 5 and 6.

[**] Michigan students have also taken National Assessment of Educational Progress exams in both science and writing. While the science scores are somewhat more encouraging, the number of Michigan students achieving “proficient” and “advanced” scores is not very high. In the NAEP writing exams, Michigan’s scores are below average.

[‡] In Helen F. Ladd, Holding Schools Accountable: Performance-Based Reform in Education (Brookings Institution Press, 1996), Ladd argued that looking at increases in spending relative to changes in NAEP scores may overstate the relationship between inputs and outcomes. Specifically, she was relating the trends in national NAEP achievement to changes in education spending between 1971 and 1992. Ladd suggested that it is necessary to deflate spending increases by a factor greater than standard deflators because education inputs have suffered from above-average inflation. In asserting that average students have not had the benefit of greater resources, she also ýclaimed that education spending has also increased due in large part to the growth of special education expenditures. On Page 3, Ladd stated, “[E]ducating children has become more challenging over time because of the increasing proportions of children who are raised in poverty or single-parent households, or both. Hence, some of the increase in resources simply reflects the fact that schools are being asked to do more today than in the past.”

Despite Ladd’s objections to this manner of discussing the relationship between ýincreased spending and flat test scores, it is undeniable that over the past 15 years student performance in Michigan has not gained ground in national comparisons and that Michigan’s spending has been high and rising over the same period. This raises legitimate questions about spending productivity. Ultimately, Ladd did admit that “most people agree that schools need to become more productive in the future” (Ladd, Holding Schools Accountable: Performance-Based Reform in Education, 3). This conclusion is exactly the focus of this primer. If a larger proportion of students is becoming more difficult to educate, it is all the more reason to find ways to make education investments work smarter to achieve results.

[***] In real terms, there were slight annual declines in total spending on primary and secondary public education at various points during the interval shown in Graphic 5. In nominal terms, total spending on public education increased every school year during this period.

[§§] This teacher salary ranking and the salary rankings in Graphic 6 are based on U.S. Bureau of Labor Statistics salary figures for each state. The figures are not adjusted for cost-of-living differences among states for three reasons.

First, a cost-of-living adjustment ratio is best applied regionally and not for a state as a whole. Second, adjusting state average salaries for the cost of living would require computing a weighted average based on the percentage of teachers in each region of each state and the corresponding adjusted average salaries for those regions — a demanding calculation involving multiple assumptions that could represent a research paper in itself. Third, a complete state-by-state comparison regarding teacher compensation would need to account for differences in employee benefits, for which complete data are not readily available. It is likely that such adjustments would not lower the relative ranking of Michigan teachers in national comparisons.

[††] The National Center for Education Statistics defines current spending as follows: “This is the sum of expenditures for Instruction, Support Services, Non-instructional Services (excluding Community Services), and Direct Program Support (excluding Support for Private school Students), and excludes Property expenditures.”