We use school-level data from the Mackinac Center’s Michigan Public School Context and Performance database to assess school-level student achievement. This dataset contains “context and performance” scores for each school based on an analysis of average standardized test scores adjusted according to the average poverty level for students enrolled in the school. Specifically, this CAP analysis adjusts state standardized test scores based on the rate of students taking the tests that qualify for the federal free lunch program, determined by their household income. Research consistently shows a statistically significant negative correlation between free lunch rates and raw achievement scores on standardized tests.
The CAP analysis uses linear regression in order to predict how each school should perform on standardized assessments given their students’ socioeconomic backgrounds[*] By comparing a school’s actual achievement to its expected achievement based on student poverty rates, CAP scores present a more accurate comparison of school-level impacts on academic outcomes.[†] The CAP score is standardized to have a mean of 100 points and a standard deviation of 15 points; a score above 100 means that the school performed better than expected based on their poverty rate and a score below 100 means just the opposite — the school performed worse than expected given their poverty rate.[‡]
CAP scores are based on academic performance data that is comprised of average standardized tests scores from multiple consecutive years. These include, for the schools serving high school students, scale scores from the Michigan Merit Examination from 2010 to 2015.[§] For schools serving elementary and middle school students, CAP scores are derived from multiple consecutive annual average scale scores from the Michigan Educational Assessment Program tests from 2012 to 2014.[**]
We calculated two overall CAP score averages at the city level: one for public charter schools and one for TPS. We only had sufficient data to calculate these average CAP scores in 71 different Michigan cities.[††]
[*] The different assessments that form the basis of CAP scores, based on the operative school year and grades tested, are the Michigan Merit Examination, the Michigan Educational Assessment Program and the Michigan Student Test of Educational Progress.
[†] Since our analysis does not represent a true experiment, selection-bias is not completely controlled for. CAP scores are created by running a regression based on the rate of students eligible for free lunches due to low family income. However, to provide a needed robustness check, we do control for many factors typically associated with student success in this current analysis, including the following observable effects in the data: gender, English language learners, special education, income disadvantage and minority status.
[‡] For fuller explanations of data and methods used to calculate CAP scores, see Audrey Spalding and Ben DeGrow, “The Michigan Context and Performance Report Card: Public Elementary and Middle Schools, 2015” (Mackinac Center for Public Policy, 2016), 2–4, https://perma.cc/ Y3MB-NCDT; Ben DeGrow and Ronald Klingler, “The Michigan Context and Performance Report Card: High Schools, 2016” (2017): 1–3, .
[§] Technically, we used CAP scores from two different reports, one that included MME scores from 2010 to 2013 and one report that was based on MME scores from 2012 to 2015. If a high school did not have a score from the report covering 2012 to 2015, we used the CAP score for the report covering 2010 to 2013.
[**] In 2015, the state of Michigan changed its assessment program to the M-STEP, replacing the MEAP and MME. Among other things, the M-STEP replaced separate reading and writing subject tests with a combined English language arts score. Accounting for that combination, a regression analysis found that a school’s 2014 MME subject test score was a reliable predictor of its performance on the 2015 M-STEP subject test, so there’s little reason to believe that recent or future M-STEP results will dramatically change a school’s CAP scores. See Ben DeGrow and Ronald Klingler, “The Michigan Context and Performance Report Card: High Schools, 2016” (2017): 33, https://perma.cc/FS3Q-ZHQU.
[††] In 21 of the cities studied, no charter schools had qualified academic achievement data. This occurred for a variety of reasons. Most of the omitted schools either had not been open long enough to report enough years of standardized test scores or they are designated as alternative education programs.