(Note: The following is an edited version of a
commentary that originally appeared on Cato-at-Liberty.Org.)
The Wall Street Journal and
The New York Times in July jumped on the release of a new study by the
National Center for Education Statistics. The Wall Street Journal’s headline was
particularly dramatic: "Long-Delayed Education Study Casts Doubt on Value of
No, it doesn’t.
And it is a failure on my part, as well as a failure of the
school choice movement as a whole, that the media don’t understand why. - more on the wisdom of this below -
Taking the study entirely at face value what the report says is
this: Private school students consistently score better in math and reading on
the National Assessment of Educational Progress than public school students, but this advantage essentially goes away if you apply a particular set of controls for the differing student characteristics between the two sectors (things like wealth, race, and so on).
Okay, you say, but if private schools don’t significantly
outscore public schools, what’s the point of school voucher programs or other
reforms that would give all parents access to the public or private school of
their choice? Why, in other words, is The Journal’s headline wrong?
It is wrong because the point of voucher programs is to
create a competitive education industry, and the existing population of U.S.
private schools does not constitute such an industry.
A vigorous free market in education requires that all
families have easy access to the schools of their choice (whether public or
private); that schools are not burdened with extensive regulations on what they
can teach, whom they can hire and what they can charge, etc.; that consumers
directly pay at least some of the cost of the service; that private schools not
be discriminated against financially by the state in the distribution of
education funding; and that at least a substantial minority of private schools
be operated for profit.
This set of conditions does not exist in any state in the
nation. Instead, American education is dominated by a 90 percent government
monopoly that is funded entirely through taxation. The private sector occupies
the remaining 10 percent niche, is almost exclusively operated on a nonprofit
basis and is forced to charge thousands of dollars in tuition in the face of the "free" monopoly schools that spend an average of $10,000 per pupil per year.
This is not a market. No study was necessary to point this out.
Competitive markets are characterized by innovation,
inexorable improvements in cost effectiveness and the quality of goods and
services, and the rapid growth of the most successful providers. None of this
has occurred in the U.S. private education sector, precisely because that sector does not constitute a competitive market.
The last great innovation to transform classroom
instruction occurred during the presidency of Thomas Jefferson (the invention of the chalkboard, around 1801). Since that time, the pace of innovation has been so slow that a student from the mid-1800s would immediately recognize a typical modern classroom. The most sought-after private schools enroll only about a thousand more students today than they did a century ago. This degree of
stagnation is unheard of outside of the education sector, because it is only in
the education sector (at least in liberal democracies) that market activity has
been so thoroughly extinguished by government monopoly provision.
Hence, this study of our small, nonmarket niche of
private schools does not allow any generalization to the sort of outcomes to be
expected from a true free market in education — and the creation of such a
market is the primary justification for vouchers and other school choice
If I were better at my job, and if the school choice
movement as a whole had a more effective media machine, this fact would be
widely understood and we wouldn’t see fallacious headlines like the one cited
That major point having been made, let’s take a look at the
study’s findings on their own merits, as an examination of the current crop of
public versus private schools.
The first problem with the study is that it collects no
data on per-pupil spending in public versus private schools. Private school
tuition, according to the NCES itself, is about half of the average public
school expenditure per pupil. While private schools have some other sources of
revenue, they still spend thousands of dollars less per pupil than public
schools, even after taking these other revenues into account. Private schools may be
dramatically more efficient even if their absolute achievement levels are
comparable to those in public schools.
Hence it is possible that if spending were equalized, private schools would raise student learning substantially
compared to current levels. While it has been shown that spending and
achievement are largely unrelated in the public sector, this has not been
demonstrated in the private sector. In fact, evidence from developing countries
suggests that higher spending in private schools DOES increase student
Next, it is worth observing the specifics of the study’s
findings. It reports that there is a small advantage to public schools in 4th
grade math, but that this advantage is not present at the 8th grade. It further
says that at the 8th grade, private school students have a small advantage over
public school students in reading. One possible interpretation of these findings
is that public school students fall behind their peers in private schools the
longer they spend in the classroom.
That, of course, is only one possibility. At any rate, it
is clear that parents are most concerned with what their children know and are
able to do at the end of their K-12 education, so if, by the later grades,
private schools confer a significant advantage, this would definitely seem to
Methodological and Data Problems
All of the above discussion takes the study’s findings at
face value. This may be ill-advised, since a preliminary review suggests that
there may be real methodological problems and potentially serious data problems.
Several of the control variables used in the model seem problematic, including
The rate of student absenteeism
It is entirely possible that sectoral differences in the
feeling of community or level of personal attention, ability of school staff to
motivate students, etc., could affect student absenteeism. So it is erroneous to
treat this as exogenous (i.e., as independent of school sector) and to control
This variable is clearly endogenous (i.e., affected by
school sector). Parents tend to prefer schools in which teachers know all the
students by name and which create a friendly, community atmosphere. This is much easier in smaller schools, and hence there is a competitive pressure not to get too large in the private education sector. No such pressure exists in the public sector, where contrary bureaucratic incentives encourage large school size. As a
result, the average public school is roughly three times the size of the average
521 students versus
182 students. It is thus unjustifiable to pretend that school size is independent of
The percentage of students in the Title I program
report by the Congressional Budget Office notes that "About 97 percent of
public schools and 45 percent of private schools participate in the school lunch
program." This vast difference in level of participation by schools may have a
significant effect on the share of eligible students who are in fact being
served by the program.
Sample specification problems
Between a fifth and a quarter of the private schools
selected for the study did not participate. The authors make no serious attempt
to analyze non-participants to determine how and to what extent they might
differ from participating schools in ways related to student performance. This
could bias their results in unknown ways.
It seems likely that public and private sector schools
apply the federal Specific Learning Disability label differentially. This label
states that children are disabled if they perform at a level below what would be
expected for students of their age and intelligence. It does account for the
possibility that poor performance may be the result of poor instruction. Roughly
6 percent of all public school students are placed in this category, making up
nearly half (43 percent) of all students classified as disabled in the
public sector. Among private schools participating in this study, a total of
3 to 4 percent of students are classified as suffering from ANY disability,
mental or physical. Because students classified as SLD can be excluded from the
test taking pool or given extra time or other accommodations, differential SLD
classification rates between the sectors may affect sectoral mean scores
(because these students, by definition, perform below the average of their
Tom Loveless has pointed out in a paper for the Brookings
Institution that the NAEP mathematics test does a poor job of measuring the
skills that it is purported to measure. Calculator use is allowed throughout, so the exam does not measure basic arithmetic ability. More advanced topics, such as
algebra with fractions, are all but absent, making it a poor test of these more
advanced skills. If the two school sectors differ in either of these important areas, the NAEP will not detect it. It is natural for scholars to
want to analyze the data they have, but readers should be aware of the
shortcomings of those data as a measure of both basic and advanced mathematical
Taking all of the above analysis together, this study’s
findings would have little bearing on market-based reforms like vouchers and
tax credits even if the research were methodologically flawless. Even as a comparison of
public schools and the existing (nonmarket) crop of private schools, this study leaves
much to be desired because it neglects to consider the substantially higher
per-pupil spending of public schools.
But the study, as noted above, is not methodologically
flawless. Several of its control variables appear to be misspecified, and so its
adjusted test score averages may be significantly biased. It makes no attempt to
assess the impact of the non-participation by between a fifth and a quarter of
all the private schools selected for participation in the study — another
probable source of bias. And the study uses a mathematics test (the NAEP) that has been
shown to do a poor job of assessing both basic arithmetic and more advanced
mathematical skills, thus obfuscating possible differences in performance in
these (rather important) areas among the students tested.
In a nutshell: This study does not say what some reporters
think it says, and it may not even say what its own authors think it says.
Andrew J. Coulson is director of the Center for
Educational Freedom at the Cato Institute in Washington and an adjunct fellow
for the Mackinac Center for Public Policy, an independent educational and
research institute headquartered in Midland, Mich.