Where and how do we look for a model education system such as the one just described? To begin with, we want a system that is generalizable, scalable, and time-tested. These criteria mean that modern U.S. experiences can only be of limited value. Every state in the nation is overwhelmingly dominated by the same sort of traditional public school system. U.S. voucher, charter, and tax credit programs are tiny, or recent, or both. Step one in our search, therefore, should be to cast a very wide net.

Formal schooling aimed at the general public dates back 2,500 years, to classical Greece, and the arrangements adopted over the ensuing 25 centuries have covered a remarkably wide gamut.

Looking beyond the United States at other wealthy industrialized nations is somewhat more helpful, since educational governance and funding structures depart from the traditional U.S. mold in countries such as Holland (national voucher-like system), Japan (vast private tutoring market), and New Zealand (national charter-school-like arrangement). Developing countries demonstrate even more variety. Some less-developed countries actually have a greater variety of education systems operating within their borders than exists between most rich countries. India alone, for example, has a public school system, a system of fully government-funded private schools, a system of government-registered but fully parent-funded private schools, a system of unregistered fully parent-funded private schools, and a significant private after-school tutoring market (serving mostly public school students in need of remedial instruction).

Another trove of alternative systems can be found by looking at the history of education. Formal schooling aimed at the general public dates back 2,500 years, to classical Greece, and the arrangements adopted over the ensuing 25 centuries have covered a remarkably wide gamut.

This broad base of historical and international precedents is helpful in meeting the sustainability and scalability criteria, but poses a problem for generalizability. There are, after all, many factors outside the classroom that affect educational performance. How can we know that a system that performed well in early 19th century England, or that is currently performing well in Japan, will work equally well in the contemporary United States? To answer that question we need a way of correctly apportioning the credit (or blame) for a society’s educational outcomes. We need a way of distinguishing the educational outcomes attributable to the school system itself from the educational effects of prevailing cultural, economic, and technological conditions.

The ideal technique would be to conduct controlled experiments in which students would be randomly assigned to one or another kind of school system. This has been done with a few small-scale voucher experiments in the United States[6], but it is rarely possible in the international context and it is patently impossible in the historical context. Comparative education policy is not the only field in which randomized experiments are difficult or impossible to implement, however. Anthropologists, epidemiologists, and cosmologists, among others, have all had to find other avenues of investigation. One of the most successful techniques adopted in these fields is the so-called “natural experiment.” The idea is to mimic controlled experiments as closely as possible using the data at hand. Jared Diamond, a proponent and practitioner of natural experimentation and author of the anthropological tour-de-force, Guns, Germs, and Steel: The Fates of Human Societies, explains that

While neither astronomers studying galaxy formation nor human historians can manipulate their systems in controlled laboratory experiments, they both can take advantage of natural experiments, by comparing systems differing in the presence or absence (or in the strong or weak effect) of some putative causative factor.[7]

My own approach to natural experimentation, applied in Market Education: The Unknown History,[8] has been to combine three separate sorts of observations:

  • Observing how a given type of school system performs across many different times and places

  • Observing how different school systems perform under comparable cultural and economic conditions

  • Observing how educational outcomes change (or don’t change) when societies move from one type of school system to another

The first sort of observation aims to isolate school system effects from extrinsic social effects, and test their repeatability, by looking for patterns of outcomes across many different settings. The second sort of observation compares the “treatment effects” of alternative school systems by holding social conditions as constant as possible. The third sort of observation uses the pre-reform society as a control group for the post-reform society, and any treatment effects suspected of being caused by the change in school system can be tested for repeatability by comparing them to the pre-reform/post-reform changes that are found in the educational transitions of other societies.

When a given approach to organizing schools is consistently associated with a certain set of outcomes across widely varying times and places, when its outcomes consistently differ from those of other school systems operating in comparable cultural settings, and when its expected pattern of outcomes is consistently manifested when societies adopt it and abandon their previous systems, it is possible to conclude with some confidence that the observed pattern of effects truly is attributable to the design of that education system and not to extraneous factors.