Question #5

Why does your research have anything new to say about the mysteries of student achievement and school performance?

There are two distinguishing qualities of our research, the first having to do with the kinds of causes of school performance we are looking at, the second having to do with the data we are using to study those causes. Research into the determinants of school performance and student achievement has been dominated by what are often called input-output studies. [32] Based on the economic concept of the production function, these studies have tried to explain educational "outputs," such as student test scores, with conventional economic "inputs," such as expenditures per student, teacher salaries, class sizes, and the caliber of school facilities. The fundamental idea behind these studies is that schools, like any economic enterprise, ought to produce their products – educated students – with varying degrees of effectiveness and efficiency as the combination of capital and labor used in production varies. Years of study now suggest, however, that schools may not be like just any economic enterprise. Since the famous "Coleman Report" of 1965, input-output studies have been unable to establish any systematic relationship between school performance and a wide range of indicators of school resources.

The research that we have been doing takes a different approach than input-output studies. It focuses more on the production process itself. It considers how schools are organized and operated – in other words, how inputs are actually converted into outputs. The production process may well be more important in public education that the economic theory of production functions would suggest. Schools are not part of a market where competitive forces can be assumed to encourage managers to organize their firms to use capital and labor efficiently. Schools are part of a political and administrative system where the forces that managers – principals and superintendents – are exposed to cannot be expected to encourage efficient organization. It therefore becomes especially important in analyzing the performance of a public enterprise such as a school to study its organization. It is also important to examine those non-economic forces that lead schools to organize as they do. While our research also considers the conventional economic determinants of school performance, our emphasis is on the production process – how it works and what causes it to work in different ways. Because of this emphasis, our research may well have something new to say.

Our research is also distinguished by the data it employs. We are far from the first researchers to suggest that school organization is important, that it can help explain the weak link between school resources and school performance. Indeed, over the last ten years many researchers have completed studies that show that successful schools have distinctive organizations. Better schools appear to be characterized by such things as clear and ambitious goals, strong and instructionally oriented leadership by principals, an orderly environment, teacher participation in school decisionmaking, and collegial relationships between and among school leaders and staff. The studies that have identified these characteristics – studies known collectively as "Effective Schools Research" – have not settled the issue of school performance, however. [33] There are serious doubts about the magnitude of the impact that school organization has on school performance and, indeed, about whether organization is a cause of performance at all: healthy school organizations may be a consequence of successful students, and not vice versa. It almost goes without saying that Effective Schools Research has provided few clues about the causes of school organization; the focus of that research has been on organizational consequences.

A primary reason for the doubts about Effective Schools Research is the methods that have been used in most of the studies. Research has been dominated by qualitative case studies of small numbers of schools, usually reputed to be unusually successful. Those few studies that have used somewhat larger numbers of schools and employed quantitative analysis have still not examined representative samples. From one study to the next there has been considerable variation in the particular organizational characteristics said to be important. And the conclusion that organization is important, however frequently it has been drawn, is still based substantially on impressionistic evidence, uncontrolled observation, and limited numbers of cases. In sharp contrast, input-output research, however negative its conclusions, is based on rigorous statistical analyses of hard data in hundreds and thousands of schools nationwide. There is consequently more reason at this point to believe that the relationship between school resources and school performance is unsystematic than to believe that school organization provides a strong link between the two.

In our research we explore how strong that link may be by employing the methods that have been used in input-output analyses. Unlike most Effective Schools Research, we investigate the resources, organization, and performance of a large random, national sample of schools in which all characteristics are measured with quantitative indicators, all relationships are estimated with statistical controls, and all inferences are careful to try to distinguish causes from effects.

Our data base is the result of merging two national surveys of American high schools – High School and Beyond (HSB), a 1980 and 1982 panel study of students and schools, and the Administrator and Teacher Survey (ATS), a 1984 survey (which we helped design) of the teachers and principals in half of the HSB schools. The merged data set includes over 400 public and private high schools – the privates providing a valuable look at school organization in a market setting – and approximately 9,000 students, 11,000 teachers, and the principals in every school in the sample. While no piece of research is ever definitive, and this is certainly true of research as new as ours, our work is a step in the right direction methodologically, and therefore a contribution that may well make a difference.