Twenty years ago, I organized a research conference at Brookings and edited a book, Holding Schools Accountable: Performance-Based Reform in Education, that highlighted the newly emerging policy of holding schools accountable for the performance of their students, as typically measured by test scores in reading and math. The following years witnessed some apparent successes with various forms of this approach at the state level, including in southern states such as North Carolina and Texas.
But the subsequent federal incarnation of this approach—the No Child Left Behind Act (NCLB)— has served to highlight the many limitations and downsides of test-based accountability of schools, at least in the form that the federal government imposed on all states. Now that NCLB has been replaced with the Every Student Succeeds Act (ESSA), states have authority to develop new approaches to school and district accountability. Based on my own research and observations from other countries, I would like to see states or districts expand well beyond the remaining test based requirements in the new law and to experiment with a different approach – the inspection and review approach used successfully in many other countries.
If they do so, I predict the quality of schools will improve, children will learn more, teachers will be happier, parents will stop opting out, and states will end up with education systems better suited to meet the needs of all children.
Consider first the limitations of test-based accountability.
The current test based approach to accountability is far too narrow. Our aspirations for education and schools are far broader than teaching children how to do well on multiple choice tests in subjects such as math and reading. Instead, we want our children to develop the knowledge and skills that will enable them not just to succeed in the labor market but to be good citizens and to live rich and fulfilling lives, and to contribute to the flourishing of others.
Research has shown that the narrow focus has led to many unintended and negative consequences. It has narrowed the curriculum, induced schools and teachers to focus on what is being tested, led to teaching to the test, induced schools to manipulate the testing pool, and in some well-publicized cases induced some school teachers and administrators to cheat.
Moreover, test-based accountability is unfair to disadvantaged students and schools. Research from the U.S. and other countries is very clear: students from disadvantaged families achieve on average at lower levels than their more advantaged counterparts. That means that schools serving concentrations of low-income students face far great challenges than those serving middle class students. Holding such schools alone responsible for student test scores is simply not fair either to the schools or the children. To be sure, the move to greater use of test score gains rather than simply achievement levels helps, but it does not fully solve the problem.
The problem in many apparently low-performing schools may not be that the schools themselves are failing, but rather that the schools alone do not have the capacity to offset the many challenges that interfere with the learning of disadvantaged children. In the absence of a broader and bolder approach to education that addresses these challenges, test-based accountability is doomed to fail.
Finally, test-based accountability has had limited success even in raising student test scores. One careful study of NCLB finds small to modest effects on 4th, and possibly 8th, grade math scores and no effects on reading scores. In the period since the introduction of NCLB in 2002, scores on NAEP, the nation’s report card, show little change in the trajectory in math and virtually no progress in reading. Indeed, the most recent 2015 scores show downturns in both subjects.
There is an alternative: An inspection and review system.
Such systems use professional inspectors to make periodic visits to schools – and ideally also to districts. The inspectors review school documents, talk to school leaders and teachers, and may also survey parents. They use a standard protocol to evaluate the quality of the policies and practices at each school, and to report on student outcomes. Their written reports are publicly available.
Such systems have been used extensively in other countries such as England, the Netherlands, and New Zealand. And similar systems, often referred to as school quality reviews, are currently in use in New York City and in Massachusetts for charter schools and low performing schools.
This approach has many potential benefits.
The individual school reports provide useful information to schools, policymakers, and also parents by highlighting both the strengths and the weaknesses of the school, with particular attention to how well the school is addressing the needs of the students it serves. In addition, the inspection office can use information from the individual reports to disseminate information on best practices.
Further, the inspections draw attention to school activities that have the potential to generate a broader range of educational outcomes than just performance on test scores. These include the skills needed for good citizenship, healthy interpersonal relationships, and personal fulfillment. The effectiveness of such systems is not easy to measure. Much of the research has been descriptive, and is based on survey responses about how the inspections affected school processes. The few studies that have focused on the single narrow measure of student test scores have found small positive effects.
Finally, this approach treats schools fairly by holding them accountable only for the practices under their control and by alerting higher level policy makers to the additional resources or capacity they may need to meet the needs of their students.
With the new ESSA, the time is ripe for change.
States now have more authority to try out new approaches. There is no single best research-based model that can be imported directly to the U.S. context. So states will need to experiment to see what works best.
Importantly, any such system will require professionally trained inspectors—perhaps former superintendents, school principals or teachers—and also well-designed inspection rubrics. That will make an inspection system far more costly than the current test-based system. The costs, though, will depend on a number of design features over which states or districts would have control, such as the scale and frequency of the reviews. In addition, some of the costs would be offset by any educational benefits that accrue from the closer monitoring of school processes.
I urge state policymakers to seize the opportunity made available under ESSA to develop and invest in school inspection systems. And I encourage federal policymakers, to help by providing funding to states to help them develop high quality systems—and to disseminate research on their effectiveness.
The potential benefits are great. The challenge is to convince policymakers that qualitative judgements, not just quantifiable outcomes such as test scores, have an important role to play in raising the quality of schools by assuring that they attend to the needs of all their students and fostering a broad range of student outcomes.
Editor’s Note: The author wishes to acknowledge that this blog post builds on ideas developed and presented in a video created for the AERA Knowledge Forum (to be released next month, we will link on this Chalkboard post as soon as it becomes available). She is grateful to the AERA Knowledge Forum participants for input on the content of this blog post and guidance regarding the communication of the argument.
