Does NCLB work?

New Study by UCLA’s Civil Rights Project:
NCLB Ignores What We Know about School Change and
Is Motivated by Politics


A new report from the Civil Rights Project at UCLA, a non-partisan research
center which has been systematically studying the implementation of the
federal No Child Left Behind Act (NCLB) since its inception, finds that some of the
basic assumptions of the law are not working and may well be mistaken. In
this study, Why High Stakes Accountability Sounds Good but Doesn’t Work-- And Why
We Keep on Doing It Anyway, commissioned by the Civil Rights Project,
Researchers Gail Sunderman and Heinrich Mintrop evaluate whether the
accountability system endorsed by NCLB is likely to succeed or fail, and
whether it is compatible with what researchers across the country have
learned about the conditions needed for lasting school reforms.

The report finds that NCLB is failing on three fronts. First, there is
little evidence that high stakes accountability under NCLB works. It has not
improved student achievement and the sanctions have had limited effects in producing
real improvement. The law also results in high numbers of schools being
mislabeled as “failing” and far outstrips the ability of states to
intervene effectively in the schools it sanctions. Third, the law has
failed to connect in a meaningful way to the educators who must implement it --
they do not see the accountability goals as realistic and consider the sanctions
to be misguided and counterproductive for improving schools.

The most important finding is the damage the NCLB is doing to our
educational system. Under NCLB, the system “works” when education systems operate
within only a basic skills framework and with low test rigor. The cost to
our nation is revealed in an educational system stuck in low-level intellectual
work.

Civil Rights Project Co-Director, Gary Orfield, concludes, “The new
administration has a unique opportunity to address the serious structural
problems of NCLB and to forge a more constructive and effective federal
role. To persist in sound-bite educational politics that sound tough but have
failed for a generation would be a tragic mistake. To claim that it would further
the civil rights of children increasingly segregated in schools that have been
officially branded and sanctioned as failures -- but not provided help that
makes a real difference -- would be a blunder.”

Even though the law is failing in some critical respects, the authors argue
that we may maintain NCLB anyway because many derive secondary benefits from
the system, specifically those who are politically and ideologically
committed to NCLB and those deriving economic or political benefits from the law.

The authors contend that after fifteen years of state and federal
sanctions-driven accountability that has yielded relatively little, it is
time to try a new approach. A system based on mandates and legal administrative
enforcement should be replaced with one that emphasizes respect for the
professionalism of educators and active involvement of communities in
developing the capacity to implement lasting changes.

The full report can be found at http://www.civilrightsproject.ucla.edu/

A copy of the Executive Summary and Foreword can be found at the end of this
advisory. Copies of CRP’s previously released NCLB reports may also be
found on our web site above. Funding for this research was generously provided by
a grant from The Charles Stewart Mott Foundation.


About the Authors:

Heinrich Mintrop, Ph.D. taught middle school and high school for over a
decade in both the United States and Germany. He received a Ph.D. in education from
Stanford University in 1996. He is currently an associate professor of
education at the University of California, Berkeley. As a researcher, he
explores issues of school improvement and accountability in both their
academic and civic dimensions. He has recently published the book Schools on
Probation. How Accountability Works (and Doesn’t Work) at Teachers College Press. At UC
Berkeley, he is involved in programs that prepare strong leaders for
high-need urban schools.

Gail Sunderman, Ph.D. is a Senior Research Scientist at the George
Washington University Center on Equity and Excellence in Education where she directs
the Mid Atlantic Equity Center (MAEC). Prior to that, she directed a five-year
study examining the implementation of the No Child Left Behind Act of 2001
for the CRP. She is co-author of the book, NCLB Meets School Realities: Lessons
from the Field (with James S. Kim and Gary Orfield, 2005) and editor of
Holding NCLB Accountable: Achieving Accountability, Equity, and School Reform,
published in 2008. She is a former Fulbright Scholar to Afghanistan and
received her Ph.D. in political science from the University of Chicago.


Executive Summary

The federal accountability system, made universal though the No Child Left
Behind Act of 2002, is at its heart a quota and sanctions system. This
system stipulates the progression of underperforming schools through a set of
increasingly severe sanctions based on meeting performance quotas for
specific demographic groups. While it includes standards, assessments, and
performance targets, sanctions are the means by which the higher levels of the system
put pressure on lower-levels of the system to take accountability seriously.
Even though the law formulates the sanctions in the language of improvement,
support, and radical renewal, the punitive core for districts and schools is
apparent: when improvement efforts fail, loss of control and threat of
organizational survival is at stake.

But whether this system is up to the job of achieving its goal of improving
the performance of persistently underperforming schools is an open question.
Using findings from the best available research, this report examines whether an
accountability system based on the imposition of sanctions is likely to
succeed or fail and, if it does persist, what the consequences may be for sustaining
an educationally rigorous system. The report asks three questions: (1) does
the system work, that is, does it produce the intended results, (2) is it
practical, that is, can it be implemented, and (3) is it legitimate, or is
it valued among those who must implement it. We conclude with a discussion of
the costs of maintaining the current sanctions system.

Does the System Work? There are two aspects to this question: does the
system as a whole produce the expected outcomes; and do the actual sanctions result
in school improvement.

Does the system produce the expected outcomes? There is little evidence
that high stakes accountability under NCLB improves student achievement.
Although state accountability systems appear to be a success since test scores
continue to rise in most systems, the picture looks far less positive when one looks
at the National Assessment of Educational Progress (NAEP). When NAEP scores are
used, gains appear to be much lower, there is substantial variation among
states, and few states have narrowed the achievement gap among racial and
socioeconomic subgroups while improving overall performance at the same
time. Given the large discrepancies between NAEP and state assessments results, it
is not quite clear what the state tests measure. By all indication, state
accountability systems with their own pressures and sanctions are successful
at focusing schools’ and districts’ attention on state assessments.

Do the sanctions work? There is also a lack of evidence that the sanctions
themselves have been successful as an effective and universal treatment for
low-performing schools. Neither the transfer option nor the supplemental
educational service provisions have been widely embraced by parents or
districts. Whether or not the transfer option produces improvements in
school performance is a moot point since the percentage of students taking
advantage of this option (about 1% of eligible students) is so low. The response to
supplemental educational services has also been low (14% of eligible
students); and third party evaluations of these services are finding small, if any
statistically significant effects of the program on improving student
achievement. The corrective action and restructuring options, such as
reconstitution, charter school conversion or take-over by education
management organizations (EMOs), may work in some limited situations but are not
effective across the board. Among the variety of corrective action and restructuring
strategies that have been tried, none stick out as universally effective or
robust enough to overcome the power of local context.

Is the Sanctions System Practical? If the NCLB system was practical, it
would identify schools in need of improvement and restructuring with high
accuracy; appropriately direct schools to pay attention to students most in need of
help; produce an intervention burden for states and districts commensurate with
capacities to provide new impetus, ideas, resources, and personnel; and
lastly, through the imposition of sanctions, create momentum for deliberate and a
well articulated improvement processes for schools and districts stuck in low
performance. NCLB fails those practical criteria.

In state systems with at least moderately high performance demands, NCLB has
led to high numbers of failing schools that by far outstrip district and
state capacity to intervene. But it is not even clear if the bulk of these
schools are in fact correctly classified. Most notably, the system has no practical
answers to address the full spectrum of student performance and learning
needs, particularly for students far-below proficient, special needs students, and
marginally performing students; moreover, it does not speak to the
predicament of low-capacity schools and districts. While it may appear that the
sanctions system has succeeded in fermenting a climate of reform, such ferment, in
many instances, is more likely to result in unproductive turbulence than
sustained school improvement.

Is the Sanctions System Legitimate? Despite an almost twenty year period in
some states, accountability systems, and particularly NCLB, continue to
encounter serious legitimacy and acceptability problems among the groups
that they are designed to target—teachers, principals, and administrators in low
performing schools and districts. In general, while standards, assessments
of performance, and consequences for low performance are widely accepted ideas
in general, research suggests that attitudes about high stakes accountability
systems are more negative. This is because accountability systems designed
around sanctions violate core professional norms of educators and produce
widespread frustration and de-moralization among those charged with carrying
out school improvement efforts. Accountability goals are often not seen as
realistic, and the sanctions are considered to be misguided and not very
useful for improving schools. In efforts to improve test scores, teachers widely
report that they must compromise standards of good teaching in order to meet
accountability goals.

What are the Costs of Maintaining a Sanctions System? The combination of
uncertain effects, loose connections to the broader educational values and
norms of educators, and the difficulties or impossibilities of carrying out
the system day-to-day makes the sanctions system a prime candidate for declaring
it a failing system. But there is a way to maintain the system, although this
way produces high educational costs. As long as states maintain low-rigor
systems that concentrate on basic skills, and the more lenient options for school
improvement or restructuring are chosen, the system can persist with
relative ease. NCLB “works” when systems place low demands on the cognitive
complexity of learning tasks and, subsequently, on teacher capacity
building. State accountability systems that operate within a basic skills framework
and with low test rigor tend to produce lower numbers of failing schools.
Because such systems tackle school improvement goals that are fairly light,
affordable, and manageable, they are more practical within the NCLB framework. Systems
that are more ambitious produce an intervention burden that makes them
unworkable.