How Bill Gates misinterprets education data

Posted at 5:00 AM ET, 03/11/2011

How Bill Gates misinterprets ed facts

This was written by Richard Rothstein, a research associate at theEconomic Policy Institute, a non-profit created in 1986 to broaden the discussion about economic policy to include the interests of low- and middle-income workers. This appeared on the institute's website.

By Richard Rothstein
Microsoft Chairman Bill Gates authored an op-ed published in The Washington Post late last month, “How Teacher Development could Revolutionize our Schools,” proposing that American public schools should do a better job of evaluating the effectiveness of teachers, a goal with which none can disagree. But his specific prescriptions, and the urgency he attaches to them, are based on the misrepresentation of one fact, the misinterpretation of another and the demagogic presentation of a third. It is remarkable that someone associated with technology and progress should have such a careless disregard for accuracy when it comes to the education policy in which he is now so deeply involved.

Gates’ most important factual claim is that “over the past four decades, the per-student cost of running our K-12 schools has more than doubled, while our student achievement has remained virtually flat.” And, he adds, “spending has climbed, but our percentage of college graduates has dropped compared with other countries.” Let’s examine these factual claims:

Bill Gates says: "Our student achievement has remained virtually flat."

The only longitudinal measure of student achievement that is available to Bill Gates or anyone else is the National Assessment of Educational Progress (NAEP). NAEP provides trends for 4th, 8th, and 12th graders, disaggregated by race, ethnicity, and poverty, since about 1980 in basic skills in math and reading (called the “Long Term Trend NAEP”) and since about 1990 for 4th and 8th graders in slightly more sophisticated math and reading skills (called the “Main NAEP”).[*]

On these exams, American students have improved substantially, in some cases phenomenally. In general, the improvements have been greatest for African-American students, and among these, for the most disadvantaged. The improvements have been greatest for both black and white 4th and 8th graders in math. Improvements have been less great but still substantial for black 4th and 8th graders in reading and for black 12th graders in both math and reading. Improvements have been modest for whites in 12th grade math and at all three grade levels in reading.

The following table summarizes these results, for the earliest and most recent years for which disaggregated data were collected.

We can see that in 4th grade math, black students now have higher average achievement than white students had when the assessments began. Average black students’ gains have been a full standard deviation, a rate of progress that would be considered extraordinary in any area of social policy. The black-white score gap has narrowed some, but not very much, because white students have also shown improvement.

Bill Gates may think that these improvements are insufficient, and perhaps he is correct. But, as Daniel Patrick Moynihan reportedly quipped, “everyone is entitled to their own opinions, but not to their own facts.” No rational reading of these NAEP data can support Bill Gates’ claim that “student achievement has remained virtually flat” over the last four decades.[†] And, to repeat, no other longitudinal data are available that describe student achievement over time.

These facts also don’t support the story that the typical teacher of disadvantaged children is ineffective. Certainly, some teachers are ineffective, and schools should do a better job of removing them. But that should not, if facts are to be believed, be the main story.

Yet it seems to be. Secretary of Education Arne Duncan recently asserted that “many, if not most, teacher-training programs are mediocre.” This may be true, but how does he know? What is his evidence? It wouldn’t seem that mediocre teacher training programs could consistently be turning out teachers who have posted the kinds of gains we’ve seen on NAEP in the last generation and more.

It is important to investigate why, in the most recent period, typical teachers have been more effective with elementary school children than with high-schoolers, but curiously, the reforms Bill Gates and like-thinking policymakers are pursuing concern elementary school teachers almost exclusively – because the student value-added scores on NCLB-required standardized tests by which they propose to evaluate these teachers are available only for elementary, not secondary school students. It is also important to investigate why teachers have apparently been more effective during most (though not all) of the last few decades in teaching math than reading, but it is difficult to motivate anyone to investigate this if our vision is clouded by the myth that all student achievement has been flat.

Bill Gates says: "The per-student cost of running our K-12 schools has more than doubled."

Here, Bill Gates is nominally correct, but misleading. When properly adjusted for inflation, K-12 per pupil spending has about doubled over the last four decades, but less than half of this new money has gone to regular education (including compensatory education for disadvantaged children, programs for English-language learners, integration programs like magnet schools, and special schools for dropout recovery and prevention). The biggest single recipient of new money has been special education for children with disabilities. Four decades ago, special education consumed less than 4% of all K-12 spending. It now consumes 21%.[‡]

Detailed tables documenting these trends are available here.

American public education can boast of remarkable accomplishments in special education over this period. Many young people can now function in society whereas, in the past, children with similar disabilities were institutionalized and discarded. But it is not reasonable to complain about the increase in spending on such children by insisting that it should have produced greater improvement in the achievement of regular children.

The increase in regular education spending has still been substantial, even if not nearly as great as Bill Gates implies. Should this spending increase have produced even greater improvement in achievement than has in fact occurred? This is a more difficult judgment to make. But in light of the actual achievement improvements documented by NAEP, it is not reasonable to jump to the facile conclusion of a productivity collapse in K-12 education. A more reasonable story is that spending has increased and achievement has increased as well. Perhaps we have gotten what we paid for.

Bill Gates says: "Spending has climbed, but our percentage of college graduates has dropped compared with other countries."

This is the Bill Gates claim that can properly be called demagogic. It attempts to agitate readers by presenting a positive development in a negative light. A climb in spending should produce an increase in the percentage of college graduates. And it has. In the last four decades,the percentage of college graduates in the United States has nearly doubled. In 1970, 16% of young adults (ages 25 to 29) were college graduates. Today, it is 31%. The improvement has been across the board: the share of African-American young adults who are college graduates has gone from 10% to 19%; for whites it has gone from 17% to 37%. Somehow, Bill Gates saw fit to present this as an indictment.

Should our college graduation rate be rising faster? Of course, that would be a good thing. Should the spending increases we have experienced have generated a faster increase in college graduation than, in fact, they have? That would be worth exploring, but Bill Gates’ phrasing suggests to the less-than-careful reader that spending increases haven’t been productive at all, because our college graduation rate has “dropped…” Would a faster increase require even greater increases in spending? That is also likely, but it is not a conclusion that Bill Gates intends to suggest.

It is commonplace to imply, as Bill Gates does in his Washington Postop-ed, that our failure to increase our college graduation rate “compared with other countries” will prevent us from “build[ing] a dynamic 21st-century economy.” Certainly, we need a sufficient number of well-trained college graduates for such an economy, but there is no reason to believe that a graduate rate in excess of 30% is too small for this purpose, or that economic dynamism can, after reaching sufficiency, increase linearly with increases in the share of young people who graduate from college. The threats to a dynamic 21st century economy are likely to come from a failure of macroeconomic policy, regulation of speculation, and investment in education, not from inefficiency in the investment we already make.

We only need to examine the list of international college graduation rates to see the absurdity of efforts to make a direct link between college graduation rates and economic success. The Organization for Economic Cooperation and Development (OECD) publishes comparative data.

One country that outranks the United States in college graduation rates is Ireland, whose economy has now collapsed because its regulation of the real estate bubble was even more careless and corrupt than ours. Another is Portugal, whose economic health is also worse than that of the United States.

Of course there are also nations on the list that are not on the verge of bankruptcy, but the chief lesson of the list is this: provided a nation has a sufficient number of college graduates for a dynamic economy, rankings above that point are irrelevant. Of course we should increase our college graduation rate, and there are many civic and cultural reasons to do so, even if we may already produce (as some analyses suggest) an apparent surplus, for economic purposes, of science, technology, engineering, and math graduates.

Education is complex, and the relationship between education and the economy even more so. Our ability to grapple with the challenges these present is not enhanced by factually inaccurate and hyperventilated appeals from those who should know better.

[*] In theory, the Long Term Trend (LTT) is distinguished from the Main Assessment because the LTT assesses the same skills, whereas the Main Assessment changes over time, as the curriculum changes. But in fact, the LTT also changes somewhat over time, and the Main Assessment is sufficiently stable to make longitudinal comparisons.

[†] If the data are further disaggregated by decade, there have been some interim periods of flatness within the overall growth. For example, gains were strongest for black elementary students in the LTT in the 1980s and 2000s, and flat in the 1990s, but on the Main Assessment they showed strong gains in the 1990s as well. Twelfth grade LTT reading scores have been mostly flat since 1990, after a dramatic leap of 24 scale points for blacks in the 1980s. Fourth grade LTT reading scores fell for blacks in the 1980s, but rebounded in the 1990s and jumped even more strongly in the 2000s. The tables showing these disaggregated data are posted here.

[‡] Detailed tables documenting these trends are available here.

-0-