Gates Foundation: No clear data to support small school reform

"We had good intentions..."--Gates Leader

“I think it’s fair to say we’ve learned a number of important lessons, and it’s quite clear that over the last six years I created a number of grant programs that were well-intentioned but had some weak assumptions.”


Tom Vander Ark, the executive director of education initiatives for the Gates Foundation, deserves some praise. It’s rare to see a corporate or private foundation leader with a self-critical attitude, especially in public.

Vander Ark is a former businessman and Federal Way, Washington, school superintendent, who now has the difficult, and enviable, job of overseeing the biggest, privately-funded school reform initiative in history. The Gates Foundation has invested more than a billion dollars (nearly twice what they spend combating malaria in Africa), to create hundreds of new, small high schools nationwide and to restructure hundreds of large, traditional high schools. However, things are not going well, according to their own commissioned study conducted by the Stanford Research Institute (SRI), proving that a billion dollars and a good idea are not necessarily sufficient when it comes to such a monumental task as changing the American high school.

In a recent Education Week article, Vander Ark was self-critical: “I think it’s fair to say we’ve learned a number of important lessons, and it’s quite clear that over the last six years I created a number of grant programs that were well-intentioned but had some weak assumptions.” Vander Ark’s contrition is commendable, but it’s important to look at just which assumptions are being reevaluated and which lessons the Gates team has learned.

The SRI evaluation must have been devastating to the Gates management team, not to mention and the hundreds of consultants they put out in the field to drive the reform effort. After a ton of money and five years of work, the study concludes: “…the quality of student work in all of the schools we studied is alarmingly low.” Yes, you read it right. It said in ALL OF THE SCHOOLS they studied. And get this, results were actually lower in the new Gates’ start-up schools than in the restructured high schools. The new charters, mostly under private management, actually had worse scores that did the conversion high schools, which according to common lore, are much more difficult to change.

Vander Ark says he “found nothing surprising” in the report. The evaluation isn’t all negative and in many ways supports the effectiveness of smaller learning communities, particularly when it comes to improved climates leading to jumps in attendance, personalized teaching/learning and respectful behavior on the part of students. But these factors didn’t seem to rate very highly with the researchers. Instead they relied heavily on math and reading scores on standardized tests and found that improved teaching and learning just didn’t happen anywhere they looked.

Think of the scope of the troubled initiative. Gates is funding more than 850 new schools and nearly 700 conversion schools. Funding goes to nearly 275 school districts (I’m assuming that as test scores don’t increase rapidly enough for the foundation, struggling districts will lose their funding as Seattle and San Francisco have done in the past few weeks ). If you just take the 1,550 schools as a group, you have the equivalent of a district twice the size of Chicago’s, but one where ALL the schools are doing low-quality work.

The Gates schools actually did worse, in terms of improved learning outcomes, than did many of our neighborhood urban schools, who made great improvements without any Gates funding and with little in the way of district support. I can show (and have shown) Vander Ark many Chicago small schools where there is a high quality of student work. A recent study of Chicago schools showed 144 of them that had made substantial improvements in reading and math over the past 10 years, without private management and with their teaching faculties and student populations left basically intact.

The Gates grants were made to school districts in partnership with other private foundations and business groups. Private management companies were brought in to start new charters. The main ones left out of the planning process were the teachers and students. Communities were rarely engaged (the Ohio initiative led by the Knowledgeworks Foundation was a notable exception). Parents had a minimal role. Instead, the Gates initiative relied mostly on “replicable models,” “scaling up,” and top-down restructuring. But are these the lessons and assumptions that Vander Ark talking about? Let’s look and see. Here are some of the changes Vander Ark is talking about implementing as part of the Gates strategy.

--He says he will build a new approach to school improvement that is in the context of a system, a school district, rather than one focused at the school level. This shift is necessary because without district-level support, no school-based change can be sustained. But seeing the district as the basic unit of change (foundation style) can also mean simply cutting deals with superintendents and politicians as they did in San Francisco, without engaging the school community.

--The foundation will now focus on “well-specified school models," i.e., Big Picture and KIPP, and scale up. But this is more of the same old replication strategy. While replication of good small schools and charters is possible on a reasonable scale, Gates has pushed schools to replicate 50 or more times in a few years, often without adequate support from school districts and with no base in communities. What's more, many of the private replicators (not the ones mentioned above) are anti-union and resistant to community engagement. This is bound to cause more divisiveness and resistance.

--The foundation will reduce its emphasis on school autonomy. Vander Ark says autonomy should be only for successful schools and not for “failing” (read poor and heavily minority) schools. This is a major shift in strategy. Previously, autonomy was seen as part of the solution to top-down imposition of stupid rules and mandates which were seen as part of the cause of failure. But as Gates becomes more tied up with the district bureaucracy, autonomy is only for the schools that don’t really need it.

--More money will be spent on professional development with a focus on content areas like math and science. But while more resources for improvement of teaching are badly needed, it begs the question: what kind of professional development? Is this just another brick in the anti-teacher wall—another thing being done TO teachers? I think so.

I hope the self-critical evaluations keep coming. But let’s go deeper, please.

Evaluation reports on the Bill & Melinda Gates Foundation's high school initiatives are posted by the organization.



Tuesday, Nov 22, 2005 - 01:59pm (PST)

Previous Post: 'Teachers are part of solution, not the problem' says my brother Fred

---------------



From: EDWEEK
Date: Wed Nov 16, 2005 6:56 am
Subject: Gates high schools get mixed review in study michaelklonsky
Offline
Send Email
“[W]e concluded that the quality of student work in all of the schools we studied is alarmingly low,” the evaluation says. “This is not surprising, however, because students cannot demonstrate high-quality work if they have not been given assignments that require … deep understanding” and higher-order thinking skills.

Published: November 16, 2005
Gates High Schools Get Mixed Review in Study

By Erik W. Robelen
A new evaluation of the Bill & Melinda Gates Foundation’s campaign to improve U.S. high schools offers a decidedly mixed picture of the early returns the foundation is getting from the roughly $1 billion it has invested in the initiative so far.

The in-depth study, commissioned by the foundation and scheduled for release this week, identifies several shortcomings in the high schools that have been started or redesigned with the philanthropy’s support, especially in instruction and student performance in mathematics. It also cites signs of progress in reading and language arts, and concludes that many Gates-funded small schools, particularly those built from scratch, offer a positive learning climate for students.
For More Info

Evaluation reports on the Bill & Melinda Gates Foundation's high school initiatives are posted by the organization.

“In summary, the first returns are promising for English/language arts but worrisome for mathematics,” the researchers say in the report.

The study’s authors caution, though, that the achievement data available to them were insufficient to draw definitive conclusions about student performance. They supplemented the data with analysis of teacher instruction and student schoolwork to provide a more meaningful portrait.

Among the most disheartening findings of that analysis—and one the researchers said also applied to comparison schools in their study that do not receive Gates support—was the lack of rigor in teacher assignments and student work, especially in math.

“[W]e concluded that the quality of student work in all of the schools we studied is alarmingly low,” the evaluation says. “This is not surprising, however, because students cannot demonstrate high-quality work if they have not been given assignments that require … deep understanding” and higher-order thinking skills.
The study, the third in a series financed by the foundation, was conducted by researchers at the Washington-based American Institutes for Research, or AIR, and the Menlo Park, Calif.-based SRI International. It comes as the foundation is awarding new grants this fall aimed at helping districts with high school improvements, including the school systems in Portland, Ore., Los Angeles, and Atlanta, with more expected in the coming weeks.

Tom Vander Ark, the executive director of education initiatives for the Gates Foundation, said the evaluation’s findings were consistent with information he’s gleaned over the past couple of years, and he said the Seattle-based philanthropy had taken steps to adjust accordingly.
He cited, for instance, a greater emphasis on districtwide measures intended to improve the quality of curriculum and instruction, as well as an emphasis on using proven school models.
“I didn’t find anything surprising in the report,” he said in an interview last week. “I think it’s fair to say we’ve learned a number of important lessons, and it’s quite clear that over the last six years I created a number of grant programs that were well-intentioned but had some weak assumptions.”
Math Found Lagging

The Gates Foundation has committed about $1 billion since 2000 to support the start-up of small high schools or the restructuring of large existing schools into smaller units.
Gates officials often emphasize that school size is not an end in itself, but a tool to build strong relationships between teachers, students, and families, and to deliver a rigorous curriculum that is relevant to students’ lives.
Foundation officials say that, in all, the philanthropy has supported more than 850 new schools and nearly 700 existing high schools. Funding has gone to schools in nearly 275 districts. (Education Week also receives funding from the foundation.)
Return on Investment
An evaluation of the Bill & Melinda Gates Foundation’s national high school initiative finds that schools started or redesigned with the philanthropy’s support are achieving mixed early results in student performance, school climate, and academic expectations. Researchers stress, however, that achievement data are too limited to draw definitive conclusions.
Student Outcomes
• Achievement appears “promising for reading and English/language arts but worrisome for mathematics.”
• Overall quality of student work in both new and redesigned schools appears “alarmingly low.”
• Students typically began high school academically behind those in other schools in the same district.
School Climate
• Learning cultures marked by “close interpersonal relationships, common focus, and mutual respect and responsibility.”
• Sustainability of schools threatened by staff burnout, teacher layoffs, and pressures to swell class sizes.
• Attendance in new schools is strong, but redesigned schools “need to address attendance problems.”
Instructional Quality
• Rigor of math assignments was typically poor and “not significantly better” than other schools studied.
• English/language arts assignments were more rigorous and relevant to students’ lives than elsewhere.
• Curriculum materials and outside guidance on teaching math are “very much needed, as are well-qualified math teachers.”
SOURCE: American Institutes for Research, SRI International
In general, the nearly 300-page study found more positive outcomes, from achievement and student work to academic climate, in schools that were newly created than in those that had undergone redesign.
“Based on the data currently available, it appears that new schools have been successful with respect to attendance, test scores, and the quality of students’ work in English/language arts,” the researchers write.
Math appears to present the biggest problem, in student test scores as well as instruction and student schoolwork.
“In general, the math achievement level of students attending new schools is on par with or lagging behind other schools in the same district,” the report says.
The report provides standardized-test data from one or two school years from four unidentified urban school systems. The researchers also examined results from four other districts for purposes of analysis.
Attendance rates at new schools were generally higher than at other schools in the district. Average reading scores were lower than the district average in some systems, but that finding was reversed, the study found, after statistically controlling for the level of students’ prior achievement.
In two of the three districts where trends on state test data could be examined, the study found larger improvements in reading and English language arts achievement over time in foundation-backed schools than elsewhere in the district. The third district saw gains in that area “on a par with the rest of the district.” The reading gains were larger in the system with new schools than in the district with redesigned schools.
And the researchers stress that the Gates-funded schools tend to serve largely disadvantaged students who enter high school well behind grade level.
“When we look at 8th grade achievement levels, we find almost universally … students start substantially behind other students going elsewhere in the district,” said David A. Rhodes, an AIR researcher involved in the study.
‘A Huge Challenge’

For redesigned schools, the researchers were still more cautious about drawing conclusions on test-score data.
They found slightly lower-quality student work than in the Gates-funded start-up schools. But in examining the rigor of teacher assignments in math, the report says the results were “similarly poor” across the two school types.
“Half of the assignments at both types of [Gates-funded] schools exhibited little or no rigor,” the report says.
Mr. Vander Ark cited several factors that he believed contributed to the problems in math.
“Five years ago, the nation’s attention was … more on literacy than on mathematics,” he said. He also noted that in small schools, there may well be extra faculty support for addressing students’ language arts difficulties, since those areas overlap with several disciplines.
“The same is less true for mathematics,” he said. “In a small school, you may have two math teachers working on their own with students that are a number of years below grade level, and they face a huge challenge with limited support.”
As for student attendance, it was relatively strong at new schools but fell short at the redesigned campuses.
The researchers cautioned that the redesigned schools have a history of underperformance that must be changed.
“You really need to think of the redesign reform as on a longer time frame,” Mr. Rhodes of AIR said. “And that’s going to emerge out of an existing stream of data that, to be blunt, is going to be dismal to start with, because a school isn’t going to reinvent itself if it’s already doing a good job.”
Some of the most promising findings had to do with what the researchers termed “creating a culture for learning.”
Still, the redesigned schools were seeing what the researchers described as “slower progress as they work to change existing structures, cultures, and beliefs.” Those schools, too, showed signs of improvement over time, most notably in fostering the creation of a school culture in which students feel known by their teachers and supported by them both in academics and personally.
Strategic Shifts

Mr. Vander Ark of the Gates Foundation said the philanthropy has been shifting its grantmaking approach, particularly in the past past two years, to help address many of the problems the research spells out.
First, he said, “we try to approach school improvement in the context of a system, a school district. ... One way to think about it is, we took school-as-the-unit-of-change too far.”
Second, the foundation is focusing its school efforts on “well-specified school models that provide really strong support,” including a well-developed curriculum and instructional approach, Mr. Vander Ark said. Models he cited included the networks of schools affiliated with the Providence, R.I.-based Big Picture Company and the San Francisco-based Knowledge Is Power Program, or KIPP.
Mr. Vander Ark said the foundation has much more explicit expectations for school-level change than it initially did.
“Another thing I got wrong at the beginning was autonomy,” he said. “I visited 100 great schools and made the observation that they were all small, autonomous, and assumed that was a path to school improvement. It turns out giving a failing school autonomy is a bad idea.”
On the subject of change throughout whole districts, he cited the foundation’s $2.3 million grant last spring for a planning effort to improve Chicago’s districtwide approach to curriculum and instruction.
Another grant just issued, $1.4 million to the Atlanta public schools, will pay for district plans to redesign high schools’ curriculum and instruction, and will consider the role of new schools in that effort.
“With an investment of $2 million in a plan, you can help a city make better use of a billion-dollar budget,” he said. “Those plans typically incorporate small schools and small learning communities, but they pay more attention to curriculum and instruction.”
The AIR-SRI evaluation offers a series of recommendations, including a call for the foundation to support professional development, technical assistance, coaching around math content and instruction, and efforts to provide curricular materials.
It also focuses on the need for sustainability, noting that new and redesigned schools are “vulnerable organizations, with limited internal capacity and numerous external challenges.”
“The foundation and its grantees may want to focus more of their energy and resources on protecting the schools that have already been started,” the report says, “even if means starting fewer new schools.”
Vol. 25, Issue 12, Pages 1,20
Subscriber Options