What International Education Rankings Don’t Measure

What International Education Rankings Don’t Measure

What International Education Rankings Don’t Measure

PISA’s reductive test score dumps prevent nuanced debate. 

Copy Link
Facebook
X (Twitter)
Bluesky
Pocket
Email

Every three years a division of the OECD called the Programme for International Student Assessments, or PISA, tests 15-year-olds around the developed world on math, reading and science. The report’s subsequent publication unfailingly stirs up a frenzy of politicians seeking soundbites, reformers grasping for useful outrages, and reporters scrounging for sensational angles.

This year the United States placed in the middle of the pack in reading and science, and below average in math, prompting headlines like “US High-School Students Slip in Global Rankings,” as The Wall Street Journal fretted. But lost in the hubbub over the performance of 6,100 US teenagers on one-off exams is an account of what’s really wrong with American education.

Mainstream outlets dutifully grab blurbs from the usual suspects in the reform crowd and their counterparts in labor and progressive education circles. One camp calls for greater accountability measures, such as Common Core standards, while the other asks us to address childhood poverty and pay teachers more. Then they return to the rankings.

The problem is that the undue focus on test scores gets in the way of nuanced debate and automatically puts opponents of accountability and market reforms on their heels, since testing sends the first volley: our kids are flubbing some huge important test.

Still, PISA’s voluminous reports provide some grist for thoughtful analysis. As Diane Ravitch and others note, it was ever thus in international test rankings for the US, which “has NEVER been first in the world, nor even near the top, on international tests,” Ravitch writes. To best improve scores, she recommends addressing the near-quarter of US children in poverty.

The PISA report leaves no question as to poverty’s importance: “socio-economic disadvantage translates more directly into poor educational performance in the United States than is the case in many other countries”—about 15 percent of variation by its calculations.

Others emphasize policy conclusions that PISA draws: children attending pre-kindergarten perform better across most OECD countries, and higher-performing countries tend to pay their teachers quite well. “The United States,” says the report, “spends a far lower proportion than the average OECD country on the salaries of high-school teachers.”

But it remains unclear whether we should even waste our breath on PISA results. Researchers Martin Carnoy and Richard Rothstein with the progressive Economic Policy Institute threw a wet blanket on the “ideological hyperventilation exercise.” Months after warning that 2009’s results betrayed sampling errors, they accused the US Department of Education and PISA of “planning a highly orchestrated event, ‘PISA Day,’ to manipulate coverage of this release.”

PISA sent advance results to a select cadre of organizations, such as the Alliance for Excellent Education, known for championing accountability measures. Writing for Slate, Dana Goldstein notes that a number of the participants in PISA Day happened to also be prominent backers of Common Core academic standards.

The affinity between PISA and US accountability mavens is no clearer than in the little remarked-upon fact that an entire chapter in PISA’s “Lessons” for the United States was penned by Jason Zimba, the “guru” behind the Common Core mathematics language and one of the standards’ co-founders. Tasked with examining the pertinence of Common Core to PISA results, he found that if the United States were to implement the standards faithfully, “PISA scores would improve.”

It’s a remarkable collaboration. A supposedly aloof transnational NGO assigned the capstone chapter of its US policy recommendations to a paid advocate and author of a politically contested education initiative. They delegated research tasks to a guy who co-founded an education technology company, then a “leading provider of assessment reporting,” that was later bought by the second-largest textbook publisher in the US.

PISA claims to provide “a powerful tool that countries and economies can use to fine-tune their education policies.” But more important is what they miss. PISA didn’t ask Bruce Baker of the labor-affiliated National Education Policy Center to write a chapter on equitable allocation of school resources, even though PISA declared “33% of the variation in mathematics performance can be explained” by how well-resourced principles consider their schools to be. A 2012 report Baker co-authored on widespread funding inequities lauded Massachusetts, which produces some of PISA’s highest achievers. It dinged Florida, which posted the weakest scores of the three individual states PISA measured, for “a substantial decline in state effort and funding level.”

PISA didn’t invite Gary Orfield of UCLA to discuss the defining feature of American education: segregation. PISA only mentions segregation to note that “school choice—and, by extension, school competition—is related to greater levels of segregation in the school system, which may have adverse consequences.” The report doesn’t specifically address the deepening segregation in US schools, which are as segregated now as they were when de jure apartheid still held sway.

Attention College Students: Get six months of The Nation’s digital edition absolutely free!

PISA didn’t ask Elaine Weiss of Bigger Bolder Approach to Education to examine the way choice and competition in the US affect school performance. BBA has released blistering reports on the failures of market-oriented solutions promulgated under the Obama administration, which has cited international test scores as justification for its initiatives. PISA, however, finds “there is no evident cross-country relationship between the degree of competition among schools and student performance.”

No one at the ACLU was asked to discuss the school-to-prison pipeline that burgeoned with the expanded use of zero-tolerance discipline in US schools. Nor was anyone called to examine US’s singular system of mass incarceration, which makes it 7.5 times more likely for black children to have a parent in prison than white children.

Nor does any part of the PISA report deign to mention arts and music education in schools, which, unlike international rankings, constitutes a legitimate crisis in US education. In the era of standardized-test based reforms, children of color in particular have seen their arts and music education opportunities dwindle to about half those of white students despite the well-established benefits of creative arts in education, even for standardized exams.

With PISA, test scores get the first word, and they’ll eventually get the last, since free-market-minded reformers will flog us with warnings of “lagging scores” and “stagnation” for years to come. Meanwhile, equity-minded public school advocates will move on to things that matter.

Pedro Noguera on the need to move beyond the Common Core.

Ad Policy
x