The Supreme Court will hear oral arguments in the University of Michigan affirmative action cases on April 1. In these cases, the white complainants argue that it is fundamentally unfair that Michigan accepts black applicants with lower SAT scores (or LSAT scores at the law school) than some whites who are rejected. But a new analysis of the SAT that I conducted reveals something startling: Every single question carefully preselected to appear on the test favors whites over blacks. These data have the potential to reframe the affirmative-action debate, especially if they spark advocates to ask the iconoclastic question, What’s wrong with admitting some black students with lower SAT scores, when every question favors whites?
On the October 1998 SAT, for example, every single one of the 138 questions (sixty math and seventy-eight verbal) favored whites over blacks. By favoring whites, I mean that a higher percentage of white than black students answered correctly every question prescreened and chosen to appear on that SAT. I call these “white preference questions.” This is not a quirk of one particular SAT form. SAT forms are designed to very strongly correlate with one another. And the pattern I’ve identified is a predictable result of the way the tests are constructed. Latino test-takers are similarly affected, faring only a bit better than blacks.
I don’t believe that ETS–the Educational Testing Service, the developer of the SAT and the source of this October 1998 test data–intended for the SAT to be a white preference test. However, the “scientific” test construction methods the company uses inexorably lead to this result. Each individual SAT question ETS chooses is required to parallel the outcomes of the test overall. So, if high-scoring test-takers–who are more likely to be white–tend to answer the question correctly in pretesting, it’s a worthy SAT question; if not, it’s thrown out. Race and ethnicity are not considered explicitly, but racially disparate scores drive question selection, which in turn reproduces racially disparate test results in an internally reinforcing cycle.
Here’s a verbal question that illustrates the SAT’s skewed test construction process:
The actor’s bearing on stage seemed _____; her movements were natural and her technique _____ .
(A) unremitting … blasé
(B) fluid … tentative
(C) unstudied … uncontrived (correct answer)
(D) eclectic … uniform
(E) grandiose … controlled
This looks like a typical SAT verbal question. Yet this question differs from others in one important respect: according to ETS, “8 percent more African-Americans than whites answered this question correctly.” I call this a “black preference question.” I don’t know why blacks did better here, but nearly all SAT questions capture something about race that can’t be determined until pretesting. Because it favored blacks, who score lower on the test overall, this “actor’s bearing” question, which was pretested by ETS in 1998, did not favor high scorers and therefore was rejected for use on the SAT. I have identified several other examples, including a black preference SAT math question, that were rejected.
My considered hypothesis is that every question chosen to appear on every SAT in the past ten years has favored whites over blacks. The same pattern holds true on the LSAT and the other popular admissions tests, since they are developed similarly. The SAT question selection process has never, to my knowledge, been examined from this perspective. And the deeper one looks, the worse things get. For example, while all the questions on the October 1998 SAT favored whites over blacks, approximately one-fifth showed huge, 20 percent gaps favoring whites. Skewed question selection certainly contributes to the large test score disparities between blacks and whites.
Supporters of affirmative action can take solace in the broad cross-section of higher education, labor and business institutions that filed amicus briefs favoring the University of Michigan in the Supreme Court cases. But defenders of affirmative action in university admissions have shied away from confronting the term “preferences” and from vigorously attacking the assumption of test neutrality. It is essential that we document the plethora of powerful white preferences throughout the admissions process, ranging from disparities in family wealth and income to unequal K-12 education–and, we now know, biases in SAT question selection. Affirmative action, at the very back end of this entire process, serves as only a partial corrective to the considerable white preferences preceding it.