Sixty-six college presidents refused to fill out the US News and World Report’s well-known college ranking survey, which was released May 3. In a letter, they called on fellow college presidents to follow their example. Each year college presidents, provosts and deans of admissions receive the peer assessment survey, which asks them to evaluate the academic standards of 260 higher education institutions across the country on a scale from “distinguished” to “marginal.” But many presidents argue that the survey is a waste of time and produces misleading results.

Many presidents say a five-point rating system is sufficiently nuanced to judge large institutions that may academically strong in some areas and weaker in others. In a letter initially signed by twelve college presidents, which is being sent to all universities involved in the survey, the rankings are described as “obscur[ing] important differences in educational mission in aligning institutions on a single scale.” The president of Goucher College in Baltimore, Maryland Sanford Ungar explains how impractical the survey is, “I would better be able to fill out a survey on refrigerators than on colleges I’ve never visited, never interacted with.”

In addition, the letter acknowledges the amount of PR, gamesmanship and even outright lying that colleges and universities engage in to boost their rankings. As the Washington Post reports every year colleges and universities send their counterparts brochures, CDs and other goodies that talk up their academic achievements. Several colleges, such as Clemson University and the University of Wisconsin, have admitted to underrating other colleges in order to achieve a higher ranking.

Inside Higher Ed published an article that shows just how subjectively each institution approaches the rating system.

The form submitted by the provost at the University of Wisconsin at Madison deemed 260 of its 262 peer institutions to be of “adequate” quality. A survey from the University of Vermont’s president listed “don’t know” for about half of the universities. The forms provided by Ohio State University’s president and provost were virtually identical. And the University of Florida’s president, like his highly publicized colleague at Clemson University, rated his own institution well above many of his competitors.

The peer assessment portion counts as 25 percent of the overall rankings. Defenders of the survey value the peer assessment because they are averaged together and are just a quarter of the entire ranking system. Additionally, some presidents believe in the accuracy of the scores if those taking the surveys only rate the institutions they are intimately familiar with and mark the “don’t know” option for the rest. University of Vermont’s Director of Institutional Studies Fred Curran said that he marked “don’t know” for 156 universities, rather than putting time into researching the institutions he knew nothing about. “I don’t think US News expects you to evaluate every institution,” he said.

In a blog post Bob Morse, the director of data research for US News and World Report, explains the reasoning behind the peer assessment survey, “U.S. News knows that peer assessments are subjective, but they are also important—a diploma from a distinguished college can help a graduate get a good job and gain admission to a top-notch graduate program.” Morse also emphasized that the rest of the ranking system, 75 percent, is “based on a formula that uses objective measures of academic quality such as graduation and retention rates, admission statistics, and financial and faculty resource data.” 

But the 66 presidents, who signed the letter boycotting the rankings, want a different assessment. Backed by the Education Conservancy, a 2004 non-profit working to develop a “robust, nuanced, and educationally sound web-based system” to accurately assess universities, the issue is finally getting some attention. The biggest challenge will be overcoming the traditionalism of a ranking system spanning more than two decades. Curran admits that opting out of the survey is inconceivable; “it’s something we’ve got to do — it’s been around long enough.”