Truth Revealed: US News Rankings Mean Nothing–and Everything

Doug Lederer and the folks at Inside Higher Ed bring us a story today of Clemson University and how it manipulates data to help move itself up in the US News & World Report annual rankings.

These ranking bug me.  As an educational consultant, I am constantly having to explain that these rankings are at best imperfect measures of institutional quality and at worst amount to a completely misleading popularity contest.  Just today I am having to respond to a client’s whining that the universities I am suggesting are “too far down the league tables” (to which I might respond:  “so how come you didn’t think of that when you were preparing for your algebra final”).  My client doesn’t want to listen to any reasoned argument that the quality of education that an individual student received in the classroom has little or no bearing on the the league tables presented in US News.  Needless to say, it’s going to be an interesting day.

What has me looking up, however, is the fact that a former institutional researcher at Clemson gave a presentation at the Association of Institutional Research that revealed in great detail the strategies that Clemson administrators were using to lift Clemson in the US News rankings.  And the participants–other institutional researchers from other universities who also report data to US News–where shocked.  Shocked, I say.

You can read the article yourself:  Doug Lederer is a great writer and the article is balanced.

The funny thing (okay, well, it’s really not so hilarious–it’s more funny-peculiar) is that colleges and universities bash the ratings when they’re  down, and then post them on their websites anytime their name is mentioned favorably in those same rankings.

Everyone wants “proof” that the quality of their educational product is somehow good–better than their peers–more worthy of your consumer dollar.  Yet colleges and universities know that the measures developed by US News are flawed.  They know that they measure institutional inputs and not educational outcomes.  They know that statistics like student-to-faculty ratios are misleading.

Still, the college administrators slavishly report their data to the magazine editors–with or without manipulation or “influencing” the data.  They know it’s a stupid game, but they play it anyway.  And clearly Clemson has the rules down pat.

As Marcellus uttered in Shakespeare’s Hamlet, something is rotten–not in Denmark–but in American higher education.

I do understand this quest for some sort of evaluation system that will help us compare one college against the next so that we can make better decisions about which college is best for which kids.  But we don’t need US News & World Reports.  We need Consumer Reports.


Mark Montgomery
Educational Consultant




Technorati Tags: college, university, ranking, rating, US News, Consumer Reports, Clemson, Association of Institutional Research, Inside Higher Ed, Doug Lederer Del.icio.us Tags: college, university, ranking, rating, US News, Consumer Reports, Clemson, Association of Institutional Research, Inside Higher Ed, Doug Lederer

Quality of the Educational Experience: Questions to Ask Admissions Officers

A couple of my previous posts have focused on student-to-faculty ratios and class size averages as indicators of educational quality.
By and large, my conclusion is that these statistics are not all that helpful in uncovering the quality of the educational experience for a prospective undergraduate.
So what to do? How can we compare and contrast the educational services offered by different institutions? How can we get beyond these simplistic numbers?
Well, one place to start is the set of questions formulated by Loren Pope in his book, Colleges that Change Lives. I have already posted those here. The point is to ask more substantive questions about how courses are taught, and not to focus solely upon the size of the class.
But there are other questions that prospective students and their parents might ask in order to understand and predict what their experience might be like if they matriculated to a particular college or university.

  1. What is the “course load” for faculty members? How many undergraduate courses is each faculty member expected to teach every year? Is this number the same for all departments, or does it vary by discipline?
  2. What percentage of courses is taught by adjunct or part-time faculty? Is this percentage the same across departments, or do some departments rely on part-timers more than others?
  3. What percentage of courses has a waiting list? How does the college (or the individual department) determine who gets off the waiting list and when?
  4. How many campus lecture halls seat 100 students? 50 students? 500 students? How many seminar rooms are there? What is the ratio of lecture halls to seminar rooms?
  5. How small must a class be before it is canceled by the administration?
  6. What is the most popular course on campus (or in your department)? Is enrollment in this course capped? Who gets in, and who doesn’t?

As you ask these questions, you should not permit admissions people to give vague answers. They will hem and haw. They will will be imprecise.
But these figures do exist on campus, and they exist in the office of institutional research. The administrators in this office crunch the numbers. They have the facts. If you really want to know this information, you need to ask to be put in touch with someone in institutional research who can provide this information to you.
Still, you must not focus solely on the statistics. Every educational institution has administrative problems like the ones cited at Dartmouth in my previous post. You will never be able to guarantee that your son or daughter will not be shut out of a class. You will not be able to ensure that every course he or she takes is a gem.
But if you dig deeply enough, you may be able to figure out how forthcoming and honest the faculty and administration is about these difficulties, and you will learn how they are working to solve them.
Mark Montgomery
Independent College Consultant and
Former College Administrator