This is a public service I’m providing as an inoculation against the annual fall media college ranking mania. It is a reminder that rankings are flawed and have their limitations. While they can provide useful information, too few people bother to look beyond the ranking order to evaluate the data used in creating the rankings.
To get an idea of what I’m talking about, let’s take a look at the following list of the 10 worst colleges:
Think something is missing? So you really do take the time to look at the methodology of the rankings when they’re released? Or is it that there’s no way you can believe the listed schools could be the worst at anything?
I’ll give you the benefit of the doubt and explain my methodology. There are 78 colleges and universities with 500 or more full-time undergraduates (excluding military academies and theological institutions) that have a four-year graduation rate of 80% or better. (That’s right, out of over 1,500 colleges, less than a 100 graduate 80% of their students in four years.) The worst colleges have the lowest percentage of undergraduates receiving Pell Grants.
Is it Flawed?
Don’t think ranking colleges on a single variable is fair? Or at the very least, I should qualify the title somehow? But US News Best Colleges Rankings doesn’t say “based on the subjective opinion of higher education officials with incentives to game the system?”
Think I should provide more information on the variable such as the average for all schools or at least the range? Yet, Money Magazine’s Rankings don’t give any averages or ranges for its data. Users have no idea of the percentage of alumni found in the PayScale database for each school. For some schools, I would think there wouldn’t be enough listings to try to adjust earnings by major although it’s an excellent idea. But if you’re curious, for the 78 schools the percentage of undergraduates receiving Pell Grants ranges from 6% to 27%.
Think that the comparison should include more than 78 colleges? But then how would I make sure that I compare colleges with similar resources? Do you really think that a public university with open admissions should be compared to an Ivy accepting less than 10% of students?
Yeah, there are all kinds of problems with my 10 worst colleges rankings. However, there’s nothing wrong with the numbers–you just may not agree with how I used them to rank colleges as the worst. Of course, that was the point of this exercise.
When you use college rankings, stop and make sure you understand
- exactly what is being ranked;
- how the information is collected;
- and if the differences really matter to you.
Just in case you’re curious, the Best 10 Colleges would be