(Updated for 2020) This is a public service I’m providing as an inoculation against the annual fall media college ranking mania. It is a reminder that rankings are flawed and have their limitations. While they can provide useful information, too few people bother to look beyond the ranking order to evaluate the data used in creating the rankings.
To get an idea of what I’m talking about, let’s take a look at the following list of the 10 worst colleges (yes, there are more than 10 because of ties):
Think something is missing? So you really do take the time to look at the methodology of the rankings when they’re released? Or is it that there’s no way you can believe the listed schools could be the worst at anything?
I’ll give you the benefit of the doubt and explain my methodology. There are 79 colleges and universities with 500 or more full-time undergraduates (excluding military academies and theological institutions) that have a four-year graduation rate of 80% or better. (That’s right, out of over 1,600 colleges, fewer than 100 graduate 80% of their students in four years.) The worst colleges have the lowest percentage of undergraduates receiving Pell Grants.
Is it Flawed?
Don’t think ranking colleges on a single variable is fair? Or at the very least, I should qualify the title somehow? But US News Best Colleges Rankings doesn’t say “based on the subjective opinion of higher education officials with incentives to game the system?”
Think I should provide more information on the variable such as the average for all schools or at least the range? Yet, Money Magazine’s Rankings don’t give any averages or ranges for its data. Users have no idea of the percentage of alumni found in the PayScale database for each school. For some schools, I would think there wouldn’t be enough listings to try to adjust earnings by major although it’s an excellent idea.
And their methodology includes Peer Quality which is split between test scores and yield rate. I don’t understand why they bother with their focus on outcomes. After all, you could argue that the best peers are the ones who get great jobs after graduation and offer plenty of networking opportunities. Do their test scores really matter then?
But if you’re curious, for the 79 schools the percentage of undergraduates receiving Pell Grants ranges from 10% to a depressingly high of 25%. The average for all colleges with at least 500 full-time undergraduates is 36%.
Think that the comparison should include more than 79 colleges? But then how would I make sure that I compare colleges with similar resources? Do you really think that a public university with open admissions should be compared to an Ivy accepting less than 10% of students?
Yeah, there are all kinds of problems with my 10 worst colleges rankings. However, there’s nothing wrong with the numbers–you just may not agree with how I used them to rank colleges as the worst. Of course, that was the point of this exercise.
When you use college rankings, stop and make sure you understand
- exactly what is being ranked;
- how the information is collected;
- and if the differences really matter to you.
Just in case you’re curious, the Best 10 Colleges would be:
You can create your own rankings using the data from the DIY College Rankings Spreadsheet.