Every year, US News compiles its law school rankings by relying heavily on reputation ratings by law professors (mainly deans and associate deans) and practitioners and judges. They are asked to assign a score (from 1 to 5) for the roughly 200 law schools on the form. A 5 is the highest score and a 1 is the lowest. While many factors that go into the US News ranking have been criticized, the reputation ratings by and large are considered one of the best components in the ranking system. But should it be?
Let’s assume a knowledgeable dean filling out the form in good faith. How is he or she to go about filling out the form?
Here’s my hypothetical dean’s stream of consciousness:
Okay, I think Yale is the top law school, so I’ll give it a 5.
What about Michigan? Great school, but not quite as high as Yale. I’ll give it a 4.
Cornell is an excellent school too, one of the best. But it’s not Yale or Harvard, so I can’t give it a 5. It’s not as good as Michigan in my view, so I can’t give it a 4. I gave Penn and Berkeley 4’s too, and I think Cornell isn’t quite at the same level. So it’s a 3.
What about USC? Another excellent school, but it’s not as high as Cornell. So it’s a 2.
Ruh-roh! I’m not even out of the top 20, and I have 160+ law schools to assign scores to, and I only have one number left. But I must go on!
How about Emory? That’s a bit lower than USC in my view, so I’ll give it a 1.
What about American? Another terrific school, but I think Emory’s better. I can’t give American a 0. What do I do? Okay, I guess I’ll give it a 1 as well.
But I’m not even out of the top 50. Yikes! I’ve run out of numbers. Maybe I’ll call Robert Morse and ask him if I can start assigning negative numbers. What do I do?
Time to try some math. To make things easy, I’ll assume there are roughly 200 law schools. And I have 5 numbers to assign — 1, 2, 3, 4, and 5. Assuming an equal number of schools assigned to each number, that’s 40 schools for each number. OMG! So I need to give schools I rank 1-40 a score of 5, schools I rank 41-80 a 4, schools I rank 81-120 a 3, schools I rank 121-160 a 2, and every other school a 1. But that’s ridiculous. The law school I think ranks #40 isn’t anywhere near the law school ranked #1. This system is impossible.
Okay, maybe I give a score of 5 to the top 10 law schools, then a score of 4 to the next 90 law schools, then 3’s and 2’s to 100-150, and 1’s to the rest. But that still lumps too many schools together. If every person filling out the form did what I did, then there would be no way to distinguish the top 10, and no way to distinguish schools in the top 100. Only real outlier scores would determine the difference.
Dear Mr. Morse — what am I to do? Please help me!
Does anyone have any advice for our poor dean? How are people to fill out the US News ranking forms in good faith to reflect accurately their sense of law school reputations?
Originally Posted at Concurring Opinions
* * * *
This post was authored by Professor Daniel J. Solove, who through TeachPrivacy develops computer-based privacy training, data security training, HIPAA training, and many other forms of awareness training on privacy and security topics. Professor Solove also posts at his blog at LinkedIn. His blog has more than 1 million followers.
Professor Solove is the organizer, along with Paul Schwartz, of the Privacy + Security Forum and International Privacy + Security Forum, annual events designed for seasoned professionals.
If you are interested in privacy and data security issues, there are many great ways Professor Solove can help you stay informed:
* LinkedIn Influencer blog
* Twitter
* Newsletter
