Summer is coming to an end, students are returning to campus like swallows to Capistrano and College rankings are released. Useless News and World Misreport just published theirs. For some, this is but the deep breath before the plunge: the long awaited NRC rankings. This will be followed by Businessweek’s ranking of B-schools. Hard on the heels of each will be a flood of blood, bile and ink to drown them all (or not).
Rather than criticize the various rankings (others have already done that) I’d like to propose my own. For each pair of colleges C(i) and C(j), record the number of students who received an offer of admission from both in given time period. Denote by N(i,j) the fraction of this group who chose C(i) over C(j) and by N(j,i) the fraction who chose C(j) over C(i). Next, construct a directed graph with a vertex for each college and insert an edge directed from C(i) to C(j) if N(i,j) > N(j,i). For expositional convenience only, ignore the case N(i,j) = N(j,i).
The resulting directed graph, called a tournament in some circles, can be used to generate clusters of `like’ colleges and a ranking among clusters. One could pick as `winners’ the usual suspects: top cycle, Banks set or uncovered set. The delightful aspect of such an approach is that it relies on the revealed preference of students themselves. One could construct a similar directed graph (by college, by department), but use faculty employment choices instead. A side benefit is that it would provide another onanism opportunity for social networks scholars.
Eran
rvohra
[...] rvohra 2009-08-22 02:45:57 Here is the original: Rankings « The Leisure of the Theory Class [...]
Larry Blume was kind enough to point to a paper that does what is suggested. If an idea is good, then others are bound to have had it earlier!
The reference is:
Avery, Christopher, Mark Glickman, Andrew Metrick, and Caroline M. Hoxby, “A Revealed
Preference Ranking of American Colleges and Universities,” NBER Working Paper, 2004.
Aren’t their incentive issues here? Suppose as a B-school on the west coast, I find that candidate X is a great candidate to have, but has a spouse working on the east coast. In the current situation, I would want to make this person an offer (doesn’t cost me much, great if they came). In the new ranking model, I have the additional ranking issue to keep in mind and it may no longer be beneficial to make this offer.
Going forward, schools may react in different ways to improve their rankings. E.g. increase application fees and decrease tuition, send out some sort of pre-offers, or invite people for longer interviews, etc.
Not to say that the current ranking system does not create any perverse incentives. :)
Kunal is correct about the small sample problem. One could deal with this by reporting the raw numbers and letting the observer decide whether the differences are significant.
One could also aggregate by school. Or, focus on grad student choices.