The American obsession with rankings and the “competition” that rankings are believed to spur is so fervent, so strong and so overwhelming that any ranking–no matter how unscientific, how thoroughly biased or how flawed–is bound to get press. The college football and basketball rankings, the US News & World Report’s annual college rankings, hospital rankings, etc., etc., etc. So many rankings exist that they have obscured the more important question of whether all universities, hospitals or whatever are competent at all. After all, overall competence and excellence is the goal of standards, not individual competence or excellence. Who cares if you have 50 good universities (or even 200), if 1000 universities are lousy? A thousand competent hospitals do much more for public health than five stellar ones.
The 14th annual  City Crime Rankings: Crime in Metropolitan America published by Congressional Quarterly threatens to be the most irresponsible of such rankings primarily because it is published by a trusted source. When a report on crime is anticipated by such criticism from the highest sources on criminology, the reasons why anyone gives it press become baffling. The criticism included in the above article include:
The study drew harsh criticism even before it came out. The American Society of Criminology launched a pre-emptive strike Friday, issuing a statement attacking it as “an irresponsible misuse” of crime data. Â
…
Critics also complain that numbers don’t tell the whole story because of differences among cities.
“You’re not comparing apples and oranges; you’re comparing watermelons and grapes,” said Rob Casey, who heads the FBI section that puts out the Uniform Crime Report that provides the data for the Quitno report.The FBI posted a statement on its Web site criticizing such use of its statistics.”These rough rankings provide no insight into the numerous variables that mold crime in a particular town, city, county, state, or region,” the FBI said. “Consequently, they lead to simplistic and/or incomplete analyses that often create misleading perceptions adversely affecting communities and their residents.”
But, most appalling is the CQ Press’s own insistence and admission that the study is worthless:
 CQ Press spokesman Ben Krasney said details of the weighting system were proprietary.   Â
Thus, the publisher asserts that it is propounding a hypothesis–their statistical model–that they refuse to have tested independently. This runs contrary to all scientific principles of inquiry. In all sciences–including social sciences like sociology–openness is the most essential ingredient. Without it, hypotheses cannot be tested independently in order to be verified. Without independent verification, there is no truth. By making the hypothesis proprietary, CQ Press is disallowing scrutiny and forbidding independent verification. Thus, CQ Press propagates an untested notion, a falsehood. In effect, it is acting as a propagandist.Â
Hopefully other silly rankings will soon be discredited by similar confession of inadequacy by their publishers.Â
Like this:
Like Loading...
