ON MY MIND
MILLARD ARNOLD: The folly of rankings
The problem with surveys based on reputation, as Malcolm Gladwell wrote, is that they’re essentially just a compendium of prejudices
On behalf of the Sunday Times, market research agency Kantar TNS recently conducted a poll of 3,500 people older than 18 to determine the best fast-food restaurant in SA. The winner was KFC, which also won the overall grand prix award, making it SA’s favourite brand.
In its media release, Kantar said that not only were 3,500 people polled to find the winners for the 32 different consumer categories, but "468 business leaders and decision makers were interviewed to establish the top brands across 12 business categories".
It was a marvellously impressive exercise with an equally satisfying result: a thorough, systematic and exhaustive methodology that identified the number one brand in SA.
As a society, we like knowing what is considered the best, so surveys resonate; we love to argue over who is first or last, who is top or who hasn’t got a clue.
So, it was with great interest that I read the Financial Mail’s annual ranking of business schools. I must say, I have a more than passing curiosity, given that I serve as the CEO of the SA Business Schools Association.
The survey was intriguing: there were numerous charts that demonstrated with some earnestness that careful consideration had gone into a process that helped determine SA’s business schools with the best reputation. A number of graduates were polled, private sector executives were questioned and the schools were interviewed.
The central theme on which the business schools survey focused was the issue of reputation. In this day and age, it may well be of paramount importance. However, if you’re convinced a good reputation is a sure-fire indication of quality, you would do well to remember that, until recently, Bell Pottinger, KPMG and McKinsey all enjoyed significant prestige.
With business schools, there is a certain gibberish in equating reputation with quality. It is quality that determines excellence. Ideally, reputation follows quality.
The difficulty with the rankings is that they fail to address the fundamental issue of what constitutes quality education. The fact is, there is no satisfactory way to measure the quality of an institution — to determine how well it does in challenging, inspiring and educating its students.
The inherent use of reputation as a synonym for quality is dangerously misleading
Unfortunately, the relationship between reputation and quality is never addressed or articulated in the Financial Mail survey. Consequently, to conflate reputation with quality is an unforgivable mistake.
I am an enormous fan of Malcolm Gladwell’s incisive intellect and penetrating analysis.
Writing for the New Yorker magazine, Gladwell wrote: "Sound judgments of educational quality have to be based on specific, hard-to-observe features. But reputational ratings are simply inferences from broad, readily observable features of an institution’s identity such as its history, its prominence in the media or the elegance of its architecture. They are prejudices."
The difficulty is that no matter how objective they try to be, rankings drive reputation. What is missing from the survey is some definitive way of measuring educational outcomes. Without some way to determine quality, as Gladwell says, "our rankings are inherently arbitrary".
Fanning the flames of elitism
Rankings are a disservice to the schools, too. The survey attempts to distil the complexities of multiple institutions down to a single figure without taking into consideration a multitude of extraneous factors, such as costs, location, context, size and student profile.
In an environment in which issues of accessibility and affordability have reached crisis point, the implicit ideology couched in reputation fans the flames of elitism.
Rankings in and of themselves are not necessarily a problem. In this case, however, the inherent use of reputation as a synonym for quality is dangerously misleading.
A survey that really addresses the value of business schools to the advancement of SA society would be a far better and more useful contribution than rankings based on reputation.
• Arnold is the CEO of the SA Business Schools Association