Academia: from inefficient effectiveness to efficient ineffectiveness

The great failure of university reform wasn’t intensifying competition, but gormlessly ignoring the prior question: what institutions and public goods govern what competitive success looks like?

This essay was previously published on The Mandarin which has now put it behind a paywall. Hence this republication.

---

As we’ve reformed one area of our economy and society after another, we followed a formula. We presumed that the more a sector was made to look like a market, the better it would be. In academia, global competition has replaced a fusty, craft-based system. But without paying attention to the sector’s particular qualities, we didn’t alleviate its weaknesses and build on its strengths. Rather the opposite. The result is a travesty of the greatness that was possible.

The central task of an academic system is to identify the best research and direct resources to it. It’s also extremely difficult. At the forefront of knowledge, the best research often takes years to bear fruit. And many of the most promising leads will turn out to be barren. So how can you tell which academics and which projects to back?

The academic system of the 1950s was built around a centuries-old institution – the university. At least in its idealised form – expressed by the conservative political theorist Michael Oakeshott – a university was “a corporate body of scholars … a place where a tradition of learning is preserved and extended”. Oakeshott’s description of the nature of scientific endeavour suggests caution in assuming that it can easily be optimised whether that’s done via an improvised market or by managerialist dictate:

Scientific activity is not the pursuit of a premeditated end; nobody knows or can imagine where it will reach. There is no perfection, prefigured in our minds, which we can set up as a standard by which to judge current achievements. What holds science together and gives it impetus and direction is not a known purpose to be achieved, but the knowledge scientists have of how to conduct a scientific investigation. Their particular pursuits and purposes are not superimposed upon that knowledge, but emerge within it.

This system identified and promoted academic merit within the traditions of professionalism. Admitted to the academic guild with basic intellectual credentials (generally a bachelors degree until the 1960s) junior members of the guild were identified for promotion on merit by more senior academics. The best-enjoyed security, internal satisfaction and the approbation of their most respected peers – the wellsprings of what Adam Smith thought drove a good life in a good society.

This system was tolerably effective at allowing the best researchers the freedom to pursue their passions. But it isn’t hard to see its limitations. Like other professions, it often put professional comfort ahead of the wider interest in efficiency and those on whose behalf it worked. And crucial public goods within the research community on which science is built – such as peer review and the replication of previous studies – went unfunded. Academic journals also tended to publish positive results, when null results (where a hypothesis was tested but couldn’t be confirmed) are often just as significant.

Reform came from the 1980s on. It sought to expand access to university education and foster increased business innovation. Though reform was ostensibly pursued to promote the public interest, and though university research was and remains overwhelmingly funded publicly and via philanthropy, reformers’ imagination didn’t run to strengthening the public goods of peer review and replication. Indeed, simply intensifying competition further undermined them as various replication crises are illustrating today.

The tragedy is that, as the magic and miracle of cooperative peer production unfolded on the internet – on Wikipedia and in open-source software – even greater miracles must surely beckon in open science. The digital world could turbocharge scientific advance at every stage of the scientific endeavour. It could foster new data collaborations between scientists, other kinds of experts, citizens and institutions such as hospitals and the sharing and repurposing of data, software resources. And the internet could foster faster, cheaper, vastly wider publication (with a major expansion in pre-publication). But reformers look little notice. Today’s open science in academia is pursued by an adventuresome minority despite, rather than because of reform. It has received fairly marginal support from governments, academic institutions and researchers, caught up, as they now are, in relentless competition.

And the vacuum created by reformers’ disinterest in the public goods of research led to a field day for private monopoly. Academics and their libraries can’t do without access to academic journals, so they’re a captive market. So journals can exploit them by price-gouging. But this went against the professional ethics of the academic institutions that owned them. But the for-profit publishers that have now bought most of the prestigious journals have had no such scruples (See Figure). It’s as if we auctioned off the oxygen in our atmosphere to the highest bidder so they could sell it back to us.

Figure: Price per page of academic journals (for-profit and not-for-profit).

Source: Right to Research Coalition.

But the most important reform move was to reimagine how to identify and rank intellectual merit. Reform didn’t start from where it was – with a tolerable solution to a hugely difficult problem which could be improved by carefully fixing bugs and adding features. Instead, it made a beeline for a God’s eye view of the problem. What would God want from the university system? Why He’d want optimality. He’s a pretty optimal kind of guy himself. So he’d like to reward the best universities and the best academics. Well, that should be pretty straightforward. Let’s look around. Publication metrics look like they do the trick. What could possibly go wrong? Lots of things as it unsurprisingly turns out. And Goodhart’s Law is ensuring that decline proceeds apace.

There’s a deep irony here. Economists exalt the way markets do without a central planner picking winners – however authoritative they might seem. Rather, winners emerge from different valuations made by numerous parties from diverse perspectives. Yet change in the higher ed sector is driven by economists’ and policymakers’ fondest imaginings that they’re moving towards a market-based system.

In all this, what’s happened is illustrated by the image above in which birds wings are fitted to a plane. The dreamers and entrepreneurs who paved the way for human flight looked to birds’ wings as a potential model. But, as one can see from his drawings, even in the fifteenth century Leonardo da Vinci understood that this was just the start or, to use Edison’s language, the one percent inspiration awaiting the 99 percent perspiration to be realised in the world. One couldn’t transfer the insights embodied in bird’s wings without an immense amount of design and engineering work – the kind of work nature did to evolve birds’ wings and the aviation industry did to develop machines that really do fly.

When it came to universities, we simply took some features of a market and grafted them onto another, quite different system. We’ve done it in one area after another. And called it economic reform.