Jubilee for Dysfunctional Institutions

AI and scientific funding

Are you involved with a dysfunctional institution, one of those founded or fattened during the postwar boom, already beginning to buckle in the 80s and 90s and now just shambling along, an incoherent mass of hotwired fixes to previous crises? Say, a medical system, a federal bureacracy, some kind of higher education?

Legitimacy produces a double bind for would-be reformers. These institutions, despite increasing irrationality, must still maintain the pretence of functionality in order to get everyone else to play along. Radical change is a repudiation of the recent past, of the people and decisions legitimated in the recent past. Unless you want to go full DOGE and embrace the de-legitimization of the institution you’re tasked with reforming, there needs to be a(n exogenous) reason why things used to be ok, but now they’re not.

So I’ve got good news. AI represents a jubilee for dysfunctional institutions! A mulligan for all of the accumulated institutional debt! You get to say: “Everything used to be fine, no bureaucratic absurdity here ;) but now we have to change everything because of AI.”

This is the healthy response. The unhealthy response is to deny that AI is changing anything but that you’re going to undertake dramatic retroactive changes to preserve the status quo functioning of the status quo. The latter strategy was undertaken by the European Research Council in changes to their flagship funding scheme, by far the most important source of academic grants in Europe.

In Things Will Have to Change, I predicted that there would be “at least a 50% increase above a linear extrapolation in the number” of submissions to the (social sceince) ERC grants. The Research Council was determined to prove me wrong, and retroactively extended the “cooldown” period for previous applicants who scored below a given threshold. In a dramatic example of cope, the justification for this change doesn’t mention AI, but it does re-affirm that the bureaucratic status quo was working fine, other than those nasty end users of the bureaucracy were cynically trying to use it normally. The text is remarkable:

The only remaining way to mitigate this situation is to reduce the number of applications. We have been relying on soft measures to try to achieve this by communicating the need for applicants to reflect carefully on the right timing of their application and the maturity of their scientific proposal. Unfortunately, the effectiveness of this approach is limited.

It’s not the granting agency’s fault — people just keep applying to (career-changing, 1.5-3.5 million euro) grants!

Some panels now have more than 250 applications to assess and we expect this upward trend to persist. [Here is maybe where you should mention AI lol.] We have increased the number of panel members, but there is an upper limit to the size of a panel, determined by practical constraints and how panels function as groups. There is also a limit to the number of proposals that can reasonably be discussed in the one-week sessions that the panels spend together.

The literature on scientific grants is clear that tortuously long single-person grants evaluated by committes that confer with each other is a hideous waste of time and a terrible way to efficiently allocate resources. The fact that the impediment to reform is the need to have “one-week sessions that the panels spend together” is just nuts: this process introduces a role for charsima, exacerbates groupthink and the possibility of corrupt log-rolling.

Once again, the functionality of the status quo is affirmed in the conclusion:

anyone applying this year should consider very carefully whether this is in fact the optimal time. In some cases, it may be wiser to postpone an application to allow the proposal to mature further and improve its chances of being selected for funding.

There are multiple claims here: that the “maturation” of a grant proposal is something that can take multiple years; that the grant evaluation procedure is legitimate enough to be able to accurately distinguish “mature” from “immature” grant proposals; and that “maturity” is somehow related to the scientific value of a grant proposal, what the researcher will actually do with it.

As far as I am aware, there is zero evidence for any of these claims. The first is tautological (maturation == getting older), and the third is meaningless (“maturity” being a bizarre concept here). But there is now compelling evidence against the second claim, at least now that everyone uses AI, from another of the European Commision’s flagship funding opportunities.

This comes from an excellent preprint by Martin Bulla and Peter Mikula analyzing the applications to the prestigious Marie Skłodowska-Curie Postdoctoral Fellowships. The percentage of proposals above a given score (out of 100) jumped in 2025, from its extremely consistent, reasonable 2018-2024 trend.¹ The time series demonstrates that in fact the system was internally consistent in the past — but that it absolutely no longer is.

Bulla and Mikula don’t pull punches; their paper is titled “When AI turns grant evaluation into a lottery.” The system now cannot distinguish between a huge percentage of applications clustered near the top of the distribution.

Grant applications are different from papers in that they don’t produce/contain knowlege themselves. Society is plausibly better off as a whole if more social science papers become better, even if it makes the peer review process more random: we’re randomly selecting from higher-quality knowledge producets. The same isn’t true for grant applications; this is purely a zero-sum game.

This is open-and-shut evidence that the system is broken — and that it specifically broke last year. I’ve heard reports from applicants at EUI that they scored 94 or 95 out of 100 and still weren’t above the bar to recieve the fellowship.

This data disproves the ERC President’s claim that “postpon[ing] an application to allow the proposal to mature further” does in fact “improve its chances of being selected for funding.” This is an egregious example of assigning indivudal blame for a systemic problem. The institution is bleeding legitimacy; we can all see that these applications are becoming lotteries.

And obviously, if it’s a lottery, the incentive for everyone is to buy as many tickets as possible. The visible degredation of the discriminatory function of the institution (that is, the ability to tell good applications from bad ones) produces exactly the wrong incentive; rather than reduce the number of applications, we’re going to see many, many more.

The second-worst response for dysfunctional institutions to AI is to ignore it and do nothing. The worst response is to ignore it and undertake blunt reforms aiming to preserve the crumbling status quo while actually making everything worse . In the ERC’s defense, they’ve retreated back to the second-worst response after serious pushback from academics…who were of course just mad that they personally wouldn’t be able to submit this year, something that many may have already started working on.

Ok, maybe I’m projecting here….I, personally, was mad that I wasn’t going to be able to submit this year, I was rejected two years ago and the proposal and been maturing so well…oh wait, no, I do social science and the social world changes. My proposal from two years ago isn’t mature, it’s completely spoiled, left to rot in the fields, dozens of hours of brutally tedious work for nothing because of the dysfunction of this system.

I’m bitter about this one — personally, it would’ve secured my current position for another four years, while now I face the prospect of needing to look for a new job as soon as 2029. And the content of the grant was both perfectly timed and substantively important; I was proposing to study political influencers on video-based social media. The grant was submitted in October 2024, a month before what has been come to be called the “podcast election” or the “influencer election”; of course, if we were serious about social science well, we would’ve had the infrastructure for studying this up and running before it became relevant…but a few months later isn’t too bad. 3 years later is in fact too bad, evidence of institutional dysfunction.

So, with that rant out of the way, let’s consider the AI Jubilee for Granting Agencies: “The previous system worked great, all of our decisions were correct, etc. But now AI is changing the dynamics of the system; we have to radically overhaul our workflow to adapt.”

And then actually adapt. Maybe use AI, maybe don’t; that’s a subject for another post. There’s a whole literature on scientific funding to investigate; now is the time to re-set the institution to the 21st century. The crisis of AI inverts the legitimacy trap for dysfunctional institutions; you lose legitimacy if you don’t undertake radical change.

1. Social science is in fact the least affected; the graph for physics is just ridiculous.