Eroding our independence

In today’s landscape, the mathematical community is split. Some are AI evangelists of various flavours and stipulations; others are AI pessimists who nevertheless use LLMs frequently in their mathematical work; still others are skeptics who abstain from LLM use entirely. Probably a plurality is undecided, waiting from orders from above to tell them where the field should go from here. Of these positions, it is the first position that strikes me as the most incoherent and indefensible. I believe that AI evangelists are making an extraordinary tactical mistake. At best, they’re naive. At worst, they are morally bankrupt and selling out our field to external parties.

AI companies view research mathematics as an opportunity to advance their own ideological and material interests. The strategy is clear. They give us free subscriptions; a handpicked cadre even gets access to frontier models. They film well-known mathematicians in product advertisements, saying things like “We lived in a world of cognitive friction until very recently where every task required us to use our brain but now we have AI and the other technologies that can bring these frictions down to zero”. They offer us scientific partnerships so that they can extract prestige from our scientific work and manufacture more consent for their products. They tell the public that LLMs have solved mathematics, though this is far from true (and also an incoherent statement in its own right). They misrepresent our field, the goals of our field, and the scientific training which is core to our work; they devalue key aspects of our labour and present mathematics as a theorem-producing factory. In their campaign of scientific misinformation, they presume to tell the public and ourselves what the real goals of research mathematics are. This messaging campaign is not inconsequential; it may genuinely mislead grant-funding agencies and universities. These are our real lives that I am talking about here. Nevertheless, some of us go along with it and amplify the rhetoric of this industry. It is appalling.

Enter your email if you want to receive these essays in your inbox.

AI companies need to execute this marketing campaign; they need to dazzle the public with newly solved Erdős problems and the messaging of scientific progress.1 They need to do this, because these companies know they are also immensely unpopular. Most people do not want their jobs to be replaced by machines which outperform humans “at most economically valuable work”.2 Most communities don’t want extractive data centres which create severe noise pollution and strain local resources; in fact, they stage grassroots campaigns to kill such projects. Most people who peel back the labour practices of this industry will find the results morally repugnant. Only recently, I discovered that data-aggregation companies are surveilling low-paid garment workers in India so that they can harvest training data in the hopes of automating them out of employment.

Will AI companies offer solutions to the problems that they are continually creating? I certainly wouldn’t rely on them to do so. Intellectual seriousness is in short supply in San Francisco, as is any kind of moral integrity. Grave concerns about the sociopolitical implications of AI remain unaddressed; instead, they are folded into the nebulous “technical” issue of alignment. At best, AI evangelists handwave at universal basic income (UBI), as if this policy addresses the issue of labour disempowerment or the baseline reality that the creation of this technology is undemocratic, unsafe, and widely unpopular.

Where are mathematicians in this? We too are in the crosshairs. At some point in the last few weeks, OpenAI held an invite-only workshop for a selection of mathematicians to discuss the future of their field. The premise of the workshop was to imagine an artificial intelligence tool which was “robustly superhuman” in mathematics; participants then discussed the implications. More details can be found in this blogpost of Daniel Litt and this Twitter thread by Lionel Levine. This sort of professional meeting should strike us as extremely worrying. Important discussions about the future of our field are being initiated at the behest of an AI company, the details of which are largely secret. It appears that OpenAI handpicked some selection of mathematicians who participated in that discussion. We should all be disturbed by the idea of a “smoke-filled backroom of mathematicians convened by” Sébastien Bubeck3 and Jacob Tsimerman, as Levine dryly puts it.4 As I write this, we still don’t know:

• who went to this conference;
• when it happened;
• the talks contributed at this conference (other than the information which Levine and Litt generously disclosed);
• or how OpenAI determined how conference attendants were selected.

Alarm bells should be going off now. This is a takeover by outside actors. No scientific community should be so intertwined with the material interests of four companies.

AI uptake erodes our independence

I believe that it is a tactical mistake for mathematicians to embrace the use of LLMs as they are available to us today. Doing so endangers not only our labour and our working conditions, it endangers our autonomy as a scientific community.

Imagine a scenario where mathematicians regularly use LLMs to:

• create novel mathematical research and prove theorems and lemmas;
• understand and digest mathematical content, as in this blogpost of Terence Tao;
• referee and peer-review papers; and
• produce teaching materials for students.

Notice that in this scenario, we depend on LLMs for basically every aspect of our scientific labour. Elsevier and Co. looks like a drop in the bucket compared to today’s cabal of AI companies. No sane person would be okay with a state of affairs where four companies have this much leverage on an independent scientific community. Yet this is the scenario that AI evangelists assure us is the “future of our field”. We need only learn to stop worrying and love the bomb.

Tactical intelligence is thin on the ground right now. I have been told in the future, journals will Lean-verify every paper before publication, so that we have independent verification that the results are correct. Of course, the Lean verification will be performed by some LLM. Of course, this mires us further in a state of financial and scientific dependence on outside actors. What is more worrying: a culture of folklore, or a scientific field that depends on AI companies for peer review and publication?

Of course, one option is that the scientific community creates its own models for scientific practice (one hopes with more guardrails). Probably we should all be talking to our colleagues who have developed computer algebra systems and open-source software about where to go from here. Even then, I feel we must be extraordinarily careful about outsourcing key aspects of mathematical practice to a technology. If we cede too much control to a technology, then the technology makes critical decisions about the practices of our scientific field. It chooses specific proof routes. It changes the macroscopic economic incentives of our field, punishing mathematicians who excel at LLM strengths (e.g. problem-solving) and rewarding mathematicians who excel at LLM weaknesses (e.g. theory-building). In doing so, it devalues human mathematical work, irrespective of the actual value of that work for human mathematical understanding. (Again: look at problem-solving. I think problem-solving ability is very good for mathematical understanding5.) If a technology is not thoughtfully designed and used, then its use may not align with the mission of our scientific community. In fact, it could directly undermine that mission.

Even overreliance on LLMs for understanding and digesting theorems can harm our mathematical practice in subtle ways. Querying LLMs replaces and undermines human-to-human mathematical communication which strengthens our community’s collective understanding. It makes our mathematical practice more isolated, solipsistic, and atomized. It may well homogenize our mathematical culture in ways we cannot perceive well as individual users. These are not theoretical harms; if you look at the activity on MathOverflow in the last 10 years, over five times fewer questions were asked during August 2026 compared to August 2020. We are talking to each other less and querying AI chatbots more.

You either care about this or you don’t. You either want to be a scientifically independent community or you don’t. To have a choice is an important thing, to me perhaps the most important thing.6 I refuse to use LLMs if it involves giving up my scientific autonomy. I do not want our field at large to be ensnared in a position of intellectual, cognitive, and financial dependence on a handful of external actors.

A message to the AI evangelists

It’s not too late. I don’t want to castigate mathematicians who have thoughtfully benchmarked these models in the past few years, tried these tools themselves, or engaged with AI companies. I know of a few AI evangelists with large social media presences who have done a lot of good in defusing widespread messaging about how “math is solved”. It may well be true that a randomly-selected mathematician who has engaged with AI tools in the last few years is more likely to be socially conscious and engaged with our wider society than one who hasn’t.

But it is time to wake up and evaluate where we are today. The time for thoughtful benchmarking has come to an end; we should not be promoting a technology that co-opts and threatens so much of our scientific practice. Stop collaborating with AI companies; stop thoughtfully engaging with them; and start taking an explicitly adversarial stance. Our independence and autonomy depends on it.

Further reading:

• AI Alignment as a Thought-Terminating Cliche, by Fernando Borretti.
• Where I stand on AI, by Vladimir Lazić. I do not agree with everything in the essay7 but found it interesting reading.

• The third footnote of “Some explicit counter-examples to Weibel’s conjecture” by the mathematician Shane Kelly:

Mathematicians are currently being instrumentalised in a large scale advertising campaign by AI companies competing for market monopoly. Public grant money is being paid to companies which then receive free advertising from us, often while showing little regard for our research priorities. More seriously, as part of this advertising campaign, some companies are pushing misinformation about the goals of mathematical research. This misinformation has the real potential to influence government funding decisions in a way that funnels money away from scientific goals, and towards private interests. This advertising campaign has more serious collateral damage. Mathematics depends on a human research ecosystem which continually generates new problems and sustains the community capable of recognising and pursuing them. Human generated research problems are a scarce and essential part of this mathematical ecosystem. Conceptually incoherent and empirically unsupported rhetoric portraying mathematicians as replaceable threatens a pipeline of early career researchers already in precarious employment conditions. Damaging either undermines the infrastructure on which future mathematical progress depends.While the author does want to be transparent about the use of computer assistance in the preparation of the current manuscript, he does not want to participate in this advertising campaign and contribute to this destruction of the commons. As such, the model and company name will remain absent from this manuscript, but are available upon request.

This article may eventually be published on Proofs and Prompts, which has been so successful that it now has a large backlog of posts. Having discussed with their editors, I have decided to post this essay on my personal blog for the time being, with comments turned on. I encourage you to follow Proofs and Prompts for communal discussions of AI and mathematics.

1

After all, their promised medical breakthroughs have yet to materialize, and it seems more difficult to safely automate a wet lab than a notebook.

2

See OpenAI’s charter.

3

Notice that OpenAI chose two former mathematicians that they employ to organize this workshop. I’m certain this is at least partially a strategic move on OpenAI’s part in order to gain more trust from the mathematical community; we shouldn’t fall for such tactics. Sébastien Bubeck is not our ally.

4

Though I find this state of affairs appalling, this certainly is not an invitation to harass the mathematicians who voluntarily disclosed their attendance at this workshop and shared their conference talk materials. I appreciate the mathematicians who did so and thank them for their transparency.

5

I feel especially bad for combinatorialists, who were already belittled by some practitioners of other fields before the advent of LLMs. I do not think we should degrade our colleagues or take comparisons about “theory-building” vs “problem-solving” too far; such comparisons only hold up to a degree before they ring outright false.

6

I have thought a lot the last few days about a 2024 essay by Ted Chiang about AI art, in particular the following quote:

Art is notoriously hard to define, and so are the differences between good art and bad art. But let me offer a generalization: art is something that results from making a lot of choices. This might be easiest to explain if we use fiction writing as an example. When you are writing fiction, you are—consciously or unconsciously—making a choice about almost every word you type; to oversimplify, we can imagine that a ten-thousand-word short story requires something on the order of ten thousand choices. When you give a generative-A.I. program a prompt, you are making very few choices; if you supply a hundred-word prompt, you have made on the order of a hundred choices.

Though we are mathematicians, not artists, we should bear in mind that when we use a technology carelessly, we are also giving up many choices. When a user prompts an LLM “Prove this conjecture! Make a breakthrough!” and digests the output, they have voluntarily given up many fine-grained decisions about how to prove the conjecture, how to communicate the proof they pursued, and how to conceptualize the proof. What would have been many micro-decisions is reduced to a single prompt. In the pursuit of a new proof certificate, the user has (unwittingly or not) ceded a considerable amount of intellectual autonomy.

7

This is generally true for all essays in my “Further reading” section.