More calls to action for the mathematical community

This post is a sequel to this polemic, where I argued that the mathematical profession is in existential crisis and that we must act rapidly to preserve the mathematical community and its values. In that piece, I proposed the following two courses of action for individual mathematicians:

1. Do not collaborate with or work for AI companies. This includes unpaid labour, part-time scientific collaborations, verifying AI-written proofs, etc.
2. A moratorium on using LLMs to prove novel mathematical theorems, until mathematicians have decided on a course of action to preserve the mathematical community. We must respond to this crisis now. We must build a new ecosystem which preserves human mathematical activity in an era of increasingly capable artificial intelligence systems. It is vital that we secure our future in collaboration with governments, universities, and non-profits. In the years to come, we should anticipate a lot of damage: an exodus of early-career researchers and potentially huge swings in funding. It is time to discuss the future we would like to live in.

I am aware that the second prescription is extremely controversial, and indeed many will see it as logistically untenable in the long-term. I do not disagree with this. The introduction of LLMs will create a prisoner’s dilemma environment in which a large number of actors are likely to defect. For this reason, it appears to me that we have no choice but to move away from what I call the “ownership model of mathematics” and the current incentive structures which determine our mathematical ecosystem. There were already serious issues with this model. Nevertheless, I believe in the near-term, we must protect and preserve the ownership model as much as possible until we have secured a future for the mathematical community. We must not defect.

Moving away from the ownership model does not necessitate that we embrace the use of AI in mathematics to the greatest extent possible. In fact, I believe we should have serious qualms about AI companies and the future that they are hurtling all of us towards. I find the prospect of an artificial intelligence that outperforms humans on all cognitive tasks (or, more weakly, all mathematical tasks) alarming. We are clearly not yet in this regime with respect to math, and we may never be, but we need to muster a response to the worst-case scenario.

For that reason, we must cooperate now. Artificial intelligence is increasingly capable of automating cognitive labour. Huge fractions of human society are directly threatened as a result, and mathematicians are far from the only professionals in this position. However, I believe that the mathematical community is uniquely well-positioned to respond to this crisis. We must hold the line and provide an exemplar of how a field can cogently and collectively respond to the advent of increasingly capable artificial intelligence systems. Below, I offer some calls to action for the greater mathematical community.

A note on Ludditism: Calling for collective action is not the same thing as asking that our field indefinitely abstain from using AI to aid in mathematical discovery. I am sure that many mathematicians want to use LLMs to enhance our mathematical understanding, and this is perfectly reasonable. Before we wholesale embrace the use of AI in mathematics, we must secure our future and protect human mathematical understanding.

What is the ownership model of mathematics?

Previously, the incentive structure for doing mathematical research was based on what I will call the “ownership model” of mathematics.1 To concisely summarize this state of affairs:

• In this model, a mathematician proves a theorem (usually by combining existing work in the literature with new ideas) and then claims attribution for it. The community then judges whether this theorem is important and interesting. The more “interesting” theorems a mathematician proves, the more likely she is to be hired by a good university with generally better working conditions (a lower teaching load, more remuneration, etc), invited to disseminate her mathematical ideas at conferences, and generally rewarded with career accolades.
• In particular, one might notice that this model heavily incentivizes proving as many theorems as possible, and devalues other contributions to human mathematical understanding, such as mathematical communication, dissemination of ideas, and teaching of undergraduates and graduate students. Sometimes the funding pressures of this model lead to a surfeit of “slop mathematics”, where people release underwhelming, poorly-written, or underverified papers so that they can maximize their number of publications.
• The current ecosystem is one of scarcity and competition: there are not enough professorships for everyone who would like to be a mathematics professor. Industry typically does not compensate people to do pure math research. Academia is therefore the only option for someone who would like to do pure math research. Thus, the number of interesting theorems you prove determines not just the success of your career but whether you will have a career at all.
• I have always had serious qualms about the process of stratification by which we decide who gets hired where, and I believe there are serious ways in which this model is detrimental to human understanding of mathematics.

When a large number of mathematicians are simultaneously using LLMs, then this model is untenable.

• First, let us state the obvious. With AI-assisted mathematics, it becomes extremely difficult to determine who can claim attribution for a result: the LLM or the human mathematician. Consider the following scenarios:
  • Scenario 1: A mathematician needs to internalize Artin-Wedderburn theory quickly for a research problem. This is a classical area of mathematics which was worked out by human beings in the previous century. The mathematician holds long Socratic conversations with an LLM and quickly learns the theory.
  • Scenario 2: A mathematician can formulate a true statement, and knows that it should follow roughly by Propositions 1, 2, and 3 in the literature, but cannot find the right references and is struggling to fill in the details of the argument. After the mathematician gives the proof sketch to an LLM, the model synthesizes the existing literature, fills in all missing details, and returns a complete proof which goes along the lines of the original proof sketch.
  • Scenario 3: A mathematician is the process of proving Main Theorem through a long and complicated argument which involves 100 pages of write-up and many intermediate propositions. After working on the problem, the mathematician becomes stuck on Technical Lemma 42, a result which on its own is not of independent interest but is necessary for the proof of Main Theorem. The mathematician has no clue on how to prove Technical Lemma 42 but has a posteriori reasons for believing that is true. The mathematician asks the LLM to prove Technical Lemma 42, and the LLM returns a complete proof of Technical Lemma 42 with no other input from the mathematician.
  • Scenario 4: A mathematician would like to resolve Famous Open Conjecture. They ask the LLM for ideas on high-level proof strategies, and the LLM returns a promising high-level proof strategy, although many technical details have not been proven and need to be verified. The mathematician then independently work to verify these technical details and completes the proof.
  • Scenario 5: A human mathematician proves Big Theorem through a convex combination of Scenarios 1, 2, 3, and 4, using the LLM as a regular collaborator throughout the mathematical process.
  • Scenario 6: A mathematician prompts an LLM the following: “Solve Famous Open Conjecture.” After 2 hours of autonomously thinking about the problem, the LLM completely resolves Famous Open Conjecture.
• Exercise: In the scenarios above, how much credit should be attributed to the human mathematician, and how much should be attributed to the LLM?
• In the future, we may face the following scenarios:
  • Scenario 6: An LLM autonomously conjectures an interesting mathematical theorem and proves it.
  • Scenario 7: An LLM autonomously develops a new subfield of math with useful and important applications, and human mathematicians endeavour to learn it.
  • Scenario 8: An LLM proves a new result after synthesizing a large body of AI-written mathematics.

What about mathematicians who abstain from using LLMs to do research mathematics? Unfortunately, I think their prospects are dismal in the current “Wild West” era, where access to LLMs coexists with the ownership model.

• Human mathematicians who do not use LLMs will be outcompeted by human mathematicians who do use LLMs, in the near term. Before new professional norms around attribution are cemented, mathematicians who collaborate with LLMs as much as possible stand to gain in the ownership model and will be able to engage in mathematical arbitrage. Consider the following.
  • Scenario: You are a human mathematician on the job market, attempting to prove Hard Theorem without AI assistance, and you are close to the finish line after a year of work. Simultaneously, another mathematician proves Hard Theorem in about 2 weeks using AI assistance to some degree (see Scenarios 1-5 above). They receive enough credit for the proof of Hard Theorem to be hired. Without Hard Theorem on your résumé, you are rejected by universities, your postdoc contract expires, and you are forced to leave math after a period of unemployment.
• If we continue in the current Wild West ecosystem, in which individual actors make highly personal choices about their use of LLMs, mathematicians with ethical qualms about using LLMs will be heavily penalized.

In my previous blogpost, I strongly urged that mathematicians take a collective and cooperative approach to the crisis we face. I believe that the use of any technology is a choice. Nevertheless, I think the following proposition is likely true.

Proposition. The ownership model will collapse even if mathematicians attempt to cooperate to protect it. The existence of highly capable LLMs introduces a prisoner’s dilemma for all mathematicians, and there will be many defectors.

Argument. Even if a vast majority of mathematicians would prefer to preserve the ownership model of mathematics, there will be huge pressures on this model.

1. In the ownership model, mathematics stand to gain by proving the largest number of important theorems possible, and LLMs are useful tools for doing so. Thus, mathematicians are highly incentivized to use LLMs covertly and take credit for the results. In other words, there will be many defectors. At best, a culture of surveillance and deception will take over our community.
2. Many mathematicians already have strong reservations about the values implicit in the ownership model. It will not prove a compelling enough philosophy to stand the external pressures that it will face.
3. Many mathematicians would like to embrace the use of LLMs to do mathematical work, for a multitude of reasons: accelerated mathematical progress; a preference for the resulting working conditions; curiosity about the mathematical world. They will be an outspoken fraction of our community, and many actors from industry and government will be sympathetic to them. If the ownership model continues, the mathematical community will be fractured along ideological lines.

Corollaries.

1. In the long-term, the current model of trying to get [x] theorem published in [y] journal and speaking at it about in [z] conference will not survive.
2. We will no longer be able to instrumentalize proving theorems for career success or make hiring decisions based on a human being’s autonomous theorem-proving ability.
3. The current funding model for mathematics is greatly jeopardized.

Some comments are in order. I am not trying to position myself as a heroic defender of the ownership model of mathematics or the publish-or-perish environment of academia more broadly. We should remember that the ownership model of doing mathematics is a very recent historical development and has only been in place the last 80-100 years. However, I believe in the importance of human understanding of mathematics, and it is clear that the system above at least incentivizes that understanding in so far as it is necessary to produce research mathematics. Will the model which succeeds it have such clear-cut incentives?

How good will LLMs become at mathematics?

Right now, we live in an intermediate regime, where LLMs are extremely useful for pushing mathematical progress forward, but human-led mathematical activity is the still the source of vast majority of novel mathematical theorems. LLMs are capable of doing novel and interesting mathematics, but they are not as good as the mathematical community, in aggregate. I certainly do not want to join AI companies in overstating current frontier model capabilities, as impressive as they are. (As an example, the recent proof that nonsofic groups exist heavily used preexisting work of Kun and Thom.)

It is possible that this intermediate regime proceeds indefinitely, and LLMs do not continue to linearly improve at proving theorems. It may be true that human mathematicians remain instrumental for proving mathematical theorems for an indefinite period of time. In this scenario, human mathematicians will continue to prove mathematical theorems, although increasingly with assistance from AI. They will remain employed by universities for nominally the same reasons—teaching, service work, and the production of original mathematical research. In fact, mathematicians may be able to point to their dramatically increased productivity as an argument for the prolongation of their profession. In all regimes, only humans with extensive mathematical training will be able to understand the output of AI-generated mathematics.

However, we should not allow the existence of our careers to be dictated by the state of AI capabilities, and I am afraid that the scenario above will not last for long. We must instead secure our future now and prepare for the following worst-case scenario:

Scenario A: Artificial intelligence outperforms humans on all mathematical tasks.

Imagine a world in which artificial intelligence is capable of proving novel mathematical theorems without any human guidance whatsoever. Moreover, the less human guidance the LLM uses, the faster and better it is at proving new, interesting theorems. Imagine a world in which artificial intelligence is capable of educating human beings about mathematical content better than mathematicians are; it can quickly curate bespoke textbooks, hold long Socratic conversations with students, and educate human beings about the most recent AI-generated mathematics. Imagine a world in which artificial intelligence has exceptional mathematical taste, and is more than capable of asking interesting questions and determining the future of a mathematical field. Imagine a world in which mathematics is no longer a collective human endeavour but instead an AI-led endeavour in which humans take the backseat.

You may notice something about this scenario: Human mathematicians will lose all bargaining power when all aspects of our labour can be automated. When I read optimistic visions of the future outlined by prominent mathematicians in the scenario above, I often find them breathtakingly naive. Will we really just sit around, going to our conferences and disseminating the latest AI-generated mathematics to each other? Will human mathematical understandingbe preserved in the scenario above? I am doubtful that selling math as a humanist pursuit will go over well. We live in a broadly anti-intellectual society. We have maintained our lifestyles of intellectual freedom, service work, and mentorship because we nominally produce original mathematical content and mathematical understanding in others. When our labour is replaceable, we should fear being replaced.

Let us all think outside of ourselves for a second. A world in which artificial intelligence outperforms humans on all mathematical tasks is not so very far from a world in which artificial intelligence outperforms humans on all cognitive tasks. I believe that we should be very frightened of this prospect, and that AI companies should probably not be building technology towards this end. Nevertheless, they are, and our entire society is throwing its resources and compute towards this endeavour. In light of that, let us imagine we are in Scenario B.

Scenario B: Artificial intelligence outperforms humans on all cognitive tasks.

What would we do if such a technology existed? This is clearly a dangerous scenario. One would hope we would not cede important decisionmaking power to such a technology. We would heavily regulate this technology and prevent actors from accessing it. We might pass inter-country agreements to avoid the proliferation of such a technology. We would affirm that the existence of a technology is not an obligation to use it.

Mathematical solidarity and holding the line

Today, mathematicians are on the bleeding edge of a cross-societal phenomenon: the automation of cognitive labour. AI has already disrupted other highly-skilled industries like animation and software engineering. It has degraded the working conditions of these fields and contributed to increased unemployment. Now we are in the crosshairs.

I believe mathematicians are uniquely well-positioned to face this challenge. We are highly educated and literate; we have a distinct subculture which gives us cultural unity; we face broadly similar working conditions and we work for the same employers. We do not have ethical obligations to use AI to accelerate mathematical progress as quickly as possible. We have extraordinary professional latitude and intellectual freedom compared to other careers. We are knowledge workers whose cognitive labour is directly threatened by AI, and we can cooperate with each other in light of this. It is our duty to preserve and increase human mathematical understanding.

I believe the greatest threat to our solidarity is self-imposed. In reading visions of the future articulated by other prominent mathematicians, I am struck by the degree to which we are afraid to selfishly defend our material interests. Well, we live fantastic lives of intellectual freedom, service work, and mentorship; we provide assistance to theorists and experimentalists in other scientific fields; we carry on an intellectual tradition which is thousands of years old. If AI makes those things obsolete or performs aspects of our labour better than us, shouldn’t we just let it? No. Stop capitulating in advance to market forces and start defending human values that you think are actually important.

I think that we have a moral obligation to resist the world that AI companies are creating. Perhaps your own material self-interests are not enough to motivate you; fair enough. What about the material self-interests of society at large? While LLMs undoubtedly have the potential to be a powerful scientific tool, I believe that we are on shaky ground. It is my opinion that AI companies are creating a future that most people do not want to live in. We can provide an exemplar of how a community can preserve itself against the automation of cognitive labour. It is incumbent on us to set an example for other industries by pre-emptively acting to safeguard our working conditions and livelihoods. We have an obligation to hold the line, not only for ourselves but for others.

Calls to action for the mathematical community

1. We need to hold a large professional meeting within the next two to three months, featuring a coalition of mathematicians from various subfields, to outline possible scenarios for the future and determine the scenario we would like to live in. We must continue litigating the details in the months and years to come. It will be professionally irresponsible if such a meeting does not materialize. The Leiden working group is a start. We have to continue.
2. Professional mathematical organizations should begin to hire mathematical advocates and emissaries, who will represent the interests of mathematicians to governments, nonprofit, and industries. This should include not only prominent senior-career mathematicians but early-career representatives whose livelihoods are most in jeopardy.
3. We must build class solidarity and stop collaborating with AI companies for the foreseeable future, until we have safeguarded our material interests and that of our larger society. We must not defect.

Setting societal precedents

We will lose bargaining power if AI technologies are equally capable of doing the mathematical labour we do—unless we set societal precedents about the kinds of labour we allow AI to do. The time to set such precedents is now, in the intermediate regime where LLMs are still unable to replace our labour.

Here are some precedents I could imagine a broad coalition of mathematicians supporting (or at minimum, discussing in the professional meeting suggested above):

1. We should not allow AI to educate our children and junior researchers in mathematics, even if it is capable of doing so. We must prevent universities from outsourcing the labour of teaching and education to LLMs.
2. We should not allow AI to decide the direction of mathematical research or the future of mathematical fields, even if it is capable of doing so.
3. We should maintain mathematics as a collective enterprise, even if it becomes more efficient to individually learn about mathematics from AI. We should continue disseminating and expositing mathematics to each other.
4. We need to secure our livelihoods. We need to convince universities and non-profits to maintain a large community of living, human, employed mathematicians who are paid to talk to each other, do mathematics, and control our field, even if originality no longer determines the quality of a piece of mathematics.
5. We must draw clear professional norms about how mathematicians are allowed to use LLMs, and abide by them.
   1. For instance, for prompts on the level of “Solve Famous Open Conjecture” with no other human assistance, I would prefer that such prompts are initiated on the basis of a collective decision of a subfield of mathematicians.
6. We must remember that the use of a technology is a choice and that we are not ethically obligated to maximize the use of LLMs for the end of mathematical progress (as measured, e.g., by the number of true theorems with Lean-typed proofs in existence.)

A sombre note

Over the last year, I have been haunted by dreams of an impending future. In this future, the frontiers of mathematical knowledge are pushed ever forward while our universities are hollowed from the inside out. Undergraduate students across scientific fields fail to master basic concepts. Difficult theorems are proven with increasingly little input from humans; in light of this, governments and industries funnel money towards buying tokens rather than hiring expensive human mathematicians. Young, talented mathematicians leave the field en masse and are pushed towards careers in finance, the military, and increasingly, AI companies. The professional mathematicians who remain fail to convince governments and industries to fund a large, active community to disseminate and pass on mathematical knowledge. After all, mathematicians already lost all bargaining power when almost all aspects of their labour could be capably performed by LLMs. The community struggles to attract and retain talented people to work on and understand the proofs of important theorems. Cognitive labour across many industries is increasingly automated, and political and financial power concentrates in the hands of a few AI companies. In general, human beings increasingly offload cognition onto LLMs, and far fewer people pursue intellectual professions as more of them are effectively replaced by AI. Humans now chiefly work in the relational sector, providing hospitality, therapy, artisanal goods, and personal services to each other. Mathematics is now a hobbyist endeavour which is done by a handful of atomized individuals, and most people don’t know very much about it. The AI industry remains poorly regulated; after all, superpowers spent the early 21st century competing to build the most powerful models possible, and they were well-aware that regulations would only slow the arms race down. Dangerous actors control frontier models and artificial intelligence is given increasing amounts of decisionmaking power, to catastrophic effect.

Is this a touch hysterical? I’m not sure. Certainly, we must not allow this future to come to pass.

About the author:

I am a rising 2nd-year PhD student at the California Institute of Technology. All the views espoused in the essay above are my own. I am 24 years old and have spent the last seven years of my life studying mathematics.

1

This turn of phrase is inspired by this blogpost of Timothy Gowers.