Mathematicians may be worried, but AI-for-science is going to be great, recursively self-improving, and we’re going to learn loads

I have a new post on my experiments with 5.6-sol as a scientific agent; you can find the original post here. I've reproduced it below, as well. I'd be grateful for any feedback, thoughts, or to hear about experiments other people have done; you're very welcome to point your own agents at the repos described below, or anywhere else you like.

As part of our Templeton-funded Proofs & Reasons project, I travelled to the ICM in Philadelphia this year, to help a collaborator run some new experiments on expert mathematicians. The ICM is an every-four-years event that you may remember as where Hilbert launched his field-defining 23 problems, or the place they announce the Fields Medals. It’s a big deal for mathematics, and we had a wonderful time both running our experiments and talking to the truly international community whose interests and abilities define modern mathematics.

At the same time that ICM was running, it was hard to ignore the vast number of AI-enabled results that were breaking, almost simultaneously, on Twitter and elsewhere. Commercially-available models were cracking, in hours, problems that had stood for decades. There are too many to summarize, but this Twitter pairinggives you a sense of both the advances AI has made, and the responses from some human mathematicians. At this point, “google for an open problem and stick it into GPT 5.6-sol on high” seems to have at least a 1% success rate—which fills some mathematicians with hope (new proofs from the alien proof machine!) and others with a bit of despair (new proofs… from an alien proof machine!)

It’s natural to ask what this looks like for science. We don’t have well-framed open questions in the same way mathematicians do, but we do have plenty of good problems. I decided to sic GPT 5-6 sol (on high) on three of my own papers, with a generic instruction to do something cool. I picked three papers that (1) I’m particularly proud of, and (2) that seemed amenable to this kind of study—empirical data…