The Shifting Value of Intelligence

On October 6 OpenAI released 722 mathematics papers, about half of them with verified proofs. Some of these proofs were for unsolved math problems that mathematicians had been working on for years.

The mathematicians were distraught, some of them crying real tears. Their purpose in life seemed to have been erased in the blink of an eye by a machine. I kind of know what it feels like having been sidelined as a programmer in a similar way. The mathematicians won’t be the last.

Some are saying that intelligence is losing its value because of AI while others say that since AI is a lever for intelligence, it’s becoming more valuable. In order to see what is actually going on we have to go beyond the general term “intelligence” and see what its constituent parts are. The generally accepted subdivision of the cluster of abilities we call intelligence is the Cattell-Horn-Carroll (CHC) model, which IQ tests are based on. The illustration below is a visualization of the CHC framework and shows what elements are used in IQ tests. The orange circle represents the skill level of a well-educated adult.

I would have preferred to use the well known CHC framework to clarify the shift in what parts of intelligence are becoming less valuable and what are becoming more valuable but that didn’t seem to work very well. The problem is that the CHC is built on what a test can measure, and that is exactly the area where AI does its best work.

I developed (with valuable input from Claude AI) a subdivision of intelligence that also includes important abilities that tests can not measure very well as is illustrated by the chart below. The top half is a regrouping from the CHC and the bottom half is the segments I added. These are the qualities that can use AI as leverage instead of being made redundant by AI.

In the plot below we see the estimated abilities of current state-of-the-art AI models. It is no coincidence that they are performing very well on the segments that can be tested for. During training there is a clear score that lets them improve. The exception is learning over time although as part of an agent / harness configuration AI will do better there as opposed to a chat interaction format.

A person who is strong in the skills marked with the green lines will be able to use AI as a multiplier. Someone who wakes up in the middle of the night with a clear and specific idea of what a film should look like or what a song should sound like has something that AI can not provide.

To prepare for a future with ubiquitous AI people could work on developing these non-testable skills. The power of these skills might not come to fruition inside an existing corporate structure as much as it would in the context of a brand new project or enterprise.

In the case of the mathematicians AI exposed a mismatch between why they claimed to work in the field (for the love of math) and what got rewarded in the field with money and reputation: proofs. A large number of mathematicians have been calling for a boycott of OpenAI because it is undermining their profession.

This exposes the tension between maintaining the status quo in academia and providing useful knowledge to society. This is not just self-interest on the part of the mathematicians. What would happen if all math were delegated to machines as is now the case with simple arithmetic?

Math is a verifiable field but not the only one. Will physics be next? What part of your job happens in the testable part of intelligence?