The Long (Self-)Correction

I propose the Long Self-Correction [1] as an alternative name/idea/concept to AI Pause and Long Reflection.

Problem with AI Pause: Pause until when, and for what purpose? Presumably to make AI (that we'll build later) safer, but the deeper problem is that humans aren't safe, and can't safely serve as builders, overseers, or alignment targets for powerful AIs.

Problem with Long Reflection: It seems to imply that the main problem with humans is that we just haven't had enough time to think, that reflection is the main thing we need to do more of, and then we can get on with building powerful AIs or other technologies. Or that if we build aligned AIs that sincerely help us think a lot more, or do the thinking for us, then things will turn out fine.

So I think we need a catchy handle for a related but distinct idea, that humans aren't ready to build AIs or other extremely powerful technologies, because we're currently too flawed, in a variety of ways, and it will take a long process (which may or may not end up succeeding) to fix those flaws.

A summary of the flaws that I have in mind:

not having a workable moral framework (consequentialism, deontology, virtue ethics all having serious problems)

being bad at philosophy and long-horizon strategy

being badly calibrated about our philosophical and strategic competence, i.e., not realizing that we're incompetent, despite overwhelming evidence (see e.g. FTX and early MIRI, and many others, trying to maximize impact while assuming their own philosophical and strategic competence)

in practice, human morality is a kind of status game that actively disvalues careful strategy and philosophy in most places

positional/zero-sum values (like power and social status) being a huge part of human motivations, but almost nobody explicitly reasons/talks about this while discussing, for example, AI safety or effective altruism or how to make the long term future turn out well [2]

being easy to manipulate (or go off-rails by onese…