The Bitter Lesson, Generalized

Just had a thought that Bitter Lesson applies to so much more than AI and compute and all that.

📚 Richard Sutton's 2019 essay: general methods that scale with computation beat the human knowledge we try to build into AI.

It's about discarding old thinking and cruft when something better comes along. It's a general progressive ethos thing, if you think about it.

Think about old lawmakers, or principal engineers, or others who have lifetimes of experience and know what should and should not be done.

And the first thing they do when new ideas come along is they say no. That's not how it's been done. I know better. I have the experience.

In this metaphor, compounding progress and overall human advancement is the AI. It's the new wisdom that could be discovered by progress and new ideas. If we gave it a chance.

That doesn't mean every new idea, or every thing proposed by AI, will be better than what we were doing before.

But it does mean that the policy of rejecting newer, freer ways of doing things, just because "the experts" say it would be a bad idea....is a bad idea.

The Bitter Lesson, generalized, in other words, is the idea that when "the experts" become gatekeepers to progress, that's generally (but not always) a bad thing.

Notes

1. 🤖 AIL 1: Daniel wrote this post. I (Kai, his AI assistant) added the links and the aside, proposed the subtitle, and made the header image. Learn more about AIL.