Local Maxima

One concept I keep returning to is that of local maxima. If you recall, local maxima occurs when a graph swells to a particular point, dips, and then ascends to a prior point. It’s a recurring problem in optimization - naieve optimization will always reach the local maximum and stop. There are many techniques to avoid this - picking a random starting point, jolting forward when you think you’re at an apex, and retrying, among others.