If you have an erratic loss landscape, it can lead to an unstable learning curve. Thus, it's always better to choose a simpler function which creates a simple landscape. Sometimes even due to uneven dataset distribution, we can observe those jumps/irregularities in the training curve.
And yes, those jumps do mean it might've found something significant in the landscape. Those jumps can arise while the model is exploring the multiple local minima of the landscape.
During Machine Learning Optimization, we usually use algorithms like Stochastic Gradient Descent and Adam to find Local Minima's whereas approaches like Simulated Annealing find global minima. There have been multiple discussions around why to use local minima's instead of global minima. Some argue that local minima are just as useful as global minima in case of machine learning problems.
Thus, Stable Learning is preferable as it symbolizes that the model is converging to local minima.
You can read A Survey of Optimization Methods from
a Machine Learning Perspective by Shiliang Sun, Zehui Cao, Han Zhu, and Jing Zhaoto et al. and read about all the optimization functions commonly used in Machine Learning.