New answers tagged

1

Can gradient-based algorithms work on those curves with their local/global minima, or do maxima lie on flat regions? Yes, with some minor caveats. All the points on the flat region are equivalent (and in your example, are all valid global minimum points). Gradients outside of the region will point correctly away from that region and gradient descent steps ...


1

From the phrasing, it seems that complicated refers to the non-convexity of the loss landscapes of neural networks. We do not have formal guarantees of convergence in general for such landscapes. This non-convexity is a property of both the function defined by the neural network, and the particular loss function we use. In practice though, non-convexity ...


Top 50 recent answers are included