0
$\begingroup$

I'm comparing 2 optimization algorithms for deep neural nets through visualizing the loss landscape. The visualization method is described here.

Besides the qualitative observation that how trajectory moves w.r.t. the loss level-sets, are there any quantitive measures to compare the two methods?

enter image description here enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

How about:

  • speed of convergence
  • stability/variance w.r.t to the initial random seed (or other sources of variance like learning rate)
  • presence/number of saddle points in your loss landscape
Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .