2
$\begingroup$

My question is regarding standard dense-connected feed forward neural networks with sigmoidal activation.

I am studying Bayesian Optimization for hyper-parameter selection for neural networks. There is no doubt that this is an effective method, but I just wan't to delve a little deeper into the maths.

Question: Are neural networks Lipschitz functions?

$\endgroup$

1 Answer 1

2
$\begingroup$

I'm not an expert in this area, but it would appear to depend on the choice of activation function:

That said, this paper appears to give some conditions (specifically for dynamic ANNs) for which networks with activation function involving $e^x$ can be Lipschitz continuous, so possibly the above is not the whole story.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .