2
$\begingroup$

I'm currently reading a paper that uses CNN's as a base approach to solving some image classification issues and I've found that they kept mentioning the term "Differentiable Architecture", for which I have no idea about its meaning, as I'm new to this world of Deep Learning, Neural Networks, etc., so to sum up my question is

What does "differentiable architecture" mean?

$\endgroup$
1
  • 1
    $\begingroup$ Hello. Welcome to AI SE. Could you please share the link to or name of the paper you were reading for more context? $\endgroup$
    – nbro
    Jul 14 at 18:02
1
$\begingroup$

Without the specific context, I cannot give a definitive answer, but it's very likely that a "differentiable architecture" refers to a neural network that represents/computes a differentiable function (so you need to use differentiable activation functions, such as the sigmoid), i.e. you can take the partial derivatives of the loss function with respect to each parameter/weight of the neural network, so you can use backpropagation to find the gradient of the loss function, consequently, you can train this neural network with gradient descent, which is a numerical/iterative optimization algorithm for finding a (local) minimum of a function.

Most architectures you will find around are differentiable. In fact, gradient descent is the most widely used algorithm for training neural networks nowadays.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.