2
$\begingroup$

I'm currently reading a paper that uses CNN's as a base approach to solving some image classification issues and I've found that they kept mentioning the term "Differentiable Architecture", for which I have no idea about its meaning, as I'm new to this world of Deep Learning, Neural Networks, etc., so to sum up my question is

What does "differentiable architecture" mean?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Hello. Welcome to AI SE. Could you please share the link to or name of the paper you were reading for more context? $\endgroup$
    – nbro
    Commented Jul 14, 2021 at 18:02

1 Answer 1

Reset to default
1
$\begingroup$

Without the specific context, I cannot give a definitive answer, but it's very likely that a "differentiable architecture" refers to a neural network that represents/computes a differentiable function (so you need to use differentiable activation functions, such as the sigmoid), i.e. you can take the partial derivatives of the loss function with respect to each parameter/weight of the neural network, so you can use backpropagation to find the gradient of the loss function, consequently, you can train this neural network with gradient descent, which is a numerical/iterative optimization algorithm for finding a (local) minimum of a function.

Most architectures you will find around are differentiable. In fact, gradient descent is the most widely used algorithm for training neural networks nowadays.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .