I am working on an optimization problem. First, I have done forward training to work the network as a surrogate model, then I freeze the output and I want to find an optimal value of input for a given output.
$\begingroup$
$\endgroup$
5
-
$\begingroup$ Hello. Please, take a look at ai.stackexchange.com/help/how-to-ask and ai.stackexchange.com/help/on-topic to know more about our site. There are several things here that are not clear to me. What do you mean by "forward training"? Do you mean that you train your neural network with gradient descent and backpropagation as usual, so, I suppose, you also have a labelled dataset? Can you explain a little bit more in detail why you want to predict the input in this way? Moreover, why did you tag this post with inverse-rl? What does this have to do with inverse RL? $\endgroup$– nbroJan 20, 2021 at 19:54
-
$\begingroup$ Yes, I have trained the network in a usual way. I am trying to use this pre-trained model as a surrogate model for my optimization problem. Now I am trying to find optimal input in order to minimize the loss function. For that, I need to find the gradient of loss function with respect to the input. So, I am looking for any inbuilt method in Keras to find gradient w.r.t. input. $\endgroup$– PreetzJan 21, 2021 at 16:52
-
$\begingroup$ Can you please edit your post and you include these details (and other details that you find necessary to understand your problem) in your post? Moreover, I would like to note that programming issues are off-topic here: For instance, asking "how to do this in library Y" is off-topic here (see ai.stackexchange.com/help/on-topic). Here, we focus on theoretical issues. That's why the answer below tries to give you the main idea or mathematical approach. $\endgroup$– nbroJan 21, 2021 at 16:55
-
$\begingroup$ In any case, it's ok to ask, in addition to your main theoretical question, if someone knows, how to do something in a specific library, but that can't be your main question, because otherwise it is off-topic here, as I said, and this type of programming issues are better suited for Stack Overflow. $\endgroup$– nbroJan 21, 2021 at 16:55
-
$\begingroup$ Once I've done that by simply adding a buffer layer after input. This layer just takes the input, adds bias to it and gives the output (biases are trainable, so instead of optimizing input, the network optimizes biases). It worked for me but I don't know if it's the best way to do so. If you need more info, let me know $\endgroup$– aminJan 22, 2021 at 12:53
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
It's just a typical optimization problem. You want to optimize function \begin{equation} f(\theta, x) \end{equation} with fixed parameters $\theta$ (network weights) and optimization variable $x$. Typically the approach is consisted of choosing a search direction and direction step. For deciding search direction you can use the gradient of objective with respect to decision variable and have an update of form \begin{equation} x = x - \alpha \nabla_x f(\theta, x) \end{equation} where $\nabla_x f(\theta, x)$ is the gradient. You can also use Newton or Quasi-Newton methods of form \begin{equation} x = x - \alpha B^{-1}\nabla_x f(\theta, x) \end{equation} where $B$ is a Hessian or approximate Hessian of $f(\theta, x)$ with respect to $x$. Step length parameter is usually decided with line search or you can also use trust region approaches which decide step direction with constraints to it's length. For more details about optimization you can consult Numerical Optimization by Nocedal and Wright
-
$\begingroup$ Thanks, I am trying to optimize by using gradient descent. But, I am looking for any inbuilt method in Keras to calculate the gradient of loss function w.r.t. input. $\endgroup$– PreetzJan 21, 2021 at 16:54