# Questions tagged [optimization]

For questions about implementing and improving optimization algorithms used in creating AI programs, or optimization in general.

194 questions
Filter by
Sorted by
Tagged with
1 vote
31 views

### Do NNs suffer from lack of efficiency in network structure and suggesting training parameters?

I am working on dynamical systems using Optimal Control theory and trying to find the connection between this field and Machine Learning. Consider a simple 2-layer Neural Network (NN) where the ...
24 views

### Are there more optimization methods like GAE for PPO [closed]

I posted about this earlier, but got the suggestion to separate the questions. I'm currently trying to "solve" the OpenAI gym "Humanoid" environment. To improve the training ...
• 13
46 views

### What method to use when optimizing an array of data

Say I have an array of data, where each element describes a shape made of points, in vector form (each vector has several hundred dimensions). Each element also has a rating that gets higher, the ...
23 views

### How to create a loss function that penalizes duplicate indices in the output tensor?

We're working on a sequence-to-sequence problem using pytorch, and are using cross-entropy to calculate the loss when comparing the output sequence to the target sequence. This works fine and ...
• 121
57 views

### How does MAML inner loop optimization works?

I started to learn meta-learning, reading the MAML paper https://arxiv.org/pdf/1703.03400.pdf In the inner loop, I am calculating adapted parameters for each task, I will be doing multiple steps of ...
• 33
27 views

### Resolving Derivation Discrepancies for Differentiating through Optimization Paths

I'm reading the paper "Optimizing Millions of Hyperparameters by Implicit Differentiation". The key contribution of the paper is to show that you can replace optimizing through the ...
25 views

### How to generate data lying in the union of different hyperplanes using a VAE

I know that a way to possibly encode prior knowledge into neural networks training is by using differentiable optimization layers (paper). I'm in the following situation, and I'm wondering if it could ...
• 197
1 vote
28 views

### Structured policies in dynamic programming: solving a toy example

I am trying to solve a dynamic programming toy example. Here is the prompt: imagine you arrive in a new city for $N$ days and every night need to pick a restaurant to get dinner at. The qualities of ...
18 views

### What is the use of utilizing q-Gaussian mutation operators in evolutionary algorithms?

There are great numbers of evolutionary algorithms for optimization of engineering problems which each of them gives its own objective function value in a defined problem. Using the q-Gaussian ...
1 vote
43 views

### Best algorithm for the Word Ladder puzzle

What would be the best performing algorithm to solve the Word Ladder problem, in terms of guaranteed finding of the shortest solution in the shortest possible time? Is it BFS, DFS, A*, IDA* or another ...
32 views

### Using neural network to find the most similar items

I am trying to design a multilayer neural network that based on item features will return x amount of the most similar items from the dataset. My current neural network takes features of 2 items, and ...