Share Your Experience: Take the 2024 Developer Survey

# Questions tagged [optimization]

For questions about implementing and improving optimization algorithms used in creating AI programs, or optimization in general.

219 questions
Filter by
Sorted by
Tagged with
1 vote
20 views

### Why completely two different algorithms are being used in Deep Q Learning?

I'm a new student in reinforcement learning. Recently, I've been studying about different algorithms of RL. But I'm quite surprized that there are some algorithms which are named as "same" ...
27 views

### How to find an argument of a NN function(which returns a distribution) to minimize a KL divergence?

Consider a neural network function $f:\mathbb{R}\to distribution$. For simplicity, maybe consider that it returns a gaussian distribution. I want to find $\arg\min_{s\in\mathbb{R}}D_{KL}(f(s),q)$ for ...
11 views

### Adaptive regret bounds in Online Convex Optimization

I have recently stumbled upon a proof in Elad Hazan's book "Introduction to Online Convex Optimization" with a step I can't quite grasp. In the second to last line it is not clear to me why ...
1 vote
22 views

Hello I'm learning optimizers now, I can understand the momentum part (similar to physics world), but confused about different learning rate of different parameters, for Adagrad/Rmsprop, if $∂L/∂w_1$ ...
• 111
27 views

### Why does Multi Objective RL exist?

I have recently posted a question here about a problem that I have controlling a robotic arm. Basically I have a dense reward for the arms position, and a sparse reward for the arms stiffness: Reward ...
• 33
1 vote
60 views

### Reward shaping for dense and sparse rewards

I am working on an RL Problem that drives me nuts. My goal is to control a robot arm in a simulator that has to do 2 things: Hold the arm in a certain position (that is easy and done) If I apply an ...
• 33
28 views

### How to estimate Time vs Memory trade-off prior to modelling

It is often the case when the time vs memory trade-off is underestimated prior to using ML/DL for solving a particular task. Taking into account the type, size and format of the available data and ...
62 views

### Can you explain the Hinton's comment "Rprop is equivalent to using the gradient, but also dividing by the size of the gradient"?

Been reviewing some old foundational material and ran into this comment by Hinton on Rprop in his old Coursera class: Rprop is equivalent to using the gradient, but also dividing by the size of the ...
• 121
33 views

### Non differentiable loss function train with actor critic style

I'm working on a project where a non differentiable loss is there. I'm thinking about how should I deal with them. My model is a very big lstm model (about 1M parameter), and after 500 steps (not sure ...
• 13
1 vote
68 views

### The SOTA of derivative-free optimization

As titled, I want to ask what is the SOTA of derivative-free algorithm. I am not familiar with this thing at all, the only derivative-free optimization algorithm I am familiar with is GA, and others ...
• 13
42 views

### How to prevent update a pretrained model if a model is optimized with backpropagation? [closed]

These are components in my model: A generator An encoder: a pretrained, and should not updated. A loss function. Input is passed to the encoder to generate X, X is then passed to generator to ...
• 109
1 vote
37 views

### Finding resulting domains after enforcing arc consistency

This question is about the below exam scheduling constraint satisfaction graph, where each node represents a course. Each course is associated with an initial domain of possible exam days (most ...
• 11
1 vote
238 views

### When to use Pruning, Quantization , Distillation and others when optimizing speed

I want to understand how to optimize models for inference speed and am seeking some advice and best practices for the same. I am a little bit aware of the concepts of pruning, quantization, and ...
• 749
1 vote
66 views

### When can we unnest the minimizations/recursions in an value function(bellman optimality equation)?

When reading the following paper(page 4): An Approximate Dynamic Programming Approach for Dual Stochastic Model Predictive Control I could see that they were able to unnest the minimization's in the ...
17 views

### Output Not Changing - Feeding Previous Outputs Back Into a Model

Full disclosure, I also posted this on Stack Overflow I have put a more theory based bent towards the question itself here I have a simple model in pytorch based on the quickstart except instead of a ...
33 views

### Multi-Task VAE, Decoder Activation Functions?

I'm working on a Multi-Task VAE with one Encoder and two Decoders. The input consists of a vector with parameters which describe a design of a fluid system. The goal is to reconstruct the parameters ...
1 vote
66 views

### Is this a bandit problem or a MDP?

I am trying to understand if this problem can be casted both as a bandit problem as well as an MDP. Lets assume that we are trying to optimize sales $y_t$ based on investments $x_{1, t}, x_{2, t}$ ...
• 53
195 views

### How to train a sample weight model for another ML model?

I'm trying to train a ML model, however the predictability of the different samples varies, i.e. some samples are inherently much harder to predict/estimate than others. Poorer predictions for these ...
• 123
1 vote
36 views

### Applicability of Holland's Schema Theorem to Genetic Algorithms with Non-Binary Individual Representations

I'm currently working on a problem formulation that requires non-binary individual representations in a genetic algorithm (GA). I've been exploring Holland's Schema Theorem as a theoretical basis for ...
• 11
26 views

### Implementing momentum is causing calculation exceptions/errors

I am developing my own neural network in order to learn about how they work. I am implementing via C++ and the Eigen library (for matrix multiplication). I have a working implementation that seems to ...
• 101
1 vote
99 views

### Maximize a scoring function within the latent space of a generative model

Given a generative model, G, trained on a dataset D. This generative model can be either GAN or Diffusion based. Supposed each sample, x_i, generated by G, can be evaluated by a readily available ...
58 views

### Best way to generate fitness landscape when using higher dimensional data

I'm using a GA to find the best set of parameters to maximize a fitness function. I want to draw a fitness landscape to visualize the effectiveness of the algorithm. The fitness function, calculated ...
17 views

### What do I need to learn to tackle the following problem: make a program that optimizes decisions in the game PlateUp!

So, recently my friends and I have been hooked on a videogame called PlateUp!. The game is kind of a management game where the objective is to succesfully run a restaurant. The game can be roughly ...
20 views

### Finding an optimal action score function for Multi-Armed Bandit Problem

Considering a multi-armed bandit problem where there are : ...
1 vote
94 views

### How does Openai's CLIP avoids dimensional collapse?

According to this paper from FAIR : https://arxiv.org/abs/2110.09348 , contrastive learning methods suffer from the problem of dimensional collapse where "the embedding vectors end up spanning a ...
1 vote
72 views

### Cheap differentiable similarity metrics of vectors

I am looking to compute the similarity between a large set of vectors during neural network training - a process that is considerably expensive when choosing the wrong metric. So far, I am making use ...
1 vote
61 views

### Why are the non-linear activations in deep nets not learned?

Why can we not parametrize and learn the non-linear activations? For example, if we look at leaky ReLu which equals to $f(y)=y$ for $y>0$ and $f(y)=\alpha y$ for $y<0$, it seems that we can ...
61 views

### Poor convergence of a neural network, which implements NMF

I'd like to understand why this simple network fails to converge. The resulting MSE error is an order of 10^4 - 10^5 bigger than what could be achieved. The task is to do a non-negative matrix ...
• 777
1 vote
75 views

### Optimizing a blackbox function with binary states

I have a non-linear black box function, which inputs a vector(size=250) and outputs a scalar value; f(x) = value. The x variable is a vector of size 250 and has ...
• 111
1 vote
150 views

### How to estimate the gradient of an argmin loss

Suppose we have a neural network $f_\theta(x)$, where $x$ is the input and $\theta$ is the network's parameters. For each $\theta$, we can minimize $f_\theta(x)$ w.r.t. $x$ and obtain the minimum ...
• 111
80 views

### How to speed up my neural network?

I would like to train an LSTM-based variational autoencoder on a large dataset (37 million sentences). However, I have calculated that my training speed as of now is too slow (on Google Colab). I am ...
1 vote
404 views

### How to determine if a decision tree is the (globally) optimal tree?

BACKGROUND: When constructing decision trees, the features are selected at various nodes based on whether it optimally splits the samples at that level (i.e., locally) using some user-chosen metric ...
• 952
1 vote
77 views

### What would be a good optimization technique for this kind of problem?

Problem Description: Since I am not sure if there is a scientific term that categorizes this problem, I will do my best to describe it thoroughly. Suppose there is a chamber that's being filled with ...
• 111
27 views

I was reading a book on Deep Learning when I came across a line, more like a few words that didn't make apparent sense. Thus, we will often settle for sampling a random minibatch of examples every ...
• 101
261 views

### How to define a fitness function to make sure the best fitness value is 'close to 9' in genetic algorithm

I am learning about genetic algorithms (GA), but I encountered a question about the definition of the fitness function used in GA. I understand that the fitness function should return a scalar value (...
105 views

### The Small Set Expansion Hypothesis, this problem was solved or is open problem yet?

I found this problem by article called "Ten Lectures and Forty-Two Open Problems in the Mathematics of Data Science" published at 2016. I`m looking for an open problem at Data Science or/and ...
128 views

### Multi-objective training involving maximization of one loss function and minimization of another

I need my model to predict $s$ from my data $x$. Additionally, I need the model to not use signals in $x$ that are predictive of a separate target $a$. My approach is to transform $x$ into a ...
1 vote
297 views

### Alternatives to Bayesian optimization

I am given a dataset $\mathcal{D} = \{\mathbf{x}_i\}_{i=1}^n$ and I need to find the point (in my case a material) $\mathbf{x}^*$ that maximizes a property $y$ (which can be obtained from a black-box ...
• 150
1 vote
304 views

### Why do we use gradient descent to minimize the loss function?

The purpose of training neural networks is to minimize a loss function, in this process we usually use gradient descent method. But in Calculus, if we want to find the global minimum of a ...
• 111
1 vote
41 views

### Do NNs suffer from lack of efficiency in network structure and suggesting training parameters?

I am working on dynamical systems using Optimal Control theory and trying to find the connection between this field and Machine Learning. Consider a simple 2-layer Neural Network (NN) where the ...
30 views

### How to create a loss function that penalizes duplicate indices in the output tensor?

We're working on a sequence-to-sequence problem using pytorch, and are using cross-entropy to calculate the loss when comparing the output sequence to the target sequence. This works fine and ...
• 121
1 vote
631 views

### How does MAML inner loop optimization works?

I started to learn meta-learning, reading the MAML paper https://arxiv.org/pdf/1703.03400.pdf In the inner loop, I am calculating adapted parameters for each task, I will be doing multiple steps of ...
• 43
28 views

### Resolving Derivation Discrepancies for Differentiating through Optimization Paths

I'm reading the paper "Optimizing Millions of Hyperparameters by Implicit Differentiation". The key contribution of the paper is to show that you can replace optimizing through the ...
1 vote
36 views

### Structured policies in dynamic programming: solving a toy example

I am trying to solve a dynamic programming toy example. Here is the prompt: imagine you arrive in a new city for $N$ days and every night need to pick a restaurant to get dinner at. The qualities of ...
1 vote
169 views

### Best algorithm for the Word Ladder puzzle

What would be the best performing algorithm to solve the Word Ladder problem, in terms of guaranteed finding of the shortest solution in the shortest possible time? Is it BFS, DFS, A*, IDA* or another ...
117 views

### Why does my regression-NN completely fail to predict some points?

I would like to train a NN in order to approximate an unknown function $y = f(x_1,x_2)$. I have a lot of measurements $y = [y_1,\dots,y_K]$ (with K that could be in the range of 10-100 thousands) ...
• 31
1 vote
35 views

### Additional Optimizations for Convolutional Models On Inferencing

I am aware of several ways to optimize a convolutional (or any) model after training to make inferencing quicker. I am currently implementing BatchNormalization Folding and removing Dropout layers ...
121 views

### What does it mean by Generalization? [closed]

Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning What does it mean by Generalization in this article?
• 331
51 views

### Are Problems in AI Usually "Ill Posed"?

I was reading the following link (https://en.wikipedia.org/wiki/Well-posed_problem) on "Well Posed Problems". Supposedly, if a problem is "Well Posed", it must meet the following ...
• 329
1 vote