# Questions tagged [optimization]

For questions about implementing and improving optimization algorithms used in creating AI programs, or optimization in general.

99 questions
Filter by
Sorted by
Tagged with
25 views

### How should I weight the factors that affect the choice of an action in a strategy board game with multiple actions?

I have written an AI that plays a strategy board game. There are lots of different types of moves (e.g. attack, defend, help ally colony, etc.). I calculate the best moves to do depending on a ...
27 views

### Is logistic regression used for unconstrained or constrained optimisation problems?

Is logistic regression used for unconstrained or constrained optimization problems, and why?
151 views

### What's the difference between RMSE and Euclidean distance, and when to use a custom loss? [closed]

I'm searching for a loss function that fits my project. Actually, I have two questions, but they are in the same direction. I take a look at the definition of the root mean squared error (RMSE) and ...
99 views

### How can I assign agents to tasks based on time and affinity?

I am working on an assignment problem. Consider $K$ agents $A_1, \dots A_K$ and $N$ tasks $T_1, \dots T_N$. Each task has a certain time $t(T_i)$ to be completed and each agent has a matching (or ...
15 views

### How does the automated temperature adjustment step work in Soft Actor-Critic?

In section 5 of the paper Soft Actor-Critic Algorithms and Applications, it is proposed an optimization problem to obtain an optimal temperature parameter $\alpha^*_t$. First, one uses the original ...
48 views

### Grey Wolf Optimization - Issue with Dimension [closed]

I'm trying to use the grey wolf optimization (GWO) for texts clustering. I used this code, https://github.com/7ossam81/EvoloPy-NN/blob/master/selector.py I tried using the dimension 30 for the GWO as ...
14 views

### Choosing best combinations from all possible combination expressions based few variables, unary operators, binary operators

I have a few financial variables of a stock universe like OHLC prices, volume, and other fundamentals with varying time-frequency. Using this set I'm creating an expression that gives the weights to ...
49 views

### What are examples of optimization problems that can be solved using a genetic algorithm?

I'm trying to learn how a genetic algorithm can solve optimization problems. I have already learned how a genetic algorithm can solve knapsack, TSP and set cover problems. I'm looking for some other ...
374 views

### What are the implications of the “No Free Lunch” theorem for machine learning?

The No Free Lunch (NFL) theorem states (see the paper Coevolutionary Free Lunches by David H. Wolpert and William G. Macready) any two algorithms are equivalent when their performance is averaged ...
37 views

### How can we conclude that an optimization algorithm is better than another one

When we test a new optimization algorithm, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,i.e., in terms of accuracy, f1 ...
79 views

### LSTM network doesn't converge, what should be changed? [closed]

I'm testing out TensorFlow LSTM layer text generation task, not classification task; but something is wrong with my code, it doesn't converge. What changes should be done? Source code: ...
32 views

### Metrics of quality of parameter space exploration

Considering a black box optimization problem on non-linear, non-convex function where we want to minimize an objective function. One way to assess the quality of an optimizer is to look at the best ...
30 views

### why the sigmoid function will be 1 and 0 if we use a fully connected layer that produce a big enough positive(res negative )output

HI I am using a fully connected network that uses sigmoid if we feed a a big enough weights the sigmoid function will finally become 1 or 0 , is there any solution to avoid this ? and will this lead ...
67 views

### Advantages of Kullback-Leibler over L1/L2?

I've recently encounter different articles that are recommending to use KL instead of L1/L2 norm when trying to minimize a probability distribution. But none of the articles are giving a clear ...
39 views

### Create optimizer object using the tf.keras.optimizers.get function

I am trying to have the type of optimizer as a variable in the hyperparameter tuning phase. For that reason I am trying to use the tf.keras.optimizer.get function ...
229 views

### Evolutionary neural architecture?

I'm working on an idea for an AI architecture, and would like to know if there are any apparent flaws, or if there is prior work in this vein. Set I/O so that the neural network can read and write ...
48 views

### Does Retina-net's focal loss accomplish its goal?

Taking out the weighting factor we can define focal loss as $$FL(p) = -(1-p)^\gamma log(p)$$ Where $p$ is the target probability. The idea being that single stage object detectors have a huge ...
28 views

### Is convergence to a local minima more likely with transfer learning?

While doing transfer learning where my two problems are face-generation and car-generation is it likely that, if I use the weights of one problem as the initialization of the weights for the other ...
13 views

### Derivation for Value Iteration of CVaR

I am reading a paper named Risk-sensitive and Robust Decision-making: a CVaR Optimization Approach. In appendix A.3 they provide a proof for their Theorem $4$. The $n=1$ case for equation (11) is ...
52 views

### Could the Jensen-Shannon divergence and Kullback-Leibler divergence be used as loss functions of non-generation problems?

If I understand correctly, the KL divergence is a measure of information loss between a ground truth distribution $P$ and a predicted distribution $Q$, and the Jensen-Shannon divergence is the mean of ...
92 views

### Can we optimize an optimization algorithm?

In this answer to the question Is an optimization algorithm equivalent to a neural network?, the author stated that, in theory, there is some recurrent neural network that implements a given ...
48 views

### Query regarding the minmax loss function formulation of the training of a Generative Adversarial Network (GAN)

Just needed a clarification on the training procedure for a standard GAN. Of my understanding the loss function to optimize is a min max (max min causing mode collapse due to focus on one class ...
55 views

### Is an optimization algorithm equivalent to a neural network?

Is an optimization algorithm equivalent to a neural network?
47 views

### Is it possible to have a dynamic $Q$-function?

I am trying to use Q-learning for energy optimization. I only wish to have states that will be visited by the learning agent, and, for each state, I have a function that generates possible actions, so ...
46 views

### Is a neural network the correct approach to optimising a fitness function in a genetic algorithm?

I've written an application to help players pick the optimal heroes during the draft phase of the Heroes of the Storm MOBA. It can be daunting to pick from 80+ characters that have synergies/counters ...
32 views

### Can we use the Tierra approach to optimize machine code?

Thomas Ray's Tierra is a computer program which simulates life. In the linked paper, he argues how this simulation may have real-world applications, showing how his digital organisms (computer ...
23 views

### Custom optimizer and word-vector evaluator lstm

I’m using Keras LSTM layers and building a model that is trained off ethics text. I have a problem of often over fitting (the network basically remembers my input corpus as it is very small). I was ...
72 views

### How does NEAT find the most successful generation without gradients?

I'm new to NEAT, so, please, don't be too harsh. How does NEAT find the most successful generation without gradient descent or gradients?
67 views

### Why is a mix of greedy and random usually “best” for stochastic local search?

I read that a mix of "greedy" and "random" are ideal for stochastic local search (SLS), but I'm not sure why. It mentioned that the greedy finds the local minima and the randomness avoids getting ...
14 views

19 views

### Feature visualization on neural networks which are not for classification

Feature visualization allows to better understand neural networks by generating images that maximize the activation of a specific neuron, and therefore understand what are the abstract features that ...