Questions tagged [optimization]

For questions about implementing and improving optimization algorithms used in creating AI programs, or optimization in general.

47 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
40 views

How to use MOPSO to align characters vertically?

I need to efficiently align characters vertically using Multi Objective PSO. Alignment is achieved by adding spaces in between a given set of characters. ...
3
votes
0answers
60 views

How does SGD escape local minima?

SGD is able to jump out of local minima that would otherwise trap BGD I don't really understand the above statement. Could someone please provide a mathematical explanation for why SGD (Stochastic ...
3
votes
0answers
71 views

Does Retina-net's focal loss accomplish its goal?

Taking out the weighting factor we can define focal loss as $$FL(p) = -(1-p)^\gamma log(p) $$ Where $p$ is the target probability. The idea being that single stage object detectors have a huge ...
2
votes
0answers
22 views

How are the lower and upper bound values of the moths determined in the Moth-Flame Optimization algorithm?

I am currently implementing the Moth-Flame Optimization (MFO) Algorithm, based on the paper: Moth-Flame Optimization Algorithm: A Novel Nature-inspired Heuristic Paradigm. To calculate the values of ...
2
votes
1answer
78 views

When training a CNN, what are the hyperparameters to tune first?

I am training a convolutional neural network for object detection. Apart from the learning rate, what are the other hyperparameters that I should tune? And in what order of importance? Besides, I read ...
2
votes
0answers
33 views

Is logistic regression used for unconstrained or constrained optimisation problems?

Is logistic regression used for unconstrained or constrained optimization problems, and why?
2
votes
0answers
113 views

How can I assign agents to tasks based on time and affinity?

I am working on an assignment problem. Consider $K$ agents $A_1, \dots A_K$ and $N$ tasks $T_1, \dots T_N$. Each task has a certain time $t(T_i)$ to be completed and each agent has a matching (or ...
2
votes
0answers
54 views

Is a neural network the correct approach to optimising a fitness function in a genetic algorithm?

I've written an application to help players pick the optimal heroes during the draft phase of the Heroes of the Storm MOBA. It can be daunting to pick from 80+ characters that have synergies/counters ...
2
votes
0answers
93 views

Is a very powerful oracle sufficient to trigger the AI singularity?

Lets say we have a oracle $S$ that, given any function $F$ and desired output $y$, can find an input $x$ that causes $F$ to output $y$ if it exists, or otherwise returns nil. I.e.: $$S(F, y) = x \...
2
votes
0answers
86 views

Which features and algorithm could optimize this air-conditioner problem?

Imagine we have 2 air conditioner systems (AA) and 2 "free cooling" systems which mix external and internal air (FC) in a closed box which always tends to warm up. For each system, we have to find ...
2
votes
0answers
36 views

Knapsack of mixture with constraints

I'm trying to find the optimized mixture for a specific set of substances. Each of those substances have characteristics that I want to optimize in the mixture (some characteristics I want to minimize ...
1
vote
0answers
10 views

Root finding in Deep Equilibrium Models

In the Deep Equilibrium Model the neural network can be seen as "infinitely deep". Training learns a nonlinear function as usual. But there is no forward propagation of input data through ...
1
vote
0answers
25 views

Why scaling down the parameter many times during training will help the learning speed be the same for all weights in Progressive GAN?

The title is one of the special things in Progressive GAN, a paper of the NVIDIA team. By using this method, they introduced that Our approach ensures that the dynamic range, and thus the learning ...
1
vote
2answers
28 views

Why does Simulated Annealing not take worse solution if the Energy Difference becomes higher?

In Simualated Annealing a worse solution is accepted with this probability: $p=e^{-\frac{E(y)-E(x)}{kT}}$. If that understanding is correct: Why is this probability function used? Because it means ...
1
vote
0answers
18 views

Which method of tree searching should be used for this board game?

Suppose the following properties of a board game: High branching factor in the beginning of the game (~500) which slowly tends towards 0 at the end of the game Evaluation of the any given board ...
1
vote
1answer
52 views

Continuous state and continuous action Markov decision process time complexity estimate: backward induction VS policy gradient method (RL)

Model Description: Model based(assume known of the entire model) Markov decision process. Time($t$): Finite horizon discrete time with discounting factor State($x_t$): Continuous multi-dimensional ...
1
vote
0answers
35 views

How many training runs are needed to obtain a credible value for performance?

I'm trying to optimize a neural network. For that, I'm changing parameters like the batch size, learning rate, weight initialization, etc. A neural network is not a deterministic algorithm, so, in ...
1
vote
0answers
43 views

Simplifying Log Loss

I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
1
vote
3answers
99 views

How to improve neural network training against a large data set of points with varying magnitude

I am currently using TensorFlow and have simply been trying to train a neural network directly against a large continuous data set, e.g. $y = [0.014, 1.545, 10.232, 0.948, ...]$ corresponding to ...
1
vote
0answers
35 views

Are there optimizers that schedule their learning rate, momentum etc. autonomously?

I'm aware there are some optimizer such as Adam that adjust the learning rate for each dimension during training. However, afaik, the maximum learning rate they can have is still determined by the ...
1
vote
0answers
19 views

Optimizer effects on neural network with two outputs

I'm confused about the following issue. Let assume that we have a neural network that takes one input and two outputs. I try to visualize my model like as follows: ...
1
vote
0answers
28 views

Imposing contraints on sequence of image classifications

Are there example implementations of networks that apply constraints across sequences of image classifications where class labels are ordinal numbers? For example, to cause the output of a CNN to ...
1
vote
0answers
109 views

How does the automated temperature adjustment step work in Soft Actor-Critic?

In section 5 of the paper Soft Actor-Critic Algorithms and Applications, it is proposed an optimization problem to obtain an optimal temperature parameter $\alpha^*_t$. First, one uses the original ...
1
vote
0answers
37 views

Is convergence to a local minima more likely with transfer learning?

While doing transfer learning where my two problems are face-generation and car-generation is it likely that, if I use the weights of one problem as the initialization of the weights for the other ...
1
vote
0answers
128 views

Could the Jensen-Shannon divergence and Kullback-Leibler divergence be used as loss functions of non-generation problems?

If I understand correctly, the KL divergence is a measure of information loss between a ground truth distribution $P$ and a predicted distribution $Q$, and the Jensen-Shannon divergence is the mean of ...
1
vote
0answers
16 views

Estimating Baselines using ALS

I am trying to figure out how ALS works when minimizing the following formula: $\\ \\$ $\text{min}_{\lbrace b_u,b_i \rbrace} \sum_{(u,i)\in \mathcal{K}} (r_{ui} - \bar{r} - b_u - b_i )^2 + \lambda_{...
1
vote
1answer
52 views

How to properly optimize shared network between actor and critic?

I'm building an actor-critic reinforcment learning algorithm to solve environments. I want to use a single encoder to find representation of my environment. When I share the encoder with the actor ...
1
vote
0answers
19 views

What is the purpose of the new neurons in the constrained neural network?

I would like to train a constrained neural network. I found a paper on this: https://papers.nips.cc/paper/4-constrained-differential-optimization.pdf. However, I don't really understand how to change ...
1
vote
0answers
29 views

Any guidance on learning rate / batch size for noisy data (high Bayes error rate)?

Is there any guidance available for training on very noisy data, when Bayes error rate (lowest possible error rate for any classifier) is high? For example, I wonder if deliberately (not due to memory ...
1
vote
0answers
22 views

Feature visualization on neural networks which are not for classification

Feature visualization allows to better understand neural networks by generating images that maximize the activation of a specific neuron, and therefore understand what are the abstract features that ...
1
vote
0answers
32 views

Classical Internet routing vs. Swarm routing (such as Ant routing)?

Is it possible to mention the drawbacks/advantages of Swarm routing (such as Ant routing etc) in comparison with classical routing algorithms in communication networks in a general view? In other ...
1
vote
0answers
18 views

What does it essentially mean if the neural network has convex error surface?

Suppose if I am building a Linear Regression model with one fully connected layer and a sigmoid with minimizing mean squared error as objective. Why would the error surface be convex? Does finding ...
1
vote
1answer
775 views

What are the advantages and disadvantages of using LISP for constraint satisfaction in 3D space

We are currently working on developing a 3D modeling software that allows designers to set spatial constraints to models. The computer then should generate a 3D mesh conforming to these constraints. ...
1
vote
0answers
48 views

Optimization step in Apprenticeship Learning via Inverse Reinforcement Learning

Why the optimization step of the algorithm a quadratic program? [See: Apprenticeship Learning via Inverse Reinforcement Learning; page 3] Isn't the objective function linear? Why don't we treat ...
1
vote
0answers
46 views

How to calculate Adaptive gradient?

In the FaceNet paper there mentions an gradient algorithm called 'AdaGrad'(Adaptive Gradient) referenced to this paper which has apparently been used to calculate the gradient of the Triplet Loss ...
1
vote
0answers
269 views

Application of Ai to task scheduling problems on heterogenous platforms

Let's say we have a cluster of 20-2000 heterogenous compute nodes. Consider for example the parallel solution of the helmholtz equation: Now we want to distribute the solution process and, to make ...
1
vote
0answers
75 views

Which algorithm would you use to solve a multiple producer-consumer problem with constraints?

I'm solving this problem similar to consumer-producer of materials (i.e. sand). This is the graph of the problem: Where Req (...
0
votes
0answers
11 views

How to determine the most appropriate metaheuristic for a problem?

I have a scheduling problem. but I don't know how to determine its complexity (NP-hard, NP-complete)? How could we justify the choice of a metaheuristic instead of other for a scheduling problem i.e. ...
0
votes
0answers
15 views

Partial pruning in counterfactual regret minimization (CFR)

I'm using CFR to solve a large imperfect-information game. One important technique for optimizing performance of this algorithm is "partial pruning", which allows the algorithm to skip ...
0
votes
0answers
26 views

Binary data clustering by Matrix factorization

I have read an article talking about binary clustering using Matrix factorization(see attached), but i would like to understand some optimization concepts: Is it reasonable to use a Frobenius norm in ...
0
votes
0answers
22 views

Finding the energy function given update rule of a single layer non-linear neural network

Consider the network with N neurons, each of which takes a $2 \times k$ input specified by the tuple $(\vec c_t, \vec \theta_t)$ to produce output $\vec{R}_t$ through an update rule on the pairwise ...
0
votes
1answer
65 views

How can I model this problem of delivering assets by choosing a route with reinforcement learning?

I would like to build a model based on reinforcement learning (RL) for the following scenario Recommend the best route (of cities listed for a given country) that satisfies the required criteria (...
0
votes
0answers
23 views

Which of these two strategies is the best to select solutions in simulated annealing?

I am using simulated annealing (SA) for an NP-hard combinatorial optimisation problem. 1) I am testing over a range of problem instances in which the objective values can be in the 100's or in the ...
0
votes
0answers
21 views

What are swarm optimization techniques used for: training the ANN by weight optimization or for feature selection?

Swarm intelligence (SI) is the collective behavior of decentralized, self-organized systems, natural or artificial. The concept is employed in work on artificial intelligence. SI-based algorithms, ...
0
votes
0answers
53 views

Can neural networks handle redundant inputs?

I have a fully connected neural network with the following number of neurons in each layer [4, 20, 20, 20, ..., 1]. I am using TensorFlow and the 4 real-valued ...
0
votes
0answers
29 views

Custom optimizer and word-vector evaluator lstm

I’m using Keras LSTM layers and building a model that is trained off ethics text. I have a problem of often over fitting (the network basically remembers my input corpus as it is very small). I was ...
-1
votes
1answer
2k views

RNN LSTM not converging with Adam

I am trying to train a RNN with text from wikipedia but I having having trouble getting the RNN to converge. I have tried increasing the batch size but it doesn't seem to be helping. All data is one ...