# Tag Info

10

A good answer to this question depends on what you want to use the labels for. When I think about "optimization," I think about a solution space and a cost function; that is, there are many possible answers that could be returned and we can know what the cost is of any particular answer. In this view, the answer is "yes"--pattern recognition is a case ...

10

This is a really common reaction after first encountering the No Free Lunch theorems (NFLs). The one for machine learning is especially unintuitive, because it flies in the face of everything that's discussed in the ML community. That said, the theorem is true, but what it means is open to some debate. To restate the theorem for people who don't know it, ...

8

I see no reason why decaying learning rates should create the kinds of jumps in losses that you are observing. It should "slow down" how quickly you "move", which in the case of a loss that otherwise consistently shrinks really should, at worst, just lead to a plateau in your losses (rather than those jumps). The first thing I observe in your code is that ...

8

TL:DR: Hyper-heuristics are metaheuristics, suited for solving the same kind of optimization problems, but (in principle) affording a "rapid prototyping" approach for non-expert practitioners. In practice, there are issues with the prevailing approach, motivating an emerging perspective on 'whitebox' hyper-heuristics. In more detail: Metaheuristics are ...

7

There are several elementary techniques to try and move a search out of the basin of attraction of local optima. They include: Probabalistically accepting worse solutions in the hope that this will jump out of the current basin (like Metropolis-Hastings acceptance in Simulated Annealing). Maintaining a list of recently-encountered states (or attributes ...

6

It is suggested that the number of hidden units in a layer should be in powers of 2 because it helps converge faster. I would quite like to see a reference to this suggestion, in case it has been misunderstood. As far as I know, there is no such effect in normal neural networks. In convolutional neural networks it might potentially be true in a minor way ...

6

You can run gradient descent without back propagation, in some cases: Simple structures such as linear or logistic regression, where the gradients can be calculated directly from the inputs and cost function value. In "black box" gradient-based learning algorithms where you don't know how (or don't want to) calculate gradient analytically, so you choose to ...

6

As @nbro has already said that Hill Climbing is a family of local search algorithms. So, when you said Hill Climbing in the question I have assumed you are talking about the standard hill climbing. The standard version of hill climb has some limitations and often gets stuck in the following scenario: Local Maxima: Hill-climbing algorithm reaching on the ...

5

Well, you are definitely mixing two different things. Here are those bits: The function that deep learning approximates is basically a function that best fits the INPUT DATA points. You should not think about its differentiability or optimization aspects. We don't care what type of function it is; we just want the best fit of input data (ofcourse ...

5

Usually you keep track of training loss and validation loss and apply proper regularization technique (L1, L2, dropout, dropconnect, ...). The more interesting technique is to observe your validation loss with respect to the number of parameters in the network (often controlled by the number of layers/feature maps). If the validation starts dropping with ...

5

I can offer two (at first sight, conflicting) perspectives on this: Firstly: If the letter string 'abc' becomes 'abd' what would "doing the same thing" to 'ijk' look like? This is just one example of a problem (so-called 'letter-string analogy problems') that is not easily framed as an optimization problem - there is a range of answers that appear ...

5

Hill climbing is not an algorithm, but a family of "local search" algorithms. Specific algorithms which fall into the category of "hill climbing" algorithms are 2-opt, 3-opt, 2.5-opt, 4-opt, or, in general, any N-opt. See chapter 3 of the paper "The Traveling Salesman Problem: A Case Study in Local Optimization" (by David S. Johnson and Lyle A. McGeoch) for ...

4

For a finite value to be 'optimal,' typically you need some benefit from more paired up with some cost for more, and eventually the lines cross because the benefit decreases and the cost increases. Most models will have a reduction in error with more training data, that asymptotically approaches the best the model can do. See this image (from here) as an ...

4

To build on Neil's answer a bit, you're right that the better your evaluation function gets, the less work your optimization function will need to perform. If your evaluation function gets good enough, you won't need to search at all. This is not just an academic idea though! It's actually fairly widely used, and has been key to solving several games. The ...

4

Gradient descent (GD) is an optimisation algorithm, that is, it is used to find a (local) minimum of a multi-variable and differentiable function $f$. GD is an iterative and numerical optimisation algorithm. It is iterative because it proceeds in iterations. It is numerical because it is not an algorithm which produces an exact solution, due to numerical ...

4

This is a very large question that could be answered in a variety of ways depending on the context. For some optimization problems operating under specific conditions you can make theoretical guarantees that your optimization will solve your problem. A specific example of this is running the gradient descent algorithm on a convex function. If the function ...

3

I understand your question to be: If some moves are compulsory, and my agent has no choice about which move to make next, do I need to perform a search, or can I just return the compulsory move? The answer depends on what your goal is. If your goal is to make an interactive agent that will play the game against you, then you are correct: there's no need ...

3

The differences you have observed between the two different versions of the TRPO paper are due to different formalizations of the problem and the objective. In the first version of the paper you linked, they start out in Section 2 by defining Markov Decision Processes (MDPs) as tuples that, among other things, have a cost function $c : \mathcal{S} \... 3 In general I agree with @nbro answer, nevertheless sticking strictly to this specific question I'd like to share some speculations: what the author of the question provides us with is the Loss Function Shape so I'll try to use the full information here to compare the 2 minima looking at the LF steepness we observe the Left LM is in a steeper region than ... 3 Empirically, I observed that algorithms like Adam and RMSProp tended to give me a final higher performance (in my case, the accuracy) on (the validation dataset) with respect to SGD. However, I also observed that Adam and RMSProp are highly sensitive to certain values of the learning rate (and, sometimes, other hyper-parameters like the batch size) and they ... 3 NEAT is a genetic algorithm (GA). A genetic algorithm maintains a population of individuals (or chromosomes) and evolves it using operations like the crossover or the mutation, so that the fittest individuals keep living and most other individuals die. The nature of the individuals depends on the problem. For example, in the case of NEAT, the individuals are ... 3 Yes it has been tried. In fact there is a whole field, dubbed Genetic Programming. There is an annual competition to obtain "Human-Competitive" algorithms, and many instances of those have been found over the years. 3 First, you need to consider what are the "parameters" of this "optimization algorithm" that you want to "optimize". Let's take the most simple case, a SGD without momentum. The update rule for this optimizer is: $$w_{t+1} \leftarrow w_{t} - a \cdot \nabla_{w_{t}} J(w_t) = w_{t} - a \cdot g_t$$ where$w_t$are the weights at iteration$t$,$J$is the cost ... 3 writing here my suggestion, because i haven't earned the right to comment yet. Your main "problem" could be your loss function. It converges, this is why your loss value is decreasing. So I suggest to let it maybe train longer. Alternatively you could change the loss function to fit your need. For example you could use: loss = tf.reduce_mean(tf.square(... 3 For the first question, RMSE and Euclidean distance have no difference, not that i know of. For the second question, you only need the common loss function for normal tasks. MSE is a common loss function used in linear regression tasks as well as loss function similar in nature like the RMSE. For classification tasks, Cross Entropy Loss is preferred. For ... 2 Honestly, nobody knows. Any talk of sentient AI's is still basically sci-fi and we can't really offer anything more than informed speculation. But think about it this way: sentience, in and of itself, doesn't necessarily involve any "goals" or "desires" or "objectives" beyond what the AI creator programmed in. Be careful not to over anthropomorphize and ... 2 A common concept in AI is "recursive self-improvement." That is, the AI 1.0 would build a version 1.01, which would build a version 1.02, and so on. This is probably not going to be thought of as the newer version 'destroying' the older version; if an AI can self-modify, it's probably going to be more like going to sleep and waking up smarter, or learning a ... 2 I'm not an expert in this area, but it would appear to depend on the choice of activation function:$e^x$is not Lipschitz continuous. See Analytic functions which are not Lipschitz continuous.$\tanh(x)$is. That said, this paper appears to give some conditions (specifically for dynamic ANNs) for which networks with activation function involving$e^x$can ... 2 In general, the larger the training set, the better. See The Unreasonable effectiveness of Data, though this article is quite dated (written in 2009). Xavier Amatriain, a researcher at Netflix has a Quora answer where he discusses that more data can sometimes hurt algorithms. For deep neural networks in particular, it does not seem that we have hit these ... 2 The error in the code is simply having a$+$rather than a$-$sign. Line 4 of the algorithm says: $$E\left[ g^2 \right]_t = \rho E\left[ g^2 \right]_{t - 1} + (1 - \rho) g_t^2,$$ but your code implements (note the$+\$ inside the brackets at the end): $$E\left[ g^2 \right]_t = \rho E\left[ g^2 \right]_{t - 1} + (1 + \rho) g_t^2.$$ A correct ...

Only top voted, non community-wiki answers of a minimum length are eligible