Hot answers tagged

20

An algorithm is sample efficient if it can get the most out of every sample. Imagine learning trying to learn how to play PONG for the first time. As a human, it would take you within seconds to learn how to play the game based on very few samples. This makes you very "sample efficient". Modern RL algorithms would have to see $100$ thousand times more data ...


15

The paper's authors needed to implement their models anyway in order to conduct their experimentations, so why not publish the implementation? Some papers and authors actually provide a link to their own implementation, but most of the papers (that I have read) don't provide it, although some third-party implementations may already be available on Github (...


11

The two tech reports below both call RNNs explicitly "recurrent net(work)s". Rumelhart, David E; Hinton, Geoffrey E, and Williams, Ronald J (Sept. 1985). Learning internal representations by error propagation. Tech. rep. ICS 8504. San Diego, California: Institute for Cognitive Science, University of California. Jordan, Michael I. (May 1986). ...


8

The equation $$\hat{y} = \sigma(xW_\color{green}{1})W_\color{blue}{2} \tag{1}\label{1}$$ is the equation of the forward pass of a single-hidden layer fully connected and feedforward neural network, i.e. a neural network with 3 layers, 1 input layer, 1 hidden layer, and 1 output layer, where the input layer is connected to the hidden layer (all scalar inputs ...


7

When crossover happens and one parent is fitter than the other, the nodes from the more fit parent are carried over to the child. This is the case as disjoint and excess genes are only carried over from the fittest parent. Here's an example: // Node Crossover Parent 1 Nodes: {[0][1][2]} // more fit parent Parent 2 Nodes: {[0][1][2][3]} Child Nodes: {[0]...


7

Although I have only partially read (or not read at all) some of the following resources and some of these resources may not cover more advanced topics than the ones presented in the book you are reading, I think they can still be useful for your purposes, so I will share them with you. I would also like to note that if you understand the contents of the ...


6

Sample Efficiency denotes the amount of experience that an agent/algorithm needs to generate in an environment (e.g. the number of actions it takes and number of resulting states + rewards it observes) during training in order to reach a certain level of performance. Intuitively, you could say an algorithm is sample efficient if it can make good use of every ...


5

I will try to give a broad answer, if it's not helpful I'll remove it. When we talk about sampling we are actually talking about the number of interaction required to an agent to learn a good model of the environment. In general I would say that there are two issues related to sample efficiency: 1 the size of the 'action'+'environment states' space 2 the ...


5

Someone can argue to some human adequate reasons, but there is a bad trend of falsified results in deep learning research papers that propose some nowel solutions or even update state-of-the-art model performance. And that's not just a few papers that lie, it's a large portion of them. And the reason for that is even more sad - most of so-called deep ...


5

The first reason described in nbro's answer can definitely be an important one; authors may have implemented their software using code that they can't share. There's a lot of research coming out of companies (large and small), and they may use all sorts of proprietary libraries that were built in the company and cannot be distributed outside. As described in ...


5

In terms of transfer learning, semantic gap means different meanings and purposes behind the same syntax between two or more domains. For example, suppose that we have a deep learning application to detect and label a sequence of actions/words $a_1, a_2, \ldots, a_n$ in a video/text as a "greeting" in a society A. However, this knowledge in Society ...


4

Examining the architecture of the DNC indeed shows many similarities to the LSTM. Consider the diagram in the DeepMind article that you linked to: Compare this to the LSTM architecture (credit to ananth on SlideShare): There are some close analogs here: Much like the LSTM, the DNC will perform some conversion from input to fixed-size state vectors (h and ...


4

Go predictions were included in the paper: The experts are far from infallible. They predicted that AI would be better than humans at Go by about 2027. (This was in 2015, remember.) SOURCE: Experts Predict When Artificial Intelligence Will Exceed Human Performance (MIT Tech Review)


4

In GE, the genotype is a linear sequence of codons. By "wrapping" it, you make it a circular sequence that never ends. It allows you to build a bigger tree, while having only a few codons. Still, it is possible to find such a combination of a genotype and a grammar that defines an infinitely deep expansion — such combinations are hardly suited for ...


4

There is actually a book called Artificial General Intelligence by Ben Goertzel and Cassio Pennachin. It's a bit out of date (from 2008), and published as a Springer-Verlag monograph (which tends to have fairly low editorial standards). This one is also an anthology, with each chapter written by a different author. It's probably not suitable as an ...


4

A non-starving policy is a (behavior) policy that is theoretically guaranteed to visit each state and take all possible actions from each state an infinite number of times, so that to always update $Q(s, a)$, $\forall s, \forall a$, an infinite number of times. In the context of off-policy prediction, this criterion implies that any trajectory will have no ...


4

The following articles Ising models for networks of real neurons (2006) by Gasper Tkacik et al. Deep neural networks for direct, featureless learning through observation: The case of two-dimensional spin models (2018) by Kyle Mills et al. Inverse Ising inference by combining Ornstein-Zernike theory with deep learning (2017) by Soma Turi, Alpha A. Lee et al. ...


4

In language theory, there are generally several admitted levels that can be studied in relation with one another or independently. The semantic level is the one dealing with the meaning of the text ("semantic" comes from the greek and means "to signify"). The semantic level is therefore generally independent from the syntax and even the language used to ...


4

This is mostly because humans already have information when they start learning the game (priors) that makes them learn it more quickly. We already know to jump on monsters or avoid them or to get gold looking object. When you remove these priors you can see a human is worse at learning these games. (link) Some experiments they tried in the study to ...


4

Most model-fitting is stochastic, so you get different parameters every time you train, and you usually can't say that one algorithm will always give you a better-performing model. However, since you can retrain many times to get a distribution of models, you can use a statistical test like the T-Test to say "algorithm A usually produces a better model ...


4

Recently arxiv.org added a Code Tab towards the end of paper descriptions. Which contains links to both the official and community code. I don't know if this is the case for all the papers or not till know. But I'm sure it'll be extended to all the papers in a short while.


4

Yes, it's guessing. In the training phase, you show it lots of coarse and detailed pictures, and the algorithm learns a mapping from course to detailed. Then you present it a new coarse image, and it executes the same mapping. The information from the original picture is gone, and it cannot be retrieved, so it's filled in by analogy to other cases. "...


3

If the gradients are noisy (does this mean that in some dimension we have small and in some high curvature or that error noise differs for very similar values of w?) Gradients being noisy means that they are "inconsistent" across different epochs / training steps. With that I mean that they'll sometimes point in one direction, later in a different ...


3

How would you implement this "Number of Steps" cost? What the paper is referring to is the reward discounting process which is a standard way of formulating RL problems, either continuous ones, or episodic ones where the goal is to complete a task in the least time (in the episodic version, a fixed cost per time step will also achieve this). As ...


3

1): The intuition is based on the concept of value iteration, which the authors mention but don't explain on page 504. The basic idea is this: imagine you knew the value of starting in state x and executing an optimal policy for n timesteps, for every state x. If you wanted to know the optimal policy (and it's value) for running for n+1 timesteps in each ...


3

Warren McCulloch and Walter Pitts talk about recurrent neural nets in their paper McCulloch, W.S., Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 5, 115–133 (1943). https://doi.org/10.1007/BF02478259. They finish their introduction with the paragraph: The nervous system contains many circular ...


3

Hopfield networks, a special case of RNNs, were first proposed in 1982: https://www.pnas.org/content/79/8/2554 Otherwise (shameless plug, I am the author) a non-technical timeline for NLP can be found here: https://blog.exxcellent.de/ki-machine-learning


3

You need 10-bits ($2^{10} = 1024$) to represent 1000 classes.


3

Dennis Soemers provides an important point that from a theoretical standpoint, this can be seen as a non-issue. However, what you bring up is an important practical issue of potential-based reward shaping (PBRS). The issue is actually worse than you describe---it's more general than $s = s'$. In particular, the issue presents itself differently based on the ...


3

I don't think the situation you're sketching should be a problem at all. If $P(s)$ is high (e.g. $P(s) = 1000$), this means (according to your shaping / "heuristic") that it's valuable to be in the state $s$, that you expect to be able to get high future returns from that state. If you then continuously take actions that keep you in the same state $s$, it ...


Only top voted, non community-wiki answers of a minimum length are eligible