22

The problem of state representation in Reinforcement Learning (RL) is similar to problems of feature representation, feature selection and feature engineering in supervised or unsupervised learning. Literature that teaches the basics of RL tends to use very simple environments so that all states can be enumerated. This simplifies value estimates into basic ...


18

The output of the policy network is as described in the original paper: A move in chess may be described in two parts: selecting the piece to move, and then selecting among the legal moves for that piece. We represent the policy π(a|s) by a 8 × 8 × 73 stack of planes encoding a probability distribution over 4,672 possible moves. Each of the 8×8 ...


16

Dennis Soemers' answer is correct: you should use a HashSet or a similar structure to keep track of visited states in BFS Graph Search. However, it doesn't quite answer your question. You're right, that in the worst case, BFS will then require you to store 16! nodes. Even though the insertion and check times in the set will be O(1), you'll still need an ...


12

Section 4.2 of "Essentials of Metaheuristics" has a wealth of information on alternative ways of encoding graph structures via Genetic Algorithms. With particular regard to evolving ANNs, I would personally not be inclined to implement this sort of thing 'from scratch': The field of neuroevolution has been around for some time, and the implementation some ...


11

No, GANs are not used. It's reinforcement learning at what it does best. The tree search is an interesting addition and assists with navigating the sheer scale of the game. Although the agent was playing itself to become better, there wasn't 2 separate networks (generator and discriminator). The agent learned through RL and didn't have the error ...


9

Assuming it is a turn-based game and, for each turn, there's an optimal choice that will lead to the winning state (zero-sum), you can basically simplify the question to "What is the optimal sequences of moves for me to win, considering the current situation that is presented on the board?". So you will need to perform your algorithm every turn as the ...


9

A common early approach to modeling complex problems was discretization. At a basic level, this is splitting a complex and continuous space into a grid. Then you can use any of the classic RL techniques that are designed for discrete, linear, spaces. However, as you might imagine, if you aren't careful, this can cause a lot of trouble! Sutton & Barto's ...


8

Can there ever be a functionally complete set of grammar rules which can parse any statement in English (locale-specific) accurately and which can be possibly implemented for use in AI-based projects? Parse it yes, accurately most likely no. Why ? According to my understanding on how we derive meaning from sounds, there are 2 complementary strategies: ...


8

You can use a set (in the mathematical sense of the word, i.e. a collection that cannot contain duplicates) to store states that you have already seen. The operations you'll need to be able to perform on this are: inserting elements testing if elements are already in there Pretty much every programming language should already have support for a data ...


8

While the answers given are generally true, a BFS in the 15-puzzle is not only quite feasible, it was done in 2005! The paper that describes the approach can be found here: http://www.aaai.org/Papers/AAAI/2005/AAAI05-219.pdf A few key points: In order to do this, external memory was required - that is the BFS used the hard drive for storage instead of RAM....


8

Yes and no! There's no inherent reason that machine learning systems can't deal with extreme events. As a simple version, you can learn the parameters of a Weibull distribution, or another extreme value model, from data. The bigger issue is with known-unknowns vs. unknown-unknowns. If you know that rare events are possible (as with, say, earthquake ...


7

What 'infinite' means here could possibly be debated at some length, but that notwithstanding, here are two conflicting answers: 'Yes': Simulate all possible universes. Stop when you get to one containing a flavor of intelligence that passes whatever test you have in mind. Steven Wolfram has suggested something broadly along these lines. Problem: the state ...


7

There certainly appear to have been research projects involving some form of text mining / information retrieval /etc. and StackExchange sites. Some examples I was able to find through google/google scholar (unlikely to be anywhere near an exhaustive list): TACIT: An open-source text analysis, crawling, and interpretation tool describes numerous text-...


7

I think the comments are basically on the right track. PID controllers are useful for finding optimal policies in continuous dynamical systems, and often these domains are also used as benchmarks for RL, precisely because there is an easily derived optimal policy. However, in practice, you'd obviously prefer a PID controller for any domain in which you can ...


6

The cleanest result we have on this issue is the "no free lunch" theorem. Basically, in order to make a system perform better at a specific task, you have to degrade its performance on other tasks, and so there is a flexibility-efficiency tradeoff. But to the broader question, or whether or not your thinking is correct, I think it pays to look more closely ...


6

Yes. For instance, the popular softmax regression gives you probability distribution for each class. Yes. Softmax is a regression over a set of discrete classes. We can use regression for classification, the most common strategy is to grab the most likely class for the prediction.


6

I don't think I can give you a true answer to the actual question as posed, as I don't have a strict definition of "general intelligence". Nor do I have a solid definition of "critical" in context. However, if we lean on our naive/intuitive understanding of what "general intelligence" and what it means to be critical, you might ...


6

Let $D_a$ be the domain for A, and $a_i$ the elements of $D_a$. Let $D_b$ and $D_c$ work similarly for $B$ and $C$ respectively. We introduce $D_t = \{t | t = (c_i - b_j, c_i)\}$ for all $c_i$ in $D_c$ and $b_j$ in $D_b$. We can see that $\{t[0]\}$ (i.e. $\{c_i - b_j, \forall i,j\}$) must be equal to $D_a$. So we can represent the constraint on $A$ by ...


5

What is OpenCog? OpenCog is a project with the vision of creating a thinking machine with human-level intelligence and beyond. In OpenCog's introduction, Goertzel categorically states that the OpenCog project is not concerned with building more accurate classification algorithms, computer vision systems or better language processing systems. The OpenCog ...


5

As Matthew Graves explained in another answer No free lunch theorem confirms the flexibility - efficiency trade-off. However, this theorem is describing a situation where you have a set of completely independent tasks. This often doesn't hold, as many different problems are equivalent in their core or at least have some overlap. Then you can do something ...


5

Randomness is typically the best one can do with ignorance, rather than a source of strength in its own right. For example, the primary use of randomness in statistics is random assignment (A/B testing, randomized controlled trials, etc.). The reason to do this is to make the influence of confounders independent from the influence of the factor under ...


5

XML, HTML and less formal languages all respond quite nicely to being transformed or interrogated within a graph framework. XML and HTML are particularly useful in that they conform strictly to a tree-structure. That means that any good data components can be measured in terms of tree-distance to any other "good" data components. If you extract your regex-...


5

The reason this is hard is because it is not trivial to understand what a law means. Many humans still have a hard time understanding laws and thus we have millions of judges and lawyers who study years to be able to even debate whether a law was broken at all. More generally to AI, the problem of understanding laws is a byproduct of the bigger problem that ...


5

Automation Levels Most cars have some Level 1 automation, such as cruise control and various skid/flip probability reduction systems. Most high volume passenger vehicles have higher levels. Some military and private air, land, and sea equipment are already at Level 5. Level 4 requires that driving be automated during normal driving conditions, with manual ...


5

Let us describe a very simple system that does something we could label as empathyc. A chatbot answers "I am sorry to hear that. What happened?" when we type "I feel bad", and it replies "I am glad to hear that. Fancy some music?" when we type "I feel good". Somehow, it perceives a human emotion, and acts accordingly. Planes fly but they do not fly as ...


5

The difference between the validation and test set in my opinion should be explained in this way: the validation set is meant to be used multiple times. the test set is meant to be used only once. I think that the misunderstanding here arise because machine learning is mostly taught focusing only on a specific part of a large pipeline, which is the model ...


4

Using evolutionary algorithms to evolve neural networks is called neuroevolution. Some neuroevolution algorithms optimize only the weights of a neural network with fixed topology. That sounds not like what you want. Other neuroevolution algorithms optimize both the weights and topology of a neural net. These kinds of algorithms seem more appropriate for ...


4

You're going to need some way to 'see' the area around the car, and to track the speed of nearby objects. Google uses a combination of LIDAR, radar, conventional cameras, and occasionally sonar (see here for a high-level overview). This technology is quite expensive, and can easily cost thousands of US dollars. However, a bigger obstacle than the expense ...


4

We're definitely nowhere near that level of AI; at best, high-tech solutions like deep convolutional neural nets can help with image recognition and some other algorithms can perform things like robotic movement adequately enough to be useful in some scenarios. None of this is even as sophisticated as the behavior of a flea, but no one refers to insects as "...


4

Some good places to start would be cognitive architectures and as mentioned in another answer intelligent agents. The question is broad but you definitely want to look into planning & decision making. You might also want to check out the L5 and L6 layers of Hierarchical Temporal Memory (As in Nupic) as it relates to feedback, behavior and attention. If ...


Only top voted, non community-wiki answers of a minimum length are eligible