9

I think you're coming at your problem slightly wrong... what you're essentially talking about is a belief network. You may want to look into existing Bayesian Learning techniques to get your head around this, but belief networks commonly use the exact scenario you're talking about; using a set of known (or uncertain facts) statements to produce some inferred ...


8

The discount factor does appear twice, and this is correct. This is because the function you are trying to maximise in REINFORCE for an episodic problem (by taking the gradient) is the expected return from a given (distribution of) start state: $$J(\theta) = \mathbb{E}_{\pi(\theta)}[G_t|S_t = s_0, t=0]$$ Therefore, during the episode, when you sample the ...


8

Both algorithms fall into the category of "best-first search" algorithms, which are algorithms that can use both the knowledge acquired so far while exploring the search space, denoted by $g(n)$, and a heuristic function, denoted by $h(n)$, which estimates the distance to the goal node, for each node $n$ in the search space (often represented as a graph). ...


7

The following post has a bit of math, which I hope helps to explain the problem better. Unfortunately it seems, this SE site does not support LaTex: Document summarization is very much an open problem in AI research. One way this task is currently handled is called "extractive summarization". The basic strategy is as follows: Split this document into ...


6

It appears to use Recurrent NNs (RNNs) that have a 'Long Short-Term Memory' (LTSM) architecture. Here's a summary of the development process that the author, Ross Goodwin, went through to create it. It seems to me (and is also observed in the above link) that the output is rather poor - simply comparable to what one might expect from Markov chains, a ...


6

Yes, this is possible. There is actually a pretty easy way that doesn't even require machine learning and can be implemented with a small amount of code. You just use a framework for image processing (e.g. PIL for Python), find the marks by going over your image with an appropriate filter and use the implemented crop function, that the framework hopefully ...


6

Neil's answer already provides some intuition as to why the pseudocode (with the extra $\gamma^t$ term) is correct. I'd just like to additionally clarify that you do not seem to be misunderstanding anything, Equation (13.6) in the book is indeed different from the pseudocode. Now, I don't have the edition of the book that you mentioned right here, but I ...


5

A closed expression refers to a formula which has no free variables [1]. This is also called sentence. In a logic system you have a set of axioms which are sentences and rules which state how to derive a sentence from this [2]. If a sentence can be derived from the axioms, this means that the axioms entail this sentence. If a sentence is not derivable, it is ...


5

Typically, Monte-Carlo Tree Search (MCTS) actually is the go-to "solution" for such problems with large branching factors. I can understand that "vanilla" MCTS may still have unsatisfactory performance, but there is a plethora of extensions/enhancements available. I don't have experience with the specific game you mentioned (Connect6), but from a quick look ...


5

As @nbro has already said that Hill Climbing is a family of local search algorithms. So, when you said Hill Climbing in the question I have assumed you are talking about the standard hill climbing. The standard version of hill climb has some limitations and often gets stuck in the following scenario: Local Maxima: Hill-climbing algorithm reaching on the ...


4

It will not be single DNN architecture, rather it will be a collection of different DNN architectures that are used together to make the final decision. Convolutions are using the images/videos from the camera. Other architectures use other sensory sources. These DNNs will be trained to compute the high-level features from their sensory sources and then ...


4

Some good places to start would be cognitive architectures and as mentioned in another answer intelligent agents. The question is broad but you definitely want to look into planning & decision making. You might also want to check out the L5 and L6 layers of Hierarchical Temporal Memory (As in Nupic) as it relates to feedback, behavior and attention. If ...


4

For future reference, I will merely point you to a technique you can implement to test the correctness or lack thereof, of your backpropagation implementation. Ps: don't feel too bad for having gotten it slightly wrong, "backpropagation is notoriously difficult to implement" - source :). In fact, there is a technique called "Gradient checking" meant ...


4

The difference between a local search algorithm (like beam search) and a complete search algorithm (like A*) is, for the most part, small. Local search algorithms will not always find the correct or optimal solution, if one exists. For example, with beam search (excluding an infinite beam width), it sacrifices completeness for greater efficiency by ordering ...


4

This will not be that hard of a problem once you have a lot of training data. But, before you have a lot of training data, you will need to get some training data one way or another. You will need a lot of training data for quite a few of the models that will give you a high accuracy. Then, you will probably want to use a Long short term memory recurrent ...


4

It's a subtle issue. If you look at the A3C algorithm in the original paper (p.4 and appendix S3 for pseudo-code), their actor-critic algorithm (same algorithm both episodic and continuing problems) is off by a factor of gamma relative to the actor-critic pseudo-code for episodic problems in the Sutton and Barto book (p.332 of January 2019 edition of http://...


4

Hill climbing is not an algorithm, but a family of "local search" algorithms. Specific algorithms which fall into the category of "hill climbing" algorithms are 2-opt, 3-opt, 2.5-opt, 4-opt, or, in general, any N-opt. See chapter 3 of the paper "The Traveling Salesman Problem: A Case Study in Local Optimization" (by David S. Johnson and Lyle A. McGeoch) for ...


4

What you are proposing is closer to a heuristic for searching than a reward for RL. This is a blurred line, but generally if you start analysing the problem yourself, breaking it down into components and feeding that knowledge into the algorithm, then you place more emphasis on your understanding of the problem, and less on any learning that an agent might ...


4

There are many computer vision (CV) algorithms and models that are used for different purposes. So, of course, I cannot list all of them, but I can enumerate some of them based on my experience and knowledge. Of course, this answer will only give you a flavor of the type of algorithm or model that you will find while solving CV tasks. For example, there are ...


3

Semantics Matters The answer depends on the definition intelligence being used. If you define intelligence as the ability to adapt, a number of things could be considered intelligent that don't normally fit under the classic AI umbrella. Nonlinear least-squares Marquardt-Levenberg curve fitting algorithm with a substantial but finite set of models, ...


3

Text approach: Use LDA (Latent Dirichlet Allocation). LDA is unsupervised. Feed it in corpuses of text from the various documents (i.e. OCR them and feed LDA the results of OCR). It will then cluster them based on the contents of the text (with or without stop words - at your discretion). If possible, you could do a supervised approach of using a bag-of-...


3

Tabu search uses memory to rule out parts of the neighborhood for local search, allowing the trajectory to typically pass through local optima instead of getting stuck in them.


3

You could parallelize the search by dividing the global space in distinct regions/subsets. Then apply in each region a local search. This way you can search the global space systematically, more exhaustively and perhaps in different ways (e.g by applying a different local search method to each region). Finally you can compare the results and choose the best ...


3

AFAIK, normally detection algorithms work in a sub-window of the image and not the whole of it. For example, for a specific size and orientation you slide a sub-window on the image and extract sub-images. Then you apply your algorithm on every sub-image for detection and report the size-and-orientations with positive results. You can have a single neural ...


3

What you are calling 'analyzing the surroundings' is generally referred to as perception. Self-driving cars sense their surroundings using cameras, radars, lidars often combining or fusing more than one sensor to paint a picture of the environment. A lot of algorithms get used for fusing the sensor data and then deriving an understanding of the surrounding. ...


3

Self-driving cars use a combination of both supervised as well as reinforcement learning. Huge amounts of sensor data are recorded in real-time. This data can be used to train all sorts of supervised classifiers, e.g. for predicting rain or switching on lights. You can also set up a model to predict pedestrians and other cars. This is supervised learning. ...


3

This two-part article by Trung Tran about Real Time Object Recognition demonstrates differences between Keras and OpenCV from a coding point of view is a reasonable treatment. It also covers implementing pre-trained VGG16 models. The application in this Campus Hippo lesson uses both Keras and OpenCV to simulate a self-driving car. This tutorial by Adrian ...


3

There is no defined rules for choosing a machine learning algorithm to learn some type of pattern. However, there are some guidelines to help you better select an algorithm which will yield a higher probability of success. Some important considerations are: Number of features: This is the number of questions that each patient had to answer. Number of ...


3

You may start assigning penalties for undesirable conditions in a state like: 1) Number of blocks outside stack 0. Supose you penalize with 10 units each block outside stack 0, then the starting state above adds 40 units to the penalty score 2) Number of blocks in the stack 0 in a position different than in the goal state. Supose you penalize with 50 ...


3

When we climb a hill: We move higher in altitude. The person who is climbing, will always look for rocks/mud on the hill that are higher, so that he can climb higher. That is what the algorithm does too. We are assuming that there is a hill of numbers. The larger numbers are placed higher than the smaller numbers. So if we want to climb up the hill, we ...


Only top voted, non community-wiki answers of a minimum length are eligible