All Questions

Filter by
Sorted by
Tagged with
1
vote
0answers
8 views

How Training of the “Attention model” in “ Attention is all you need” paper done? What are Keys, Values? [closed]

I have recently encountered the paper on NLP. It is very new to me and I am still unable to see how that works. I have used all the resources over there from the original paper to Youtube videos and ...
2
votes
1answer
17 views

Is there any difference between reward and return in reinforcement learning?

I am reading Sutton and Barto's book on reinforcement learning. I thought that reward and return were the same things. However, in Section 5.6 of the book, 3rd line, first paragraph, it is written: ...
0
votes
0answers
7 views

Confusion about the proof that optimizing InfoNCE equals to maximizing mutual information

In the appendix of Representation Learning with Contrastive Predictive Coding, van den Oord et al. prove that optimizing InfoNCE is equivalent to maximize the mutual information between input image $...
1
vote
0answers
22 views

Can we solve an $8 \times 8$ sliding puzzle using hill climbing?

Can we solve an $8 \times 8$ sliding puzzle using a random-restart hill climbing technique (steepest-ascent)? If yes, how much computing power will this need? And what is the maximum $n \times n$ that ...
1
vote
0answers
9 views

Optimal critic in WGAN

The Kantorovich-Rubinstein duality for the optimal transport problem implies that the Wasserstein distance between two distributions $\mu_1$ and $\mu_2$ can be computed as $$W(\mu_1,\mu_2)=\underset{f\...
1
vote
0answers
11 views

What are some good loss functions used to minimize extreme errors in regression and time series forecasting?

I'm working on a time series forecasting task, and, in some specific cases, I don't need perfect accuracy, but the network cannot by any means miss by a lot. So, in detriment of a smaller mean error, ...
0
votes
0answers
14 views

How can I build a model that replaces a feature of one image with another feature?

I would like to build a neural network (using TensorFlow) that is able to take two animals, and replace a feature in the second with one in the first. For example, if given a dog and cat, the cat's ...
2
votes
1answer
27 views

True online TD($\lambda$) with dutch trace

In the RL textbook by Sutton & Barto section 7.4, the author talked about the "True online TD($\lambda$)". The figure (7.10 in the book) below shows the algorithm. My question is: at the end of ...
0
votes
0answers
14 views

How to detect forgery on scanned document images?

I am trying to detect forgeries done after a document is scanned by a scanner. I already tried to access the metadata, and, if it is edited with any software after scanning, then it is easily ...
1
vote
0answers
9 views

Sampling from deep belief networks

Deep belief networks (DBNs) are generative models, where, usually, you sample by thermalising the deepest layer (as it's a restricted Boltzmann machine), and then forward propagating a sample towards ...
2
votes
0answers
22 views

Is better to spend parameters on weights or bias?

If a neural network has a limited number of neuron parameters to find, -let's say only 1000 parameters-, it is generally better to spend the parameters on weights or neuron bias? For example, if each ...
0
votes
0answers
10 views

Train a model using a multi-column text-filled excel sheet

I have an excel sheet filled with my own personal appreciations of movies I've watched, and I want to use it to train an AI model so that it can predict if I'll like a specific movie or not, based on ...
1
vote
2answers
32 views

Are there any good tutorials about training RL agent from raw pixels using PyTorch?

Is there any good tutorials about training reinforcement learning agent from raw pixels using PyTorch? I don't understand the official PyTorch tutorial. I want to train the agent on the atari ...
0
votes
0answers
17 views

Conditional Variational Autoencoder - NON Image Data

First I would like to expand an issue I've been dealing with way too long: Creating a conditional Variational Autoencoder with continuous variables in non-image data ( more specifically, time series). ...
1
vote
1answer
37 views

Should I use minimax or alpha-beta pruning?

Should I use minimax or alpha-beta pruning (or both)? Apparently, alpha-beta pruning prunes some parts of the search tree.
2
votes
0answers
16 views

Video recognition (specifically video, not individual frames)

There are libraries for recognizing individual video frames, but I need to recognize an object in motion. I can recognize a person in every single frame, but I need to know if the person is running or ...
0
votes
0answers
17 views

Help with deep Q learning for 2048 game getting stuck

I am having trouble making a reinforcement algorithm than can win the 2048 game. I have tried with deep Q (which I think is the simplest algorithm that should be able to learn a winning strategy). ...
10
votes
2answers
982 views

How does one prove comprehension in machines?

Say we have a machine and we give it a task to do (vision task, language task, game, etc.), how can one prove that a machine actually know's what's going on/happening in that specific task? To narrow ...
0
votes
0answers
10 views

Is there any advantage to using a non-diagonal covariance matrix for a policy distribution?

For reinforcement learning implementations with a gym.spaces.Box action space, which is the product of $k$ real closed intervals, it is common (actually more like ...
0
votes
1answer
26 views

What is correct update when the some indexes are not available?

To update the Q table Q-learning takes the arg max of the Q values - the state, value mappings. For example, in tic tac toe the state XOX OXO -X- contains two ...
2
votes
0answers
17 views

What are the applications of hierarchical softmax?

Apart from its use in word embeddings (e.g word2vec algorithm), are there any other applications of hierarchical softmax? If yes, can you please give me some reference papers?
1
vote
0answers
10 views

How to understand the matrices used in the Attention layer?

Attention-scoring mechanism seems to be a commonly-used component in various seq2seq models, and I was reading about the original "Location-based Attention" in Bahadanau well-known paper at https://...
0
votes
0answers
11 views

How to padding input data in tfjs [closed]

how to padding input data in tensorflowjs ? like in python we use keras pad_sequesces ...
0
votes
0answers
39 views

Reinforcement Learning DQN target : How can the target rely on untrained parameters?

I'm trying to understand DQN, I understand where the loss function comes from I'm just unsure about why the target function works in practice. Given the loss function $$ L_i(\theta_i) = [(y_i - Q(s,a;\...
1
vote
1answer
37 views

What is the difference between reinforcement learning and evolutionary algorithms?

What is the difference between reinforcement learning (RL) and evolutionary algorithms (EA)? For some problems, you could presumably co-evolve two "species" populations using evolutionary algorithms ...
2
votes
0answers
16 views

Are the final states not being updated in this $n$-step Q-Learning algorithm?

I am reading this paper and in algorithm 3 they describe an $n$-step Q-Learning algorithm. Below is the pseudo-code. From this pseudo-code, it looks as though the final tuples that they would ...
0
votes
0answers
14 views

Fuzzy Rules making for sleep quality estimation

I have read this paper (its PDF): Sleep behavior assessment via smartwatch and stigmergic receptive fields At fuzzy classification part of paper said and referenced to this paper (19 REF.its PDF ...
1
vote
1answer
29 views

What is meant by the rank of the scoring function here?

I've been reading this paper on Knowledge Graph Reasoning for Explainable Recommendation lately, and I don't understand a particular section: Specifically, the scoring function $f((r,e)|u)$ maps ...
1
vote
1answer
50 views

Is there any programming practice website for beginners in Reinforcement Learning [closed]

I am doing an online course on Reinforcement Learning from university of Alberta. It focus too much on theory. I am engineering and I am interested towards applying RL to my applications directly. ...
2
votes
0answers
29 views

Why do we need recurrent neural networks instead of feed-forward neural networks? [duplicate]

Why do we need recurrent neural networks instead of feed-forward neural networks? What are the advantages of RNNs compared with FFNNs?
0
votes
0answers
6 views

Identifying and Labeling multiple letters in image

While I attempt to learn AI/ML I have taken on the task to create a Boggle solver. The idea is that a system could take an image of a Boggle arrangement of letters and identify the letters (and the ...
1
vote
0answers
25 views

What is the most compressed audio that I can feed an AI?

The problem I currently have is that I want to train an AI to produce music, like music that contains voices etc... However, the problem is that with a WAV file, one second of audio can be up to 48,...
0
votes
0answers
7 views

LSTM for imbalanced panel data

The available tutorials are most focused on time series prediction. I am wondering how shall we prepare the input data when it is an imbalanced data? Here is how data looks like. ...
1
vote
0answers
32 views

Calculating the advantage 'gain' of actions in model-free reinforcement learning

I have a simple question about model-free reinforcement. In a model I'm writing about, I want to know the value 'gain' we'd get for executing an action, relative to the current state. That is, what ...
0
votes
0answers
46 views

Why is my neural network not able to approximate this function?

I'm trying to approximate the following function with a neural network (in Python). ...
1
vote
0answers
14 views

How to learn how to select a subgraph via reinforcement learning?

I have the following problem. I am given a graph with a lot (>30000) nodes. Nodes are associated with a low (<10)-dimensional feature vector, and edges are associated with a low (<10)-...
1
vote
1answer
22 views

What are the real applications of hierarchical temporal memory?

What are the real applications of hierarchical temporal memory (HTM) in machine learning (ML) these days?
2
votes
1answer
26 views

Can the agent wait until the end of the episode to determine the reward in SARSA?

From Sutton and Barto's book Reinforcement Learning (Adaptive Computation and Machine Learning series) (p. 99), the following definition for first-visit MC prediction, for estimating $V \sim V_\pi$ is ...
3
votes
1answer
66 views
+50

How do I keep my system (online) learning if I can get ground truth labels only for examples flagged positive?

I have a binary classifier (think of it as a content moderation system) that is deployed after having being trained via batch learning. Once deployed, humans review and check for correctness only ...
0
votes
1answer
18 views

How can I draw a conceptual dependency for the statement “Place all ingredients in a bowl and mix thoroughly”?

I stumbled across a question asking to draw a conceptual dependency for the following statement: Place all ingredients in a bowl and mix thoroughly My attempt so far Explanation: Both the sender ...
0
votes
0answers
5 views

How to exclude sections of bad data from time-series data before training an LSTM network

I am using LSTM network for predicting IOT time-series data receiving from un-reliable devices and networks. This results in several multiple sections [continuous streak of bad data for several days ...
1
vote
1answer
70 views

What is a RAM state in the gym's breakout-ram environment?

I have encountered the gym environment and decided to create AI that plays breakout. Here is the link: https://gym.openai.com/envs/Breakout-ram-v0/. The documentation says that the state is ...
1
vote
1answer
66 views

Can tabular Q-learning converge even if it doesn't explore all state-action pairs?

My understanding of tabular Q-learning is that it essentially builds a dictionary of state-action pairs, so as to maximize the Markovian (i.e., step-wise, history-agnostic?) reward. This incremental ...
0
votes
0answers
12 views

Is there any difference between bounded sum and bold union fuzzy set operations?

Is there any difference between bounded sum and bold union fuzzy set operations? What about the difference between bounded difference and bold intersection? In some books, I found no difference, and,...
1
vote
1answer
22 views

What is the main contribution of the paper Disentangling by Factorising?

Considering the paper Disentangling by Factorising, in addition to introducing a new model for Disentangled Representation Learning, FactorVAE (see figure), what is the main theoretical contribution ...
1
vote
0answers
28 views

Actor-Critic implementation not learning

I've implemented a vanilla actor-critic and have run into a wall. My model does not seem to be learning the optimal policy. The red graph below shows its performance in cartpole, where the algorithm ...
1
vote
0answers
31 views

How to prevent deep Q-learning algorithms to overfit?

I have recently solved the Cartpole problem using double deep Q-learning. When I saw how the agent was doing, it used to go right every time, never left, and it did similar actions all the time. Did ...
2
votes
0answers
31 views

How does SGD escape local minima?

SGD is able to jump out of local minima that would otherwise trap BGD I don't really understand the above statement. Could someone please provide a mathematical explanation for why SGD (Stochastic ...
1
vote
1answer
35 views

If deep Q-learning starts to choose only one action, is this a sign that the algorithm diverged?

I'm working on a deep q-learning model in an infinite horizon problem, with a continous state space and 3 possible actions. I'm using a neural network to approximate the action-value function. ...

15 30 50 per page
1
2 3 4 5
133