All Questions
843 questions
2
votes
0
answers
111
views
When are traditional image processing methods preferable to machine learning and why?
By traditional image processing I understand, e. g. using filters to improve the image, extracting edges and then classifying objects using template matching.
My current decision criteria are:
large ...
- 21
0
votes
1
answer
722
views
How do I create an AI controller for Pacman?
How do I create an AI controller, which can play pacman - by taking in pixel values (or some other data by represents the state) which perhaps runs on a separate thread, which can control the game?
It ...
- 9
1
vote
1
answer
399
views
Is the described Q-table considered large?
I never saw any rule of thumb as to what size is said as large for a q-table but I have a Q-table with like 2500 entries. Is it considered large for a tabular approach? Anyone from experience can ...
0
votes
1
answer
285
views
How to compare memory requirements for tabular Q-learning vs deep neural network?
I want to compare the space complexity/memory requirement of tabular Q-learning v.s. deep neural Q-network (DQN). I think DQN would be faster and Q-table has a disadvantage at large table sizes but ...
10
votes
1
answer
9k
views
What is the difference between the triplet loss and the contrastive loss?
What is the difference between the triplet loss and the contrastive loss?
They look same to me. I don't understand the nuances between the two. I have the following queries:
When to use what?
What ...
- 373
1
vote
0
answers
117
views
Is item-based collaborative filtering the same thing as content-based filtering?
According to this Google dev page
content-based filtering
Uses similarity between items to recommend items similar to what the
user likes.
collaborative filtering
Uses similarities between queries ...
- 11
0
votes
0
answers
96
views
What if we modify some Q-values while taking the action?
Just a passing thought about Q-learning. In the tabular Q-learning, what if I play around and modify any Q-values as I am using them to take actions? Would it be a violation of any (1) theoretical ...
0
votes
1
answer
263
views
Is it possible to add states to the Q-table after the game has started?
I would like to implement Q-learning in a game.
Here is the board:
It's a 2 player game. At each turn, each player can put a pawn on a line of their choice. They can't choose the column. The right ...
- 103
0
votes
1
answer
508
views
How to deal with changing rewards in Q-learning? DQN?
I read the working of Q-learning through a grid-based taxi routing wherein a taxi has to pick and drop off a passenger from source to destination. Likewise, I have a routing problem and hence, I tried ...
0
votes
1
answer
80
views
Is there a way to improve the low-quality data?
I'm on a robotics team and we've been tasked to write a program to differentiate between a live and dead fish. We've been given ~15 minutes of training footage and it's absolutely terrible. It's low ...
user55176
2
votes
1
answer
882
views
Is Q-learning only capable of learning a deterministic policy?
I was following a reinforcement learning course on coursera and in this video at 2:57 the instructor says
Expected SARSA and SARSA both allow us to learn an optimal
$\epsilon$-soft policy, but, Q-...
0
votes
1
answer
30
views
Using reinforcement learning for human-robot interaction [closed]
I have a scenario where a user is wanting to exercise and improve over time. They attend around 10 exercise sessions, doing 20 repititions of an exercise each session.
I want to develop a ...
- 101
1
vote
2
answers
68
views
What are all the possible usages of 'multilayer perceptron'?
The term 'multilayer perceptron' has been used in literature in various ways in the literature.
I am presenting some of them below
As a feed-forward neural network [1].
As a fully connected feed-...
- 3,990
2
votes
0
answers
61
views
What are the specific differences between vision transformers variants?
I have tried 4 different types of attacks on vision transformers (ViT small and tiny, DeiT small and tiny) but the attack successes on smaller versions are higher than the tiny versions. My ...
- 31
0
votes
1
answer
234
views
How to manage impossible actions? [closed]
I am using Q-learning in julia language.
Because of the solver’s configuration, actions have to be defined as the whole action space and impossible actions have to be also considered. It means that I ...
- 33
0
votes
1
answer
280
views
Is the case of a big state space, should we use a softmax exploration policy rather than $\epsilon$-greedy for Q-Learning?
In Reinforcement Learning, epsilon-greedy policies are the most used exploration policies, but in case there is a big state space with impossible actions, wouldn't it be better to use soft-max ...
- 33
0
votes
1
answer
129
views
Could Softmax Action Selection be useful to solve an episodic task with more than 100000 possible states and 2000 actions?
I am new in the field of RL. I am trying to use tabular methods, Q-Learning for solving a problem that takes a lot of time for computation, so I would like to know if there are more efficient methods ...
- 33
2
votes
1
answer
437
views
Is it possible learning convergence is lost in Reinforcement Learning as the state space grows?
I am new in the AI field and I am trying to use Reinforcement Learning. Specifically, I am using tabular Q-Learning and SARSA algorithms to solve a sequential decision making problem. (I am using <...
- 33
2
votes
2
answers
5k
views
What is the difference between features and inputs in machine learning?
I have seen many places that features and inputs have been used interchangeably when talking about machine learning especially deep neural networks. I want to know if they are indeed the same thing or ...
- 145
2
votes
2
answers
3k
views
What are the differences between BLEU and METEOR?
I am trying to understand the concept of evaluating the machine translation evaluation scores.
I understand how what BLEU score is trying to achieve. It looks into different n-grams like BLEU-1,BLEU-2,...
- 373
1
vote
2
answers
1k
views
What is the difference between a policy and rewards?
I don't understand the difference between a policy and rewards. Sure, a policy tells us what to do, but isn't the output of a neural network trained on rewards basically a policy (i.e. choose the ...
0
votes
1
answer
77
views
Does $S_{t+1}$ denote the future information in Q-learning?
In Q-learning, $Q(S_t,a)$ is updated by the Bellman equation. $Q(S_t,a) = r + \max_{a'}(Q(S_{t+1},a'))$ where $S_{t+1}$ is the future state.
Let's say $S$ denotes the stock price, does it mean we are ...
- 1
0
votes
1
answer
1k
views
What is the difference between CNN-LSTM and RNN?
I'm starting to study RNN for a project of video prediction, but I encounter these CNN-LSTM models. Initially, I thought that is another name for RNN, but I think I get it wrong. Since I'm a beginner ...
0
votes
1
answer
486
views
When calculating the max in DQN, do I have to calculate the Q for every possible action for a particular state?
I'm trying to implement the DQN paper using python/pytorch for my needs (https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf). I'm studying the main algorithm:
I am a bit confused about the $\gamma* \max ...
3
votes
3
answers
1k
views
Why are Siamese Neural Networks used instead of a single neural network?
Siamese Neural Networks are a type of neural network used to compare two instances and infer if they belong to the same object. They are composed by two parallel identical neural networks, whose ...
- 63
2
votes
0
answers
28
views
What are the benefits of using spectral k-means over simple k-means?
I have understood why k-means can get stuck in local minima.
Now, I am curious to know how the spectral k-means helps to avoid this local minima problem.
According to this paper A tutorial on Spectral,...
- 121
2
votes
0
answers
119
views
When to model decision-making problem as single agent vs multi-agent problem?
I understand the goals and purposes of RL in the case of a single agent and the underlying model, i.e. MDPs, for RL problems (or sequential decision making with uncertainty in general).
My question is ...
- 121
8
votes
2
answers
10k
views
What is the difference between a loss function and reward/penalty in Deep Reinforcement Learning?
In Deep Reinforcement Learning (DRL) I am having difficulties in understanding the difference between a Loss function, a reward/penalty and the integration of both in DRL.
Loss function: Given an ...
- 195
2
votes
1
answer
171
views
Can directly using expert policy in epsilon-greedy speed-up Q-learning?
In deep Q-learning we typically use epsilon-greedy policy during training. We choose a random action for a certain probability $\epsilon$, and choose the action that maximize the current Q-value ...
- 223
2
votes
1
answer
704
views
Is using Monte-Carlo estimate of returns in Deep Q Learning possible?
In all the tutorials of deep Q-learning (using neural networks) I have read so far, the state-action value function $Q(s,a)$ is learned by temporal difference learning. However, in policy gradient ...
- 223
-1
votes
1
answer
817
views
what does the OpenAI ALE/Breakout-RAM-V5 observation return [closed]
I haven't been able to understand the output that OpenAI gym return for observation from this snippet
...
2
votes
1
answer
297
views
Deep Q-Learning Model Effectiveness Improves then Crashes
I am implementing a Deep Q-Learning Algorithm. The model appears to improve but after awhile it just crashes and does just as well as if an agent was making random decisions. Shouldn't the behavior ...
- 23
3
votes
1
answer
2k
views
What is the difference between a greedy policy and an optimal policy?
I am struggling to understand what is the difference between an optimal policy and a greedy policy.
Let $F(r_{t+1},s_{t+1}| s_t,a_t)$ be the probability distribution accorting to which, given action $...
- 33
3
votes
1
answer
2k
views
What is multi-head attention doing mathematically, and how is it different from self-attention?
I'm trying to understand the difference between the concept of self-attention and multi-head attention. The latter is not actually too clear to me.
I understand that, in the case of self-attention, we ...
- 307
1
vote
0
answers
63
views
How to compare RL algorithms with different NN sizes?
I wanted to run some tests with some RL algorithms in a continuous control task, namely PPO-clip and SAC.
When comparing their NN structures described in their papers, SAC used 2 layers with 256 ...
- 119
2
votes
1
answer
240
views
Is logic AI a complement to learning AI?
I want to know the relation between logic AI and learning AI.
Logic AI here refers to the branch of AI that is based on mathematical logic. Learning AI refers to the branch of AI that is based on ...
- 3,990
4
votes
2
answers
691
views
Is the optimal policy the one with the highest accumulative reward (Q-Learning vs SARSA)?
I was looking at the following diagram,
The reward obtained with SARSA is higher. However, the path that Q learning chooses is eventually the optimal one, isn't it? Why is the SARSA reward higher if ...
- 282
1
vote
1
answer
1k
views
What is the difference between Mean Teacher and Knowledge Distillation?
I recently read two papers:
BYOL Bootstrap your own latent: A new approach to self-supervised Learning
DINO Emerging Properties in Self-Supervised Vision Transformers.
I am confused about the terms ...
1
vote
0
answers
195
views
How to use Actor-Critic RL with a categorical, state-dependent action space?
I have a problem where the agent is given an embedding vector to represent the state. Then it is also given a set of possible actions in the environment, let's say that the actions are each ...
- 454
0
votes
1
answer
138
views
How many layers and neurons in a FFNN do I need to make it equivalent to a CNN?
I started to learn machine learning early, and I studied the convolutional neural network and its ability to understand images and how it helps to reduce the number of parameters that need to be tuned....
1
vote
0
answers
34
views
What is the difference between the $Q_a$ calculated to update delta and those to select next action in the exploitation phase?
As the title suggests, I have a doubt about the computation of the $Q_a$ used to update the delta and the $Q_a$ used to select the next action in the exploitation phase, as shown below (source of ...
2
votes
0
answers
175
views
Watkins' Q(λ) with function approximation: why is gradient not considered when updating eligibility traces for the exploitation phase?
I'm implementing the Watkins' Q(λ) algorithm with function approximation (in 2nd edition of Sutton & Barto).
I am very confused about updating the eligibility traces because, at the beginning of ...
13
votes
2
answers
2k
views
Is there a fundamental difference between an environment being stochastic and being partially observable?
In AI literature, deterministic vs stochastic and being fully-observable vs partially observable are usually considered two distinct properties of the environment.
I'm confused about this because what ...
- 255
2
votes
1
answer
74
views
Can teacher forcing in RNN ensure Turing completeness?
RNN has the same capability as a universal Turing machine. But I am confused whether RNN holds the same capabilities if we use teacher forcing.
Consider the following excerpts from paragraphs taken ...
- 3,990
1
vote
0
answers
81
views
Is the capability of RNN more than the capability of MLP?
Consider the following excerpt paragraph taken from the section titled "Recurrent Neural Networks" of the chapter 10: Sequence Modeling: Recurrent and Recursive Nets of the textbook named ...
- 3,990
4
votes
1
answer
255
views
Is there any relation between the recursive neural network and recurrent neural network?
Recurrent neural networks, abbreviated as RNNs, are widely used in deep learning literature, especially for text processing.
Are they related to recursive neural networks in any way?
I am asking for ...
- 3,990
1
vote
1
answer
662
views
How is $Q(s', a')$ calculated in SARSA and Q-Learning?
I have a question about how to update the Q-function in Q-learning and SARSA. Here (What are the differences between SARSA and Q-learning?) the following updating formulas are given:
Q-Learning
$$Q(s,...
- 266
1
vote
1
answer
270
views
Are the capabilities of connectionist AI and symbolic AI the same?
The universal approximation theorem says that MLP with a single hidden layer and enough number of neurons can able to approximate any bounded continuous function. You can validate it from the ...
- 3,990
1
vote
1
answer
385
views
What is meant by "two action selections" in SARSA?
I have some difficulties understanding the difference between Q-learning and SARSA. Here (What are the differences between SARSA and Q-learning?) the following updating formulas are given:
Q-Learning
$...
- 266
5
votes
1
answer
856
views
What is the difference between an on-policy distribution and state visitation frequency?
On-policy distribution is defined as follows in Sutton and Barto:
On the other hand, state visitation frequency is defined as follows in Trust Region Policy Optimization:
$$\rho_{\pi}(s) = \sum_{t=0}^...
- 379