16Aghnar
  • Member for 3 years, 3 months
  • Last seen more than a week ago
1 answers
6 votes
1k views
Learning Rate Decay and Exploration Rate Decay
Accepted answer
6 votes

Welcome to AI.SE ! First of all, I'd say that there is a reason to give Learning Rate (LR) and Exploration Rate (ER) the same decay : they play at the same scale (the number of successives batches you'...

View answer
3 answers
7 votes
320 views
What kinds of problems can AI solve without using a deep neural network?
3 votes

A nice example Markov Decision Processes, which can be solved by classic reinforcement learning techniques like Q learning. A Markov Decision Process consists of A set of discrete states (or ...

View answer
1 answers
1 votes
46 views
Deep Reinforcement Learning Atari: how does the agent understand motion?
Accepted answer
3 votes

In the article Playing Atari with Deep Reinforcement Learning, Mnih et al, 2013, which was a major outbreak in Deep Reinforcement learning (especially in Deep Q learning), they don't feed only the ...

View answer
1 answers
3 votes
101 views
How can we teach a neural net to make arbitrary data associations?
Accepted answer
3 votes

In a nutshell : Memorizing is not Learning So, first let's just remind the classical use of a neural net, in Supervised Learning : You have a set of $(x_{train}, y_{train}) \in X \times Y$ pairs, and ...

View answer
2 answers
4 votes
534 views
Why is the derivative 0 if the policy is deterministic?
3 votes

Well, I'd rather comment, but I don't have yet this privilege, so here are some comments. First, having a deterministic policy inside the log would do create trivial terms. Secondly, for me, in ...

View answer
3 answers
7 votes
320 views
What kinds of problems can AI solve without using a deep neural network?
2 votes

Image Segmentation with Unsupervised Learning Deep Learning is now widely used for image classification and segmentation. However, for segmentation, some algorithms are still really effective. For ...

View answer
2 answers
4 votes
467 views
Why does Clipped Surrogate Objective works in Proximal Policy Optimization
Accepted answer
2 votes

Ok, so I think I have a better understanding of this now. Firstly, let's remind the main idea of the PPO : staying close to the previous policy. It's the same idea than in TRPO, but the L function is ...

View answer
1 answers
1 votes
72 views
What do the state features of KukaGymEnv represent?
Accepted answer
1 votes

Here's an incomplete answer, but it may help. Your state is read by the function getExtendedObservation(). This function makes two things : it calls the function getObservation() from this source code,...

View answer
1 answers
1 votes
1k views
How is dropout applied to the embedding layer's output?
Accepted answer
1 votes

It doesn't drops rows or columns, it acts directly on scalars. The Dropout Layer keras documentation explains it and illustrates it with an example : The Dropout layer randomly sets input units to 0 ...

View answer
1 answers
3 votes
1k views
Tuning of PPO metaparameters: a high level overview of what each parameter does
1 votes

Some investigation about the memory dict: The current type is latest, which means you're not using a memory replay, but a latest memory. Switching to replay may help. Also, include_next_state means ...

View answer
1 answers
1 votes
118 views
How do I segment each part of a DICOM image?
0 votes

I looked at some picture from DICOM, and it seems that these images usually don't have many local minima, so I guess that's why watershed isn't optimal. However, it seems that in DICOM the color ...

View answer
1 answers
0 votes
256 views
Using 3D Points as Inputs to a Neural Net
Accepted answer
0 votes

A few thoughts : 10x3 matrix for each example is really a small amount of data. FCNN could do a good job on that. As a result, I'm not sure CNN is appropriated. The smallest dimension is 3, so it'll ...

View answer