Questions tagged [papers]

For questions related to artificial intelligence research papers. So, you should use this tag if you want someone to clarify something in a research paper.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

How does Google's 2016 GNMT architecture work?

I'm trying to read this paper describing Google's LSTM architecture for machine translation from 2016. However, I'm getting stuck as certain things are described too vaguely for me. This is a picture ...
1
vote
0answers
6 views

Does this diagram represent several LSTMs, or one through several timesteps?

I'm trying to read this paper describing Google's LSTM architecture for machine translation. It features this diagram on page 4: I'm interested in the encoder block, on the left. Apparently, the pink ...
0
votes
1answer
53 views

What is the meaning of these equations in Noise2Noise paper?

I am trying to understand what is meant by following equations in the Noise2Noise paper by Nvidia. What is meant by the equation in this image? What is $\mathbb{E}_y\{y\}$? And how should I try to ...
2
votes
2answers
146 views

What is the gradient of the objective function in the Soft Actor-Critic paper?

In the paper Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor, they define the loss function for the policy network as $$ J_\pi(\phi)=\mathbb E_{s_t\...
2
votes
1answer
55 views

What is the difference between success rate and reward when dealing with binary and sparse rewards?

In OpenAI Gym "reward" is defined as: reward (float): amount of reward achieved by the previous action. The scale varies between environments, but the ...
2
votes
1answer
29 views

How is MuZero's second binary plane for chess defined?

From the MuZero paper (Appendix E, page 13): In chess, 8 planes are used to encode the action. The first one-hot plane encodes which position the piece was moved from. The next two planes encode ...
1
vote
2answers
515 views

What does “reimplementations of deep learning algorithms which replicate performance from the papers” mean?

In the OpenAI's Machine Learning Fellow position, it is written We look for candidates with one or more of the following credentials: ... Open-source reimplementations of deep learning algorithms ...
2
votes
1answer
64 views

What are proxy reward functions?

The understanding I have is that they somehow adjust the objective to make it easier to meet, without changing the reward function. ... the observed proxy reward function is the approximate solution ...
3
votes
2answers
1k views

What should I do when the potential value of a state is too high?

I'm working on a Reinforcement Learning task where I use reward shaping as proposed in the paper Policy invariance under reward transformations: Theory and application to reward shaping (1999) by ...
1
vote
1answer
76 views

Can entire neural networks be composed of only activation functions?

Inverse Reinforcement Learning based on GAIL and GAN-Guided Cost Learning(GAN-GCL), uses a discriminator to classify between expert demos and policy generated samples. Adversarial iRL, build upon GAN-...
0
votes
0answers
10 views

How does scheduled sampling for transformers work?

I was reading this paper which applies a modified version of the transformers for traffic forecasting. I am somewhat familiar with the transformer architecture and how it functions, but, in the paper, ...
12
votes
1answer
2k views

How would DeepMind's new differentiable neural computer scale?

DeepMind just published a paper about a differentiable neural computer, which basically combines a neural network with a memory. The idea is to teach the neural network to create and recall useful ...
1
vote
0answers
29 views

In the MINE paper, why is $\hat{G}_B$ biased, and how does the exponential moving average reduce the bias?

While reading the Mutual Information Neural Estimation (MINE) paper [1] I came across section 3.2 Correcting the bias from the stochastic gradients. The proposed method requires the computation of the ...
1
vote
0answers
84 views

SeqGAN - Policy gradient objective function interpretation

Could someone clear my doubt on the loss function used in SeqGAN paper . The paper uses policy gradient method to train the generator which is a recurrent neural network here. Have I interpreted the ...
3
votes
1answer
345 views

What is the significance of this Stanford University “Financial Market Time Series Prediction with RNN's” paper?

Researchers at Stanford University released, in 2012, the paper Financial Market Time Series Prediction with Recurrent Neural Networks. It goes on to discuss how they used echo state networks to ...
1
vote
1answer
35 views

Why is this variable in equation 2 of the SQAIR paper a random vector of $n$ ones followed by a zero?

I've been reading the SQAIR paper lately, and the mathematics involved seems a bit complicated. Some background, about the paper: SQAIR stands for Sequential Attend, Infer, Repeat - the paper does ...
11
votes
4answers
6k views

Where can I find the original paper that introduced RNNs?

I was able to find the original paper on LSTM, I was not able to find the paper that introduced "vanilla" RNNs. Where can I find it?
3
votes
0answers
42 views

Why is it necessary to divide the priority range according to the batch size in Prioritized Experience Replay?

According to DeepMinds's paper Prioritized Experience Replay (2016), specifically Appendix B.2.1 "Proportional prioritization" (p. 13), one should equally divide the priority range $[0, p_\...
0
votes
0answers
33 views

Is the “mlpconv” layer in the “Network in Network” paper computing $1 \times 1$ convolutions or not?

I am reading the Network in Network paper. This is the equation they introduce for their mlpconv layer (equation 2 in the paper, page 3): $$ \begin{aligned} f_{i, j,...
2
votes
1answer
102 views

How to perform back-propagation in Decoupled Neural Interfaces?

I am attempting to create a fully decoupled feed-forward neural network by using decoupled neural interfaces (DNIs) as explained in the paper Decoupled Neural Interfaces using Synthetic Gradients (...
5
votes
1answer
74 views

How does the network know which objects to track in the paper “Label-Free Supervision of Neural Networks with Physics and Domain Knowledge”?

I was reading the paper Label-Free Supervision of Neural Networks with Physics and Domain Knowledge, published at AAAI 2017, which won the best paper award. I understand the math and it makes sense. ...
0
votes
0answers
13 views

What BERT predicts when the token supposed to be masked is not masked?

I am reading the BERT paper. In the paper, they say that: Although this allows us to obtain a bidirec- tional pre-trained model, a downside is that we are creating a mismatch between pre-training and ...
2
votes
0answers
25 views

How are the lower and upper bound values of the moths determined in the Moth-Flame Optimization algorithm?

I am currently implementing the Moth-Flame Optimization (MFO) Algorithm, based on the paper: Moth-Flame Optimization Algorithm: A Novel Nature-inspired Heuristic Paradigm. To calculate the values of ...
6
votes
2answers
157 views

Why are reinforcement learning methods sample inefficient?

Reinforcement learning methods are considered to be extremely sample inefficient. For example, in a recent DeepMind paper by Hessel et al., they showed that in order to reach human-level performance ...
19
votes
2answers
7k views

What is sample efficiency, and how can importance sampling be used to achieve it?

For instance, the title of this paper reads: "Sample Efficient Actor-Critic with Experience Replay". What is sample efficiency, and how can importance sampling be used to achieve it?
1
vote
1answer
88 views

What are some alternatives to “Papers with Code”?

There are lots of research papers available that are worth reading. We can read papers easily, but the associated code (not necessarily the official one developed by the authors of the paper) is often ...
1
vote
1answer
66 views

Are mult-adds and FLOPs equivalent?

I am comparing different CNN architectures for edge implementation. Some papers describing architectures refer to mult-adds, like the MobileNet V1 paper, where it is claimed that this net has 569M ...
0
votes
1answer
38 views

Do the rows of the design matrix refer to the observations or predictors?

I attempt to understand the formulation of dictionary learning for this paper: Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution Multimodal Task-Driven ...
2
votes
3answers
485 views

What is the definition of a heuristic function in the BayesChess paper?

I am reading BayesChess: A computer chess program based on Bayesian networks (Fernandez, Salmeron; 2008) It is a chess-playing engine using Bayesian networks. The following is mentioned about the ...
1
vote
0answers
25 views

Why scaling down the parameter many times during training will help the learning speed be the same for all weights in Progressive GAN?

The title is one of the special things in Progressive GAN, a paper of the NVIDIA team. By using this method, they introduced that Our approach ensures that the dynamic range, and thus the learning ...
2
votes
1answer
61 views

How should I take into consideration the number of steps in the reward function?

I am currently implementing the paper Active Object Localization with Deep Reinforcement Learning in Python. While reading about the reward scheme I came across the following: Finally, the proposed ...
1
vote
0answers
26 views

What is the purpose of the DAMSM loss for the generators in AttnGAN?

I am confused about the training part in AttnGan. If you observe page 3. There are two types of losses for generator network: one involving the Deep Attentional Multimodal Similarity Model (DAMSM) ...
0
votes
1answer
56 views

What does $r : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ mean in the article Hindsight Experience Replay, section 2.1?

Taken from section 2.1 in the article: We consider the standard reinforcement learning formalism consisting of an agent interacting with an environment. To simplify the exposition we assume that the ...
1
vote
0answers
23 views

How do non-local neural networks relate to attention and self-attention?

I've been reading non-local neural networks as explained in the original paper. My understanding is that they solve the restrained reception of local filters. I see how they are different from ...
0
votes
0answers
21 views

How does the second phase of the evaluation function described in this article work?

I am trying to create an evaluation function for a general game player based on the research from this article An Automatically-Generated Evaluation Function in General Game Playing. I can't ...
1
vote
1answer
90 views

How can I get the predicted box in Faster R-CNN?

The RPN loss in Faster RCNN paper is $$ L({p_i}, {t_i}) = \frac{1}{N_{cls}} \sum_{i} L_{cls}(p_i,p_i^*) + \lambda \frac{1}{N_{reg}} \sum_i p_i^* L_{reg}(t_i, t_i^*) $$ For regression problems, we ...
1
vote
0answers
28 views

How to understand this NN architecture?

I was reading a paper Multi-Agent Reinforcement Learning for Adaptive User Association in Dynamic mmWave Networks and I was stuck understanding the deep neural network architecture that was used. The ...
3
votes
1answer
71 views

How are the observations stored in the RNN that encodes the state?

I am a bit confused about observations in RL systems which use RNN to encode the state. I read a few papers like this and this. If I were to use a sequence of raw observations (or features) as an ...
4
votes
1answer
159 views

How does the Ornstein-Uhlenbeck process work, and how it is used in DDPG?

In section 3 of the paper Continuous control with deep reinforcement learning, the authors write As detailed in the supplementary materials we used an Ornstein-Uhlenbeck process (Uhlenbeck & ...
0
votes
0answers
16 views

How to calculate the attention loss in the paper “Tell Me Where to Look: Guided Attention Inference Network”?

I have been reading the research paper Tell Me Where to Look: Guided Attention Inference Network. In this paper, they calculate the attention loss, but I didn't understand how to calculate it. Do we ...
3
votes
1answer
91 views

Understanding proof of lemma 1 (policy improvement bound) of the “Trust Region Policy Optimization” paper

In the Trust Region Policy Optimization paper, in Lemma 1 of Appendix A, I did not quite understand the transition from (21) from (20). In going from (20) to (21), $A^\pi(s_t, a_t)$ is substituted ...
2
votes
1answer
54 views

What is the surrogate loss function in imitation learning, and how is it different from the true cost?

I've been reading A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning lately, and I can't understand what they mean by the surrogate loss function. Some relevant ...
0
votes
1answer
67 views

How do you perform a gradient based adversarial attack on an SVM based model?

I have an SVM currently and want to perform a gradient based attack on it similar to FGSM discussed in Explaining And Harnessing Adversarial Examples. I am struggling to actually calculate the ...
0
votes
0answers
14 views

Limbs for PAFs in OpenPose

What limbs are used in openpose for the PAF? If you look at the skeletal reconstruction one would assume it is $ears - eyes$, $eyes - nose$, $nose-neck$, $neck - hips$, $shoulders - elbows$, $elbows- ...
0
votes
0answers
21 views

How are the step size and covariance matrix updated in CMA-ES?

I've been following the tutorial The CMA Evolution Strategy: A Tutorial to try and understand the CMA-ES, but I'm having trouble understanding how the step size and the covariance matrix are been ...
2
votes
0answers
86 views

What is the memory complexity of the memory-efficient attention in Reformer?

When I read the paper, Reformer: The Efficient Transformer, I cannot get the same complexity of the memory-efficient method in Table 1 (p. 5), which summarizes time/memory complexity of scaled dot-...
1
vote
0answers
76 views

How can a de-noising auto-encoder act as an anomaly detection model?

In some research papers, I have seen that, for training the autoencoders, instead of giving the non-anomalous input images, they add some anomalies to the normal input images, and train the auto-...
2
votes
0answers
47 views

What is meant by “arranging the final features of CNN in a grid” and how to do it?

In the paper What You Get Is What You See: A Visual Markup Decompiler, the authors have proposed a method to extract the features from the CNN and then arrange those extracted features in a grid to ...
1
vote
0answers
77 views

Ways to keep up with the latest developments in Machine Learning and AI?

With over 100 papers published in the area of artificial intelligence, machine learning and their subfields every day (source), accounting for ~3% of all publications world wide per year (source) and ...
2
votes
1answer
43 views

What is convergence analysis, and why is it needed in reinforcement learning?

While reading a paper about Q-learning in network energy consumption, I came across the section on convergence analysis. Does anyone know what convergence analysis is, and why is convergence analysis ...