Questions tagged [papers]

For questions related to artificial intelligence research papers. So, you should use this tag if you want someone to clarify something in a research paper.

Filter by
Sorted by
Tagged with
0 votes
1 answer
28 views

What do we mean by this type of notation?

I was going through this VIT paper, what will it look like in torch , if we are trying to write this expression.
user avatar
0 votes
0 answers
12 views

How is the union bound being used?

I am trying to understand the assumption proof of Theorem 2(Page -$7$) in the paper "A Universal Law of Robustness via isoperimetry" by Bubeck and Sellke. Inequality 1 \begin{align} \mathbb{...
user avatar
  • 101
1 vote
0 answers
21 views

What are semantic word spaces in NLP?

In the abstract of this paper, it's written Semantic word spaces have been very useful but cannot express the meaning of longer phrases in a principled way. I would like to understand what semantic ...
user avatar
  • 121
1 vote
0 answers
161 views

In the MuZero paper, how does backprop in the MCTS account for the immediate reward from each edge?

On page 12 of this paper: Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, it describes how MCTS works for the MuZero algorithm. It states in equation 4 that during the 'backup' ...
user avatar
0 votes
1 answer
34 views

What is a "canonical space"?

I am reading the paper on 3D reconstruction, ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction, and I encountered the term "canonical space". What is a "...
user avatar
0 votes
0 answers
20 views

Why is the projection reasonable in C51 algorithm?

In this paper, there is an equation about the projection. The author said we project the sample Bellman update $\hat{T}Z_θ$ onto the support of $Z_θ$ Why is the original support reasonable? And why ...
user avatar
  • 3
0 votes
0 answers
16 views

If two functions are close apart can I proof the difference of their empirical loss is also small?

I am trying to understand the proof of Theorem 3 in the paper A Universal Law of Robustness via isoperimetry by Bubeck and Sellke. Basically, there exist at least one $w_{L,e}$ in $\mathcal{W}_{L,e}$ ...
user avatar
  • 101
0 votes
1 answer
21 views

Is the "Helvetica scenario" mentioned here related to Artificial Intelligence?

Consider the following sentence from the original GAN paper titled Generative Adversarial Nets in particular, $G$ must not be trained too much without updating $D$, in order to avoid "the ...
user avatar
  • 3,099
2 votes
1 answer
42 views

How to decode P bits that represent a random weight generator?

So I've been tasked by my neural network professor at university to replicate the following research: Intelligent Breast Cancer Diagnosis Using Hybrid GA-ANN. Each chromosome represents a possible net,...
user avatar
3 votes
1 answer
45 views

What is the total number of actions and rewards count

Reading this two articles about Reinforcement Learning: Deep Reinforcement Learning with Double Q-learning by Hado van Hasselt et al. Human-level control through deep reinforcement learning by ...
user avatar
  • 133
0 votes
0 answers
15 views

Recursive Memory Optimized Gradient Graph Explained?

I'm reading the paper Training Deep Nets with Sublinear Memory Cost by Tianqi Chen, et. al. The paper is known for the $O(\sqrt n)$ memory cost to train a $n$-layer neural network. My problem is ...
user avatar
  • 111
0 votes
1 answer
20 views

What is the right way to find the alphas in this equation?

In the Grad-CAM++ paper the following equation (7) is posed (written here without the relu function): $$ Y^c = \sum_k \Bigl( \Bigl\{ \sum_{a,b} \alpha_{ab}^{kc} \cdot \frac{\partial Y^c}{\...
user avatar
  • 131
0 votes
1 answer
56 views

What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?

I have a question about the Grad-CAM++ paper. I do not understand how the following equation (10) for the alphas is obtained: $$ \alpha_{ij}^{kc} = \frac{\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}} {...
user avatar
  • 131
2 votes
1 answer
39 views

What is a 'degenerate run' in evaluating model performance?

I've recently come across a paper that uses the term "degenerate run", but I'm not sure if I understand what it means. The idea is that when they report the average performance of running ...
user avatar
  • 121
0 votes
0 answers
24 views

What does "position" in "each position in the decoder" denote in the Transformer's original paper?

I am reading Attention is All You Need and I feel confused about the word "position" in this paper, by the way I'm not native English speaker which may cause my confusion which has confused ...
user avatar
  • 1
1 vote
0 answers
86 views

How are 4D cost volumes constructed for DL based stereo matching?

I read a paper on Stereo Matching using Pyramid Cost Volumes (paper link: Semantic Stereo Matching with Pyramid Cost Volumes). At some point, in the proposed architecture, after: Feature extraction ...
user avatar
0 votes
0 answers
20 views

Derivation in paper Deep Neural Networks as Gaussian processes in ICLR 2018

I am trying to understand the derivation of the main equation in the seminal paper titled Deep Neural Networks as Gaussian processes (in ICLR 2018). I have asked this question in https://math....
user avatar
0 votes
0 answers
19 views

How does spatial pyramid pooling really work?

I went over the SPP paper (by Kaiming He) and understood in general that SPP can solve the fixed input size of the network problem (mainly due to the FC layer and not the CNN itself). Furthermore, the ...
user avatar
  • 1
1 vote
1 answer
86 views

What are the steps to derive the original GAN loss function from the generalized version?

I am trying to understand how the loss function from the original GAN paper $$\min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log D(\boldsymbol{x})]+\...
user avatar
  • 13
0 votes
1 answer
139 views

When calculating the max in DQN, do I have to calculate the Q for every possible action for a particular state?

I'm trying to implement the DQN paper using python/pytorch for my needs (https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf). I'm studying the main algorithm: I am a bit confused about the $\gamma* \max ...
user avatar
0 votes
0 answers
20 views

Is there any sort of 'best practice' for giving the general public access to deep learning models to accompany academic papers?

Is there any sort of 'best practice' for giving the general public access to deep learning models to accompany academic papers?
user avatar
1 vote
1 answer
45 views

Is the initial teacher model in the Noisy Student algorithm noised?

Reading through the paper on the Noisy Student algorithm, I have a quick question about how the initial teacher model is built. In step 1 of the algorithm, the loss function is defined such that it ...
user avatar
1 vote
1 answer
69 views

How is AI used in Internet of Things?

I would really appreciate it if someone would explain how AI is used in IoT. In the papers that I have found, half of the paper itself is about what IoT is and very few information about how AI is ...
user avatar
0 votes
0 answers
54 views

How to reduce loss of Bi-LSTM handwriting recognition model?

I am currently training an bi-LSTM model which predicts the handwriting of an individual. I am hitting a current min loss of 1.2 and I think it is not a problem with the model because I copied a model ...
user avatar
0 votes
1 answer
97 views

PPO advantage estimate - Why does advantage estimate have $r_t+\gamma V(s_{t+1})-V(s_t)$

So I've been looking at this formula for advantage estimate \begin{equation} \begin{aligned} & \hat{A}_t = \delta_t + (\gamma \lambda)\delta_{t+1} + ... + (\gamma \lambda)^{T-t+1}\delta_{T-1}\\ &...
user avatar
1 vote
0 answers
80 views

What do state features mean in the context of inverse RL?

I am reading Zeibart's Inverse RL paper, and it states - The agent is assumed to be attempting to optimize some function that linearly maps the features of each state, $f_{sj} \in \mathbb{R}^k$, to a ...
user avatar
1 vote
1 answer
48 views

What does the complexity equation constitute exactly in “Why Should I Trust You?” LIME paper

I've recently been reading this paper on LIME, an algorithm to interpret ANY machine learning model. I encountered this equation (in red) on page 4 and have just been having a hard time deciphering ...
user avatar
0 votes
1 answer
35 views

Does "number of actions" refer to the number of actions taken or size of the action space?

In the original DDQN article (https://arxiv.org/pdf/1509.06461.pdf,) the phrase "number of actions" is used twice; First, in the following context: Secondly in Theorem 1. I have a hard ...
user avatar
0 votes
0 answers
74 views

How to use EfficientDet for semantic segmentation?

In the EfficientDet paper, section 5.2. 5.2. EfficientDet for Semantic Segmentation, the authors say we modify our EfficientDet model to keep feature level $\{P2, P3, ..., P7\}$ in BiFPN, but only ...
user avatar
  • 11
0 votes
0 answers
23 views

Is smoothing wrong in temporal predictions?

I found this paper from 2003 about predicting Forex rates: Using Recurrent Neural Networks To Forecasting of Forex. At the end of page 11, they say The network we built had two inputs and one output. ...
user avatar
0 votes
0 answers
25 views

What are the "two new constraints" that the authors of the "TeEther" paper are referring to?

Smart contracts (SC) are programs developed for Ethereum Blockchain initially. They are used for transferring Ether and subsequently can be applied in place of Banking transactions and credit cards. ...
user avatar
2 votes
1 answer
737 views

Does higher FLOPS mean higher throughput?

I understand that FLOPS means floating-point operations per second, and throughput is the number of inputs (for example, images) per second. If a model has higher FLOPS, it means it performs faster. ...
user avatar
  • 181
1 vote
0 answers
32 views

Generative systems based on Schmidhuber's compression framework

In Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes ...
user avatar
  • 111
0 votes
1 answer
54 views

What is the meaning of the shaded area in the reinforcement learning literature graphs?

In most of the reinforcement learning literature, I see that there is a shaded area in the graphs. I couldn't understand what it exactly represents? For example, from the A3C paper: Or another ...
user avatar
6 votes
0 answers
92 views

How can a neural network distinguish a rotated 6 and 9 digits?

Rotated MNIST is a popular dataset for benchmarking models equivariant to rotations on $\mathbb{R}^2$, described by $SO(2)$ group or its discrete subgroups like $\mathbb{Z}^{n}$: Group equivariant ...
user avatar
0 votes
0 answers
19 views

How to compute an AUROC score using the Mahalanobis distance?

I was reading this paper: A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. The core idea is, given a test sample $x$, and a set of classes $C$, to compute ...
user avatar
1 vote
1 answer
50 views

Do we use two distinct layers to compute the mean and variance of a Gaussian encoder/decoder in the VAE?

I am looking at appendix C of the VAE paper: It says: C.1 Bernoulli MLP as decoder In this case let $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$ be a multivariate Bernoulli whose ...
user avatar
  • 203
2 votes
3 answers
127 views

Are the authors of the VAE paper writing the PDFs as a function of the random variables?

Usually, I see the conventions: discrete random variable is denoted as $X$, the pmf is written as $P(X=x)$ or $p(X=x)$ or $p_{X}(x)$ or $p(x)$, where $x$ is an instance of $X$ a continuous random ...
user avatar
  • 203
4 votes
2 answers
425 views

How should I read a deep learning paper?

I have a background in mathematics and I am accustomed to reading papers with lemma and proofs. When I see a deep learning paper, they seem to be of practical nature. How can I improve my reading and ...
user avatar
0 votes
1 answer
32 views

Which is more popular/common way of representing a gradient in AI community: as a row or column vector?

Consider the following remark about writing gradients from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning by Marc Peter Deisenroth et al. Remark (...
user avatar
  • 3,099
1 vote
1 answer
30 views

Does Seq2Seq decoder take a special vector or the weights of the last encoder cell as an output?

I'm reading Sequence to Sequence Learning with Neural Networks and there's a thing that I couldn't quite grasp. Paper says the encoder outputs a vector to be fed to the decoder. More precisely Our ...
user avatar
  • 145
1 vote
0 answers
98 views

Where do the characteristics of self-attention come into play in Linformer's proof that self-attention is low rank?

In Linformer's proof that self-attention is low rank in their paper, I don't see how it doesn't generalize to every matrix. They don't utilize any specifics of self-attention (the entire proof feels ...
user avatar
  • 2,249
2 votes
0 answers
105 views

Why does the number of input tokens to an LSTM have an impact on the convergence of Integrated Gradients?

Background I am computing the attribution scores for a simple LSTM model using Integrated Gradients. This method defines the contribution of a feature to a model prediction by integrating over the ...
user avatar
  • 71
3 votes
1 answer
114 views

Why exactly was previously believed that the deterministic policy gradient did not exist?

I'm reading the paper Deterministic Policy Gradient Algorithms, David Silver et al. First of all, in the introduction, the author says that It was previously believed that the deterministic policy ...
user avatar
  • 133
1 vote
1 answer
164 views

How is the VAE related to the Autoencoding Variational Bayes (AEVB) algorithm?

I am familiar with the variational autoencoder, but not totally clear on what exactly the AEVB is. In the original VAE paper (by Kingma and Welling), he uses both the terms variational autoencoder and ...
user avatar
  • 203
1 vote
0 answers
131 views

Why is the Graph Isomorphism Network powerful?

I am reading a paper known as GIN, How powerful are graph neural networks?, Xu et al. 2019 The paper, Lemma 5 and Corollary 6, introduces Graph Isomorphism Network (GIN). In Lemma 5, Moreover, any ...
user avatar
  • 125
1 vote
1 answer
140 views

What is a filter in the context of graph convolutional networks?

In Section 2.1 of the research paper titled Semi-Supervised Classification with Graph Convolutional Networks by Thomas N. Kipf et al., Spectral convolution on graphs defined as The multiplication of ...
user avatar
  • 125
1 vote
1 answer
206 views

How does the VAE learn a joint distribution?

I found the following paragraph from An Introduction to Variational Autoencoders sounds relevant, but I am not fully understanding it. A VAE learns stochastic mappings between an observed $\mathbf{x}$...
user avatar
  • 203
1 vote
1 answer
199 views

How are GCN doing semi-supervised learning?

In Semi-Supervised Classification with Graph Convolutional Networks, the authors say that GCN is an approach for semi-supervised learning (SSL). But a GCN is making predictions using only the graph ...
user avatar
0 votes
0 answers
57 views

In this paper, how does scaling the filter instead of the image generate saliency maps of the same size and resolution as the input image?

In this paper, in section 3.1, the authors state Scaling the filter instead of the image allows the generation of saliency maps of the same size and resolution as the input image. How is this ...
user avatar

1
2 3 4 5
7