Questions tagged [papers]

For questions related to artificial intelligence research papers. So, you should use this tag if you want someone to clarify something in a research paper.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Computing the mean attention distance for ViT

Recently I came across the paper that introduces the Vision Transformer (ViT) "AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE". The thing I don't really ...
1
vote
0answers
9 views

What is language-conditioned visual reasoning?

Can anyone explain what language-conditioned visual reasoning is? I saw this term in this paper and I searched on the internet but I couldn't find a proper explanation.
1
vote
0answers
11 views

What does the lambda parameter in the paper “Interpretable Explanations of Black Boxes by Meaningful Perturbation” do?

I do not understand the purpose of the $\lambda$ parameter in equation 3 of the paper Interpretable Explanations of Black Boxes by Meaningful Perturbation. $$m^{*}=\underset{m \in[0,1]^{\Lambda}}{\...
0
votes
0answers
15 views

What is the primary paper demonstrating that CNNs struggle with datasets containing ambiguities? [closed]

It is known that neural networks, such as convolutional neural networks, struggle with pattern recognition if training sets contain ambiguities (i.e. several labels can correspond to one and the same ...
1
vote
0answers
23 views

Is label-embedding similar to one-hot encoding?

In one-hot encoding, a vector is given to each class label. For each class, only one entry of the vector is equal to 1 and the remaining entries are zeros in this encoding. Thus, in one-hot encoding, ...
1
vote
1answer
26 views

Could someone help tell what the labels are pointed out by red rectangles?

The following figure comes from the paper The perceptron: A probabilistic model for information storage and organization in the brain I can tell the labels pointed out by blue rectangles are: "...
0
votes
0answers
11 views

What are some of the ideas to solve Learning-to-choose problem?

Suppose I want to predict cats and dogs, but with a twist: the model can choose the image to predict. For example: Given a list of 10 images (with both dogs and cats), the model need to choose one ...
1
vote
3answers
160 views

Why is AI Super Resolution Reconstruction more than just guessing?

I saw a video on Youtube about AI and Super Resolution Image Reconstruction with TecoGAN. I must say I am impressed. Now, I am wondering how reliable this is. I have learned at university that you ...
0
votes
0answers
15 views

Naming convention for deep learning layer sequences (“FC7”, “Conv-1-3”)

I was looking at the deep learning paper A Target-agnostic Attack on Deep Models and saw this figure (figure 3 on paper) demonstrating the performance of a transfer-learning-based adversarial attack ...
0
votes
1answer
41 views

How does CURL extract labels from logits?

While going over the pseudocode of the CURL paper, the method to identify labels from the logits wasn't clear to me. I believe this technique might be common in other PyTorch/Deep Learning tasks. I ...
1
vote
0answers
15 views

Why class embedding token is added to the Visual Transformer?

In the famous work on the Visual Transformers, the image is split into patches of a certain size (say 16x16), and these patches are treated as tokens in the NLP tasks. In order to perform ...
1
vote
1answer
17 views

What to do when the ROIs are smaller than $227 \times 227$ in R-CNN?

As English is not my native language, I have some hard time understanding the following sentence: Regardless of the size or aspect ratio of the candidate region, we warp all pixels in a tight ...
1
vote
0answers
21 views

Is there any work that applies the approach in “Finite-Sample Convergence Rates for Q-Learning and Indirect Algorithms” to standard Q-learning?

I am trying to mathematically characterize the finite sample convergence rates for Q-learning. To this end, I have read the following papers Learning rates for Q-learning, by Eyal Even-Dar et al.; ...
0
votes
1answer
61 views

What is the difference between the definition of “accuracy” in machine learning and federated learning?

What is the difference between the definition of "accuracy" in machine learning and federated learning? In particular, how is the accuracy calculated in the following paper: Cai, Lingshuang,...
1
vote
0answers
12 views

What's mutual exclusivity in meta-learning?

What do we mean by mutual exclusivity of tasks? This work (E Pan, 21) and this one (M Yin, 20) state that most classification meta-learning algorithms fail for non-mutually exclusive tasks as the ...
0
votes
0answers
25 views

How to choose the reward in reinforcement learning? [duplicate]

I am solving a combinatorial optimization problem, where I do not have a global optimum, so the goal is to improve the objective function as much as possible. So, to do this, I was inspired by this ...
0
votes
0answers
12 views

Why do the authors of this paper down-sample by $ds_1 / 2$ (in the context of coarse-to-fine segmentation)?

This question is a follow-up of this post and based on this paper. In section 2.2, the authors write: In the first level, the 3D FCN is trained on images of the lowest resolution in order to capture ...
2
votes
0answers
19 views

What is meant by Hinton when he refers to “Part-Whole Hierarchies” in his GLOM framework

I was recently reading Hinton's GLOM idea How to represent part-whole hierarchies in a neural network, and I am simply unsure about what exactly he means when he says parsing images into "part-...
0
votes
0answers
10 views

How exactly is masking performed in the training part of the paper “Semi-Supervised Classification with Graph Convolutional Networks”?

I am struggling to understand the training part of the paper Semi-Supervised Classification with Graph Convolutional Networks (2017) by Thomas Kipf and Max Welling. The GitHub repo is here. I do not ...
1
vote
1answer
25 views

Why the non-exploitation of edge labels in current graph convolutions “results in an overly homogeneous view of local graph neighborhoods”?

I am currently reading a paper called Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs (2017, CPPR), and I cannot understand the following sentence: We identify that the ...
1
vote
1answer
33 views

Understanding Generalized Advantage Estimate in reinforcement learning

I was reading the paper on Generalized Advantage Estimate. It first introduces a generalized form of policy gradient equation without involving $\gamma$ and then it says the following: We will ...
3
votes
2answers
65 views

Why some neural network models in the 1980s shown as circuit models

I am familiar with the currently popular neural network models that have weights and are trained with backpropagation and gradient descent. However, I came across a different type of neural network ...
1
vote
1answer
24 views

How is the variational lower bound for hard attention derived in Show, Attend and Tell

How is the jump from line 1 to line 2 done in equation 10 of Show, Attend and Tell? While we're at it, another thing that might be muddying the waters for me is that I'm not clear on what the sum is ...
0
votes
0answers
23 views

Do $V_\theta$ and $V_s$ represent partial or total derivatives in the paper “Learning Continuous Control Policies by Stochastic Value Gradients”?

I was reading up on the Stochastic Value Gradients paper by Heess et al. In the paper, they describe a recursive process to calculate path-wise derivatives via equations (3) and (4), at the bottom of ...
1
vote
0answers
29 views

What is a Hebbian linear classifier?

I was reading Deep Learning of Representations for Unsupervised and Transfer Learning, and they state the following: They have only a small number of unlabeled examples (4096) and very few labeled ...
6
votes
2answers
94 views

What does “semantic gap” mean?

I was reading DT-LET: Deep transfer learning by exploring where to transfer, and it contains the following: It should be noted direct use of labeled source domain data on a new scene of target domain ...
0
votes
0answers
61 views

Comparing heuristics in A* search and rescue operation

I was reading a research paper titled A Comparative Study of A-star Algorithms for Search and rescue in Perfect Maze (2011). I have some doubts regarding it: 1. The Evaluation Function of $\mathrm{A}^...
1
vote
2answers
70 views

In variational autoencoders, what does p(x|z) mean?

If $x \sim \mathcal{N}(\mu,\,\sigma^{2})$, then it is a continuous variable, and therefore $P(x) = 0$ for any x. One can only consider things like $P(x<X)$ to get a probability greater than 0. So ...
1
vote
0answers
17 views

Clonal operator in Immune Clonal Strategy

I was reading about Immune Clonal Strategy, specifically about Monoclonal operator from Immunity clonal strategies, and it goes as follows: Here $a_i $ is a point and $a_i = \{ x_1, x_2, \cdots, x_m \...
1
vote
0answers
23 views

What is the meaning of “Our current objective weights every token equally and lacks a notion of what is most important to predict” in the GPT-3 paper?

On page 34 of OpenAI's GPT-3, there is a sentence demonstrating the limitation of objective function: Our current objective weights every token equally and lacks a notion of what is most important to ...
2
votes
0answers
38 views

How many papers about AI / ML were published in the recent years?

I am trying to formulate an argument at work saying the disruption in AI/ML is very high and that it is hard to stay "state of the art". I would like to support that hypothesis by numbers. ...
1
vote
0answers
25 views

The MLP output of a neural network can be written as $\|x\|\|w_l\|\cos(\theta_l)$: why is the norm easier to maximize?

The MLP output of a neural network is a dot product between the weights and the input and therefore can be written as $\|x\|\|w_l\|\cos(\theta_l)$ (see this for more details), where $x$ is the input, $...
0
votes
1answer
21 views

How does high entropy targets relate to less variance of the gradient between training cases?

I've been trying to understand the Distilling the Knowledge in a Neural Network paper by Hinton et al. But I cannot fully understand this: When the soft targets have high entropy, they provide much ...
0
votes
0answers
56 views

How to evaluate a Genome in NEAT

I am trying to implement NEAT from scratch by going through the original NEAT paper. I implemented a Genome class which consists of a list of Node Genes and ...
4
votes
1answer
74 views

What is the relation between self-taught learning and transfer learning?

I am new to transfer learning and I start by reading A Survey on Transfer Learning, and it stated the following: according to different situations of labeled and unlabeled data in the source domain, ...
1
vote
0answers
38 views

Why does $E_q[\log p(\mathbf{w}|\mathbf{z},\beta)]=\sum_{n=1}^{N}\sum_{i=1}^{k}\sum_{j=1}^{V}\phi_{ni}w_n^j\log \beta_{ij}$ hold in LDA?

I'm having trouble understanding an equality that comes up in the original LDA paper by Blei et al.: Consider the classical LDA model, i.e. for every document $\textbf{w}=(w_1,\ldots,w_N)$ in a text ...
0
votes
1answer
52 views

K dimensional WL test in K-WL GCN

I am reading a paper on the K-WL GCN. I did not complete the paper yet but I just skimmed over it. There I am trying to understand the K-WL test (page 3 Weisfeiler-Leman Algorithm). I think my ...
1
vote
2answers
89 views

Bayesian Perceptron: How is it compatible to Bayes Theorem?

I found a very interesting paper on the internet that tries to apply Bayesian inference with a gradient-free online-learning approach: [Bayesian Perceptron: Bayesian Perceptron: Towards fully Bayesian ...
0
votes
1answer
61 views

Bayesian Perceptron: Why to marginalize over neuron's output instead of it's weights?

I found a very interesting paper on the internet that tries to apply Bayesian inference with a gradient-free online-learning approach: Bayesian Perceptron: Towards fully Bayesian Neural Networks. I ...
0
votes
0answers
24 views

How is the discounted maximum entropy objective obtained for soft-q-learning and SAC

In the soft q-learning paper, they provide an expression for the maximum entropy objective that takes discounting into account. My main question is: can someone explain how they incorporated ...
1
vote
1answer
23 views

Why does this paper say that the Nash-equilibrium of GAN is given by a discriminator which is 0 everywhere on the data distribution?

I am facing difficulty in understanding the bolded portion of the following statement from this paper GANs are defined by a min-max two-player game between a discriminative network $D_\Psi(x)$ and ...
0
votes
1answer
34 views

What to do with a GAN that trained well but got worse over time?

I am training a WGAN-GP network based on the following paper, though I am using a different dataset. Now, for the first ~ 60-70 epochs, my network trained really well, which I could see in the loss ...
1
vote
1answer
52 views

Is there a full and precise formulation of Theorem 1 in the Integrated Gradients paper?

Theorem 1 (page 5) in the paper about Integrated Gradients states that Integrated gradients is the unique path method that is symmetry-preserving. What I miss is A precise formulation of the ...
0
votes
0answers
10 views

Is there any method that combines temporal action proposals with multiple actions' classifiers?

I am trying to classify actions in untrimmed videos. These videos contain a very imbalanced set of actions (where the background class is the majority). I have previously tried frame-wise action ...
3
votes
0answers
28 views

What's up with Neural Stochastic Differential Equations from a practical standpoint?

I've spent a few days reading some of the new papers about Neural SDEs. For example, here is one from Tzen and Raginsky and here is one that came out simultaneously by Peluchetti and Favaro. There are ...
0
votes
0answers
28 views

How to implement the deconv which is used in “Visualizing and Understanding Convolutional Networks”

I'm trying to understand the deconv referenced in the paper Visualizing and Understanding Convolutional Networks The paper states (section 2, p. 3): the deconvnet uses transposed versions of the same ...
0
votes
0answers
34 views

Are regret values in each block of MC external sampling stored in each node of the block we are traversing down (denoted by $\{Q_1,…, Q_n\}$)?

Everything I know about Monte Carlo counterfactual regret minimization (CFR) comes from the paper Monte Carlo Sampling for Regret Minimization in Extensive Games by Marc Lanctot et al. So, I will use ...
0
votes
1answer
29 views

Converting age and sex variables to a 64-unit dense layer

I am studying a preprint for my own learning (https://www.medrxiv.org/content/medrxiv/early/2020/04/27/2020.04.23.20067967.full.pdf) and I am befuddled by the following detail of the neural network ...
1
vote
0answers
52 views

BlackOut - ICLR 2016: need help understanding the cost function derivative

In the ICLR 2016 paper BlackOut: Speeding up Recurrent Neural Network Language Models with very Large Vocabularies, on page 3, for eq. 4: $$ J_{ml}^s(\theta) = log \ p_{\theta}(w_i | s) $$ They have ...
1
vote
1answer
117 views

How exactly is Monte Carlo counterfactual regret minimization with external sampling implemented?

I have read many papers, such as this or this, explaining how external sampling works, but I still don't understand how the algorithm works. I understand you divide $Q$, which is the set of all ...

1
2 3 4 5