Questions tagged [papers]

For questions related to artificial intelligence research papers. So, you should use this tag if you want someone to clarify something in a research paper.

Filter by
Sorted by
Tagged with
6
votes
1answer
85 views

How does the network know which objects to track in the paper “Label-Free Supervision of Neural Networks with Physics and Domain Knowledge”?

I was reading the paper Label-Free Supervision of Neural Networks with Physics and Domain Knowledge, published at AAAI 2017, which won the best paper award. I understand the math and it makes sense. ...
0
votes
1answer
50 views

What is the difference between the definition of “accuracy” in machine learning and federated learning?

What is the difference between the definition of "accuracy" in machine learning and federated learning? In particular, how is the accuracy calculated in the following paper: eq (3.12) https:...
0
votes
0answers
24 views

How to choose the reward in reinforcement learning? [duplicate]

I am solving a combinatorial optimization problem, where I do not have a global optimum, so the goal is to improve the objective function as much as possible. So, to do this, I was inspired by this ...
1
vote
0answers
12 views

What's mutual exclusivity in meta-learning?

What do we mean by mutual exclusivity of tasks? This work (E Pan, 21) and this one (M Yin, 20) state that most classification meta-learning algorithms fail for non-mutually exclusive tasks as the ...
1
vote
1answer
543 views

Are mult-adds and FLOPs equivalent?

I am comparing different CNN architectures for edge implementation. Some papers describing architectures refer to mult-adds, like the MobileNet V1 paper, where it is claimed that this net has 569M ...
2
votes
0answers
18 views

What is meant by Hinton when he refers to “Part-Whole Hierarchies” in his GLOM framework

I was recently reading Hinton's GLOM idea How to represent part-whole hierarchies in a neural network, and I am simply unsure about what exactly he means when he says parsing images into "part-...
2
votes
1answer
70 views

What is a heatmap in the CornerNet paper?

I have been working on understanding how CornerNet works, but I couldn't figure out a few parts about the architecture. First, the authors mention that there are 3 distinct parts to be predicted as a ...
0
votes
0answers
11 views

Why do the authors of this paper down-sample by $ds_1 / 2$ (in the context of coarse-to-fine segmentation)?

This question is a follow-up of this post and based on this paper. In section 2.2, the authors write: In the first level, the 3D FCN is trained on images of the lowest resolution in order to capture ...
0
votes
0answers
9 views

How exactly is masking performed in the training part of the paper “Semi-Supervised Classification with Graph Convolutional Networks”?

I am struggling to understand the training part of the paper Semi-Supervised Classification with Graph Convolutional Networks (2017) by Thomas Kipf and Max Welling. The GitHub repo is here. I do not ...
1
vote
1answer
330 views

In Faster R-CNN, how can I get the predicted bounding box given the neural network's output?

The RPN loss in Faster RCNN paper is $$ L({p_i}, {t_i}) = \frac{1}{N_{cls}} \sum_{i} L_{cls}(p_i,p_i^*) + \lambda \frac{1}{N_{reg}} \sum_i p_i^* L_{reg}(t_i, t_i^*) $$ For regression problems, we have ...
3
votes
1answer
85 views

How are the “Link Strength true”, “Link Strength blind” and “Mutual Information” calculated in this report on Bayesian networks?

I'm trying to understand how to calculate the strength of every arc in a Bayesian Network. I came across this report Measuring Connection Strengths and Link Strengths in Discrete Bayesian Networks, ...
1
vote
1answer
21 views

Why the non-exploitation of edge labels in current graph convolutions “results in an overly homogeneous view of local graph neighborhoods”?

I am currently reading a paper called Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs (2017, CPPR), and I cannot understand the following sentence: We identify that the ...
1
vote
1answer
31 views

Understanding Generalized Advantage Estimate in reinforcement learning

I was reading the paper on Generalized Advantage Estimate. It first introduces a generalized form of policy gradient equation without involving $\gamma$ and then it says the following: We will ...
4
votes
1answer
67 views

What is the relation between self-taught learning and transfer learning?

I am new to transfer learning and I start by reading A Survey on Transfer Learning, and it stated the following: according to different situations of labeled and unlabeled data in the source domain, ...
0
votes
0answers
61 views

Comparing heuristics in A* search and rescue operation

I was reading a research paper titled A Comparative Study of A-star Algorithms for Search and rescue in Perfect Maze (2011). I have some doubts regarding it: 1. The Evaluation Function of $\mathrm{A}^...
1
vote
1answer
34 views

Why is Adam trapped in bad/suspicious local optima after the first few updates?

In the paper On the Variance of the Adaptive Learning Rate and Beyond, in section 2, the authors write To further analyze this phenomenon, we visualize the histogram of the absolute value of ...
2
votes
2answers
394 views

What are some alternatives to “Papers with Code”?

There are lots of research papers available that are worth reading. We can read papers easily, but the associated code (not necessarily the official one developed by the authors of the paper) is often ...
0
votes
1answer
19 views

How does high entropy targets relate to less variance of the gradient between training cases?

I've been trying to understand the Distilling the Knowledge in a Neural Network paper by Hinton et al. But I cannot fully understand this: When the soft targets have high entropy, they provide much ...
1
vote
2answers
80 views

Bayesian Perceptron: How is it compatible to Bayes Theorem?

I found a very interesting paper on the internet that tries to apply Bayesian inference with a gradient-free online-learning approach: [Bayesian Perceptron: Bayesian Perceptron: Towards fully Bayesian ...
3
votes
2answers
65 views

Why some neural network models in the 1980s shown as circuit models

I am familiar with the currently popular neural network models that have weights and are trained with backpropagation and gradient descent. However, I came across a different type of neural network ...
1
vote
1answer
24 views

How is the variational lower bound for hard attention derived in Show, Attend and Tell

How is the jump from line 1 to line 2 done in equation 10 of Show, Attend and Tell? While we're at it, another thing that might be muddying the waters for me is that I'm not clear on what the sum is ...
3
votes
1answer
94 views

How are the observations stored in the RNN that encodes the state?

I am a bit confused about observations in RL systems which use RNN to encode the state. I read a few papers like this and this. If I were to use a sequence of raw observations (or features) as an ...
0
votes
0answers
22 views

Do $V_\theta$ and $V_s$ represent partial or total derivatives in the paper “Learning Continuous Control Policies by Stochastic Value Gradients”?

I was reading up on the Stochastic Value Gradients paper by Heess et al. In the paper, they describe a recursive process to calculate path-wise derivatives via equations (3) and (4), at the bottom of ...
6
votes
2answers
88 views

What does “semantic gap” mean?

I was reading DT-LET: Deep transfer learning by exploring where to transfer, and it contains the following: It should be noted direct use of labeled source domain data on a new scene of target domain ...
1
vote
0answers
27 views

What is a Hebbian linear classifier?

I was reading Deep Learning of Representations for Unsupervised and Transfer Learning, and they state the following: They have only a small number of unlabeled examples (4096) and very few labeled ...
1
vote
1answer
52 views

Is there a full and precise formulation of Theorem 1 in the Integrated Gradients paper?

Theorem 1 (page 5) in the paper about Integrated Gradients states that Integrated gradients is the unique path method that is symmetry-preserving. What I miss is A precise formulation of the ...
3
votes
1answer
114 views

Understanding proof of lemma 1 (policy improvement bound) of the “Trust Region Policy Optimization” paper

In the Trust Region Policy Optimization paper, in Lemma 1 of Appendix A, I did not quite understand the transition from (21) from (20). In going from (20) to (21), $A^\pi(s_t, a_t)$ is substituted ...
1
vote
2answers
68 views

In variational autoencoders, what does p(x|z) mean?

If $x \sim \mathcal{N}(\mu,\,\sigma^{2})$, then it is a continuous variable, and therefore $P(x) = 0$ for any x. One can only consider things like $P(x<X)$ to get a probability greater than 0. So ...
1
vote
0answers
17 views

Clonal operator in Immune Clonal Strategy

I was reading about Immune Clonal Strategy, specifically about Monoclonal operator from Immunity clonal strategies, and it goes as follows: Here $a_i $ is a point and $a_i = \{ x_1, x_2, \cdots, x_m \...
1
vote
0answers
25 views

The MLP output of a neural network can be written as $\|x\|\|w_l\|\cos(\theta_l)$: why is the norm easier to maximize?

The MLP output of a neural network is a dot product between the weights and the input and therefore can be written as $\|x\|\|w_l\|\cos(\theta_l)$ (see this for more details), where $x$ is the input, $...
1
vote
0answers
23 views

What is the meaning of “Our current objective weights every token equally and lacks a notion of what is most important to predict” in the GPT-3 paper?

On page 34 of OpenAI's GPT-3, there is a sentence demonstrating the limitation of objective function: Our current objective weights every token equally and lacks a notion of what is most important to ...
2
votes
0answers
36 views

How many papers about AI / ML were published in the recent years?

I am trying to formulate an argument at work saying the disruption in AI/ML is very high and that it is hard to stay "state of the art". I would like to support that hypothesis by numbers. ...
0
votes
1answer
125 views

How do you perform a gradient based adversarial attack on an SVM based model?

I have an SVM currently and want to perform a gradient based attack on it similar to FGSM discussed in Explaining And Harnessing Adversarial Examples. I am struggling to actually calculate the ...
0
votes
0answers
56 views

How to evaluate a Genome in NEAT

I am trying to implement NEAT from scratch by going through the original NEAT paper. I implemented a Genome class which consists of a list of Node Genes and ...
2
votes
0answers
90 views

What is the purpose of the DAMSM loss for the generators in AttnGAN?

I am confused about the training part in AttnGan. If you observe page 3. There are two types of losses for generator network: one involving the Deep Attentional Multimodal Similarity Model (DAMSM) ...
0
votes
1answer
39 views

K dimensional WL test in K-WL GCN

I am reading a paper on the K-WL GCN. I did not complete the paper yet but I just skimmed over it. There I am trying to understand the K-WL test (page 3 Weisfeiler-Leman Algorithm). I think my ...
0
votes
1answer
56 views

Bayesian Perceptron: Why to marginalize over neuron's output instead of it's weights?

I found a very interesting paper on the internet that tries to apply Bayesian inference with a gradient-free online-learning approach: Bayesian Perceptron: Towards fully Bayesian Neural Networks. I ...
0
votes
0answers
24 views

How is the discounted maximum entropy objective obtained for soft-q-learning and SAC

In the soft q-learning paper, they provide an expression for the maximum entropy objective that takes discounting into account. My main question is: can someone explain how they incorporated ...
1
vote
1answer
23 views

Why does this paper say that the Nash-equilibrium of GAN is given by a discriminator which is 0 everywhere on the data distribution?

I am facing difficulty in understanding the bolded portion of the following statement from this paper GANs are defined by a min-max two-player game between a discriminative network $D_\Psi(x)$ and ...
2
votes
1answer
47 views

In the MINE paper, why is $\hat{G}_B$ biased, and how does the exponential moving average reduce the bias?

While reading the Mutual Information Neural Estimation (MINE) paper [1] I came across section 3.2 Correcting the bias from the stochastic gradients. The proposed method requires the computation of the ...
3
votes
2answers
144 views

What is different in each head of a multi-head attention mechanism?

I have a difficult time understanding the "multi-head" notion in the original transformer paper. What makes the learning in each head unique? Why doesn't the neural network learn the same ...
0
votes
1answer
29 views

What to do with a GAN that trained well but got worse over time?

I am training a WGAN-GP network based on the following paper, though I am using a different dataset. Now, for the first ~ 60-70 epochs, my network trained really well, which I could see in the loss ...
0
votes
0answers
10 views

Is there any method that combines temporal action proposals with multiple actions' classifiers?

I am trying to classify actions in untrimmed videos. These videos contain a very imbalanced set of actions (where the background class is the majority). I have previously tried frame-wise action ...
3
votes
1answer
368 views

What is the significance of this Stanford University “Financial Market Time Series Prediction with RNN's” paper?

Researchers at Stanford University released, in 2012, the paper Financial Market Time Series Prediction with Recurrent Neural Networks. It goes on to discuss how they used echo state networks to ...
1
vote
1answer
36 views

Why is this variable in equation 2 of the SQAIR paper a random vector of $n$ ones followed by a zero?

I've been reading the SQAIR paper lately, and the mathematics involved seems a bit complicated. Some background, about the paper: SQAIR stands for Sequential Attend, Infer, Repeat - the paper does ...
3
votes
0answers
28 views

What's up with Neural Stochastic Differential Equations from a practical standpoint?

I've spent a few days reading some of the new papers about Neural SDEs. For example, here is one from Tzen and Raginsky and here is one that came out simultaneously by Peluchetti and Favaro. There are ...
0
votes
0answers
28 views

How to implement the deconv which is used in “Visualizing and Understanding Convolutional Networks”

I'm trying to understand the deconv referenced in the paper Visualizing and Understanding Convolutional Networks The paper states (section 2, p. 3): the deconvnet uses transposed versions of the same ...
3
votes
1answer
119 views

How to perform back-propagation in Decoupled Neural Interfaces?

I am attempting to create a fully decoupled feed-forward neural network by using decoupled neural interfaces (DNIs) as explained in the paper Decoupled Neural Interfaces using Synthetic Gradients (...
0
votes
0answers
32 views

Are regret values in each block of MC external sampling stored in each node of the block we are traversing down (denoted by $\{Q_1,…, Q_n\}$)?

Everything I know about Monte Carlo counterfactual regret minimization (CFR) comes from the paper Monte Carlo Sampling for Regret Minimization in Extensive Games by Marc Lanctot et al. So, I will use ...
0
votes
1answer
29 views

Converting age and sex variables to a 64-unit dense layer

I am studying a preprint for my own learning (https://www.medrxiv.org/content/medrxiv/early/2020/04/27/2020.04.23.20067967.full.pdf) and I am befuddled by the following detail of the neural network ...

1
2 3 4 5