Questions tagged [papers]

For questions related to artificial intelligence research papers. So, you should use this tag if you want someone to clarify something in a research paper.

Filter by
Sorted by
Tagged with
1 vote
0 answers
101 views

Question about the Conditioning Augmentation technique?

In the paper StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks, the goal is to convert text descriptions into images. The text encoder encodes the ...
David's user avatar
  • 51
0 votes
0 answers
32 views

Why is masked self attention necessary on GPT decoder

I'm currently reading the paper for the first GPT model and I'm confused about why masked self attention is necessary and I haven’t found any good answers online. The consensus seems to be that we don'...
Kiran Manicka's user avatar
1 vote
0 answers
39 views

Are there neural networks that compute weights dynamically based on geometric attributes of neurons?

I am interested in exploring neural network architectures where the weights are not stored but are computed dynamically based on certain attributes or "dimensions" of the connected neurons. ...
Deadbeef Development's user avatar
1 vote
1 answer
108 views

How do Multimodal LLMs of 2023 score on the ARC benchmark (in 2020: 20% Accuracy)

Q: I wonder if anyone has tried to solve the ARC tasks with one of the state-of-the-art Multimodal LLMs? Can LLMs that can process graphics input do the following? (This question is not about the The ...
knb's user avatar
  • 165
0 votes
0 answers
20 views

Training Diffusion Probabilistic Models

My question is in regards to the training process for Diffusion Probabilistic Models by Sohl-Dickenstein et. al. and also Ho et. al. and this blog post For the derivation of the model log-likelihood ...
diffusionQuestion's user avatar
0 votes
0 answers
54 views

Question about feature matrix and notation in the paper Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning

Let me know if this is not the place for this question. I'll take it down happily if that's the case. Also, I emailed the authors, but it seems like I won't be getting a response, so that's why I am ...
Schach21's user avatar
  • 242
0 votes
0 answers
48 views

Understanding the functionality of the switch in the latent diffusion models: Does conditioning information pass to both cross attention and $z_{T}$?

Consider the following diagram from the paper titled High-Resolution Image Synthesis with Latent Diffusion Models by Robin Rombach et. al., In the context of this diagram, I'm uncertain about the ...
hanugm's user avatar
  • 3,612
0 votes
0 answers
24 views

Understanding the decreasing influence of text embedding in Text-to-Image diffusion models: A Mathematical perspective

I've been reading the paper titled eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers by Yogesh Balaji et. al. Consider the following excerpt from the abstract of the paper ...
hanugm's user avatar
  • 3,612
0 votes
0 answers
8 views

Clarification regarding the training status of 'domain specific encoder' in stable diffusion

I am currently studying the paper titled High-Resolution Image Synthesis with Latent Diffusion Models by Robin Rombach et al. Specifically, I am focused on the section 3.3. Conditioning Mechanisms. In ...
hanugm's user avatar
  • 3,612
1 vote
1 answer
38 views

In-depth understanding of formulation and guidance mechanisms in Diffusion models

I've been reading a research paper titled High-Resolution Image Synthesis with Latent Diffusion Models by Robin Rombach et al. and came across an a concept related to diffusion models (DMs). In the ...
hanugm's user avatar
  • 3,612
1 vote
0 answers
32 views

Can a Fully Connected Neural Network represent all Neural Networks of smaller size?

A fully connected Neural Network architecture can be characterized by a vector $\mathbf a = (a_0,a_1,\ldots,a_L)\in\mathbb N^{L+1}$ and an activation function $\sigma :\mathbb R\to\mathbb R$. In this ...
Stratos supports the strike's user avatar
1 vote
0 answers
30 views

Meaning of a symbol in trace

I am reading this paper Not All Samples Are Created Equal: Deep Learning with Importance Sampling. In the paper there is a deviation shown below. I can understand everything except the $V$ symbol in ...
onexpeters's user avatar
0 votes
1 answer
342 views

In the original diffusion model paper, why do they sample the first step with the same loss?

In the original diffusion model paper by Sohl-Dickstein et al., they explain very little about calculating the loss and training and network to learn the diffusion process. They did publish a ...
Grieverheart's user avatar
0 votes
0 answers
8 views

Filter distribution of Latent variable models

In this paper https://arxiv.org/pdf/1907.00953.pdf, about stochastic latent variable models, the paper says "We use the reparameterization trick to sample from the filtering distribution". I ...
chadmc's user avatar
  • 15
0 votes
2 answers
86 views

BYOL: Why is there a prediction network in the online network but not in the target network?

In the BYOL paper, the following architecture is presented: Why is a prediction network added to the online network, which is not present in the target network? How are the online and target compared ...
Robin van Hoorn's user avatar
0 votes
0 answers
44 views

Layer Questions regarding Bidirectional VAE (D3VAE)

I am currently trying to figure out how D3VAE are working, but I can't seem to understand the network architecture given. The paper can be found here: https://openreview.net/pdf?id=rG0jm74xtx The ...
Patrick Lehnen's user avatar
0 votes
1 answer
58 views

What does crop size refer to in DeepLabv3 paper?

In the paper in which DeepLabv3 is presented, the authors are mentioning that: "For atrous convolution with large rates to be effective, large crop size is required; otherwise, the filter ...
Manuel Maior's user avatar
3 votes
1 answer
366 views

Difference between dot product attention and "matrix attention"

As far as I know, attention was first introduced in Learning To Align And Translate. There, the core mechanism which is able to disregard the sequence length, is a dynamically-built matrix, of shape ...
Gulzar's user avatar
  • 749
-1 votes
1 answer
152 views

Understanding self attention - How come there is no connection between different states?

During trying to understand transformers by reading Attention is all you need, I noticed the authors constantly refer to "self attention" without explaining it. The original attention ...
Gulzar's user avatar
  • 749
1 vote
1 answer
56 views

In the Dropout paper, why would increasing the dropout increase the error rate if the capacity is constant?

In the original paper on dropout, in section 7.3.2, we see that while keeping $pn$ constant, we get a (test) error increase by decreasing retainment below 0.6. Why would that happen? If $pn$ is ...
Apples14's user avatar
0 votes
0 answers
26 views

How can I transform the LSTM output to an embedding matrix of actions?

Page 3 of the paper Feudal Networks for Hierarchical Reinforcement Learning describes producing an 'embedding matrix' $U$ of size $(|a| \times k)$ from the output of an LSTM, given a $(d \times 1)$ ...
Telf's user avatar
  • 1
0 votes
0 answers
49 views

YOLO v1 confidence score during inference

I am studying the paper of Yolo v1. For the training part, everything is clear to me. I cannot understand how does the confidence works during the inference stage, since Pr(Object) and ground truth ...
Lorenzo Epifani's user avatar
1 vote
0 answers
112 views

Has anyone here tried to implement MADDPG for a different environment and succeeded?

Has anyone tried implementing the multi-agent RL algorithm MADDPG (I've linked the paper below)? The paper seems to have a good amount citations, and they do have their code on github. However, a few ...
Confuse's user avatar
  • 111
0 votes
0 answers
45 views

Are there papers that do an empirical investigation on DRL hyperparameters?

I am looking for papers that perform a study on DRL hyper-parameters. This paper does a fantastic job of describing the hyperparameters for on-policy algorithms. It would be great to get similar ...
desert_ranger's user avatar
5 votes
2 answers
1k views

InstructGPT: What is the sigma in the loss function and why $\log(\cdot)$ is being used?

InstructGPT: What is the sigma in the loss function and why $\log(\cdot)$ is being used? $$ \operatorname{loss}(\theta) = -\frac{1}{\binom{K}{2}}E_{(x,y_w,y_l)\sim D}[\log(\sigma(r_{\theta}(x, y_w) - ...
Nathan B's user avatar
  • 161
0 votes
0 answers
23 views

How to understand the definition of $\lambda_i$ used in the return estimator proposed in the paper "Human-level Atari 200x faster"?

I'm reading article called "Human-level Atari 200x faster" Agent57 uses Retrace (Munos et al., 2016) to compute return estimates from off-policy data, but we observed that it tends to cut ...
Kari's user avatar
  • 260
1 vote
1 answer
41 views

What does this bracket notation $\langle\phi(x),v\rangle$ mean?

I found it at the bottom of page 2 of the paper Intriguing properties of neural networks (2014), in the form of $$\underset{x\in\mathcal{I}}{\mathrm{arg\,max}}\langle\phi(x),v\rangle$$
Nils André's user avatar
1 vote
1 answer
668 views

Are there Explainable GNN methods for node regression tasks?

I am wondering if there are gnn explainable methods for a regression task (e.g., traffic forecasting) where nodes have numerical features and the predicted output is a numerical value. Most of ...
Achiles Br's user avatar
2 votes
2 answers
135 views

Where does the proximal policy optimization objective's ratio term come from?

I will use the notation used in the proximal policy optimization paper. What approximation is needed to arrive at the surrogate objective (equation (6) above) with the ratio $r_t(\theta)$? Put ...
user's user avatar
  • 203
2 votes
1 answer
326 views

Why do activation functions in neural networks have to be non-polynomial to approximate any function?

Can someone give me the main idea of the paper Multilayer Feedforward Networks With a Nonpolynomial Activation Function Can Approximate Any Function? I'm having trouble understanding it.
Alicia Chi's user avatar
1 vote
0 answers
14 views

Where exactly is permutation happening in equation 5 of the paper "Learning with Sets in Multiple Instance Regression Applied to Remote Sensing"?

I am reading the article Learning with Sets in Multiple Instance Regression Applied to Remote Sensing about creating an embedding which is order-invariant to inputs ($m_{l}$). They referred to order-...
Oculu 's user avatar
  • 43
1 vote
1 answer
54 views

What is the meaning of $ (I - \gamma P^{\pi})^{-1} \left[\frac{\mu(a|s)}{\hat \pi_{\beta}(a|s)} \right](s, a)$?

In Theorem 3.1 of the conservative q-learning paper, what is the meaning of $$ (I - \gamma P^{\pi})^{-1} \left[\frac{\mu(a|s)}{\hat \pi_{\beta}(a|s)} \right](s, a)$$? I thought $(I - \gamma P^{\pi})^{-...
hongshan.li's user avatar
5 votes
0 answers
69 views

What exactly is non-delusional Q-learning?

Problems occur when we combine Q-learning with a function approximator. What exactly is the delusional-bias and non-delusional Q-learning? I am talking about the neurIPS 18 best paper Non-delusional Q-...
wrek's user avatar
  • 183
1 vote
0 answers
65 views

How does having zero advantage help with identifiability?

I am reading the D3QN paper and they have the following paragraph - Equation (7) is unidentifiable in the sense that given $Q$ we cannot recover $V$ and $A$ uniquely. To see this, add a constant to $...
desert_ranger's user avatar
1 vote
1 answer
813 views

Papers on Prompt Engineering

I am into AI in general and NLP in particular. Besides, I have a background in philosophy, and the new LLMs like GPT-3 seem to have exciting capabilities. I want to study prompt engineering (for ...
Mehdi Abbassi's user avatar
0 votes
1 answer
32 views

Is there any work done on topic agnostic binary topic classification?

In the recent preprint paper Tree-based Focused Web Crawling with Reinforcement Learning a new model is introduced to classify web pages called KwBiLSTM. The input to this model is a featurized ...
Kroshtan's user avatar
  • 249
1 vote
0 answers
178 views

Exact definition of WRN-d-k (Wide ResNet)

I am a little confused about the WRN-d-k notation from Wide Residual Networks. To quote the paper, In the rest of the paper we use the following notation: WRN-n-k denotes a residual network that has ...
nalzok's user avatar
  • 271
0 votes
1 answer
954 views

In-batch negative training Improves the results

I have read Dense passage retrieval for Open Domain Question Answering, and in page 6 it talks about in-batch negative training, it states the following: We find that using a similar configuration (7 ...
Kais Hasan's user avatar
0 votes
1 answer
55 views

What do we mean by the notation $\mathbf{x}_{p} \in \mathbb{R}^{N \times\left(P^{2} \cdot C\right)}$?

I was going through this VIT paper, what will it look like in torch , if we are trying to write this expression.
TheExorcist's user avatar
1 vote
0 answers
91 views

What are semantic word spaces in NLP?

In the abstract of this paper, it's written Semantic word spaces have been very useful but cannot express the meaning of longer phrases in a principled way. I would like to understand what semantic ...
Hermi's user avatar
  • 121
2 votes
1 answer
305 views

In the MuZero paper, how does backprop in the MCTS account for the immediate reward from each edge?

On page 12 of this paper: Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, it describes how MCTS works for the MuZero algorithm. It states in equation 4 that during the 'backup' ...
Matrix001's user avatar
0 votes
1 answer
2k views

What is a "canonical space"?

I am reading the paper on 3D reconstruction, ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction, and I encountered the term "canonical space". What is a "...
Trong-Thang Pham's user avatar
0 votes
1 answer
188 views

Is the "Helvetica scenario" mentioned here related to Artificial Intelligence?

Consider the following sentence from the original GAN paper titled Generative Adversarial Nets in particular, $G$ must not be trained too much without updating $D$, in order to avoid "the ...
hanugm's user avatar
  • 3,612
2 votes
1 answer
69 views

How to decode P bits that represent a random weight generator?

So I've been tasked by my neural network professor at university to replicate the following research: Intelligent Breast Cancer Diagnosis Using Hybrid GA-ANN. Each chromosome represents a possible net,...
JOSEPH CAROÈ's user avatar
4 votes
1 answer
261 views

What is the total number of actions and rewards count

Reading this two articles about Reinforcement Learning: Deep Reinforcement Learning with Double Q-learning by Hado van Hasselt et al. Human-level control through deep reinforcement learning by ...
Jigberto's user avatar
  • 143
0 votes
1 answer
24 views

What is the right way to find the alphas in this equation?

In the Grad-CAM++ paper the following equation (7) is posed (written here without the relu function): $$ Y^c = \sum_k \Bigl( \Bigl\{ \sum_{a,b} \alpha_{ab}^{kc} \cdot \frac{\partial Y^c}{\...
mlerma54's user avatar
  • 141
0 votes
1 answer
83 views

What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?

I have a question about the Grad-CAM++ paper. I do not understand how the following equation (10) for the alphas is obtained: $$ \alpha_{ij}^{kc} = \frac{\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}} {...
mlerma54's user avatar
  • 141
2 votes
1 answer
276 views

What is a 'degenerate run' in evaluating model performance?

I've recently come across a paper that uses the term "degenerate run", but I'm not sure if I understand what it means. The idea is that when they report the average performance of running ...
Pedram's user avatar
  • 121
1 vote
0 answers
118 views

What does "position" in "each position in the decoder" denote in the Transformer's original paper?

I am reading Attention is All You Need and I feel confused about the word "position" in this paper, by the way I'm not native English speaker which may cause my confusion which has confused ...
zenga's user avatar
  • 11
1 vote
0 answers
290 views

How are 4D cost volumes constructed for DL based stereo matching?

I read a paper on Stereo Matching using Pyramid Cost Volumes (paper link: Semantic Stereo Matching with Pyramid Cost Volumes). At some point, in the proposed architecture, after: Feature extraction ...
Manuel Maior's user avatar

1
2 3 4 5
7