Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [papers]

For questions related to artificial intelligence research papers. So, you should use this tag if you want someone to clarify something in a research paper.

Filter by
Sorted by
Tagged with
0 votes
1 answer
8 views

How would I approach finding the source code for the paper that is being discussed in this article?

I have read a news article relating to recent research into graph theory-based fraud detection using machine learning, but cannot find the source code for the paper discussed. The paper can be found ...
Rasai Stewart's user avatar
2 votes
0 answers
47 views

Should intelligent machines have a body? (Reference request)

I was studying AI when a question came to my mind. I have to write a 6-pages essay on a chosen topic in AI, and I think it can be interesting to reflect on the measure in which machines should have a ...
Amanda Wealth's user avatar
0 votes
1 answer
49 views

What is the difference between RAG-Sequence Model and RAG-Token Model?

When I start to learn about RAG, I can't understand the difference between the RAG-Sequence Model and RAG-Token Model. First of all, I see that for RAG-Sequence Model we use just one document to ...
LAILA EL OUEDEGHYRY's user avatar
0 votes
0 answers
26 views

Should we sum or mean reduce the KL loss in Bayes by Backprop?

I'm unsure if you're supposed to use sum or mean reduction of KL loss for Bayes by Backprop. For example, the BayesianTorch library does both: it reduces by mean across each individual tensor (as seen ...
profPlum's user avatar
  • 424
0 votes
0 answers
17 views

What are some good resources to understand the code for 3D Gaussian Splatting?

I am looking for some good resources like videos or blogs (or other githubs!) that go through the code of Gaussian Splatting and explain the major components and how they are working. Haven't found ...
ChaoS Adm's user avatar
  • 101
1 vote
0 answers
15 views

2 Different attentions mentioned in DeBERTa

I am trying to understand the disentangled self-attention in DeBERTa paper, but I can't understand at which point of the paper they stop from introducing background knowledge about already known ...
Mahammad Yusifov's user avatar
4 votes
1 answer
138 views

Notation used in paper on Continuous Time Reinforcement Learning

I am working on implementing the learning shown in this paper: https://homes.cs.washington.edu/~todorov/courses/amath579/reading/Continuous.pdf In the paper, the authors devise a continuous time ...
Derick Diana's user avatar
1 vote
1 answer
77 views

What does the notation $\hat{A}_t\left(s_{0: \infty}, a_{0: \infty}\right)$ appearing in Generalized Advantage Estimation mean?

In general, the advantage function is defined as: $A^\pi\left(s_t, a_t\right):=Q^\pi\left(s_t, a_t\right)-V^\pi\left(s_t\right)$ So far, I understand this formular like this: With the advantage ...
kklaw's user avatar
  • 185
2 votes
1 answer
66 views

What does "aligned" across domains in domain adaptation?

within Delving into Local Features for Open-Set Domain Adaptation in Fundus Image Analysis paper. I got trouble in understanding their cluster-aware contrastive adaption $\mathcal{L}_\text{cca}$. I ...
chickensoup's user avatar
0 votes
0 answers
27 views

What is the Input to the agent in the paper "Model Based Reinforcement Learning for Atari", and why does the world model run at inference time?

I am currently reading the paper Model based Reinforcement Learning for Atari. However, they do not specify what exactly they use as input for the agent. I believe it to be the observation space - ...
Nick's user avatar
  • 1
0 votes
1 answer
74 views

Overcoming the quadratic scaling in transformer architecture

Do you know any papers that try to overcome quadratic scaling problems by attending lower dimensional representations in the dimension of tokens? For example, let's say that the input to the ...
Andy Yermakov's user avatar
0 votes
0 answers
14 views

Are There Scientific Papers on Methods for Robustifying LLMs and Current Challenges They Face?

I'm currently researching Large Language Models (LLMs) and am particularly interested in the recent advancements and challenges in this field. My focus is on understanding the methods being developed ...
Iman Mohammadi's user avatar
0 votes
0 answers
96 views

Example VQ-VAE code for audio in paper "Neural Discrete Representation Learning"

I want to replicate the paper "Neural Discrete Representation Learning" by van der Oord et al (2018). DeepMind provides an example for CIFAR images on GitHub. It seems that the model for ...
emonigma's user avatar
1 vote
0 answers
160 views

Question about the Conditioning Augmentation technique?

In the paper StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks, the goal is to convert text descriptions into images. The text encoder encodes the ...
user avatar
1 vote
0 answers
58 views

Are there neural networks that compute weights dynamically based on geometric attributes of neurons?

I am interested in exploring neural network architectures where the weights are not stored but are computed dynamically based on certain attributes or "dimensions" of the connected neurons. ...
Deadbeef Development's user avatar
1 vote
1 answer
289 views

How do Multimodal LLMs of 2023 score on the ARC benchmark (in 2020: 20% Accuracy)

Q: I wonder if anyone has tried to solve the ARC tasks with one of the state-of-the-art Multimodal LLMs? Can LLMs that can process graphics input do the following? (This question is not about the The ...
knb's user avatar
  • 165
0 votes
0 answers
23 views

Training Diffusion Probabilistic Models

My question is in regards to the training process for Diffusion Probabilistic Models by Sohl-Dickenstein et. al. and also Ho et. al. and this blog post For the derivation of the model log-likelihood ...
diffusionQuestion's user avatar
0 votes
0 answers
58 views

Question about feature matrix and notation in the paper Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning

Let me know if this is not the place for this question. I'll take it down happily if that's the case. Also, I emailed the authors, but it seems like I won't be getting a response, so that's why I am ...
Schach21's user avatar
  • 242
3 votes
1 answer
182 views

Understanding the functionality of the switch in the latent diffusion models: Does conditioning information pass to both cross attention and $z_{T}$?

Consider the following diagram from the paper titled High-Resolution Image Synthesis with Latent Diffusion Models by Robin Rombach et. al., In the context of this diagram, I'm uncertain about the ...
hanugm's user avatar
  • 3,890
0 votes
0 answers
32 views

Understanding the decreasing influence of text embedding in Text-to-Image diffusion models: A Mathematical perspective

I've been reading the paper titled eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers by Yogesh Balaji et. al. Consider the following excerpt from the abstract of the paper ...
hanugm's user avatar
  • 3,890
0 votes
0 answers
12 views

Clarification regarding the training status of 'domain specific encoder' in stable diffusion

I am currently studying the paper titled High-Resolution Image Synthesis with Latent Diffusion Models by Robin Rombach et al. Specifically, I am focused on the section 3.3. Conditioning Mechanisms. In ...
hanugm's user avatar
  • 3,890
1 vote
1 answer
51 views

In-depth understanding of formulation and guidance mechanisms in Diffusion models

I've been reading a research paper titled High-Resolution Image Synthesis with Latent Diffusion Models by Robin Rombach et al. and came across an a concept related to diffusion models (DMs). In the ...
hanugm's user avatar
  • 3,890
1 vote
0 answers
38 views

Can a Fully Connected Neural Network represent all Neural Networks of smaller size?

A fully connected Neural Network architecture can be characterized by a vector $\mathbf a = (a_0,a_1,\ldots,a_L)\in\mathbb N^{L+1}$ and an activation function $\sigma :\mathbb R\to\mathbb R$. In this ...
Stratos supports the strike's user avatar
1 vote
0 answers
35 views

Meaning of a symbol in trace

I am reading this paper Not All Samples Are Created Equal: Deep Learning with Importance Sampling. In the paper there is a deviation shown below. I can understand everything except the $V$ symbol in ...
onexpeters's user avatar
1 vote
1 answer
590 views

In the original diffusion model paper, why do they sample the first step with the same loss?

In the original diffusion model paper by Sohl-Dickstein et al., they explain very little about calculating the loss and training and network to learn the diffusion process. They did publish a ...
Grieverheart's user avatar
0 votes
0 answers
8 views

Filter distribution of Latent variable models

In this paper https://arxiv.org/pdf/1907.00953.pdf, about stochastic latent variable models, the paper says "We use the reparameterization trick to sample from the filtering distribution". I ...
chadmc's user avatar
  • 15
0 votes
2 answers
166 views

BYOL: Why is there a prediction network in the online network but not in the target network?

In the BYOL paper, the following architecture is presented: Why is a prediction network added to the online network, which is not present in the target network? How are the online and target compared ...
Robin van Hoorn's user avatar
0 votes
1 answer
65 views

What does crop size refer to in DeepLabv3 paper?

In the paper in which DeepLabv3 is presented, the authors are mentioning that: "For atrous convolution with large rates to be effective, large crop size is required; otherwise, the filter ...
Manuel Maior's user avatar
4 votes
1 answer
649 views

Difference between dot product attention and "matrix attention"

As far as I know, attention was first introduced in Learning To Align And Translate. There, the core mechanism which is able to disregard the sequence length, is a dynamically-built matrix, of shape ...
Gulzar's user avatar
  • 759
-1 votes
1 answer
186 views

Understanding self attention - How come there is no connection between different states?

During trying to understand transformers by reading Attention is all you need, I noticed the authors constantly refer to "self attention" without explaining it. The original attention ...
Gulzar's user avatar
  • 759
1 vote
1 answer
61 views

In the Dropout paper, why would increasing the dropout increase the error rate if the capacity is constant?

In the original paper on dropout, in section 7.3.2, we see that while keeping $pn$ constant, we get a (test) error increase by decreasing retainment below 0.6. Why would that happen? If $pn$ is ...
Apples14's user avatar
1 vote
0 answers
148 views

Has anyone here tried to implement MADDPG for a different environment and succeeded?

Has anyone tried implementing the multi-agent RL algorithm MADDPG (I've linked the paper below)? The paper seems to have a good amount citations, and they do have their code on github. However, a few ...
Confuse's user avatar
  • 111
5 votes
2 answers
1k views

InstructGPT: What is the sigma in the loss function and why $\log(\cdot)$ is being used?

InstructGPT: What is the sigma in the loss function and why $\log(\cdot)$ is being used? $$ \operatorname{loss}(\theta) = -\frac{1}{\binom{K}{2}}E_{(x,y_w,y_l)\sim D}[\log(\sigma(r_{\theta}(x, y_w) - ...
Nathan G's user avatar
  • 161
1 vote
1 answer
41 views

What does this bracket notation $\langle\phi(x),v\rangle$ mean?

I found it at the bottom of page 2 of the paper Intriguing properties of neural networks (2014), in the form of $$\underset{x\in\mathcal{I}}{\mathrm{arg\,max}}\langle\phi(x),v\rangle$$
Nils André's user avatar
1 vote
1 answer
827 views

Are there Explainable GNN methods for node regression tasks?

I am wondering if there are gnn explainable methods for a regression task (e.g., traffic forecasting) where nodes have numerical features and the predicted output is a numerical value. Most of ...
Achiles Br's user avatar
2 votes
2 answers
221 views

Where does the proximal policy optimization objective's ratio term come from?

I will use the notation used in the proximal policy optimization paper. What approximation is needed to arrive at the surrogate objective (equation (6) above) with the ratio $r_t(\theta)$? Put ...
fool's user avatar
  • 203
2 votes
1 answer
486 views

Why do activation functions in neural networks have to be non-polynomial to approximate any function?

Can someone give me the main idea of the paper Multilayer Feedforward Networks With a Nonpolynomial Activation Function Can Approximate Any Function? I'm having trouble understanding it.
Alicia Chi's user avatar
1 vote
0 answers
14 views

Where exactly is permutation happening in equation 5 of the paper "Learning with Sets in Multiple Instance Regression Applied to Remote Sensing"?

I am reading the article Learning with Sets in Multiple Instance Regression Applied to Remote Sensing about creating an embedding which is order-invariant to inputs ($m_{l}$). They referred to order-...
Oculu 's user avatar
  • 43
1 vote
1 answer
69 views

What is the meaning of $ (I - \gamma P^{\pi})^{-1} \left[\frac{\mu(a|s)}{\hat \pi_{\beta}(a|s)} \right](s, a)$?

In Theorem 3.1 of the conservative q-learning paper, what is the meaning of $$ (I - \gamma P^{\pi})^{-1} \left[\frac{\mu(a|s)}{\hat \pi_{\beta}(a|s)} \right](s, a)$$? I thought $(I - \gamma P^{\pi})^{-...
hongshan.li's user avatar
5 votes
0 answers
78 views

What exactly is non-delusional Q-learning?

Problems occur when we combine Q-learning with a function approximator. What exactly is the delusional-bias and non-delusional Q-learning? I am talking about the neurIPS 18 best paper Non-delusional Q-...
wrek's user avatar
  • 183
1 vote
0 answers
69 views

How does having zero advantage help with identifiability?

I am reading the D3QN paper and they have the following paragraph - Equation (7) is unidentifiable in the sense that given $Q$ we cannot recover $V$ and $A$ uniquely. To see this, add a constant to $...
desert_ranger's user avatar
1 vote
1 answer
1k views

Papers on Prompt Engineering

I am into AI in general and NLP in particular. Besides, I have a background in philosophy, and the new LLMs like GPT-3 seem to have exciting capabilities. I want to study prompt engineering (for ...
Mehdi Abbassi's user avatar
0 votes
1 answer
35 views

Is there any work done on topic agnostic binary topic classification?

In the recent preprint paper Tree-based Focused Web Crawling with Reinforcement Learning a new model is introduced to classify web pages called KwBiLSTM. The input to this model is a featurized ...
Kroshtan's user avatar
  • 259
2 votes
1 answer
228 views

Exact definition of WRN-d-k (Wide ResNet)

I am a little confused about the WRN-d-k notation from Wide Residual Networks. To quote the paper, In the rest of the paper we use the following notation: WRN-n-k denotes a residual network that has ...
nalzok's user avatar
  • 321
0 votes
1 answer
1k views

In-batch negative training Improves the results

I have read Dense passage retrieval for Open Domain Question Answering, and in page 6 it talks about in-batch negative training, it states the following: We find that using a similar configuration (7 ...
Kais Hasan's user avatar
0 votes
1 answer
59 views

What do we mean by the notation $\mathbf{x}_{p} \in \mathbb{R}^{N \times\left(P^{2} \cdot C\right)}$?

I was going through this VIT paper, what will it look like in torch , if we are trying to write this expression.
TheExorcist's user avatar
1 vote
0 answers
96 views

What are semantic word spaces in NLP?

In the abstract of this paper, it's written Semantic word spaces have been very useful but cannot express the meaning of longer phrases in a principled way. I would like to understand what semantic ...
Hermi's user avatar
  • 121
2 votes
1 answer
326 views

In the MuZero paper, how does backprop in the MCTS account for the immediate reward from each edge?

On page 12 of this paper: Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, it describes how MCTS works for the MuZero algorithm. It states in equation 4 that during the 'backup' ...
Matrix001's user avatar
  • 123
0 votes
1 answer
3k views

What is a "canonical space"?

I am reading the paper on 3D reconstruction, ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction, and I encountered the term "canonical space". What is a "...
Trong-Thang Pham's user avatar
0 votes
1 answer
258 views

Is the "Helvetica scenario" mentioned here related to Artificial Intelligence?

Consider the following sentence from the original GAN paper titled Generative Adversarial Nets in particular, $G$ must not be trained too much without updating $D$, in order to avoid "the ...
hanugm's user avatar
  • 3,890

1
2 3 4 5
7