Skip to main content
2 votes

In-depth understanding of formulation and guidance mechanisms in Diffusion models

You can think as if the network learns the gradient of the data distribution... For example, think about having some points in 1D which are distributed as 2 Gaussians: Learning the gradient means ...
Alberto's user avatar
  • 2,293
2 votes

Exact definition of WRN-d-k (Wide ResNet)

I also found the WRN-$n$-$k$ notation confusing, but I think I can explain it: $n$ is the total number of convolutions in the model. So to understand the architecture associated with each $n$, we ...
Kale Kundert's user avatar
2 votes

How would I approach finding the source code for the paper that is being discussed in this article?

There is one good place to look if the paper is in arXiv, and the one you referenced is here. After the abstract there are several tabs, one of which is labelled "Code, Data, Media", which ...
Neil Slater's user avatar
  • 32.7k
1 vote

What is the difference between RAG-Sequence Model and RAG-Token Model?

In the first equation, you can see that in the sequence likelihood (the big product) $z$ is constant, which means that you first find which are the interesting documents, than you give your LLM these ...
Alberto's user avatar
  • 2,293
1 vote

Notation used in paper on Continuous Time Reinforcement Learning

Overall your understanding of dimensions here is correct. The function $a_k(\mathbf{x})$ measures the exponential of the negative squared Euclidean distance between the input state $x$ and the center $...
cinch's user avatar
  • 2,277
1 vote

What does the notation $\hat{A}_t\left(s_{0: \infty}, a_{0: \infty}\right)$ appearing in Generalized Advantage Estimation mean?

In policy gradient methods reducing bias is crucial to obtain more accurate gradient estimates for updating the policy in a stochastic gradient ascent fashion. The usual advantage function as you ...
cinch's user avatar
  • 2,277
1 vote

What does "aligned" across domains in domain adaptation?

To provide some context for this answer, the referenced paper is dealing with the problem of unsupervised domain adaptation (UDA). In UDA, there are two (or more) datasets, each drawn from a different,...
Lynn's user avatar
  • 141
1 vote
Accepted

Overcoming the quadratic scaling in transformer architecture

Yes. What you're describing seems to be similar to the Hourglass Transformer, the Funnel Transformer, among others. Looking at the first paper, they consider two ways of downsampling tokens (...
Alexander Wan's user avatar
1 vote
Accepted

How do Multimodal LLMs of 2023 score on the ARC benchmark (in 2020: 20% Accuracy)

I have gathered a few notes about ARC, with links to projects that use LLMs. Too many links to post them all here. As of 7-may-2024, Jack Cole+Mohamed Osman solves 34 of the 100 hidden tasks. This is ...
neoneye's user avatar
  • 126
1 vote

Understanding the functionality of the switch in the latent diffusion models: Does conditioning information pass to both cross attention and $z_{T}$?

Section 4.3.2 of the paper (on p. 7 in v2) answers this question: By concatenating spatially aligned conditioning information to the input of $\epsilon_θ$, LDMs can serve as efficient general purpose ...
Josiah Yoder's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible