# Tag Info

### In-depth understanding of formulation and guidance mechanisms in Diffusion models

You can think as if the network learns the gradient of the data distribution... For example, think about having some points in 1D which are distributed as 2 Gaussians: Learning the gradient means ...
• 2,293

### Exact definition of WRN-d-k (Wide ResNet)

I also found the WRN-$n$-$k$ notation confusing, but I think I can explain it: $n$ is the total number of convolutions in the model. So to understand the architecture associated with each $n$, we ...
• 121

### How would I approach finding the source code for the paper that is being discussed in this article?

There is one good place to look if the paper is in arXiv, and the one you referenced is here. After the abstract there are several tabs, one of which is labelled "Code, Data, Media", which ...
• 32.7k
1 vote

### What is the difference between RAG-Sequence Model and RAG-Token Model?

In the first equation, you can see that in the sequence likelihood (the big product) $z$ is constant, which means that you first find which are the interesting documents, than you give your LLM these ...
• 2,293
1 vote

Overall your understanding of dimensions here is correct. The function $a_k(\mathbf{x})$ measures the exponential of the negative squared Euclidean distance between the input state $x$ and the center $... • 2,277 1 vote ### What does the notation$\hat{A}_t\left(s_{0: \infty}, a_{0: \infty}\right)appearing in Generalized Advantage Estimation mean? In policy gradient methods reducing bias is crucial to obtain more accurate gradient estimates for updating the policy in a stochastic gradient ascent fashion. The usual advantage function as you ... • 2,277 1 vote ### What does "aligned" across domains in domain adaptation? To provide some context for this answer, the referenced paper is dealing with the problem of unsupervised domain adaptation (UDA). In UDA, there are two (or more) datasets, each drawn from a different,... • 141 1 vote Accepted ### Overcoming the quadratic scaling in transformer architecture Yes. What you're describing seems to be similar to the Hourglass Transformer, the Funnel Transformer, among others. Looking at the first paper, they consider two ways of downsampling tokens (... • 1,289 1 vote Accepted ### How do Multimodal LLMs of 2023 score on the ARC benchmark (in 2020: 20% Accuracy) I have gathered a few notes about ARC, with links to projects that use LLMs. Too many links to post them all here. As of 7-may-2024, Jack Cole+Mohamed Osman solves 34 of the 100 hidden tasks. This is ... • 126 1 vote ### Understanding the functionality of the switch in the latent diffusion models: Does conditioning information pass to both cross attention andz_{T}? Section 4.3.2 of the paper (on p. 7 in v2) answers this question: By concatenating spatially aligned conditioning information to the input of\epsilon_θ\$, LDMs can serve as efficient general purpose ...
• 171

Only top scored, non community-wiki answers of a minimum length are eligible