pi-tau's user avatar
pi-tau's user avatar
pi-tau's user avatar
pi-tau
  • Member for 1 year
  • Last seen this week
6 votes
Accepted

Which situation will helpful using encoder or decoder or both in transformer model?

6 votes
Accepted

Why use a "square root" in the scaled dot product

5 votes

Can someone help me understand the intuition behind the query, key and value matrices in the transformer architecture?

4 votes
Accepted

Why in Multi-Head Attention implementation should we use $3$ linear layers for Q, K, V instead of $3 * h$ layers?

4 votes

Is there a proper initialization technique for the weight matrices in multi-head attention?

4 votes

Why does this multiplication of $Q$ and $K$ have a variance of $d_k$, in scaled dot product attention?

3 votes

One to one relation between state + action -> reward

2 votes
Accepted

Understanding KL Stopping and KL Cutoff for the PPO algorithm

2 votes
Accepted

How do I code so that the embedding output and input share the same weight matrices?

2 votes

Filling replay buffer with expert trajectories for PPO/DQN

2 votes

How to expand reconstruction error to mean squared error in Variational AutoEncoder?

2 votes

If a residual block learns to approximate the identity function, should we reinitialize the weights?

1 vote

Why can decoder-only transformers be so good at machine translation?

1 vote

Why are some Neural Networks more forgiving on Quantization?

1 vote

How is the padding mask incorporated in the attention formula?

1 vote

Layer Norm in a ResNet MLP

1 vote

Confusion About Triangle Mask in Transformer Decoder

1 vote

Residual Blocks - why do they work?

1 vote
Accepted

2D convolution with channels versus 3D convolution for layers of a map?

1 vote

In vanilla policy gradient is the baseline lagging behind the policy?

1 vote
Accepted

What does the branching factor mean in the time complexity of Breadth-First Search (BFS)

1 vote

When training a CNN, what are the hyperparameters to tune first?

1 vote

Is there a notion of location in Transformer architecture in subsequent self-attention layers?

1 vote

diagonal of the hypercube in cross-attention

1 vote

What if we drop the causal mask in auto-regressive Transformer?

1 vote

What is the difference between densenet and resnet?

1 vote

Why do Transformer decoders use masked self attention when producing new tokens?

1 vote

Practical differences between the different RL architectures

0 votes

How does the agent derive the state from the observations in Reinforcement Learning?

0 votes

What is the meaning of log p(x) in VAE math and why is it constant