# Tag Info

Accepted

### Going beyond intent classification

Found it! After a lot of searching and reading through different papers, this was the exact one I needed. "BERT for Joint Intent Classification and Slot Filling" There's also an example ...
• 101

### How does transformer models like GPT generate valid meaningful response for meaningless garbage input?

If you give a human some input that doesn't seem to convey any meaning they will probably ask you for clarification. Presumably there are a lot of examples of this in the ChatGPT training data so that ...
• 171

### Where to find the source code for the research paper "Attention is all you need"?

You may find the code related to the paper at https://paperswithcode.com/paper/attention-is-all-you-need.

### What is the weight matrix in self-attention?

In my mind there are two weight matrices, the one you get prior to applying softmax: $$\alpha_{i,j} = \frac{\langle q_i, k_j \rangle}{\sqrt{d}}$$ the other you get after applying the softmax:  \...

### Is the multi-head attention in the transformer a weighted adjacency matrix?

Short answer, yes I believe we can! Now, here's the long answer. Suppose $X \in \mathbb{R}^{d \times n}$ has as columns $X_i$ the $d$-dimensional embeddings of the $n$-tokens $x_1, x_2, ..., x_n$ from ...

### Why don't people use nonlinear activation functions after projecting the query key value in attention?

It seems like doing this would lead to much-needed nonlinearity, otherwise, we're just doing linear transformations. Attention is broadly defined as a following operation ($\text{softmax}$ is ...
• 2,248