Skip to main content
1 vote

Will an AI LLM learn a language if fed during training with a large corpus of undeciphered language?

Theoretically, if you had a very large corpus of only Minoan, you could train an LLM on it. The LLM would be able to generate Minoan texts, either from scratch, or as completions of supplied starting ...
Neil Slater's user avatar
  • 32.7k
1 vote
Accepted

Can transformer attention make predictions based on analogy?

To complete the pattern based on the given examples, the last statement should follow the same structure. Each example follows a clear pattern where the second person in the first part of the sentence ...
Kulin Patel's user avatar
1 vote

Spatial vs spatiotemporal methods for object counting in low frame-rate videos

It depends. The more information you input for a sample allows the model to better understand it. However, due to differences on data quality, dataset size, model size, etc., no one can guarantee your ...
t4rf9's user avatar
  • 11
1 vote
Accepted

Why are the Q and K matrices two separate matrices in attention?

Your calculations are correct, though training a whole $W$ is not always equivalent. We can agree that $W_q, W_k \in R^{n\times m}$, thus $W_{qk} = W_qW_k^T \in R^{n\times n}$ Now, check the parameter ...
Alberto's user avatar
  • 2,293

Only top scored, non community-wiki answers of a minimum length are eligible