New answers tagged transformer
0
votes
Positional Encoding of Time-Series features
I think your input should have shape [batch_size, time_points, n_features] which would correspond to ...
- 168
0
votes
Accepted
When exactly does the split into different heads in Multi-Head-Attention occur?
The queries, keys and values are calculated then chunked so that each chunk depends on (is a linear combination of) all values of the input embedding.
As for understanding an implementation, I didn't ...
- 168
0
votes
How "Patch Merging" works in SWIN-Transformers?
What happens during the patch merging? Concatenationating is only one part of the whole operations. Below, I quote the code from the original implementation and explain step by step what happens.
<...
Top 50 recent answers are included