New answers tagged transformer
0
votes
Positional Encoding of Time-Series features
I think your input should have shape [batch_size, time_points, n_features] which would correspond to ...
0
votes
Accepted
When exactly does the split into different heads in Multi-Head-Attention occur?
The queries, keys and values are calculated then chunked so that each chunk depends on (is a linear combination of) all values of the input embedding.
As for understanding an implementation, I didn't ...
0
votes
How "Patch Merging" works in SWIN-Transformers?
What happens during the patch merging? Concatenationating is only one part of the whole operations. Below, I quote the code from the original implementation and explain step by step what happens.
<...
Top 50 recent answers are included
Related Tags
transformer × 157natural-language-processing × 57
attention × 56
deep-learning × 39
neural-networks × 26
bert × 18
recurrent-neural-networks × 14
papers × 12
pytorch × 12
positional-encoding × 12
machine-learning × 11
gpt × 11
sequence-modelling × 10
machine-translation × 9
computer-vision × 7
long-short-term-memory × 7
vision-transformer × 7
convolutional-neural-networks × 6
comparison × 6
python × 6
time-series × 6
reference-request × 5
terminology × 5
tensorflow × 5
keras × 5