New answers tagged

0 votes

Positional Encoding of Time-Series features

I think your input should have shape [batch_size, time_points, n_features] which would correspond to ...
user avatar
0 votes
Accepted

When exactly does the split into different heads in Multi-Head-Attention occur?

The queries, keys and values are calculated then chunked so that each chunk depends on (is a linear combination of) all values of the input embedding. As for understanding an implementation, I didn't ...
user avatar
0 votes

How "Patch Merging" works in SWIN-Transformers?

What happens during the patch merging? Concatenationating is only one part of the whole operations. Below, I quote the code from the original implementation and explain step by step what happens. <...
user avatar

Top 50 recent answers are included