Skip to main content

Questions tagged [vision-transformer]

The tag has no usage guidance.

12 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2 votes
0 answers
63 views

Is it possible for original Vision Transformer (ViT) to do fine-grained semanantic segmentation? if so, how?

As far as I know, in the original ViT, the image is first divided to a fixed size of patch (16x16, for example) then they are flattened and treated as tokens and fed into Transformer. Without using ...
wanburana's user avatar
2 votes
0 answers
55 views

What are the specific differences between vision transformers variants?

I have tried 4 different types of attacks on vision transformers (ViT small and tiny, DeiT small and tiny) but the attack successes on smaller versions are higher than the tiny versions. My ...
Craving_gold's user avatar
1 vote
0 answers
20 views

Can a transformer detect same object with different sizes?

Suppose a vision transformer has trained to detect this cat picture Next we show it another picture of a zoomed-in cat (taken from the same image) and asked it to identify the picture The linearized ...
James's user avatar
  • 157
1 vote
0 answers
284 views

Glass Degradation Video Prediction

We are working on the subject above, where a sequence of $n$ glass frames forms an example with an associate target that is a video of the glass future state. We would like to know if there exists an ...
Filippo Portera's user avatar
0 votes
0 answers
29 views

can Vision transformers be used to retain the relevant features (drop unrelated features from the clutter in image) and map to the specific query

Background, I have good understanding of ML 101 (supervised, unsupervised, tensorflow etc), however just getting into transformers & gen-AI. I have recently started looking into Transformers/ViT ...
cyborgt8's user avatar
0 votes
0 answers
24 views

Can the vision transformer solve object permanence?

Suppose a ball is rolling down an incline as follows: In the middle of the path, a curtain blocks the camera view of the ball momentarily. Soon the ball reappears after passing behind the curtain. ...
James's user avatar
  • 157
0 votes
0 answers
33 views

Where U-Net and Convolutional layers are settled in Stable Diffusion model?

When I read about Stable Diffusion model, they usually talk about adjusting convolution layers or U-Net weights. I believe they both should be related together and the U-Net is the part that accepts ...
best_of_man's user avatar
0 votes
0 answers
18 views

How to train ViT on smaller datasets?

I know ViTs aren't made for small datasets and low resolution. But have you ever reached traditional CNN accuracy using ViT on CIFAR10/100. I have been playing around with ViT on CIFAR10 and 100. But ...
v1998199904's user avatar
0 votes
0 answers
85 views

ViT fails to detect a white pixel in a black image

I wanted to report you to some experiments in the context of Deep Learning for Computer Vision, in particular for visual reasoning. The main question I am trying to answer is the difference between ...
Fiorenzo Parascandolo's user avatar
0 votes
0 answers
25 views

What algorithms could I use if I want to increase the accuracy of matched keypoints in an image pair?

Let's say that I used a keypoint detector like SIFT or SuperPoint to detect keypoints in image 1 and 2. Afterwards, I used a keypoint matcher to match corresponding keypoints in this image pair. The ...
Surzilla's user avatar
  • 101
0 votes
0 answers
127 views

How do i approach creating a masked auto-encoder for feature extraction

I trained Masked Autoencoder-based models in order to use the encoder as a backbone for another task. Pretraining has been done in a Self-Supervised manner on image data. Now that it comes to my ...
Mitch's user avatar
  • 1
0 votes
0 answers
181 views

How does Position embeddings work in Vision Transformer

I'm a bit confused how the position embedding in happened to each patch in the transformer. I thought Ideally we'd want each patch to have a value of (1, 2, 3, 4....) to describe the position of the ...
a__ys's user avatar
  • 3