0
$\begingroup$

This is a question our Prof gave us as exam preparation, but I don't know why the Autoencoder should contain more info than the Teacher Student model.

Teacher Student Models are a class of models in Self-supervised Learning and Knowledge Distillation. Popular examples include DINO, BYOL, BoWNeT etc.

Teacher Student Example from Emerging Properties in Self-Supervised Vision Transformers (DINO)

Example from Emerging Properties in Self-Supervised Vision Transformers (DINO)

My intuition is that it's due to the fact that Autoencoder need to reconstruct the original image but I'm very uncertain about this.

$\endgroup$
0

0

You must log in to answer this question.

Browse other questions tagged .