As long as the first training is unsupervised and the second one supervised it does fall into the category of self-supervised learning.
I would say pre-training will use my training data to learn the dependencies between the features
This sentence is quite vague so let's clarify it a bit. Supervised and unsupervised losses are much different from each other statistically speaking. Supervised losses learn to predict conditional probabilities $p(y|x)$, i.e. the best predictions given an input. Unsupervised losses have no labels to compute such probabilities so they try to learn $p(x)$ directly, i.e. they try to learn the prior distribution of you data.
Of course between the two tasks the latter is much harder. Usually you would leverage labels to compute some distance or difference between predictions and labels but since you have no labels the only thing that can be done is to compute distances or differences between predictions and the same input that was fed to the model. So what an unsupervised model learn is not per se dependencies or correlations of features, is more generic than that, it's about learning to return similar predictions, usually embedding vectors, for similar inputs. Some approaches like contrastive learning try to go beyond that by imposing a margin between different predictions, so learning not only embedding vectors similar for similar inputs, but embedding vectors that are also dissimilar for dissimilar inputs.
This key aspect of unsupervised learning is the reason why it can be useful sometimes for pretraining supervised models. Supervised losses (almost all) are not designed to learn good embeddings. Problems like shift invariance and miss calibration are examples of consequences of that.
