How does the distribution of the parameters change in logistic regression?

I have my own data to train a logistic regression model (for a multi-class classification task), and I want to know how the distribution of weight parameters changes after each update with gradient descent.

For example, let's say that there are $$f$$ many features for each input, and the weight $$W$$, which is a $$c \times f$$ matrix, where $$c$$ is a number of classes, is initialized with uniform distribution $$U(-1/\sqrt{f}, 1/\sqrt{f})$$, which is LeCun uniform initialization.

For each step of gradient descent with Cross-Entropy loss, it will be updated as $$W_{t+1} = W_{t} - \alpha \frac{\partial \mathcal{L}}{\partial W_{t}}$$ where $$\alpha$$ is a learning rate and the gradient is given $$\frac{\partial \mathcal{L}}{\partial W_{t}} = \frac{1}{n} (\mathbf{y} - \mathbf{p})^{T}\mathbf{X}$$ where $$\mathbf{X} \in \mathbb{R}^{n\times f}$$ is an input matrix, $$\mathbf{y} \in \mathbb{R}^{n \times c}$$ is one-hot encoded labels, and $$\mathbf{p} = \mathrm{softmax}(\mathbf{X}W_{t}^{T}) \in \mathbb{R}^{n \times c}$$ is the model's output (predicted probability for each example & class).

What I want to know is how the distribution of $$W_{t}$$ changes as $$t$$ increases, if some information about $$\mathbf{X}$$ is known. More precisely,

1. Is it possible to get a bound of $$\mathbb{E}[||W_{t}||_{\infty}]$$ in terms of $$t, \alpha, ...$$? We may assume that the input $$\mathbf{X}$$ is also bounded (in the sense that $$||\mathbf{X}||_{\infty} \leq M$$ for known $$M$$.) I have a really rough bound for $$||W_{t}||_{\infty}$$, but it is not good enough.
2. Are there any works in this direction for other models, such as MLPs?

When I plotted the value $$||W_{t}||_{\infty}$$ for each $$t$$, then it seems that it increases sub-linearly in $$t$$, but