Adam is known as an algorithm that has an adaptive learning rate for each parameter. I believe this is due to the division by the term $$v_t = \beta_2 \cdot v_{t-1} + (1-\beta_2) \cdot g_t^2 $$ Hence, each weight will get updated differently based on the accumulated squared gradients in their respective dimensions, even though $\alpha$ might be constant. There are other StackOverflow posts that have said that Adam has a built-in learning rate decay. In the original paper also, the authors of adam paper says that the learning rate at time step $t$ decays based on the equation $$\alpha_t = \alpha \cdot \frac{\sqrt{1-\beta_2^t}}{{1-\beta_1^t}}$$
Is the second equation the learning rate decay that has been built into the Adam algorithm?
