You can check the Open AI Introduction to RL series, they explain pretty neatly there what is the Policy Optimization and how to derive it. I think, that usually when we are talking about REINFORCE algorithm, we are talking about the one described in Sutton's book on Reinforcement learning. It is described as the policy optimization algorithm maximizing the Value Function $v_{\pi(\theta)}(s) = E[G_t|S_t = s]$ of initial state of the agent. Here $G_t = \sum_{k=0}^\infty \gamma^k R_{t+k+1}$ is the $\gamma$ discounted return from given state, time $s, t$. Or, shortly put.
$$
J(\theta) = v_{\pi(\theta)}(s_0) = E[G_t|S_t = s_0]\\
\nabla J(\theta) = E_\pi\left[G_t\frac{\nabla \pi (A_t |S_t, \theta)}{\pi (A_t |S_t, \theta)}\right]
$$
But in the RL series of Open AI, the algorithm that is described as Vanilla policy gradient (If it is the one you are talking about) is optimizing finite-horizon undiscounted return $E_{\tau \sim \pi} [R(\tau)] $, where $\tau$ are possible trajectories. e.g.
$$
J(\theta) = E_{\tau \sim \pi} [R(\tau)] \\
\nabla J(\theta) = E_{\tau \sim\pi}\left[\sum_{t=0}^T R(\tau) \frac{\nabla \pi (A_t |S_t, \theta)}{\pi (A_t |S_t, \theta)}\right]
$$
They do look very similar in objective functions, but they are different. The way the gradient ascent is performed differs strongly since in REINFORCE method the gradient ascent is performed once for each action taken for each episode and the direction of ascent is taken as
$$
G_t\frac{\nabla \pi (A_t |S_t, \theta)}{\pi (A_t |S_t, \theta)}
$$
so the update becomes
$$
\theta_{t+1} = \theta_{t} + \alpha G_t\frac{\nabla \pi (A_t |S_t, \theta)}{\pi (A_t |S_t, \theta)}
$$
but in VPG algorithm the gradient ascents performed once over multiple episodes and direction of ascent taken as average
$$
\frac{1}{|\mathcal{T}|}\sum_{\tau\in\mathcal{T}} \sum_{t=0}^T R(\tau) \frac{\nabla \pi (A_t |S_t, \theta)}{\pi (A_t |S_t, \theta)}
$$
and gradient ascent step is
$$
\theta_{t+1} = \theta_{t} + \alpha \frac{1}{|\mathcal{T}|}\sum_{\tau\in\mathcal{T}} \sum_{t=0}^T R(\tau) \frac{\nabla \pi (A_t |S_t, \theta)}{\pi (A_t |S_t, \theta)}
$$
which looks a lot like what you have stated as REINFORCE algorithm.
I admit, that some form of mathematical equivalence can be derived between them, since expectation over policy and expectation over the trajectory sampled from the policy looks practically the same. But approaches differ at least in the way, the ascent is computed.
