From your description, it seems that you are implementing a version of an algorithm called REINFORCE. This algorithm belongs to a family called Policy Gradient methods, which directly optimizes the policy network $\pi(a_t|s_t)$ from rewards without ever worrying about estimating a value function. This type of algorithm is usually pretty slow and presents high variance.
The methods that you recognize as the ones using two neural networks correspond to a family called Actor-Critic methods. This type of algorithm uses the trajectories of rewards to estimate a value network $q(s_t,a_t)$ (called the critic), and, contrary to the previous family of methods, it uses the value network to train the policy network $\pi(a_t|s_t)$ (called the actor), instead of directly using the trajectory of rewards. This indirect dependence usually makes variance smaller and also learning faster. I recommend you have a look at chapter 13 of the book An Introduction to Reinforcement Learning.
So, to answer your first question: it seems you are missing the family of Actor-Critic methods. I recommend you learn about them since they are very powerful (e.g., read about DDPG or SAC).
About your second question, the standard method to "reward" a policy network is not by training it. Usually, you have a reward function $r(s_t,a_t)$ that depends on your current state $s_t$ and action $a_t$ and you modify the parameters $\theta$ of your network in such a way that the probability of an action $\pi(a_t|s_t)$ increases if the reward is positive or decreases if it is negative. More specifically, you perform stochastic gradient ascend steps like this one:
$$\theta_t\leftarrow\theta_t+\alpha\mathbb E\left[\sum_{k=t}^{T+t} \gamma^{k-t}r(s_k,a_k)\right]\nabla \log\pi(a_t|s_t,\theta_t)$$
What this formula says is that if in the time-step $t$ you take an action $a_t$ in the state $s_t$, wait $T$ steps and collect the rewards from $r(s_t,a_t)$ to $r(s_{t+T},a_{t+T})$, then, you should modify your parameters in the direction that the policy increases the most (i.e., $\nabla \log\pi(a_t|s_t,\theta_t)$) if the expected return $E\left[\sum_{k=t}^{T+t} \gamma^{k-t}r(s_k,a_k)\right]$ is positive, or in the direction that the policy decreases the most if that expected return is negative.