If you want, you can do training and testing in RL. Exactly the same usage, training for building up a policy, and testing for evaluation.
In supervised learning, if you use test data in training, it is like cheating. You cannot trust the evaluation. That's why we separate train and test data.
The Objective of RL is a little different. RL trying to find the optimal policy. Since RL collects the information by doing, while the agent explores the environment (for more information), there might be lost in the objective function. But, it might be inevitable for a better future gain.
Multi-arm bandit example, If there are 10 slot machines. They will return random amounts of money. They have different expected returns. I want to find the best way to maximize my gain. easy, I have to find the machine with the greatest expected return and use only the machine. How to find the best machine?
If we have a training and testing (periods), For example, I will give you an hour of the training period, so it doesn't matter if you lose or how much you earn. And in the testing period, I will evaluate your performance.
What would you do? In the training period, you will try as much as possible, without considering the performance/gain. And in the testing period, you will use only the best machine you found.
This is not a typical RL situation. RL is trying to find the best way, Learning by doing. All the results while doing are considered.
suppose... I tried all 10 machines once each. And, the No.3 machine gave me the most money. But I am not sure that it is the best machine, because all the machines provide a RANDOM amount. If I keep using the No.3 machine, it might be a good idea, because according to the information so far, it is the best machine. However, You might miss the better machine if you don't try other machines due to randomness. But if you try other machines, you might lose an opportunity to earn more money. What should I do? This is a well-known Exploration and Exploitation trade-off in RL.
RL trying to maximize the gain including the gains right now and the gains in the future. In other words, the performance during training also considered as its performance. That's why RL is not unsupervised nor supervised learning.
However, in some situations, you might want to separate training and testing. RL is designed for an agent who interacts with the environment. However, in some cases, (for example), rather than having an interactive playground, you have data of interactions. The formulation would be a little different in this case.