Should I build an environment from scratch myself or it is not always needed?

Question

I am inspired by the paper Neural Architecture Search with Reinforcement Learning to use reinforcement learning for optimizing a child network (learner). My meta-learner (controller or parent network) is an MLP and will take as the reward function a silhouette score. Its output is a vector of real numbers between 0 and 1. These values are k different possibilities for the number of clusters (the goal is to cluster the result of the child network which is an auto-encoder, embedded images are the input to the meta-learner).

What I am confused about is the environment here and how to implement this network. I was reading this tutorial and the author has used gym library to set the environment.

Should I build an environment from scratch myself or it is not always needed?

I appreciate any help or hints or links to a source that helps me understand better RL concepts. I am new to it and easily gets confused.

Answer 1

I guess it would always be better if you can reuse existing environments to make it work for yourself. Since most of the environment codes is anyway opensourced, you can always edit it to your liking.

If you want a custom environment, you can add an environment to gym like this.

Answer 2

Gym environments were designed first of all as a benchmark for RL algorithms. Therefore, you should ask yourself: Do I want to benchmark my RL algorithm or do I want to solve my problem with RL?

If the answer is "I want to benchmark my algorithm", you should probably go with existing environments. Using existing environments as benchmarks you will be able to compare your algorithm with other existing approaches.

If the answer is "I want to solve my problem", then yes, perhaps the best way to continue is to build your custom environment.

Finally, if you want to understand RL better, I would suggest to start with existing environments to gain a better grasp of RL concepts.