I am inspired by the paper Neural Architecture Search with Reinforcement Learning to use reinforcement learning for optimizing a child network (learner). My meta-learner (controller or parent network) is an MLP and will take as the reward function a silhouette score. Its output is a vector of real numbers between 0 and 1. These values are k different possibilities for the number of clusters (the goal is to cluster the result of the child network which is an auto-encoder, embedded images are the input to the meta-learner).

What I am confused about is the environment here and how to implement this network. I was reading this tutorial and the author has used gym library to set the environment.

Should I build an environment from scratch myself or it is not always needed?

I appreciate any help or hints or links to a source that helps me understand better RL concepts. I am new to it and easily gets confused.