# Can reinforcement learning be used to learn an unknown analytical function (for example, $y = x^2$ )?

• Are there any examples for RL to learn analytical functions (for example, $$y=x^2$$)?
• What are the considerations when constructing the environment?
• Are there any literature that analyzes the difficulty/validity of the problem?
• Which algorithm is best for this type of problem? Why?
class Env(gym.Env):
def __init__(s, config = {}):
s.observation_space = spaces.Box(0, 1, [1, ])
s.action_space = spaces.Box(0, 1, [1, ])

def reset(s):
s.state = np.random.rand(1)
return s.state

def step(s, action):
reward = - abs(s.state ** 2 - action)[0]
return s.state, reward, True, {}

• trained with PPO for 100 iterations, result NOT good:
+---------------------+------------+-------+--------+------------------+--------+------------+----------------------+----------------------+--------------------+
| Trial name          | status     | loc   |   iter |   total time (s) |     ts |     reward |   episode_reward_max |   episode_reward_min |   episode_len_mean |
|---------------------+------------+-------+--------+------------------+--------+------------+----------------------+----------------------+--------------------|
| PPO_Env_e2752_00000 | TERMINATED |       |    100 |          815.105 | 400000 | -0.0105903 |         -1.54313e-08 |            -0.262791 |                  1 |
+---------------------+------------+-------+--------+------------------+--------+------------+----------------------+----------------------+--------------------+


• What is your purpose for doing this? RL is a bad choice for learning functions in the abstract, you may as well collect the same number of observations for supervised learning if the next state is not under the agent's control. Aug 3 '21 at 8:03
• my purpose is to understand RL better. The direct motivation is: suppose I provide observation $x$ to the agent, from educated interpretation, it would be helpful to provide the agent with $x^2$, I wonder whether I need to provide $x^2$ or can the agent figure out itself
– Rick
Aug 3 '21 at 10:26
• for example, the direct observation is an array arr with shape [n, ], suppose I design the environment such that the best action is argmax(arr**2), can the agent figure out itself or do I calculate arr**2 for the agent as observation?
– Rick
Aug 3 '21 at 10:29
• To understand RL better, you should use one of the standard RL toy environments. Your environment design is missing key things that RL is made to solve, such as time steps and state progression. Try CartPole, Taxi, FrozenLake or similar simple environments in OpenAI gym Aug 3 '21 at 10:39
• To partially answer your title, then yes RL can be used to learn unknown analytical functions, provided an agent is allowed to operate the function as a "black box" and observe results. However, this is because RL is a very general learning algorithm that can be made to fit many scenarios. It is not the best choice, and not even a very good choice for the problem as you have framed it. You will not learn much about the normal use of RL by solving this problem with RL methods. Aug 3 '21 at 10:46