1
$\begingroup$

I am new to RL and wish to realize a RL control for an industrial process. The goal is to control the temperature and humidity in a vegetal food production chamber.

States: External temperature and humidity, internal temperature and humidity, percentage of the proportional valves controlling heater, cooler and steam for humidity. The goal is to keep temperature and humidity in the chamber (measures) as close as possible to the desired values (the setpoints).

Agent actions: Increase/decrease the percentage of the proportional valves controlling the actuators.

Rewards: Deviation between measure and setpoints (small deviation => high reward, high deviation => low reward).

I have data available, the history of states and actions from a real system. The actions are made by several PID controllers (some of them in cascade). So far I have about 3 month every minutes (with some stops sometimes when a chamber is for example cleaned). The data are continuously logged and every month I get more data. The data includes bad/unwanted states.

For training the RL agent, I am planning to simulate the environment using a supervised learning model (with the predict function), probably XGboost. Is it feasible, are there pitfalls to avoid in this case?

Improve this question
$\endgroup$
13
  • $\begingroup$ Are you asking if it's feasible to simulate the environment using SL? Anyway, it's not clear (to me) how you want to simulate the environment given only a labelled training set. $\endgroup$
    – nbro
    Commented Apr 19, 2020 at 23:38
  • 1
    $\begingroup$ @nbro: OP could train a predictive model for $p(s'|s,a)$ - or maybe $p(s', r|s,a)$, depending on whether reward is imposed by starting goals, or is some measure of output of the industrial process. The simplest environment "simulation" is pretty much experience replay, you would use SL instead if you hoped to generalise from the data set, separately from simply training e.g. Q-learning. Choices such as trying to create a sampling model vs distribution model, and limits of approximate/learned envirobment models might be useful parts of any answer. Also, whether there is any benefit to the idea $\endgroup$
    – Neil Slater
    Commented Apr 20, 2020 at 10:41
  • $\begingroup$ @NeilSlater Ha, right. Good points! But I think the OP should be more specific and ask a clearer and more specific answer. $\endgroup$
    – nbro
    Commented Apr 20, 2020 at 11:23
  • $\begingroup$ @nbro Thanks for your answer. Yes the question is is it feasible. $\endgroup$
    – brz
    Commented Apr 20, 2020 at 14:26
  • $\begingroup$ @brz OP refers to the asker of the question. It's an acronym for "Original Poster" and in this case refers to you, so it's nothing to do with RL. Anyway, I think you should clarify how your dataset looks like and try to be more specific regarding what you want to be feasible. $\endgroup$
    – nbro
    Commented Apr 20, 2020 at 14:50

1 Answer 1

Reset to default
0
$\begingroup$

RL is a generic technique that can be applied to any MDP system. From the looks of it you have data to produce a state-space model of your system (system identification, excluding your existing control loops) and then you can use that to drive a simulated exploration of your process and discover a control policy. As this is a continuous process, some discretization will be required

so yes, your quest is feasible, but not trivial!

I experimented sometime ago with the cartpole balancing system, which is a toy, but I found that a good old PID was much better than AI :)

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .