1
$\begingroup$

I'm trying to train a function for a industrial-process-control-like system. This is my first attempt at a custom training, so feel free to point out any invalid assumptions.

I've got one input and one controlled output, which I'm trying to optimise. I've reduced the problem with some values normalisation to:

  • the first half of input looks like a sin() raise from 0 to max value then dropoff - with lots of noise on top (let's say up to +/-10% at each measurement)

  • I don't know what the max is, but it's roughly predictable (input goes from 0 to between 0.5 and 2)

  • the output cannot go down (well, it can by a tiny bit, but I'd ignore that here), and cannot go higher than the input value

  • the goal is to get the output value as high as possible

Currently the best non-NN approach I've got is to start a few % below input and at each step run output = A*input + (1-A)*previous_output, so the result looks like this (input in blue, output in orange)

tracking_input

I wanted to check if some RNN can improve on this, so I'm planning to check an LSTM doing this instead. I'm struggling to come up with a loss calculation which is viable for training here.

I considered making the input and output for the network an absolute change from the previous value (or input as input-last_output difference), then as the loss using some kind of inverted value of sum(output changes) from step 0 to the crossover point period to reward higher maximums while ignoring the distance. (so discarding anything that happens past the crossover)

But... that doesn't relate to the input really, so tensorflow wouldn't be able to train based on this, if I understand it correctly. Am I going in the wrong direction? Are there some known ways to solve this problem?

$\endgroup$

1 Answer 1

0
$\begingroup$

This turns out to be less about the loss function and more about the approach. There's a number of them implemented in the tf_agents package.

Choosing the DDPG agent for outputting the increments and implementing a PyEnvironment to run the simulation and return the highest reached output as a reward seems to work just fine. Very similar to the blackjack example.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .