0
$\begingroup$

I've read in this discussion that "reinforcement learning is a way of finding the value function of a Markov Decision Process".

I want to implement an RL model, whose state space and action space dimensions would increase, as the MDP progresses. But I don't know how to define it it terms of e.g. Q-learning or some similar method.

Precisely, I want to create a model, that would generate boolean circuits. At each step, it could perform four different actions:

  • apply $AND$ gate on two wires,
  • apply $OR$ gate on two wires,
  • apply $NOT$ gate on one wire,
  • add new wire.

Each of the first three actions could be performed on any currently available wires (targets). Also, the number of wires will change over time. It might increase if we perform fourth action, or decrese after e.g. application of an $AND$ gate (taking as input two wires and outputting just one).

$\endgroup$
8
  • 1
    $\begingroup$ It's not fully clear to me what you mean by "targets" here. Can you clarify that? $\endgroup$
    – nbro
    Commented Dec 12, 2020 at 17:23
  • $\begingroup$ Action $a_0$ might be for example an AND gate and targets would be specific circuits representing bits. So AND action would act on two targets - bits, while e.g. NOT action would act just on one target. $\endgroup$ Commented Dec 12, 2020 at 19:23
  • $\begingroup$ I still don't get what is a target in RL terms. What would a target correspond to in RL or MDPs? $\endgroup$
    – nbro
    Commented Dec 12, 2020 at 20:31
  • $\begingroup$ @nbro: My understanding is that the possible targets are part of the state, and both the state space and action space dimensions may increase as the MDP progresses. $\endgroup$ Commented Dec 12, 2020 at 20:50
  • $\begingroup$ @NeilSlater That's exactly what I wanted to say, but I couldn't formulate it as clearly as you did. I edited my question. $\endgroup$ Commented Dec 12, 2020 at 21:45

0

You must log in to answer this question.