I have a data problem with no direct reward mechanism,(test/train) good and fault solutions.
Though over a long time period good decisions might be made.

I've been searching for days now for an agent solver, and some example code. As most neural networks go from input to output on static data sets (train/test). Essentially one problem, with known results or classifications.

Since there is no direct good/bad reward. I'm looking for some code example perhaps using multiple evolving agents over time to compete against eachother over previous runs, with a reward that's based on multiple cycli.. Though I'm not after numeric pattern estimators (ea not RNN or LSTM's) ... what else to use?

as I don't have direct rewards straight backpropagation is not something I can use.

Though there do exist more ways to solve things.
For example:

  • simple Fuzzy logic >>though easy thermostat alike rules don't exist for the data.
  • Q-Learning a form logic that dynamically builds a logic table.
    It's mostly used in games, where there is a kinda binary logic (wall, enemy, ..)
    And the output are a few choices.

    however my data isn't in a common game environment (wall, enemy, .. ).

  • Genetic-based neural networks, start with randomized weights.
    Train and mix and add noise, and move weights towards the best result for each generation.

    comes close though I don't have a good/fault set, it's still a hard problem.

  • Agent-based Genetic neural network, I've seen some evo sims around the idea

    The evo sims still have quick rewards, though they come close to it. But my rewards happen randomly and rarely, not at a fixed interval. But maybe i'm wrong and i've seen the wrong samples of it.
    A good python example of this would still be great.

So far I'm a bit unlucky and I wonder, maybe a differeent NN design like :
another type of network exists, one that's a like a normal DNN, but can have X outputs re-used as inputs besides its data input. This sounds a lot alike LSTM but i'm thinking of a small neural net in which 1 output is for action, the other outputs get a delayed or averaged backpropagation after x cycles, also those redirection to input might have some delay on their feedback loop. (a genetic net still can solve such circulair networks). (Oh and its not a straight LSTM i'm after here).

How are such networks called?
Is it possible in pytorch / kerras ?.
python examples ??

  • 1
    $\begingroup$ I was reminded of the neural cellular automata a bit here (might be way off of your problem), but you don't provide enough information (at least for me) to get ideas. Can you provide any kind of objective, what is the learning task here? You say you neither have a loss nor a reward. Essentially, what do you expect the model to do? $\endgroup$
    – Chillston
    Commented Nov 11, 2022 at 13:47
  • $\begingroup$ I do have rewards but they happen later, its about personal medical date. $\endgroup$
    – Peter
    Commented Nov 12, 2022 at 14:39
  • $\begingroup$ not date ...data. $\endgroup$
    – Peter
    Commented Nov 12, 2022 at 14:47
  • 1
    $\begingroup$ Sorry, I still don't understand what you are after. So far what you have given us is that you are looking for a specific ANN architecture and that you found some already, but the important part is missing - what do you want the network to do? If you don't want to disclose it all, could you abstractly describe what the input and the output of the ANN are going to be? $\endgroup$
    – Chillston
    Commented Nov 14, 2022 at 9:16
  • $\begingroup$ Well, there needs to be found a medication 'moments' (over a longer period) so that I stay within the limits, while also using minimal and best guess, the medication has directly an impact on the next measurements since they affect my body. inputs as heart rate, electric skin resistance, blood oxygen level and a few more. The output would be 'now is the best moment to take medicine, but it might also not advice to take medicine (its not life treathning medication) $\endgroup$
    – Peter
    Commented Nov 15, 2022 at 16:17


You must log in to answer this question.