I have a lengthy timeseries datasets which contains several variables (from sensors etc) to be classified as actions or states. Providing they are successfully done, I want to learn a control policy using DDPG. But I have no knowledge of the environment. How can I learn my policy off-line only by using these datasets without having any model of the environment? After learning off-line first, then the policy can then be used to learn and control online later in a certain real-world environment.
First, I know that experience buffer can be used to store the datasets. How should you set the buffer size in this case? From what I understand, DDPG needs lots of data to be used for learning. Should I build an environment model using the specified datasets? Or I don't really need this step?
All of these will be implemented in Python and maybe with the help of another tools if needed. There are some implementation of DDPG available so it is not the main problem, but this implementation must be tweaked to solve my proposed problem. Normally the implemented DDPG in Python requires a Gym-environment as an input so I must change it to satisfy my needs as I don't need Gym for my use case. And these implementations in Python are somehow on-line codes so you need to interact directly with the environment model for the algorithm to be working.
Can someone help me tackle this problem or give me some advice regarding this? I can help giving more details if needed. Thank you.