1
$\begingroup$

I am playing around with a stock trading agent trained via (deep) reinforcement learning, including memory replay. The agent is trained for 1000 episodes, where each episode consists of 180 timesteps (e.g. daily stock prices).

My question is concerning the sampling of episodes for training.

Assuming I've got daily stock prices going back 3 years, that's about 750 trading days/prices.

How should I sample this data set to get enough episodes for training?

With an episode length of 180 and an episode count of 1000, I'd need 180k "days" to choose from, if I wouldn't want any duplication.

Do I even need to sample 1000 non-overlapping windows from my dataset or can I sample my episodes using a sliding window approach? Could I even just randomly sample the dataset for episodes? For example, calculate a random date and build the episode from the 180 days following that random starting date?

The reward for each action is calculated as follows, p are the prices and t is the current timestep of the episode.

  • CASH: 0
  • BUY: p(t+1) - p(t) - fee
  • HOLD: p(t+1) - p(t)
Improve this question
$\endgroup$
10
  • $\begingroup$ Are you want to predict the stock price using RL? $\endgroup$
    – Swakshar Deb
    Aug 10, 2020 at 13:33
  • $\begingroup$ No, the model predicts one out of three actions (open/close/hold). $\endgroup$
    – Scarysize
    Aug 10, 2020 at 14:16
  • $\begingroup$ Why don't use the sliding window and iterative the whole data over and over. You can also go to investing.com and find more training data. $\endgroup$
    – Swakshar Deb
    Aug 10, 2020 at 14:23
  • 1
    $\begingroup$ I may be wrong about your problem, but I think you are trying to make action (open/clsoe/hold) based on previous days stock price. If this is the case the next state is independent of our current action if you take sliding window aproach. How you formulated the MDP? I think it's a contextual bandit problem for the sliding window approach. Same goes to other methods. $\endgroup$
    – Swakshar Deb
    Aug 10, 2020 at 15:32
  • 1
    $\begingroup$ Make the state as follows $s_{t} = (p_{t},p_{t-1},,...,p_{t-n}, cash \ in \ hand)$, then it will be a complete RL problem. $\endgroup$
    – Swakshar Deb
    Aug 10, 2020 at 16:13

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .