0
$\begingroup$

I've written a Double DQN-based stock trading bot using mainly time series stock data. The internal network of the Double DQN is a LSTM which handles the time series data. An Experience Replay buffer is also used. The objective function is cumulative stock return over the test period. My epsilon used for exploration is 0.1 (which I think is already very high).

My trading bot has a very simple action space, trade or no-trade.

-- When it decides to trade, it sends a signal to buy and own a stock for a day. I'd get a positive return if stock price has gone up from today to tomorrow; equally would get a negative return if stock price has gone down.

-- When it decides to not trade, I own no stock and the daily return is 0 because there is no trading. Strangely, my algorithm gives a daily 'no trade' signal most of the time when I run the algo through a number of different test periods.

Very often, after giving a 'no trade' signal for many days, the algo would finally give a 'trade' signal but the next day reverse back to giving 'no trade' right away.

My questions:

Why am I getting this phenomenon? Most importantly, what can I do to make the algo not stuck in giving out 'no trade' signal most of the time?

Improve this question
$\endgroup$
5
  • $\begingroup$ It could be simply a bug. If it's not a bug it's property of your data. If input give zero (or negative) expectation of reward for any action it's perfectly normal to do no action. By other word network unable to extract any useful information from input. Which is correct if your data are realistic - stock prices unpredictable from their history on medium frequency trading. $\endgroup$
    – mirror2image
    Jun 12 '20 at 4:51
  • $\begingroup$ Hi mirro2image, I see your reasoning. Since I am training the network using sections of time series data over 20 years, I'd be very surprised that most of these data make the network decide there is zero expectation of reward. I am guessing it's something else, but I don't know what it might be. $\endgroup$
    – ZXY
    Jun 12 '20 at 20:51
  • $\begingroup$ PS: You can try simple time-series prediction without RL with the same (or close) architecture and see if it works $\endgroup$
    – mirror2image
    Jun 13 '20 at 5:18
  • $\begingroup$ Interesting. I built a very similar agent. It would initially start trading very frequently but then would converge on no trading. My interpretation was that it was being rewarded for not losing money and refusing to take a position because it was unable to come up with a profitable trading strategy... just like me. :). Have you tried just rewarding it for some profit target? Also, from the literature, it appears to me that targeting the log of the returns is the way to go. $\endgroup$
    – Joshua DeFord
    Mar 2 at 21:47
  • $\begingroup$ @JoshuaDeFord Thanks for sharing your experience. Could you share the literature you've read that used log of the returns please? Also in your own experiments how long was your trading window? I am doing intraday trading, trading every minute (if an action is taken), lasting for 100 minutes. $\endgroup$
    – ZXY
    Mar 4 at 11:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.