0
$\begingroup$

If we take simple financial timeseries data(stock/commodity/currency prices), State(t+1) does not depend on the action that we choose to take at State(t) as in Maze or Chess problem.

Simple example: as states we can have the sum of the daily returns of 5 different ETFs. Based on that, we want to take action - either buy(go long) or sell(go short) in another ETF. No matter what we choose however, our action will not determine what the next state would be (we do not have any control of what the returns of those 5 ETFs will be tomorrow).

In that case of simple financial time series data, would multi-armed-bandit approach be more suitable?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

Your agent's actions will (probably) not have much impact on the observed financial time series. However, they will make a large difference to other things - namely what stock your agent is holding and the account balance.

If you are happy to ignore the agent's current portfolio and balance as not part of your problem, effectively treating these items as infinite sinks, then yes a multi-armed bandit might be a reasonable solution. But then so might any other sequence-predicting algorithm, if what you are searching for is some kind of financial prediction of good times to buy or sell.

If the portfolio and cash balance are an important part of your problem, you should add them to the state and use reinforcement learning. You might do this if your goal is to model a single investor playing the markets.

Note that although it may be possible to use machine learning techniques to analyse markets, and base investments on advice of a trained AI agent, it is a very risky venture. There are lots of ways you can fool yourself into believing too strongly in your solution and you stand to lose significant amounts of money.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you @NeilSlater! While playing with on-policy Monte Carlo algo I start to think that the same results could be achieved simply by taking the average return for every possible action(long/short) in every possible state. I mean we don't even need to explore/exploit... just find those values, perform walk-forward backtesting or s.th. like that and done. All that with RL starts to not make sense for me in terms of generating buy and sell trading signals. $\endgroup$
    – kobo
    Commented Jan 4, 2023 at 17:47
0
$\begingroup$

The question is whether the rewards are i.i.d. In such time-series problems, the rewards are not i.i.d; the reward at timestep t+1 depends on the reward at timestep t. Therefore, even though the actions/states may be somewhat independent, the rewards should i.i.d in order to apply MA bandits. Otherwise, the problem is about RL.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .