I've gotten curious about this topic and am wondering what the stack exchange community has to say about it. Also, does anyone know of any professors/researchers who have published papers pertaining to this?
This is a highly relevant question as market trends have become more emphasized over the fundamentals of individual companies, and algorithmic trading has proven to be quite effective, particularly in areas such as high-frequency micro-trading.
This 2013 Forbes article estimated nearly 80% of stock trading volume in the U.S. is conducted by automated trading systems. More recently, Bloomberg published an article on the subject: The U.S. Stock Market Belongs to Bots.
The massive adoption of algorithmic trading in a fairly short span is a pretty good indicator that these systems are effective. However,
- My feeling with any predictive system is that it works until it doesn't
Imperfect and Incomplete information are persistent in real-world scenarios. The implication is that there are always unknown factors that can diminish quality of analysis, and even lead to disastrous outcomes.
No matter how complex an algorithm becomes, reality is more complex, (at least for the foreseeable, pre-singularity future;)
Adding to that, systems in general tend to be imperfect, and when engaged in massively repetitive functions, small errors can grow exponentially and produce catastrophic effects.
"Flash Crashes" are a recent phenomena, and a result of algorithmic, high-frequency, black-box trading.
A couple of thoughts:
Humans can't reliably predict trends in the stock market, so expecting AI's to do so is probably unreasonable.
The above would be more true if it were proven that the movement of stock prices is really a random walk, but my understanding is that the current thinking is that stock movements aren't completely random... but just really close to random.
If there are some trends there that represent a useful signal, and if somebody has found that, they're very unlikely to share that information, as the market would then immediately price that information in and they would lose their edge.
There is quite some research done by Hans-Georg Zimmermann, who has programmed Neural Networks for Siemens since some 20 years in order to predict Stock markets. He wrote some books on it, too, though I don't know if they are any good in English.
This article gets to the point a bit faster than the video, I hope it helps.
edit: I think this interview gives a good short introduction to his work.