There was a recent question on adversarial AI applications, which led me to start digging around.
Here my interest is not general, but specific:
How do you game an automatic trading system by messing with data, as opposed to hacking the algorithm itself?
I don't have a proper source for this, as i've only read this from an online forum: everything i say is just hearsay and I am very uneducated on the subject. With that being said...
As you may know algorithmic trading relies on strategies, i.e. I trade a certain way once I see certain indexes move in a certain way. If your procedure is known by other people for some reason, then other adverse agents can release information or influence the market to manipulate your bot into making trades that the adverse agent then takes advantage over.
I'm not sure how this works in detail, and frankly I find it quite unlikely that a single adverse agent can target a single known strategy with the entire market in-between the two. But I've heard that a process known as the alpha algorithm can be used to discover hidden trading strategies. But again, I wasn't able to find any articles that directly linked this method towards financial trading. So my usual warning of me talking out my butt again applies.
Just a fun thought. A friend of mine told me (unfortunately, again I wasn't able to find a proper source for this) that Jane Street found that the most effective trading strategies were based on relatively simple ML techniques (I recall the specific example mentioned was just a common form of regression). The important follow question would be "What are their input metrics", I don't think I was told this, but I would believe that it would also just be something computed by relatively well-founded stock market indices. If this is the case, than again it would be very difficult for a single adverse agent to influence that particular bot since the force of the entire market is in between them.
On the opposite end, we have this redditor(finally, a source!), who amazingly analyzed about 130+ trading strategies, each of which were "big brained" in kind. This redditor concluded that all papers were essentially p-hacked, and none of the results were significant. Of course there is the possibility that authors with profitable results would not publish them, but the conclusion remains: keep your strategies simple. Again, this implies that it would be harder to "hack" a specific strategy used in the market by a particular individual.
P.S. There are other "big brained" methods that are kept well secret and thus difficult to analyze: I heard a rumour that a Chinese company paid people to take pictures inside shopping malls, to use to analyze shopping behavior, and how vibrant the economy is; I also hear that some people use satellites photos to look at, say, car dealerships, to see how much of their stuff is being purchased. Again its unclear if these techniques are profitable or not, but certainly a lot of money is being thrown into it.
