For example using a neural network to predict a coin toss. Can a trained neural network to predict it with more than 50% accuracy?
This is a question of marginal vs. conditional distribution
The marginal distribution of the coin may be a Bernoulli random variable with 50% probability for either outcome. However, the conditional distribution of the outcome given information about other factors (e.g. the angle, throw height, ... see other answers) may look entirely different. Provided these features determine the outcome in some way, a neural network can absolutely predict the outcome with more than 50% accuracy.
A neural network could not exceed 50% accuracy, if
- The information determining the throw outcome is not available
- The function is of a nature that can not be learnt by the neural network
- The coin toss is truly random
A coin toss is often used as a casual example of a "truly random" event, so in this sense the answer to your question is "No". In reality however, it is very hard to find any truly random events (at least outside quantum mechanics), which is why random number generation is a big challange and neural networks can predict a lot of things.
If there are no patterns, relations or correlations in your data, AI can do nothing to improve what essentially is just guessing.
My last 5 tosses were Heads, Tails, Tails, Heads, Tails. Can you predict the next toss outcome? How would you explain your guess? If you give AI this same data, it cannot do better than just guessing.
The question changes if you have data that is related to the outcome of the coin toss, such as the direction and force the coin was tossed before it lands. In this case, it isn't "an event with a statistical probability of 50%" anymore. If you measured everything perfectly, you could have 99.9% accuracy on what the outcome of the coin toss would be.
AI can only produce accurate results if a super smart human could theoretically also produce accurate results.
If you obtain information about the force and angle of the throwers thumb striking the coin at release, that would give insight into how many times the coin would be expected to rotate. Combine this with what faces up when the coin releases, and you should be able to do better than 50/50.
I don’t have a firm source (perhaps there is something on Skeptics), but it seems that people have trained themselves to flip coins to reliably land on one of the sides, so there are some features that dictate how the coin rotates.
Really, this is kind of the point of regression. You think some process has a 50/50 chance of the two outcomes, but once you know a bit more (features), you can sharpen that estimate. Formalizing this mathematically involves the conditional vs marginal distribution discussed in the answer by Scriddie.
You need to ask yourself, what is the limiting factor in the accuracy for whatever you are trying to predict.
- If the limiting factor is in the quality of the algorithm being used to calculate the prediction, then perhaps you could find a better algorithm that would improve the accuracy.
- If the limiting factor is in the very nature of the problem itself,
such as a coin flip, then there is no method of calculation that
could improve the accuracy.
Although the question is a little vague, I'll treat it as a statement about the mapping of inputs and outputs in the underlying random process - no matter what conditions/inputs/features we observe, there is not a consistent mapping from input to output. A statistical probability of 50% suggests in two cases with identical inputs, we may find different outputs. A traditional deterministic neural network cannot do this, as it is really just a mathematical function, which by definition maps every possible input to exactly one output - it is not possible to use the same inputs and get different outputs. Because of this, a deterministic neural network can't achieve more than 50% accuracy in the long run in this case. No matter what set of features is input, there are in reality two possible outcomes, but the neural network can only return one outcome for any particular input. On average, the neural network will return the correct output only half the time - it can't achieve more than 50% accuracy.
If a neural network was only able to be as reliable as random guessing, they wouldn't be much use!
Let's suppose there's an election on, and the result is finely balanced between the yellow party and the purple party. At a top level, 50% of people will vote for each colour. If you know nothing else about the people, "Who will the next person in the polling station vote for?" is intrinsically an even guess.
It would still be possible to use a neural network (or a decision tree, or a human!) to predict at much better than 50% if you have additional input. For example, if you look at them and can read their apparent wealth, race, gender presentation, or whether they come accompanied or alone, it may then be possible to identify membership of a sub-population which is more likely to vote yellow.
The fundamental limit on performance isn't the baseline 50:50 probability. Instead it is the component which is caused by some inputs (which may but doesn't have to be true randomness) that are simply not available to the network. Suppose for example that there's a 5% chance that someone has been bribed to flip their vote and the network can't know that. In this case it won't get to more than 95% reliable predictions, but can still do much better than 50% with demographic data.