A phone can capture an image that lies on the front of the screen. It is also possible to manipulate input of the touchscreen using various programs and external devices.
If we combine these two elements of technology together, it's possible to have the phone aware of what's happening on the screen and do dictated commands.
This could be extremely useful to those who want to play a mobile game without the grinding effort, or those who do menial tasks on the phone but don't want to spend the time navigating across the phone.
For example, a phone could automatically open up an app on its own, press buttons that consistently appear on a screen for daily log in bonuses, auto-mode, etc, and collect the rewards for playing that app, without a human wasting time.
Such a behavior could even be recognized by the phone's memory, and perhaps linked to AI (like Google Now) so that the behaviors could be remotely activated on command, or on a timer.
Now my question is - do we have apps that are able to recognize what happens on screen, and provide self input from the background?