Most robotics challenges like Robocup soccer, micromouse and the Amazon picking challenge are oriented on fully autonomous systems. A motion controller gets started, the operator takes away his hands from the keyboard and then the system is solving the task by it's own. Such a technology might be state-of-the-art but my interests go more into the direction of low-tech robotics. That's a control paradigm which is working with a mouse and a keyboard no Artificial Intelligence is in the loop. This is called a teleoperated robot and the human takes his hands never away from the keyboard.
Realizing a plain teleoperation system is not that hard. Because the user input has to be transmitted only to the servo motor. To make things more complicated the system should provide annotations. That means, the human operator is doing a task and in the status bar a natural language description of the detected events is shown. For example “gripper pushes object”, “object in gripper”, “gripper releases object”.
The only problem is, that i don't how to annotate a teleoperation robot system. Is some kind of best practice method available for creating a semantic event recognition? Are neural networks are great choice?