AI is developing at a rapid pace and is becoming very sophisticated. One aspect will include the methods of interaction between AI and humans.

Currently the interaction is an elementary interaction of voice and visual text or images.

Is there current research on more elaborate multisensory interactions?


This is one of the main research areas of my lab which researches intelligent prosthetics which also give sensory feedback such as touch and kinaesthesia (the feeling of a limb moving in space) to the user. We use reinforcement learning to bridge the gap in control and have preliminary work in communicating to the user predictions made by the artificial agent.

  • $\begingroup$ Interesting. Is there an artificial intelligence aspect to your research? $\endgroup$ – Frank Kurka Mar 10 '17 at 12:00
  • $\begingroup$ Yes, we work with Richard Sutton (AKA the godfather of RL) and other researchers to have intelligent agents control the arm and provide feedback. $\endgroup$ – Jaden Travnik Mar 10 '17 at 14:52

Probably these days it's still under the umbrella of "man-machine interaction" in CS, i.e. there is a (sub-) field for interactions between humans and machines in CS, but I am not aware that it has split again to create a sub-sub-field for AI/man interactions.

  • $\begingroup$ You might want to look at robotics too, they might be dealing with that in their field. $\endgroup$ – Frank Feb 20 '17 at 3:16
  • $\begingroup$ If you are more on the AI to human side, perhaps clarifying your question would help getting more answers. "Between" is bidirectionnal, no? Anyway, interesting question. $\endgroup$ – Eric Platon Apr 10 '17 at 11:57
  • $\begingroup$ Agree. There are multiple steams of research in human-machine interfaces and they are the closest one to answer the question of how will we interact with AI. I would only emphasis one direction of research: direct brain-computer interface. This field seems the most promising to me, especially due to recent interest from giants forming Neuralink. $\endgroup$ – Alex Mar 5 '18 at 11:45

At risk of seeming out-of-scope, I think it's worth mentioning there may be a case that human vs. AI play of serious games games, such as Chess and Go, represents the "deepest" level of human/AI interaction to date. (Game theory is also important because it underlies all optimization, including, I have no doubt, expanding and optimizing AI-to-human interaction.)

I bring this up because in terms of human-to-computer interaction, I doubt there is a more effective means of engagement than computer games, whether "serious" or otherwise. Thus you might find it useful to look at the concept of Gamification, which naturally lends itself to the type of multisensory input and output you're interested in:

"Gamification can improve an individual's ability to comprehend digital content"


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.