I am researching affective computing. Particularly, I'm studying the part of emotion recognition, i.e. the task of recognising the emotions that are being felt by the user/subject. For example, affectiva can be used to this end. I have concerns not in the validity of these models, but in what we are going to do with them.

What about responding to emotions? Will computers be able to really understand user emotions?


2 Answers 2


Will computers be able to understand user emotions?

The term Understand is multidimensional, so characterizing the degree of understanding — emotional or otherwise — is a slippery task. Nonetheless, some forms of AI emotional understanding are possible.

An Interesting Simple Case

Even the embedded programs of velocity sensitive musical keyboards are emotionally aware at a primitive level. Using the key depression velocity to drive the note parameters of attack, timbre, and volume allows the accomplished user's emotions to pass through the software into the emotional content of the music, which can be experienced by listeners just as with instruments that are not digital in nature.

More Advanced Emotional Capabilities

At the other extreme is a wide array of human emotional capabilities. One example is the set of complex recognition functions underneath the typical listening skill set of a counselling professional.

The term affective computing was mentioned in the question. A counselor may note the affect of the counselled, reading tone of voice, facial expression sequences, and body language. From this, a clearer picture of the internal emotional state of the counselled can be understood. These recognition and analysis abilities can determine the subject's internal emotional state more accurately and comprehensively than a masterful analysis of a functional MRI or the detection of neuro-chemical metabolites in blood.

There seems to be no theoretical principle that limits computers from mastering the front end of a counselor's skill set, the recognition of emotion to produce a coded sequence of affects. The degree to which software can interpret the sequence of affects (in combination with concurrent natural language expressions) and determine the emotional condition (and potential market-oriented decision patterns) of the human is yet to be determined.

There are other emotional capabilities beyond the skills of emotional recognition and interpretation. Compassion is largely the logical integration of ethics in the absence of self-centered motive with the results of emotional recognition and interpretation. The addition of emotional imitation yields empathy.

The development of such capabilities in automata is more important than the shallow and therefore myopic motives of e-Commerce. There may be applications for toys, education, entertainment, and remedy for the growing coldness of a technological society. (Such is discussed further below.)

Functions of recognition, interpretation, the application of rules or meta-rules in the ethical domain to these interpretations, and imitation may well be within the capabilities of digital systems.

Isaac Asimov introduced the possibility of telepathy to the feature set of both the human mind and automata. It may seem like telepathy is confined to the domain of fiction, however an emoticon can be the tele-form of a component of affect, such as a smile. In this way, consensual technology assisted telepathy has actually become commonplace.

With the erosion of privacy in a culture with increasing interaction with digital systems by an increasing segment of the public, less consensual telepathic techniques may be developed and mastered.

Deeper Aspects of the Question

The doubt lies in the existence of a soul or some other autonomous and non-deterministic aspect of the human person. The existence of these things and whether the simulation of them can be accomplished with a Turing machine and practically deployed to the Von Neumann architecture or a collection of them have neither been proven or dis-proven formally. If autonomous and non-deterministic elements exist in people, then, even if they can be simulated, we cannot infur they can be realized as independently autonomous.

Challenging the implications of the MIT coined term Meat Machine, the notion that humans are capable of intention by fiat is not disproven, yet a pure Turing machine clearly cannot intend in that way.

This is important in the context of the question in that a computer may only be able to simulate frustration when an intention is thwarted by the conditions of life. Actual frustration may not be possible.

Current Publicly Experienced Progress

One interesting entry point of computer science into the domain of human emotions is the story based memory and reasoning model purported by Roger Schank of Yale U in the 1990s. Although the machinery used by Amazon underneath the hold of its ever improving ability to recommend movies in a buyer specific way is Company Confidential to Amazon, one must wonder.

Can e-Commerce develop user profiling regarding books and movies to comprehend the story plot or the arc of protagonist's emotional development? Can it do so sufficiently to match user viewing and purchase with other products? Is that what Amazon is beginning to do? Short of an unethical Amazon employee, we can only guess. Certainly, what is transpiring in leading edge e-Commerce has moved beyond word or phrase matching or tracking interest in particular authors, screenwriters, directors, producers, and stars.

If Roger Schank's story based memory and reasoning has penetrated into e-Commerse, emotional analysis of individuals of within public is well underway, since story plot and protagonist developmental arcs are bound in climaxes and setting up climaxes, all related to emotional states. If not, emotional analysis of groups within the public has been underway since public relations functions entered into the roles of enterprise IT systems. (I know for a fact that such occurred decades ago.)

Even if the likes of Facebook, Google, and Amazon are attempting to approximate in various ways the emotional states of individuals in real time, the matching of movie plots of protagonist arcs and such pattern matching and naive categorization is far from what a good counselor or a self-actualized friend or family member might do.

Computer systems to which the public is exposed do not yet appear to accurately developing comprehension of the buyer's emotion. That is one area where even the most progressive interfaces are still shy. Furthermore, the computer interface itself is still as dry as it was in the late 20th century.

Short of breaches of confidentiality, what capabilities to recognize, compare, analyze, or simulate emotions may exist in proprietary labs is solely a mater of conjecture.

Warm Interfaces

Returning to the question of the general coldness of cybernetic interfaces in a technological society, there is much for the computer science community to learn. For one thing, the soft skills of actual technical people often leaves much to be desired, so a cultural shift from cold development teams to culturally progressive and warm teams might be necessary.

Furthermore, one who has engaged with various PaaS (personality as a service) devices such as Siri, Google Assistant, Alexa, Cortana, and Microsoft's Office Assistant, Clippy, might find the paperclip personality of Clippy to be the warmer from among the list. Perhaps the Clippy didn't see a higher level of acceptance because of the disparity between the coldness of a grid of cells in a spreadsheet and the attempted warmth of the animated character was too much, and the little guy tends to get in the way of some portion of the interface, which can be annoying. (Integration of concepts was poor.)

Nonetheless, the animated style of Clippy is intended to create a mental association with cartoon characters, which is brilliant. Cartoon character design and storytelling through comic and cartoon media is a creative area of technology that had mastered computer generated warmth long before intelligent assistants began to appear.

One can't help thinking, upon general survey of intelligent assistant history, that the more the system tries to be human, the less human it seems. Human response to emotional content in a cybernetic interface is like that of color matching or synthesized musical instruments. If one is attempting to synthesize, the synthesis has to be good. If it can't be fully convincing, then there is more authenticity in inventing something that is not pretending to be something real. A caricature of real life objects (like Sponge Bob or Road Runner) avoids exposing an unacceptable margin of error in the imitation of real life. [1]

Our sensitivity in this area is so acute that if a real person, working in a customer service department, acts too mechanically such that we have even the slightest doubt as to whether we are talking to a human being, the impression is negative and we want to hang up and look for the function we wish to initiate on the company's web site.

It is unclear whether the trend for people to become more like cogs in a machine or whether usage patterns will drive companies to design their user interfaces and the mechanisms behind them to exude something like hospitality or congeniality. Although Jaques Ellul seems to be correct in his proposition that technology has long since been driving humanity rather than the other way around, the DNA nature of the human mind will perhaps force humanity to remain remarkably human. Thus the evolution of cybernetic interface professionals may continue to seek the meaning of and simulation of warmth.

The problematic, controvertible, and somewhat bizarre pursuit of Turing Testing will probably endure.

Concerns About Emotional Modelling Uses

Whether an emotional recognition, modelling, analysis, and comparison suite, obviously a powerful tool-set, is used for the betterment of humanity or its detriment is a question like that of any other powerful tool-set. Mass psychology, nuclear science, statistics, and genetics have in common the potential for great good or great evil.

In the context of this question, broadcast or more targeted information dissemination can be characterized as propaganda, which may not necessarily be perilous. Propaganda was used for women's suffrage and along the path of U.S. independence from Europe. Elements of broadcast and targeted control have created much geopolitical trouble, a topic outside of the scope of this social network, but pertinent and somewhat obvious, such that further mention is unnecessary anyway.

Individual profiling can be of great good too. Such can be like a combination of the old library subject indexed Dewey Decimal cards in combination with a knowledgeable librarian when looking for information or items of entertainment or personal expression for borrowing, rental, or purchase.

Of course, profiling in combination with statistics can lead to the ability to obtain covert control over a mass of people, not just for the profit maximization of a company. An appropriately self-deluded megalomaniac or a collection of them in some kind of secret society could use emotional modelling and comprehensive user profiling for a large sample of national or global population for the injection of emotionally powered systems of belief. Such a person or group could theoretically succeed in covert and systematic sublimation of liberty to a kind of virtual authoritarian dictatorship.

How technology is used is always a concern. Transparency and accountability are required features of a society.

Just as in the other areas of science and technology mentioned above, it is up to we who are skilled enough to realize these systems to be intelligent and active elements within our society's accountability mechanisms. Whether a corporation or governmental department is leading humanity into perdition is an item that can be judged during a job hunt and during the initial work period.



This is perhaps the strongest critique of the Turing's Imitation Game philosophy. Cartoons, comics, and sci-fi creatures demonstrate that people don't seem to care if intelligent beings can be distinguished from humans, and there is no proof that human intelligence is the only or the best example of intelligence.

Along these same lines, heroes are representations of intelligent behavior that epitomize some aspect of humanity that actually doesn't naturally occur in pure form as they do with these protagonists. Yet actors are branded and marketed as stars because of this idyllic expression of character. It may be that humans don't really like human intelligence and constantly try to transcend it through fiction, but results in only some public awareness that is never more than an anthropological fantasy.

Even AI is not an imitation game. It is a transcendence game. For this reason, the Turing Test is of limited value.

  • $\begingroup$ I like that you start with a keyboard. The first thing I thought of, over and above the pressure sensitivity was the "Mood Organ" musical device at the beginning of Do Androids Dream of Electric Sheep. *(Don't know if this was intended, but the impact of music on emotions is undeniable, and it occurred to me recently that music may be thought of as emotions translated into algorithms. Your connection with physical interaction of the human with the keyboard is quite useful information. I have only a little music theory, and rudimentary piano, but know exactly what you're talking about!) $\endgroup$
    – DukeZhou
    Feb 14, 2018 at 18:46

I was an Undergraduate Research Scholar - I and my team developed an algorithm to detect Human Emotions from touch Screen - which is under further improvement and development by PHD scholors of my guide .

From studying literature I can say that the reverse is a difficult task - Detecting Affect . Even more difficult to detect human emotions with a light weight solutions - i.e without using heavy and wired hardware . - which is the need of hour .

Once its done - there is a plethora of applications - focused ad marketing , enhancing User experience , game development etc.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .