Would people go far with Artificial Intelligence and machine learning to the point where machines could learn during a long period of time to distinguish what's 'good' from 'bad' according to people living in a restricted geographical area, and then the machines take control and turn what was learnt into a set of 'rules' and 'laws' (think of it as an effective machine of 'politics') that match the majority of the people's view of issues.

That should be accepted by everyone, since a contract set at the beginning says: "Everyone is ok".

  • $\begingroup$ Before machines can learn..Do you yourself actually know what is good and bad? Didn't your perception change from child to adulthood? $\endgroup$ – DuttaA Jun 27 '18 at 15:11
  • $\begingroup$ It did. Things the machine set the first time should be scalable, and not only the first time, but each time it's necessary to update something. Lets call it continueous learning. $\endgroup$ – Yla NC Jun 27 '18 at 16:09
  • $\begingroup$ Defining good and bad should be based on a formula that figures in the contract, every user, oh sorry one, approved by confidence. I can assume that for a fool of people, we can always find a majority and a minority, the simplest decision rule could be : majority rules. $\endgroup$ – Yla NC Jun 27 '18 at 16:11

Even if you could do that (which I believe is a long way off), what would be the point?

If I understand you correctly, you want an AI system to learn through observation of human beings what their 'rules of interaction' are. It sees a person killing someone else, and then that person is punished by the community, so the AI learns that killing people is not right. However, that is already codified in laws... So what it would pick up are social behaviours, which hopefully be things like "be nice to other people", "don't do anybody any harm", "be honest and truthful", etc.

The first question then is how an AI could assess events[*] as to whether they are 'good' or 'bad'. If everybody lies and steals, that would be learnt as normal behaviour, and the AI would not be able to pick up that most people would see this as 'bad'[**]. Causal relations are also hard to grasp. Somebody steals something. Then later someone else buys him a drink. So stealing stuff means other people will buy you drinks? These are really hard problems to solve. You need to know about people's motivations, and the multitude of 'threads' of interaction happening at the same time, even in a very limited area.

So, recognising events and causal links, plus a moral evaluation of them is pretty difficult. I don't think we will get there anytime soon. Unsupervised learning of behavioural is also pretty difficult, as you only have unlabelled observations and no real criteria to classify them. Plus, many actions are morally ambiguous. Killing people is generally seen as bad. What about the officers who tried to assassinate Hitler in 1944? Life is complex, and our artificial models are not anywhere near that.

So even if you were able to do all that, what would you end up with? An AI system that has picked up a lot of unwritten rules about human behaviour, and then postulates that as laws? So everyone has to behave the same way? I just don't see the point of that, even as a thought experiment.

[*] leaving aside here the question how you determine what an 'event' is in the first place
[**] Please note that even if people steal things, they steal can view theft as a bad thing.

| improve this answer | |
  • $\begingroup$ Many thanks, I found some interesting points on your reply. But what if we think of a mechanism that builds a sort of consciousness ; for example, the act of killing might be seen as a 'good' event, the first time, but the next 10 times , it's not. Some functions could be implemented to decide as better as possible if something is good or bad. When it comes to people lives, ofc it's a bit dramatic, but in other situations, the idea might be of help. $\endgroup$ – Yla NC Jul 6 '18 at 3:02

Applying Scientific Rigor

When we extend the limits of science, we must employ the precision science requires. If we create excellent models that, as John von Neumann stated, "are expected to work," the working results is the technology advancement.

In the case of this question, it is clear what artificial means, however neither the term emotional nor the term intelligence are precisely defined in a way that leads to scientific rigor. A survey of a hundred books and papers reveals a hundred different definitions for each. We cannot dismiss them as being undefinable, but there is not yet a sufficient basis in cognitive science for authors in academia to reach agreement.

Because we have precise meanings for degrees Kelvin, proportion of a population, Volts, seconds, and Megabytes, we can include those properties in mathematical models. The ambiguity of the terms intelligence and emotion are challenges inherent in this question.

Clarification of the Objective

The body of the question contains the moral linearization embodied in the ideas of right and wrong. This indicates that the word emotional may not be precisely what the question author meant, which makes the question more manageable. It is easier to define moral intelligence than emotional intelligence, emotions being a slippery concept in cognitive science.

LSTM networks in combination with semantic nets have shown some ability to synthesize the appearance of emotion in animated characters or chat, yet these AI systems are merely synthesizing an internal emotional experience. The AI may externally act as if it feels, but it does not feel with the kind of consciousness subjectively experienced by humans. This superficiality may someday be overcome through the emergence of true AI cognition.

Demystifying Morality

Spiritualizing right and wrong is not necessary. Ethics is an attempt to demystify morality, however it is not precise in the way that an AI system requires, given the current state of AI research. One can arrive at a precise definition of right that can be used in conjunction with current technology.

We sometimes hear the term, "The greatest good for the greatest number," which has a ring of truth to it, but it is not quite a precise formulation. It is far to ambiguous to lead directly to a meaningful mathematical relation, but it can be a conceptual starting point.

Actions and Sustainable State of Affairs

It is important to make a distinction here between right and wrong as opposed to good and bad. Right and wrong are usually attributes of actions whereas good and bad are usually attributes of trends of states — trends because one cannot state that something is good or bad unless there is some continuity to the goodness or badness of the state.

The inaccuracy and unreliability inherent in prediction in a complex system imposes a more important distinction between the measure of rightness and the measure of sustained goodness. A person or AI system can do something wrong but the effect may be good, and a person or AI system can do something right but the effect may be bad.

With partial information in situ, what the reinforcement and robotics literature calls partial observation, the model used for decisioning may be an inaccurate representation of objective reality. This inaccuracy may lead to decisions that have no effect or an effect opposing the objective.

A Possible Formulation

We can consider this expression, where the environment includes all people and their surroundings, and the goodness of the environment is estimated as a quantity $\mathcal{E}$.

$$ G = \sum_{n=1}^{N} \mathcal{B}(p_n) = \mathcal{E}(a_1, a_2, ..., a_h) \text{,} $$

In this model of goodness $G$ is the sum of benefit to $N$ people, each benefit measured through approach $\mathcal{B}$. Goodness can also be expressed as an environmental reaction to actions $a_1$ through $a_h$ which can be of benefit to the people or detract from it.

With this we can define right as a class of actions that are likely to lead to a greater good, defined as the sum of benefits. Wrong becomes a class of actions that are likely to lead to a reduction of benefits. The basis is no action, and all measurements of good and bad are relative to a pure dark vacuum in which nothing happens.

The converse can be defined simply as bad conditions of the population $B$,

$$ B = - G $$

The Caveat With a Linear Composition of Benefits

A flaw in this model is the potential of significant inequality. By this model, one can justify the killing, maiming, or plunging into poverty or pain a small number of people so that the vast majority of people can benefit insignificantly. This model provides some balance such that benefit withheld from people that lack opportunity or ability and bestowed upon those who ran into opportunity or were born with critical ability is not considered as good as a buffered system where there is a somewhat more equally distributed benefit. This is one possible expression of this egalitarian morality.

$$ G = \sum_{n=1}^{N} \mathcal{B}(p_n)^{0.5} = \mathcal{E}(a_1, a_2, ..., a_h) \text{,} $$


The challenge is to find an approach to measuring benefit $\mathcal{B}$, from which the the attributes of environmental state $\mathcal{E}$ that produces functional learning conditions. However, once an approach is chosen, it is possible to begin to consider expectations and apply reinforcement learning to simulated conditions based on social data and study this idea further in the laboratory.

The output of this research could, theoretically be applied to sensory recognition, the AI learning objective being to detect the impact of actions on $G$ based on the assessment of the summation of $\mathcal{B}$. This will be an estimate, as is most often the case with prediction in complex systems such as social ones.

Politics and Religion as Attractors

Under these conditions and with multiple AI systems performing these predictions of the effects of actions in the effort to do right things, there may be distinct classes of expectations, which we can call poles. The result might be the emergence of artificial politics, platforms, perspectives regarding current affairs, and religions. This is likely because any effective learning system applied to this model of goodness would naturally learn what some call the ripple effect. The idea of paying good things forward would become an option, which may be a basis for AI altruism.

In chaos theory, these emergent patterns, neither polynomial nor purely periodic are called attractors.

Distinct from Rule Based Overall Approaches

Note that this overall approach is distinct from earlier formulations from the great science fiction writer Isaac Asimov that formulated good AI behavior through a set of three rules, the implementation of which is not very feasible at the current time. The feasibility of this overall approach is solely reliant on the ability to measure benefit via $\mathcal{B}$, which is considerably more tenable and a vastly reduced problem to solve.

Geographic Limitations

The limitation of the geographical area may not be appropriate in a global economy with global climate concerns and a global Internet. Everything from agricultural products and energy resources to labor and information are involved in global trade.1 We may not necessarily want to introduce AI systems into a global culture and economy that considers justice in a provincial way.

Laws Follow Morality

The emergence of law is the result of reaching agreements about what class of actions reduce badness $B$ and prohibition of them. Economic incentives are often a widely sweeping set of action classes that tend to increase goodness $G$. The formation of these things in human society is like this.

Social consciousness
Written code
Legislative and judicial process
Statistical control through economic incentivization

To build artificial moral intelligence, it may be prudent to follow this same development path in the absence of any better ideas for a path.

Saving the Day with AI

The possibility of reaching a consensus by adding AI systems into the political mix is low. The environment $E$ is complex and the interaction of multiple entities intending to find the relation between classes of actions and $G$ adds complexity. A highly bifurcated chaos is probably unavoidable.

The only way to stop vacillation is to obliterate not only life, but the entire universe, which is not only infeasible but it limits $G$ to only one possible value, zero.

There may be other positive values to adding AI into culture — $G$ may increase if the approach to measurement of benefit $\mathcal{B}$ is well developed, but utopia is an absolute. The dynamics of the biosphere, even if mater and energy follow absolute rules of operation, is characterized by enormous complexity. Ideals are excellent in that they are objective toward which society may reach, but individuals, including those we synthesize out of plastic, metal, and silicon will probably never reach a unanimous consensus.

One could argue that the maximization of $G$ would be compromised if unanimity was reached and held as a constant. Isn't that the great paradox of the stability of tradition and the occasional necessity of revolution, the cycle of things coming and things going, of life and death?


[1] The exceptions regarding globalized information are those isolationist elements in most nations that abhor the corruption of traditional values and consider an open Internet a kind of virtual colonialism to resist with censorship and the sword.

| improve this answer | |

The current state of research in AI is more concentrated towards more applicable and meaningful tasks, such as drug discovery, making better image understanding models, etc. Building an AI that can classify between good and bad is far from our understanding.

We have only started to understand the true potency of neural-network based AI models in this decade, so we simply can't predict what happens next as well.

"Good" and "Bad" are abstractions created by humans. There is no universal definition for such things, so you can’t mathematically define good/bad. It comes due to your consciousness. An AI system should have consciousness in the first place to do such things, and our current research shows no hope in that direction.

| improve this answer | |
  • $\begingroup$ Thank you very much for your answer. But I still think that good/bad distinction could be achieved mathematically, for example by considering a happiness degree, something that measures utility of many 'events' ( = percepts + actions + reactions ) happening together in the the environment the AI lives in. $\endgroup$ – Yla NC Jul 6 '18 at 2:51

The most interesting scientific field in terms of relevance to this question is probably "affective computing".

There are several problems with the model that you suggested. It is questionable if AGI should learn in the same way humans learn. In addition, there are several ethical problems surrounding this question, perhaps even metaethics, because the question peeks beyond human ethics.

A hardcoded ethics protocol might be possible to implement, similar to Isaac Asimov's "Three Laws of Robotics". The possibility of hacking is an issue that one would have to think proactively about - especially in this case.

If humanoid AGIs or robots are very similar to humans, they should probably be treated that way. For example, ethiologist Frans de Waal has studied empathy and social behavior in monkeys and suggested that we are very similar in behavior, so we are probably similar in terms of feelings (if it quacks like a duck and so forth). Perhaps we need an ethiology for androids too?

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.