Basic Fairness
If by ethics the question means that the system design is to foster basic ethics in the context of a conversation, then at least three attributes can be designed into the system.
- Egalitarianism — no favor or disregard on the basis of gender, age, beliefs, financial status, ancestry, or culture, even if those features of the conversant are known and useful to know in personalizing the conversation
- Temporal fairness — ensuring that the interaction with new correspondents and veteran correspondents provide these with equal access to relevant information, even if the establishment of trust is an evidence based process controlled by the system (recommended)
- Sharing — balancing the objectives of the system to serve the interests of others as much as the interests of the system and its stakeholders — The Golden Rule: Treat others as you wish to be treated.
These attributes can be encoded in fuzzy rules, incorporated into the value function of an ongoing learning algorithm, or expressed in the loss function used during training, directing what is learned by a network. All of these strategies require the tracking of metrics and the uses of them in adjusting behavior, a kind of social awareness.
There are obvious ways to do this in the mathematical expressions of values, assets, advantages, affirmations, attentions, losses, errors, pains, or other learning feedback approaches are assembled. In generative network topologies, a discriminative network would need to be able to recognize unfairness and selfishness using the above principles. Such places specific requirements on the designs of metrics and how fairness and the balance of interests are quantified.
A sum of squares across the dimensions (features) of potential inequity that can develop during a conversation is a possibility. Using the fourth power instead of squaring would imbue permissiveness for smaller inequities but a drastic aversion to greater inequities.
$$\ell = \sum {(b_i - b_j)}^4 + \dots$$
The variable $\ell$ is the loss function result and $b$ is the benefit to conversation participants $i$ and $j$ where the two might be a human and the chatbot, a human and a stakeholder of the chatbot production server, or two humans that are in separate conversations with the chatbot, or two humans that are in the same conversation with the chatbot. Note the ellipsis indicates more terms that affect loss as needed.
Cognitive Ethical Sophistication
If by ethics the question means the kind of ethics passed from families that value honor and truth to their children or taught at Harvard Law, the system design may have to wait until cognitive abilities are further developed in computing machinery. Otherwise the question author will have to further AI to achieve the ethical sophistication.
Research into semantic acquisition, abstraction, automated assembly of causal models, application of these abstractions and models in planning and execution, and the integration of these with natural linguistics is ongoing. Broad cognition and What some might call wisdom escapes the products of AI as of this writing.
A sufficiently deep network may approximate dialog that demonstrates what appears to be ethical awareness if trained with data that represents stories and conversational actions deemed to be ethical in those stories. However, compacting learning the layers of comprehension that humans acquire through books, movies, other media, family life, and community events over a few decades (if they acquire it at all) into a computing project may be an overoptimistic objective at this time.
Although a collection of classic works might demonstrate much of a high ethical standard, it is not labeled data. One would need to identify which character was transformed in a way that demonstrates the valuable ethical standards and then configure the learning process to learn from the character arcs of ethical characters in the stories.
Inherent Feedback
It has no feedback loops of any kind as of right now, though reinforcement learning with a loop might be helpful.
There is no way to adapt without feedback. If the system is adapting, some feedback mechanism is correcting conditions that are maladaptive, even if the mechanism is opaque to the observer or identified by some other name. In the case of GANs, the feedback creates a balance through the discriminative and generative network components.
$$\Bigg(\Big((G, E_1) \Rightarrow D\Big) \Rightarrow G, E_2\Bigg) \Rightarrow D \dots$$
That does not exclude the possibility that additional feedback paths may provide additional design advantages.
Hard Coding Ethics
Can judicious behavior be hard coded?
The idea is that there would be some sort of unchanging knowledge base for the chatbot to use that has hard-coded ethical statements and values that the bot has no ability to change.
Will this even be effective at allowing it to generate its own responses yet still be confined to the ethical standards given it?
It is not unchanging knowledge in the domain of ethics that leads to justice, but unchanging principles that guide the detection of what is not ethical that guides a continuous improvement of ethical standards. That is why even constitutions have rules for amendment. In probabilistic terms, without the balance of doubt and certainty, no ethical growth can occur.
No one would read a book or watch a movie where the protagonist has no character arc because it is unremarkable. When a story is remarkable it is because ethics are adaptive and the adaptation must be a new kind that is not already covered many times in previous books or movies. People like to be surprised by these adaptations. They must occur against all fears and opposing external forces that are trapping the protagonist in a prison of mediocrity.
The story climax is when the protagonist, against all the building forces that keep them unremarkable, takes a path that discards all previous planning and forges a new radically different but fundamentally better one.
Excellent Objective in the Question
The intention to develop a system that is ethical is honorable and ethical in itself, so the question author may imbue some of those qualities into the system being designed. In my opinion, such would be a triumph of greater value to the field of AI than all other progress up until now.
To make such an advancement, the framing of the problem will require thinking in new directions. Consider the stated concern.
It may begin to ignore these "facts" over time, and they may become irrelevant.
Two principles may prove elucidating in regard to this concern.
- Every language in common use has the equivalent of the English words
should and ought, indicating a universal sense that something is awry. This detection and identification of imbalance is central to the definition of heroism and honor, which is making at least an attempt to re-balance it. Even the Oracle in the Matrix, after creating imbalance, sits on the park bench talking to Sati in the warmth of a new balance. Identifying what needs disruption and re-balancing and constructing a strategy to do so is the one thing in common across all ethical behavior. Ethics is remedial. Some ethical choices require unlearning, and that is the most timeless feature of ethics.
- Some of the most ancient principles, such as the Golden Rule or the idea of transparency – that right thought and right action are inescapably coupled – are more pertinent to this project than anything one can find in a book about machine learning. The computer cannot hide a malicious intention and then converse ethically. The objectives of the stakeholders of the system will need to also be ethical.
First Decision in Approach
The decision to make is the level of abstraction to apply. Should this chatbot be ethical or be ethically minded?
The first would require the imbuing of ethical character into the bot. The second would require the imbuing of the acquisition of ethical character into the bot. As difficult as the more abstract second choice may seem, the first may be unachievable. We don't have an example of it. We do, however, have one proof of concept for the second choice: Good people.
The other sub-question to address is regarding selection of AI components.
So which would likely be better [generative or reinforcement]? Or perhaps I should try a combination of the two?
If you are serious about this objective beyond the simple tracking of basic fairness metrics, then you may have to use elements of both and possibly new things that extend the list of AI components and principles to use.
Take One Step Back
One thing to understand about generative topologies is that they are not really adversarial, even if that word is in the name of a seminal paper. They are usually highly symbiotic and involve equilibria. What would the discriminative network be without the generative one? What would the generative one generate without the discriminative one?
The two collaborate to create a feedback system exhibiting a balanced equilibrium. This is like biological stasis, which is the genius of it. It is also like chemistry. Salt in saturated suspension in water involves both dissolving and crystallization in a constant expression of opposing reactions at a molecular level. Similarly, all ethical decisions are decisions are made within the context of at least one equilibrium, and the details matter.
Every court case where a judge or jury is to apply the public perception of ethics to the case has two sides presented. They may appear to be adversaries, but the entire legal process is a sophisticated and symbiotic design to force collaboration in the interest of justice.
Every educated journalist understands the importance of avoiding sensational reporting. Dramatically presenting only one side of the story sells news and is a constant temptation. Journalistic integrity demands striking an equilibrium. The result of the ethical choice determines whether the news item educates the public or adds polarization to a culture that may already be partially crippled by extremism.
Take Another Step Back
Reinforcement is an odd name to give what is essentially Agile planning via the probabilistic projection of expected advantages. Should lean methods for strategic approaches to conversation be in this ethical chatbot? If ethical behavior is defined to include telling stories that convey wisdom, then yes. Roger Schank's work is of great interest in that case. His work in story based reasoning may be a worthy direction of study.
Adapting the foundations of ethics to the ever changing story inherent in every significant conversation may require planning. It is possibly unavoidably inherent every conversation when the intention of the participants is to do more than exchange pleasantries.
