How would I go about designing a (relatively) simple AI that discovers and invents random more complex concepts on its own?

For example, say I had a robot car. It doesn't know it's a car. It has several inputs and outputs, such as a light sensor and the drive motors. If it stays in the dark, it's score drops (bad), and if it moves into the light, it's score rises (good). It'd have to discover that it's motor outputs cause the light input to change (because it's moving closer or farther away from a light source), and that brighter light means higher score.

Of course, it'd be easier to design an AI that does specifically that, but I want its behaviour discovery system to be more generic, if that makes any sense. Like later on, it could discover a way to fight or cooperate with other robots to increase its score (maybe other robots destroy light sources when they drive over them, and they can be disabled by driving into them), but it'd have to discover this without initially knowing that another robot could possibly exist, how to identify one, what they do, and how to interact with one.

Also, I want it to be creative instead of following a 'do whatever is best to increase your score' rule. Like maybe one day it could decide that cooperating with other robots is another way to increase its score (it finds out what love is), but if it's unable to do that, it becomes depressed and stops trying to increase it's score and just sits there and dies. Or it could invent any other completely random and possibly useless behavior.

How hard would it be to make something like this, that essentially builds itself up from a very basic system, provided I give it lots of different kinds of inputs and outputs that it can discover how to use and apply to its own evolving behavior?


This is not an answer. I couldn't comment, so here are some remarks about your question: This is a very broad question, and considered The Holy Grail for building artificially intelligent systems - meaning that some scientists have been dreaming about this since time immemorial.

Some homework is warranted from your side; you could have offered some of your solutions or maybe identified the multiple layers to your query, as they invoke concepts from many fields within the study of AI (or AGI to be general).

For example, the following layers are taken up around the big-words in the question, though rhetorically -

  1. On evolution - what registers as evolution (is only genetic mutation evolution or should it involve some form of natural selection or seeking a niche for increasing the chances of survival or etc). How would evolution of software look like? Should it be able to modify its own code in the process of evolution?

  2. On concept - what constitutes a concept, identifying from the environment a concept by calculating its relevance (w.r.t other concepts), coming with a process of selecting the concept for use in a given environment (natural or artificial).Referring to the example in the question, is the road a relevant concept or the trees and the sky and the bees pollinating some flowers along the roadside? What is a more fundamental concept - the trees or the bees, and how does one measure that?

  3. On rewards - for us humans, rewards are maximization of survival of genetic machinery they carry (translating to reproductive success). what reward system should we come up that machines could use to increase their chances of survival in the physical world? what value should be put on a reward for moving the car on a straight line, or moving from darkness towards light? Shouldn't moving from darkness to light be more fundamental action (concept) and therefore rewarded higher than moving along the straight line? However, given that the car has learned to move from darkness to light, shouldn't the value of the reward be lowered so that it can learn other actions/concepts?

As can be seen from this brief detailing, there are layers within layers. Therefore, it is proper to establish now that there are no simple answer to this essay-requiring soul-searching time-eating heavy-research-wanting question. However appropriate directions can be given about work that is being done by some of the most prominent scientists of our times. The thing that one is looking for are known as Universal Problems Solvers, such as Gödel Machine (by Jürgen Schmidhuber) and AIXI - Artificial Intelligence (AI) based on Solomonoff's distribution ξ (by Markus Hutter).

Here is a quote lifted from Wikipedia page on AIXI that is pretty self-explanatory on how it maximizes the rewards over time.

AIXI is a reinforcement learning agent. It maximizes the expected total rewards received from the environment. Intuitively, it simultaneously considers every computable hypothesis (or environment). In each time step, it looks at every possible program and evaluates how many rewards that program generates depending on the next action taken. The promised rewards are then weighted by the subjective belief that this program constitutes the true environment. This belief is computed from the length of the program: longer programs are considered less likely, in line with Occam's razor. AIXI then selects the action that has the highest expected total reward in the weighted sum of all these programs.

Gödel Machine goes further - it allows the agent to modify its own code that allows it to maximise the rewards of its actions - which is to modify its own code that allows it to maximize the rewards of its actions - as so on. This is kind of a recursive definition, that simulates evolution (a rapid one) by choosing to evolve that code/state of the agent that converges towards a code that is superior to the code that the agent is currently running.

Here is a quote lifted from page on summary of Gödel machine. See that Hutter is also referenced (discovered of AIXI above).

Our Gödel machine will never get worse than its initial problem solving strategy, and has a chance of getting much better, provided the nature of the given problem allows for a provably useful rewrite of the initial strategy, or of the proof searcher. The Gödel machine may be viewed as a self-referential universal problem solver that can formally talk about itself, in particular about its performance. It may "step outside of itself" (Hofstadter, 1979) by rewriting its axioms and utility function or augmenting its hardware, provided this is provably useful. Its conceptual simplicity notwithstanding, the Gödel machine explicitly addresses the `Grand Problem of Artificial Intelligence' by optimally dealing with limited resources in general environments, and with the possibly huge (but constant) slowdowns buried by previous approaches (Hutter, 2001, 2002) in the widely used but sometimes misleading O()-notation of theoretical computer science.

The main limitation of the Gödel machine is that it cannot profit from self-improvements whose usefulness it cannot prove in time.

Reading the papers of the stated research would certainly be useful in finding answers to some of the layers in the question. There is also good work going on in the AGI community that somewhat aligns with the direction. Hope this helps.

  • 1
    $\begingroup$ This starts with "this is not an answer", then goes on to be an excellent answer that turns around the premises of the question. That is called a "frame challenge" and IMO a perfectly acceptable way to answer when the OP presents a top-level idea that needs a step back and context added. $\endgroup$ – Neil Slater Mar 29 at 8:44

The idea of pull yourself out of the swamp doesn't work. That means, the robot can't determine by it's own what the desired behavior is. That means, it's a “hen egg problem”: The robot doesn't know which actions are resulting into additional score, and without measuring the score, it's not possible to determine which actions are needed next.

Overcoming the problem is possible by recognizing the boundary of the system. The robot is the inner system and the environment is the outer system. The outer system formulates requirements in natural language. The requirements are given by a project. For example, if the competition is called micromouse, than the robot has to drive in a maze. It's up to the robot to fulfill the requirement, this allows him it to increase the score.

A potential alternative is to evolve the robot's behavior and the requirements of the outer system in parallel. That means, in step 1 the game is easy to solve and only if the robot has won the level, the system will make the next level a bit harder. In level 2, the robot has to adapt to the new requirements. The inner and outer system are in a progressive loop.



Self-evolving AI, although they wouldn't have used those terms, was part of the visions of Leibniz, Church, Turing, Shannon, Minsky, and others. It is the basis for genetic algorithm research and the foundation of the fantasy of the singularity — fantasy because, if we honestly evaluate our postmodern culture, technology long ago achieved domination over humanity in the realm of autonomy.

Very few people decide their technology. They may chose their brand or decide on a car versus a train or an iPhone versus an Android, but technology is the more autonomous of technology and humanity in the common scenario. Jaques Ellul proved that in a hundred different ways in his Technological Society years ago.

The singularity is also farcical because its conception is based on the idea of general intelligence, a concept that is fundamentally flawed. Humans do not exhibit strictly general intelligence and machines cannot. Such is easily provable, and the outline of the proof is remarkably simple, especially for those that have studied abstract mathematics. In lay terms, it is reasonably intuitive:

There is always at least one problem for which the generality previously exhibited falls short of being sufficiently general.

This is a consequence of Godel's incompleteness theorems if not self-evident from simply thinking through some specific cases.

Still, much can be accomplished with computers, and AI capabilities have emerged and will continue to do so. Great progress has been made with artificial networks, rules engines, fuzzy logic control, automation in flight systems, brain imaging, comprehension of genetics, and comprehension of the nature of phenomena in the biosphere.

In that regard, we cannot assume self-evolution is possible in AI. It does not occur in nature. The DNA life-print of an organism is indetachably bound to the conditions of the biosphere as experienced by the organism and its entire ancestry.

It is also impossible to discover a more complex concept randomly. There is a mystical belief that random walks lead to discovery, but that is because the walk appears random. The walk, the use of the senses in determining direction, the decision to stop and study — these are all far from random. But if we change the intention slightly, feasibility appears.

How would one go about designing a simple AI system that enumerates concepts in a domain with regard to finding a solution to a specific problem or set of problems and then evaluates their feasibility and the likely return on the time investment into developing the solution?

The technological foundations that are likely to lead to this monumental advancement are these.

  • Genetic algorithms
  • Directed graph structures, traversals, and related search optimization
  • Representation of semantics and associations in graph structures
  • Ways to compare such representations
  • Ways to represent generality
  • Ways to generate examples from those generalization representations
  • Recursive artificial network strategies to develop semantic from time series data
  • Fuzzy logic engines (since actual cognition is marked with doubt about rules)
  • Parallel computing models to drastically reduce the cost of computation
  • Learning circuits that are analog in forward path but digitally controlled
  • Computer language constructs that are more declarative and less explicitly algorithmic
  • $\begingroup$ Fuzzy logic claims to be the answer to everything, also for self-evolving systems. Sometimes the papers are published in so called predatory journals which is equal to charging publication fees to the authors without any peer-review. $\endgroup$ – Manuel Rodriguez Jan 20 at 22:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.