9
$\begingroup$

An AI agent is often thought of having "sensors", "a memory", "machine learning processors" and "reaction" components. However, a machine with these does not necessarily become a self-programming AI agent. Beyond the parts mentioned above, is there any other elements or details necessary to make a machine capable of being a self-programming AI agent?

For example, a paper from 2011 declared that solving the optimization problem of maximizing the intelligence is a must-have feature for the self-programming process, as quoted below:

A system is said to carry out an instance of self-programming when it undergoes learning regarding some element of its "cognitive infrastructure", where the latter is defined as the fuzzy set of "intelligence-critical" features of the system; and the intelligence-criticality of a system feature is defined as its "feature quality," considered from the perspective of solving the optimization problem of maximizing the intelligence of a multi-feature system.

However, this description of "optimization of intelligence" is vague. Can anyone give a clear definition or better summary for the necessary components for self-programming agents?

This question is from the 2014 closed beta, with the asker having a UID of 23.

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
5
$\begingroup$

At the highest level, all it needs is for the various systems already discussed to incorporate code objects. If it can interpret its source code / model architecture from the formatted text objects underpinning them, can 'understand' them in terms of having a useful ML model, and alter the code with its reaction, then it can self-program.

That is, the basic loop behind a recursively improving intelligence is simple. It examines itself, writes a new version, and then that new version examines itself and writes a new version, and so on.

The difficult component comes at lower levels. We don't need to invent a new concept like 'sensor,' what we need to do is build very, very sophisticated sensors that are equal to the task of understanding code well enough to detect and write improvements.

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ Although the knee-jerk computer science reaction to statements about systems that understand their own code is often to cite the halting problem, it turns out that AI approaches do have something useful to say about that: cs.stackexchange.com/questions/62393/… $\endgroup$
    – NietzscheanAI
    Commented Aug 18, 2016 at 20:06
  • 3
    $\begingroup$ Right, the Halting Problem is a no-go theorem for fully understanding all possible code, but doesn't stop one from having a good understanding of most code that you actually come across. $\endgroup$
    – Matthew Gray
    Commented Aug 18, 2016 at 20:18
  • $\begingroup$ Realistically the halting problem only really applies to 'turning machines' which are pure mathematical constructs that can't really exist (they require an infinite tape for unlimited memory for example) and can run for an infinite time. Real world computers have limited amounts of memory. There are ways to write software that can be formally verified (Idris, Coq). Using dependent types. Limit the size of an array (ie < the amount or ram). Not allowing a program to modify itself in memory in a way that could violate the formal proofs. No infinite loops. No byte loop/divide by zero. Etc... $\endgroup$
    – David C. Bishop
    Commented Nov 1, 2016 at 7:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .