1
$\begingroup$

I have a basic understanding of how neural networks work, and I have always thought that those chatbots work in a similar way (but I might be wrong): they take an input, shape it in a way that can be processed, go through a series of layers in a neural network model (with weights based on their previous training), and then give an output.

In the published conversation with Blake Lemoine, the LaMDA chatbot makes a reference to a “previous conversation”: is this memory somehow stored in the weights of the neurons, or is there a physical memory from which it can read?

What goes out of my understanding in this particular situation, is how the notion of a "previous conversation" can be processed by a similar system: is this memory somehow stored in the neurons of the neural network (in this case the training would continue also during the conversation), or do they have some kind of memory storage that they can access during the evaluation of their inputs?

Here is the article where I've taken the link to the conversation: https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/

$\endgroup$
0

1 Answer 1

3
$\begingroup$

What goes out of my understanding in this particular situation, is how the notion of a "previous conversation" can be processed by a similar system: is this memory somehow stored in the neurons of the neural network

Although I cannot answer what specific "memory" LaMDA may have (but I suspect it is either nonexistent or very simple), I think I can address this point.

The AI likely has no "notion" of anything here. It has access to a very large and complex statistical language model, where conversations that allude to prior conversations may happen from time to time. It is therefore capable of outputting text with similar-looking references, adjusting for the current context in the flow of conversation.

If you asked the AI how much it enjoyed the summer of 2015, depending on what previous prompts (and any pre-prompt warm up*) were, it would likely make a good attempt at "remembering" what it did then, even though it did not exist. It would draw statistically on other texts related to that time, and might invent places, refer to real events that it may of attended etc. A lot will depend on how much pre-prompting is applied that forces the AI to answer questions logically "knowing" that it's an AI, but that pre-prompting can often fail later in a conversation.

Typically a chatbot built using one of the current crop of cutting-edge language models will fail to be self-consistent and coherent in the long term unless the person interacting with it is helping it along or willing to ignore the failures. With older chatbots this could happen very quickly. With GPT-3 and probably LaMDA, it may take a slightly longer conversation.

I find it interesting that the journalist in your linked story managed to get a complete opposite of the Engineer's experience:

In early June, Lemoine invited me over to talk to LaMDA. The first attempt sputtered out in the kind of mechanized responses you would expect from Siri or Alexa.

“Do you ever think of yourself as a person?” I asked.

“No, I don’t think of myself as a person,” LaMDA said. “I think of myself as an AI-powered dialog agent.”

Afterward, Lemoine said LaMDA had been telling me what I wanted to hear. “You never treated it like a person,” he said, “So it thought you wanted it to be a robot.”

The above simple exchange is pretty good proof that the engineer is suffering from confirmation bias, IMO.

Related: The Eliza Effect.


* A pre-prompt warm up is a piece of text that "frames" the start of interaction with a large language model so that it can perform a specific task. At the start of any conversation, a paragraph of context is fed into the language model to set the statistics for a certain kind of document. Open AI's GPT-3 uses this a lot to support different tasks.

A simple pre-prompt might look like this:

The following is a conversation between a human and AI. The AI is knowledgable, friendly and helpful.

AI: Hi, I am AI! How can I help you today?

Human:

At that point control will pass to the human - outside of the AI - so they will add their first sentence that will become part of the growing prompt history. Note that the AI, if left to its own devices would begin to speak for the human too, naively continuing to predict the conversation unitil its end! In fact it does often do so, but the chatbot code is set to catch obvious end points such as newlines, or the AI actually outputting "Human: ", and will force the mode of conversation back to looking like a chat. It is really quite crude.

LaMDA's architecture may not do exactly the same as GPT-3, in fact I would suspect it is different in some key ways. However, it is very likely that there are still two "layers" in play - a very large sophisticated language model trained on gigabytes of human text, and a control layer which makes the language model conform to a chat scenario. Even if the control layer is more sophisticated than GPT-3's pre-prompting, it will have similar limitations.

$\endgroup$
3
  • 2
    $\begingroup$ Good example of using pre-prompting on GPT-3 here: youtube.com/watch?v=mgEYP_OS418 $\endgroup$ Jul 7, 2022 at 17:18
  • $\begingroup$ Why wouldn't Lemoine's argument also apply to his own conversation with LaMDA? That would invalidate his thesis. LaMDA tells Lemoine it is a person because that is what the engineer wants to hear, anyone could tell. $\endgroup$ Jul 28, 2022 at 22:42
  • $\begingroup$ @JaumeOliverLafont Yes, exactly my point. Confirmation bias very much on show there. There's probably a better term for rejecting/altering evidence that opposes a view (confirmation bias is usually quoted when accepting only evidence that supports a view). $\endgroup$ Jul 29, 2022 at 8:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .