Could we code in Natural language once we pass the Turing test?

Question

Technically speaking, could we code in natural language once we pass the Turing test? Would passing the Turing test at least simplify programming languages' syntax?

Answer 1

Yes, we can program in natural language.

Specifications in natural language are normally neither precise nor complete. But when using a large language model like GPT-4, that is not a problem: The model makes assumptions for parts you did not specify.

Sometimes that is good enough, but in general, you need to refine your specification until it is complete and precise enough to fit the requirements you try to express in natural language.

Note that you do not need to fully understand your actual requirements from the start, you can incrementally refine them too.

To illustrate it, I will give a example using GPT-4:

(All prompts are shown, there is no further prompt engineering, it is zero-shot. Prompt: is what the programmer writes, GPT-4:, Output: is the output from the current version of the program executed as python.)

---

Prompt:

Write a python function that can concatenate some strings, and print the result with 10 characters per line.

GPT-4:

def concat_and_print(strings):
    concatenated = "".join(strings)
    for i in range(0, len(concatenated), 10):
        print(concatenated[i:i+10])

# Example usage
strings = ["hello", "world", "this", "is", "a", "test"]
concat_and_print(strings)

Output:

helloworld
thisisates
t

Prompt:

I want to give the strings as args.

GPT-4:

def concat_and_print(*strings):
    concatenated = "".join(strings)
    for i in range(0, len(concatenated), 10):
        print(concatenated[i:i+10])

# Example usage
concat_and_print("hello", "world", "this", "is", "a", "test")

Output:

helloworld
thisisates
t

Prompt:

add spaces

GPT-4:

def concat_and_print(*strings):
    concatenated = " ".join(strings)
    for i in range(0, len(concatenated), 10):
        print(concatenated[i:i+10])

# Example usage
concat_and_print("hello", "world", "this", "is", "a", "test")

Output:

hello worl
d this is 
a test

---

Note how the specification is not only incomplete, but also imprecise. Much less exact than natural langue could be. The specification add spaces does not say where the spaces should be added. Not even that they should be added to the output. It is not specified that they should be between the words, but the interpretation of it is what I meant.

(This is a real chat session with GPT-4, not cherry picked or edited.)

Answer 2

Controlled Natural Language (of the second type) can be used for programming, with no need to pass the Turing test.

Answer 3

No. Humans pass the Turing Test but cannot express themselves with enough precision and little enough ambiguity to code well in English (natural language). No machine will improve on that until it can greatly improve on human-level clarity in expressing their purpose when programming.

Is it possible to program in natural language? Yes, I think so. But it would require superhuman intelligence to anticipate all the possible confusions that might arise when a word is not sufficiently precise or accurate to represent an software activity or mechanism. This probably would require adopting a rigorous convention in choosing a subset of unambiguous words to form a "natural" programming language (a kind of creole). Coding consistently using only those words would require discipline that exceeds the ability of most humans, methinks.

Answer 4

I think this is a very interesting question. It's highly hypothetical, so my simple answer will also, necessarily, be hypothetical.

• If the AI can pass the Turing Test, it can be assumed it has a command of the human language(s) used in that process

Self-awareness in this context would seem to be irrelevant, as would the question of whether the AI actually "understands" the content, or is merely imitating natural language.

• If the AI has a command of the human language(s), it would be a reasonable assumption that it could translate natural language instructions into machine code

It might be best to think of the problem in the simplest terms. If you asked this hypothetical AI to "draw a red square on the screen", it's hard to see that task as impossible, or even difficult.

Obviously, as the instructions became more complicated, there would be a greater margin for error, and it would be useful to integrate precise, formal terms related to functions, as Randy points out.