Do full-text translators such as DeepL or Google Translate fall under the term "Generative AI"?

Question

My question relates to full-text translators that are not specifically based on LLMs. My current understanding is that the term Generative AI goes beyond LLMs and that the full-text translators (especially those which are based on artificial neural networks) also fall into this category.

In the Wikipedia article about Generative AI, I could only find the statement that generative AI systems such as ChatGPT are also used for translations. That is quite obvious, but this does not answer the question of whether other full text translators are also commonly referred to as Generative AI.

IBM research defines Generative AI as

Generative AI refers to deep-learning models that can generate high-quality text, images, and other content based on the data they were trained on.

AFAIK DeepL, Google Translate, Bing Translate but also a few not so well-known systems falls clearly into that definition.

My question was triggered by an answer on Meta Stack Overflow that contained the following sentence:

Besides, the banner clearly states: "Answers generated by artificial intelligence tools". Translations aren't generated answers. They're translations.

with the implicit conclusion that it is clear to everyone that pure translators do not count as generative AI, hence it does not need any further explanation. Other participants in that thread seem to agree to that point of view. However, I think their use of terminology is not the typical use in the field of AI, and I would like to hear what the experts say.

Answer 1

Generative AI, as defined by IBM research, refers to deep-learning models capable of creating new content, be it text, images, or other media, based on their training data. This definition indeed encompasses models like GPT-3 or GPT-4, which can generate text in various styles and formats, including translations.

However, when it comes to full-text translators like Google Translate, DeepL, or Bing Translate, there's a nuanced difference. These systems are typically based on neural machine translation (NMT) models, a specific application of deep learning tailored for the task of translating text from one language to another. While these NMT systems are indeed 'generative' in the sense that they produce new text in a target language, their primary function is not to create original content but to convert existing content from one language to another as accurately as possible.

Answer 2

The distinction between "generative" and "non-generative" AI isn't an especially useful one. A language model (to first approximation) takes a sentence as input and tells you how probable it is that a native speaker would phrase it that way. Any remotely decent translation system is going to contain a language model, either as a separate component or integrated into the translation model proper. This helps ensure that the text it outputs is grammatically correct and scans well. Without a language model, you get a word-for-word translation that won't sound natural at all.

But if you have a probability distribution over the next word given the previous words, not only can you answer how probable an existing sentence is, but you can take a series of words and sample the next word in proportion to its probability under that model. You then put that word in the context and generate a new one, and so on. Any model that can evaluate the probability of a sentence can be trivially modified to generate a sentence. Similarly, you can take any generative model and output the distribution over the next word instead of just selecting a single one; the product of these successive distributions will be the joint distribution over sentences of that length. The models are exactly the same in either case; only the user interface differs.

(Of course, when setting policies, those user interface details might actually matter, since we might want to encourage users to fix their grammar but discourage them from posting answers when they have no idea if they're correct. But those details are entirely uninteresting mathematically.)

Historically, the use of language models in things like machine translation and voice recognition are the serious uses, and using them to generate text is a neat side task that can be used to sanity check the sort of sentences it thinks are representative of the model. (Text generators can also be fun toys, of course, which is how they're being used in things like ChatGPT.)

Answer 3

Actually it depends, generative models are specific kind of machine learning models. Generative often means a model that models the probability distribution of data $p(x)$, you cannot do translation using this kind of model.

But also you can build conditional generative models, where the probability of data points $x$ conditioned to some input $y$ is modeled $p(x \, | \, y)$, and in this case it is possible to perform a translation task using a generative model, where $x$ is the translated text, and $y$ is the original text.

So it all depends on the model you use for the task, if you use a (conditional ) generative model to perform translation, assuming that the model is trained for that task (like ChatGPT), then yes, translation can be done using generative models.

The concept of translation in AI/ML is more general, to just produce an output given an input, for example, CycleGAN is a generative model that translates between two sets of unpaired images, from horse to zebra and viceversa. And GANs are generative models.

About the specific question on DeepL and Google Translate, it depends on what underlying model they use, if at some point they use generative models to perform translation, then they should also be flagged as generative AI.

In general the concept of Generative AI has... deformed over time... (due to hype). Generative AI has existed long before LLMs and Difussion models like Dall-E and Stable Difussion, just look at Generative Adversarial Networks (GANs),

Answer 4

The original question has been updated so my answer is being updated to reflect this.

The question being asked can be boiled down to just "What is it that makes something a generative AI?" I am quite familiar in my field of computer vision with Generative Adversarial Networks, but I believe after looking at Large Language Models that they operate on a similar principle. I will go over both of these.

Generative Adversarial Networks

First I will go over the GAN which I know well, though I have not implemented one in almost 8 years. In a GAN, you have 2 machine learning models, one is generative, and one is not. There is a non-generative model called the discriminator who's job is to determine if an image belongs in the dataset or not. Then you have the generative model who is considered an attacker or adversary, who is there to try to fool the discriminator, by producing an image that the discriminator thinks belongs to the dataset, but it does not.

During training we cycle between training the discriminator on the dataset until it cannot be fooled. Then we train the adversary until it can fool the discriminator. Then we add the adversary's images to the dataset and go back to the start again. We continue this process until the discriminator can no longer be trained to tell the difference any more, and the adversary wins. Now the GAN is complete and we discard the discriminator.

The GAN can be altered to have an input parameter where you can select a class, and it will attempt to create an output of that class to fool the discriminator with that class. This is an optional feature, but something many GANs try to implement.

LLM vs GAN

LLMs on the other hand looks to only have one model. If you dig into it though, you will find out there really is 2!.

Creating LLM Vector Stores

The Vector store is a model too! It is a generative model that provides an output to the final model that interprets the output. So you have a model that is trained on a large language database, and a vector store that is trained to output segments of words similar to the input it is given. Both the input you give and the input the vector store outputs is given to the regular model, and it takes that and provides an output to you.

So mechanically, what ties these two types of generative AI together? How can we create a mechanical definition? You have a normal network, either a discriminator, or interpreter, and then you apply to it a generative network, either a adversary or tokenizer(?) respectively during training. When you are done, either you keep them together in the latter case, or you discard the discriminator and publish the adversary in the former case, and you have a generative AI.

For older translation models, they likely did not have the processing power, or technology available to perform these kinds of tasks. This is a relatively new field. They would not have been able to be generative AI. They at most would be comprised of one model during both training and publishing.

Old answer continues below:

> [arXiv:2307.15208][4] states "Generative AI refers to a set of
artificial intelligence techniques and models designed to learn the underlying
patterns and structure of a dataset and generate new data points that plausibly
could be part of the original dataset."

Considering this, what we see is that a generative AI differs from a traditional AI in that it creates new information where none was before. A traditional AI might detect which paintings belong to Raphael, while a generative AI might try to create a new painting in Raphael's style.

I think your misconception here is that all neural network AIs produce information of some kind, but they do not produce new information. The information was already there in the input in some form in a traditional AI. 

 - The location of the dog was in the picture already, the neural network just pointed to it and said, "Here it is." 
 - The tune matched a Metallica song in a database already, the neural network just correctly identified it.
 - The information was already encoded in a foreign language, the neural network just decoded the information and then reencoded it into a new language of your choice.

Now if the model had instead read the foreign language, decoded the information, and used it as a prompt to write a novel in the language of your choice, the contents of which included new information never before seen in that input, now we are talking about generative AI. This is in fact what they do.

A lot of the media talks about how chat GPT can hallucinate it's answers when you ask it straight forward questions, but the truth is, it is generative AI. It is giving you answers that could have conceivably been in the answer set. They just aren't. This is what some experts mean by it hallucinating up new answers. It's pulling them out of the void like a generative AI is supposed to do. It just isn't what the average person expects this type of program to do.

In short, what the difference between a traditional AI and a generative AI is that a traditional AI learns a dataset, and tries to tell you something about the input with respect to the dataset. The generative AI takes the input and tries to produce something similar that it thinks would be from the same grouping in what it believes the dataset should be.

Answer 5

No, they are not "generative".

As clearly explained by Wicket on the GenAI Beta site, sourced from SirBenet's answer on the same site, Google Translate isn't "generative":

output could be considered too tightly defined by the input text

These translation tools don't generate new text. They translate.
Sure, they use AI to infer meaning to result in better translations, but you can't tell them to "Translate Macbeth to Swahili"...

You'll never get more out of the tools than you put into them.