3
$\begingroup$

Is there a such thing (described in title), at least in research papers (not actual models)?

So far all LLMs that I know are autoregressive models.

$\endgroup$
1
  • $\begingroup$ The question you want to ask is "Do LLMs based on a diffusion model (as opposed to an autoregressive model) exist?" $\endgroup$
    – agemO
    Dec 26, 2023 at 10:11

1 Answer 1

7
$\begingroup$

This is a false dichotomy. Most diffusion models, including Dalle 2 and 3, already are transformers.

However, assuming you meant to ask if any language models use diffusion as opposed to a GPT, the answer is yes. For example, see this recent paper: https://arxiv.org/abs/2305.09515

$\endgroup$

You must log in to answer this question.