Training of LLM aka GPT models is clear on how is trained but can't find any info how is "mapped" query to internal tokens and generates response tokens more precise inference phase which consists of natural language processing and dialog management, with focus on how user input is computed to generate response tokens/content(dialog management part would be perfect but would work a general GPT inference too), mathematics and phases/steps are similar to traditional NLP models (matrix of vectors mapped to vocal vector)? Checked more resources but none provides a clear picture on how do work internally tokens, chatGPT and OpenAI chatGPT

  • $\begingroup$ Would you add more explain of "mapping" and "generating back"? Are you looking for Positional Encoding? Or is the mapping you mentioned that some functions at Tokenize data section at keras.io/examples/generative/text_generation_gpt? $\endgroup$
    – Cloud Cho
    Sep 13, 2023 at 18:12
  • $\begingroup$ Is more how user input is actually computed to generate response, kinda response back, how is computed (matrix and vectors in distributed manner to tind starting of sequence and keep generating based on tokens and then a sum of product on response to generate desired length token response) ? $\endgroup$
    – n1tk
    Sep 13, 2023 at 19:45
  • $\begingroup$ So you're looking inferencing (not training)? $\endgroup$
    – Cloud Cho
    Sep 13, 2023 at 21:40
  • 1
    $\begingroup$ Exactly, and in the question I do point out that training is very clear mathematically for me but can’t find math explanation related to inference phase which consists of natural language processing and dialog management … $\endgroup$
    – n1tk
    Sep 14, 2023 at 1:28
  • $\begingroup$ Is this what you looking for pi-tau.github.io/posts/transformer/#inference? $\endgroup$
    – Cloud Cho
    Sep 16, 2023 at 0:23


You must log in to answer this question.

Browse other questions tagged .