How to combine several chatbots into one?

Question

I'm in the middle of a project in which I want to generate a TV series script (characters answering to each other, scene by scene) using SOTA models, and I need some guidance to simplify my architecture.

My current intuition is as follows: for a given character C1, I have pairs of sentences from the original scripts where C1 answers other characters, for example, C2 (C2->C1). These are used to fine-tune a data-driven chatbot. At inference time, the different chatbots simply answer each other, and, hopefully, the conversation will have some sense.

This is however unpractical and will be kind of a mess with many characters, especially if I use heavy models.

Is there an architecture out there that could be used for conversational purposes, which could be trained only once with the whole dataset while separating the different characters?

I'm open to any ideas!

Answer 1

What you want to do is you have some character like hulk, thor, caption America, iron man, etc. and you want to Train a response generator for each character like for thor on his dataset, for hulf on his dataset etc... and then you wanted to make a conversation. If I Understood you well then.

• You can fine-tune GPT-2 small or GPT-2 Medium using your Dataset for each character. Reference for Finetune

• You can decode using simple nucleus sampling at each time step. Use greedy nucleus sampling multiple times in parallel to generate multiple responses. You can generate 30 such responses and also use the last 3-5 dialogue turn as a context or Short term memory. Decoder reference

• To Choose the best response out of all generated responses(30 responses as an example). you can use cosine similarity between generated responses and query + last 3-5 dialogue turn.

  def cosine_similarity_nd(embd1, embd2):
  numerator = np.multiply(embd1, embd2)
  numerator = np.sum(numerator, axis=1)
  eucli_norm_1 = np.sqrt(np.sum(np.power(embd1, 2), axis=1))
  eucli_norm_2 = np.sqrt(np.sum(np.power(embd2, 2), axis=1))
  denominator = np.multiply(eucli_norm_1, eucli_norm_2)
  denominator = denominator + 1e-10  # remove zeros
  cosine_similarity = np.divide(numerator, denominator)
  return cosine_similarity.reshape((-1))
  

• Or you can train a reverse model for either all character or mix all character dialogue and use this model to calculate loss of all responses with your query. Reference reverse model

• Or you can Combine result of cosine and reverse model to find response out of all the responses.

Answer 2

Define features that describe C1 and C2 (like their ids) and add these features to inputs of your model. Thus your only model will be able to generate the next line for any pair of (C1, C2).