Let's assume we collect a high quality amount of training data for machine translation for example parallel corpus data from the european parlament combined with other data. We store these texts in a database, like a graph data base, which is a fast operation even for data of size 100 gb.

Now, when we have an input text that we want to translate into an other language, we lookup the relevant training data in the graph and limit it to some constant size. Maybe, we also additionally collect some other constant size of texts to learn the general grammar.

Now, we have collected relevant training data in constant runtime and can use this to train a model at inference time using slow state of the art solutions that become fast enough for small inputs.

We have created a fast to train and adaptive model for machine translation that will be slower at inference time but fast enough.


  • What are the downsides of this approach?
  • Can it nearly achieve the quality like training the whole data set
    • for machine translation
    • for other text related problems
    • for other problems like image-related

Thanks in advance!

  • $\begingroup$ What is your reference implementation for "a fast to train and adaptive model for machine translation". The question title mentions neural networks. These are usually very slow to train, so I would like to see some example of the kind of system you have in mind. Also what kind of response time you are targeting - is it under a second, a few seconds, up to an hour? $\endgroup$ Commented Jul 1 at 7:06
  • $\begingroup$ With "fast" I mean fast enough for a normal user, like 2 minutes, at worst 10 minutes, at best less than 60 seconds. The model should use a normal state of the art approach like neural-network-based or seq2seq (I'm no expert here, but we could just look one up based on new papers and comparisons for the specific task). Since the training data is guaranteed to be constant time and rather relevant due to graph retrieval, the model is assumed to have train+inference in constant time and yield a quite ok answer. We assume an average notebook hardware. $\endgroup$ Commented Jul 1 at 7:15
  • $\begingroup$ So, to be clear, you don't know of any constant-training-time neural network suitable for the task, you are simply hoping that one can be made? You are also using "constant time" really loosely, when you mean "fast", for all the components. $\endgroup$ Commented Jul 1 at 7:39
  • $\begingroup$ I think you misunderstand the title. Constant time does not mean that the neural network training is constant time. Insertion Sort is the slowest possible sorting algorithm for average N, but the fastest for small N. Training a neural network (obviously) requires O(N*p) runtime where p depends on the parameters. Since N is guaranteed to be constant, the training time is constant and independent on the size of our training data. Thus, I say, it does not matter at all, which model we use, we just use some slow state of the art model, which becomes fast (1 to 10 minutes) for small N. $\endgroup$ Commented Jul 1 at 7:45
  • $\begingroup$ Let's just assume that runtime is not a problem (it won't be a problem). My question is mainly about the quality of the results (answers). It's also ok if someone can give me the name of this approach maybe with a link to a paper that shows the downsides. In my Data Science courses I never heard this method. We learned: train all data first to one model, wait a few days, than have fast inference. I don't want to "wait a few days". $\endgroup$ Commented Jul 1 at 7:50


You must log in to answer this question.