# Language Model from missing data

I want to learn how a set of operations (my vocabulary) are composed in a dataset of algorithms (corpus).

The algorithms are a sequence of higher level operations which have varying low-level implementations. I am able to map raw code to my vocabulary, but not all of it.

e.g. I observe a lossy description of an algorithm that does something:

X: missing data Algo 1: BIND3 EXTEND2 X X ROTATE360 X PUSH Algo 2: X X EXTEND2 ROTATE360 

The underlying rotate operation could have very different raw code, but effectively the same function and so it gets mapped to the same operation.

I want to infer what the next operation will be given a sequence of (potentially missing) operations (regions of code I could not map).

i.e. I want a probability distribution over my operations vocabulary.

Any ideas on the best approach here? The standard thing seems to throw out missing data, but I can still learn in these scenarios. Also, the gaps in the code are non-homogenous--some could do many things, The alternative is to contract the sequences and lose the meaning of the gaps, or to learn an imputation.

• The question is trying to address too many things at once which should be handled as separate problems. The core idea is to connect a vocabulary to underlying sourcecode which is called in the literature “language grounding”. A common implementation for this task is done with if-then-statements. The second problem in the OP is about inferring the next action in a sequence. This can be implemented with hypothesis tracking in which different options are mapped against observation. – Manuel Rodriguez Jul 23 at 7:09