# How to exploit translational symmetry for extrapolation in video generation using machine learning

I'll try to rephrase my problem in the context of video processing. Imagine that initial frame of video has some translational symmetry. The frame evolves according to an update rule.

I generate a time series for how an edge, say right up edge, of the frame evolves. I generate another time series for how a larger edge, including the smaller right up edge, evolves. Since there is translational symmetry, I should be able to find how the smaller edge is related to the larger edge. The final goal is to use the obtained correlation to extrapolate to larger edges. I want to find the correlation between these two multivariate time series using machine learning (ML) methods.

I want to know

1 - which one of ML methods can be used in general for this task?

2 - if I use neural networks, the input and output shapes would be (values at time steps, number of variables). For the input it makes sense, but how can I define the output layer (for example, for LSTM in tensorflow)?

• Surely the sample correlation is just a direct calculation, with no ML needed.What do you mean by "correlation" ? And why do you think ML is needed? – Peter K. Jan 26 at 1:24
• @PeterK. I am not sure if the correlation is the right word. I updated the post to make it clear. – New Developer Jan 26 at 8:03