We are working on the subject above, where a sequence of $n$ glass frames forms an example with an associate target that is a video of the glass future state. We would like to know if there exists an algorithm (ConvLSTM, Trasformer, etc…), that could be trained and that it is able to predict glasses breaking videos. In this moment, we are testing SimVPv2 (on MMNIST).

  • $\begingroup$ How do the glass images look like, are there like initial fractures somewhere, giving a hint that it may break in near future? $\endgroup$
    – NikoNyrh
    Commented Mar 15 at 16:47
  • $\begingroup$ Thank you! Yes, the glasses have already fractures. One of the initial tasks is to classify them between "broken" and "intact". Then a regression algorithm could be used to estimate the degree of rupture. The final task is to have a predictor that will give the next frame of the glass given a degradation video (perhaps with a ConvLSTM or a Transformer algorithm). $\endgroup$ Commented Mar 15 at 16:53
  • $\begingroup$ At present we are trying OpenSTL-SimVPv2 on MMNIST. We are unable to run in parallel the VPTR learner, due to our 32 GB RAM limit. $\endgroup$ Commented Mar 17 at 10:48


You must log in to answer this question.