6
$\begingroup$

We have the popular TextRank API which given a text, ranks keywords and can apply summarization given a predefined text length.

I am wondering if there is a similar tool for video summarization. Maybe a library, a deep model or ML-based tool that given a video file and a length, it ranks frames, or video scenes/shots. I'd like to generate a short summary of a video with visual features.

$\endgroup$
2
  • $\begingroup$ as in video -> text or frame ranking? $\endgroup$
    – mshlis
    Commented Aug 13, 2019 at 22:07
  • $\begingroup$ Frames, or Shots of the video. $\endgroup$
    – Mary
    Commented Aug 13, 2019 at 22:17

1 Answer 1

1
$\begingroup$

From "Deep learning-based video summarization — A detailed exploration" by Surya Remanan:

Video summarization can be considered as the process of distilling a raw video into a more compact form without losing much information. In a general video summarization system, image features of video frames are extracted, and then the most representative frames are selected through analyzing the visual variations among visual features. This is done either by taking a holistic view of the entire video or by identifying the local differentiation among the adjacent frames. Most of those attempts rely on global features such as colour, texture, motion information, etc. Clustering techniques are also used for summarization. Video summarization can be categorized into two forms:

  • Static video summarization (keyframing) and
  • Dynamic video summarization (video skimming)

Static video summaries are composed of a set of keyframes extracted from the original video, while dynamic video summaries are composed of a set of shots and are produced taking into account the similarity or domain-specific relationships among all video shots.

Following is an attention-based video summarization model - PyTorch implementation of the ACCV 2018-AIU2018 paper Video Summarization with Attention

There is a video summarisation focused models using reinforcement learning - Unsupervised video summarization with deep reinforcement learning (Theano)

There is an LSTM - GAN based approach to video summarization - video summarization lstm-gan pytorch implementation

Microsoft Bing Search has come up with a video summarization technique using thumbnails. Intelligent Search: Video summarization using machine learning

$\endgroup$
2

Not the answer you're looking for? Browse other questions tagged .