This is a follow-up question from my previous question here. I'm new to ML/DL, and one thing I need to do is to use a machine or deep learning video attention model which as the name suggests, can tag which parts of a video is probably more interesting and absorbs more viewer attention.
Do we have an available model to do that? If not, how to do it?
A video attention model isn't only detecting the image itself, but it simulates the movement of the human eye on a given picture. It can't be realized as a technical algorithm alone but has to implemented as a cognitive model because the common goal is to replicate human eye movements. Creating a video attention model is usually done with the help of an eye‐tracking dataset which is feed into a neural network.
The longer explanation is, that at first a human participant is needed, who has to the task to do a task on the screen. During the task, his eye movements are recorded with a hardware device who monitors at which position the human is watching. The generated corpus is the ground truth and the attention model has the obligation to replicate the movements. At first, for the original image, but it should also simulate the human eye movement for novel pictures unseen before.
In most cases, gaze tracking is combined with cognitive architectures like SOAR and ACT-R because it's not possible to simulate the eye tracking process alone and ignoring other aspects of human cognition like short term memory. The quality of the attention model is measured how well the software replicates the eye tracking corpus.
