4
$\begingroup$

Every week I will get a lot of videos from a game that I play, outside the game where you throw wooden skittle bats at skittles, and then I will cut videos, so that, at the end. there is video only about throws.

The job is simple and systematic. I have a lot of videos, so I was wondering:

  • Is it possible to teach AI to cut videos from the right place?

I was thinking to ask help or guidance where to start to solve this problem.

I can't use only sound, because sometimes you can hear skittles hit from outside of the video. I also can't just use movement activity, because sometimes there are people moving around the field. Videos are always filmed from a fixed stand, so it should make it easier. So, is it possible and where to start?

Here is another example: https://www.youtube.com/watch?v=sHu6yMBV3xU

$\endgroup$
  • $\begingroup$ Hi and welcome to this community! Let me try to understand your problem. You have a lot of videos while people are playing a certain game. So, you want to create an AI that edits your videos so that you have only the parts related to the "throws". You're saying that you can't use only the sound and the movement in the videos to train an AI, because it might not learn the expected patterns. Is this correct? $\endgroup$ – nbro Aug 11 at 21:56
  • 2
    $\begingroup$ The general rule of thumb is if a human could identify it with no problem, normally an AI can be trained to somewhat succeed. This sounds like there are some very simple and easily recognised identifying features (rapid movement of a limb, bright colour moving) so if I had to say, I would think an AI would be able to do this task with relative ease. As to how you program it is up to you. $\endgroup$ – Recessive Aug 12 at 3:54
  • $\begingroup$ @Recessive This is just what I thought. Some thing like this. I'm just wondering where to start this project.@nbro The sound and movement can help to find the right spot, where to cut, but they can't be the onlyones that triggers AI to cut video. $\endgroup$ – pinq- Aug 12 at 7:35
  • 1
    $\begingroup$ @pinq I've never doe anything with a video before, so I'm not sure where you would start. As a guess, I would look in to RNN's, probably LSTM's (as you will want to analyse a bunch of frames, say every 10th frame or something, every frame would likely take too long) mixed with a CNN. Look up projects people have done for feature recognition in videos before to get a grounding. $\endgroup$ – Recessive Aug 13 at 4:48
  • $\begingroup$ @pinq- This is a very exciting problem indeed. +1 $\endgroup$ – naive Aug 13 at 6:18
2
$\begingroup$

I believe the answer is yes, but the video edition is mostly programmatic. The AI part comes with detecting the right spots to cut.

  • You want to detect the right portion when someone picks a wooden skittle
  • you want to detect when the Skittles stop moving on the ground (to detect this, also when they start moving)

These will give you timestamps on the video. The next step is to add some padding and run the drop-down frame operations in the video, which can all be done with a simple script.

You might find this video very similar in concept, and maybe even as code source: https://youtu.be/DQ8orIurGxw

For the recognition of the objects in the video frames, there are lots of options. You might look into computer vision or object motion detection.

$\endgroup$
  • $\begingroup$ Yeah, I think this is the right way to proceed. You are right, I dont' need IA to "cut" it, but just to learn those two thing( picks skittle and stop moving). Have to start learn how to teach AI to recognize those. $\endgroup$ – pinq- Aug 19 at 6:45
  • $\begingroup$ @pinq- I'm glad to be of help. Depending on your level of knowledge you might want to train your own model or use existing products. Azure, Google Cloud and Amazon Web Services all provide great tools for AI development. $\endgroup$ – Alpha Aug 19 at 11:01
1
$\begingroup$

I'm thinking, you can input the video you are trying to edit and make it output the timestamps to cut. You will probably have to manually type in the timestamps that you would cut for that video, or maybe a keylogging program of some sort.

This works in theory, kinda, but I'm not sure how exactly though. The inconsistent input length is easy enough to deal with, but I'm pretty sure you cant have inconsistent output lengths (otherwise I would be using it long ago). Maybe just make like 100 nodes and make it output (timestamp, confirm) where if confirm is > 0.5 it is an actual cut it wants to make.

But inputting whole videos is never a good idea. It takes way too long and takes like 30 gigs of RAM(I had to use virtual RAM which is ridiculously slow) for like a 5 min video and crashes all the time. Any suggestions though?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.