1
$\begingroup$

I have an AI/ML challenge in relation to video analysis and am unsure where to start.

I am investigating an application that will grade students performance of carrying out a task, based on analysis of a video of them carrying out the task.

The problem has not been sufficiently defined yet but to get a high level idea, imagine video showing close up of a trainee doctor performing stitches to close a wound. the AI model would be trained using many videos of someone performing the stitches correctly and score the trainee on a number of criteria.

Most frameworks will allow detection of objects but taking a video of a person carrying out a task and assessing their success using an AI/ML model feels a step above regular object analysis.

Assumption is we will create the training material, having professionals video themselves carrying out the task successfully, which will also be graded by other professionals to provide a rubric of scores.

I understand this is not something that can be simply answered but an idea of where to start would be very helpful.

  • are there specific areas of AI i should investigate?
  • are there frameworks that can actually do this ( i have not found any)?

Appreciate any advice.

Thank you

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

You have not described exactly what the tasks will be, but there are some open source libraries for real time pose tracking. For example, OpenPose is one that can be configured to track the body, the hands and the face. However, this is only going to give you predicted pose information for each frame. If the subjects are meant to be doing specific tasks, e.g. picking up objects and moving them, OpenPose won't help and you may need to do further image recognition/analysis or object tracking in order to interpret what is going on.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the reply. the problem has not been sufficiently defined yet but to get a high level idea, imagine video showing close up of a trainee doctor performing stitches to close a wound. the AI model would be trained using many videos of someone performing the stitches correctly and score the trainee on a number of criteria. Ive updated the original post with this information. $\endgroup$
    – wilson_smyth
    Apr 23, 2021 at 10:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .