What AI concepts, topologies1, algorithms, or SaaS can be used to recognize a person eating a chocolate.

For this question, image recognition draws from a real time feed, validating each of these steps in sequence:

  1. Using a camera app, the user begins recording.
  2. The user focuses on the subject's face, at which time the app attempts to recognize the person2
  3. Person holds a single piece of chocolate (i.e M&Ms) to the camera lens, at which time the app attempts to recognize it.
  4. The user puts the chocolate into their mouth, chews, and swallows, at which time the app attempts to recognize that AS AN ACTION.
  5. The app gives a completion message indicating success or failure.

I understand that we can use real time recognition for each step, but I don't know if there are concepts proposed or tested to validate the scene as a sequence or any of the three recognition steps individually.

The app should invalidate the scene if the subject is swapped with another subject, if the subject does not swallow the chocolate, or there is some other deviation from the expected sequence above.


[1] By topology in this context is meant the standard mathematical meaning of the term applied to the higher level connections that are likely needed to recognize sequences of actions. In this sense, the use of the term topology is not at the level dimensions of neuron layers or convolution kernels. Since process topology is normally considered prior to considering library dependencies or deployment concerns, the term topology is more appropriate than architecture in this question. (First things first.)

[2] I've already identified SaaS options for facial recognition.

  • 6
    $\begingroup$ @DouglasDaseeco: I'm not voting either way, but the issue here seems to be the gap between problem statement (which I agree is well-defined in the question) and the solution which is left very open-ended. A question that had most of the architecture and technology decisions covered, but was struggling to put them together might work. As would one that focused on a single step of the process. But as it stands, this is a project problem statement asking for either an architect's solution, or a shopping list of applicable resources. $\endgroup$ Jul 23 '18 at 13:32
  • 2
    $\begingroup$ @DouglasDaseeco: There are 3 close votes so far. You still have time to attempt an answer here. Although that doesn't guarantee the question won't be closed, I have seen cases where a good answer turns round perceptions of the whole Q&A pair. Often the case where other experts think the original question was too broad or difficult, but the answer demonstrates it was not. $\endgroup$ Jul 23 '18 at 13:49
  • 3
    $\begingroup$ Welcome to AI.SE, Ahmad! Feel free to look over the edits made to your question and make your own adjustments if the current state doesn't convey what you're asking. Everyone else: long discussions in comments can be difficult to follow, so I encourage you to use Artificial Intelligence Meta or chat to discuss tags or question suitability. $\endgroup$
    – Ben N
    Jul 23 '18 at 17:12
  • $\begingroup$ @BenN Thank you! The edits made by DouglasDaseeco are valid, thank you for the edits, I could not rephrase it better than this. $\endgroup$ Jul 25 '18 at 14:23
  • $\begingroup$ If the question to be closed because what I am asking about is vague or very wide open, I am fine to close it. I had to do more googling after posting the question to find there are white papers written around a concept called action recognition which is a concept that might be applied to what I am asking for (maybe?), as it seems it will detect actions in the whole scene and based on it to produce outcome if the steps are valid and lead to "person eating M&Ms chocolate" scene. github.com/jinwchoi/… $\endgroup$ Jul 25 '18 at 14:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.