From the paper (1) Facial expression analysis using CNN, the results using CNN show a 65% and 62% accuracy respectively, for emotion classification and state of mind identification.

Proposed Method:


The state of mind identification process is incorporated along with the Learning Management System (LMS) to have real-time learning adaptation. The web camera takes the video of the learner, from which frames (image of the learner) are grabbed at a frequency of 1 frame/sec. The video was split into image frames, whereby each image was analysed through a standard emotion detection API (Windows Azure). The emotion detection module uses the trained CNN classifier to detect the different emotions in the image. (CNN is used as a feature extracter and classifer).The state of mind identification module takes a sequence of 6 images as a window frame to identify the emotion pattern of the learner for the last 6 seconds. The learner's state of mind is identified from the derived emotion pattern.

Further, the identified state of mind is sent to the LMS as feedback of learner's emotion. Based on the identified state of mind of a learner, the LMS adapts the instructional or pedagogical method in real-time.

Factors believed to have decreased the accuracy of model:

  • Some of the captured images do not contain the whole face or the frontal view.
  • Frequent Movement of the learner lead to the instability in face recognition.
  • Poor lighting or very sharp lighting on face
  • Use of spectacles

Question: What all methods should I employ to improve the accuracy of the model which identifies the state of the learner using (2) Microexpressions ? What kind of preprocessing does the model require?

Currently, I'm thinking of using TensorFlow + Keras for training the CNN model using CASEME II dataset (drawback: limited data)

Reference papers:

(1) Facial expression analysis using CNN

(2) Microexpressions


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.