Currently, i'm processing a image classification problem about facial emotional classification. I am using 2 extract methods: HOG and Facial Landmark. My idea is using HOG to find the gradient magnitude and oriented of the image and use facial landmark to find face keypoint. I thought i can fuse 2 method to make a better feature. But the new feature worse than HOG and better than facial landmark (same model to evaluate). I have some question:

  1. I wonder how i can fuse these two method where HOG normalization before and facial landmark return 68x2 pairs point interger.

  2. If can, should i normalize or something before fuse ? Which method i can try to fuse them (concat, add, multiply, ...) ?

  3. Is there anyway how to measure my method will be better or evaluate it ? I am also try to fuse HOG and SIFT (Bag of visual word) too.

I had tried fuse HOG and Facial Landmark feature but it get worse than HOG and better than Facial Landmark in the same model. I also fuse (SIFT) bag of visual word and HOG but it still worse than HOG and better than bag of visual word. Here is the code i use:

x_hogp_train = pca.transform(x_hog_train)[:,:382]
x_hogp_valid = pca.transform(x_hog_valid)[:,:382]
x_hogp_test = pca.transform(x_hog_test)[:,:382]

scaler = StandardScaler() # scale bovw feature
x_scale_bovw_train = scaler.transform(x_bovw_train)
x_scale_bovw_valid = scaler.transform(x_bovw_valid)
x_scale_bovw_test = scaler.transform(x_bovw_test)

# fuse them use concat
x_fused_train = np.concatenate((x_hogp_train, x_scale_bovw_train), axis=1)
x_fused_valid = np.concatenate((x_hogp_valid, x_scale_bovw_valid), axis=1)
x_fused_test = np.concatenate((x_hogp_test, x_scale_bovw_test), axis=1)



You must log in to answer this question.

Browse other questions tagged .