# How to train my model using transfer learning on inception_v3 pre-trained model?

I am trying to train my model to classify 10 classes of hand gestures but I don't get why am I getting validation accuracy approx. double than training accuracy.

My dataset is from kaggle:
https://www.kaggle.com/gti-upm/leapgestrecog/version/1

My code for training model:

print(x.shape, y.shape)
# ((10000, 240, 320), (10000,))

# preprocessing
x_data = x/255
le = LabelEncoder()
y_data = le.fit_transform(y)
x_data = x_data.reshape(-1,240,320,1)
x_train,x_test,y_train,y_test = train_test_split(x_data,y_data,test_size=0.25,shuffle=True)
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

# Training

base_model = keras.applications.InceptionV3(input_tensor=Input(shape=(240,320,3)),
include_top=False,
weights='imagenet')
base_model.trainable = False

CLASSES = 10
input_tensor = Input(shape=(240,320,1) )
model = Sequential()
model.compile(loss='categorical_crossentropy',

history = model.fit(
x_train,
y_train,
batch_size=64,
epochs=20,
validation_data=(x_test, y_test)
)


I am getting accuracy like:

Epoch 1/20
118/118 [==============================] - 117s 620ms/step - loss: 2.4571 - accuracy: 0.1020 - val_loss: 2.2566 - val_accuracy: 0.1640
Epoch 2/20
118/118 [==============================] - 70s 589ms/step - loss: 2.3253 - accuracy: 0.1324 - val_loss: 2.1569 - val_accuracy: 0.2512



I have tried removing the Dropout layer, changing train_test_split, but nothing works.

EDIT:

On changing the dataset to color images from https://www.kaggle.com/vbookshelf/v2-plant-seedlings-dataset , I am still getting higher validation accuracy in initial epochs, is it acceptable or I am doing something wrong?

• If you set the validation set size to 50% instead of 25% do you get the same behaviour? Could be that dropout is turned off for validation, so validation loss will be slightly higher, perhaps for the first few epochs anyway – Mike NZ Apr 20 at 5:12
• @MikeNZ Yes I get same behavior on setting validation size to 50% – Shubham Agrawal Apr 20 at 11:04

The problem is that you're not creating a model with InceptionV3 as the backbone. What you want to do is this:

input_tensor = Input(shape=(240,320,1) )

base_model = tf.keras.applications.InceptionV3(input_tensor=Input(shape=(240,320,3)),
include_top=False, weights='imagenet')

base_model.trainable = False

x = base(input_tensor)
x = tf.keras.layers.GlobalAveragePooling2D()(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(N_CLASSES, activation='softmax')(x)

...

model = tf.keras.Model(inputs = input_tensor, outputs = x)

model.compile(...)


The inputs and outputs parameters are of importance here.

• The problem is I am using grayscale dataset but inception_v3 accepts rgb images, that's why I converted grayscale to 3 channel images by convolution operation model.add(Conv2D(3,(3,3),padding='same')) before inserting inception model layers in my model. – Shubham Agrawal Apr 20 at 10:38