I have written my own basic convolutional neural network in Java as a learning exercise. I am using it to analyze the MIT CBCL face database image set. They are a set of 19x19 pixel greyscale images.
Network specifications are:
Single Convolution Layer with 1 filter: Filter Size: 4x4. Stride Size: 1
Single Pooling Layer 2x2 Max Pooling
3 layer MLP(input, 1 hidden and output) input = 64 neurons hidden = 15 neurons output = 2 neurons learning rate = 0.1
Now I am getting reasonable accuracy(92.85%), but my issue is that it is being achieved at very different points in the epoch count across network runs:
Epochs Training Accuracy Test Accuracy Validation Accuracy
Run 1 415 93.13 92.44 93.35 Run 2 515 92.44 93.18 92.84 Run 3 327 93.83 92.05 92.38
I am using the Java random class with the same seed for every run to initialize the kernel, the MLP weights and break the input data into 3 sets.(training is being done using the 33-33-33 method)
I am a loss as to what is causing this variation in epoch count to achieve the highest point in validation accuracy. Can anybody explain this?