Why is the validation accuracy lower in case of CNN?

Question

I fed the same set of 1.4 million data to two different models:

1. MLP
2. CNN model

In both cases, I used the same parameters and hyperparameters.

The CNN is showing comparatively lower accuracy (80%) than that of the MLP (82%).

Why?

And, also, what does this experiment tell us?

Edit:

Is the data images, video or audio, or other grid-based signals (same signal repeated at multiple locations with a meaningful distance metric between them) in your case?

It is protein C-alpha distances data that has 3 classes (helix, strand, coil) and n-number of features, where n is an even number.

In fact, what is it, what was your expectation of the relative performance of the two models, and why?

I thought CNN would be more efficient and thereby would demonstrate better test/validation accuracy.

Answer 1

To get a full understanding of your problem, one would like to know what approximately the $n$-features are.

Whether, it is about the geometrical structure, protein is described by a graph, where vertices correspond to atoms and edges to bonds within them - I would consider use of GraphNN, there is some research, that has demonstrated the success of GraphNN for protein prediction:

• Structure-based protein function prediction using graph convolutional networks
• Protein Interface Prediction using Graph Convolutional Networks

In case, your data is some general form tabular data with features of different nature and form - like some continuous features, binary features, categorical features, whatever, there is no notion of proximity between different features.

The efficiency of CNN is based a lot on the ability of them to have a notion of locality, aggregate the information from the neighborhood, and construct the hierarchy of low-level (for several neighboring pixels) features and global (that understand the image or a signal as a whole) features.

Two neighboring pixes from the cat's ear have a notion of proximity and relatedness, whereas the red color and square shape do not have.

For unstructured tabular data I would recommend to start from some tree ensembling approach:

• Random forest
• Gradient boosting

EDIT

Seems like what you receive as input is a matrix of pairwise distances $\rho(i, j)$ between all possible amino residues - Protein contact map.

I would think about it as a weighted adjacency matrix. However, there is a rather special structure,

Therefore, there is no need for generic GNN, most likely.

In the literature, the recent research solves this problem with the help of CNN:

• Improved protein structure prediction using predicted interresidue orientations
• A fully open-source framework for deep learning protein real-valued distances

Probably, you should tune some hyperparameters, or introduce residual connections, if there are no such at the moment.