Neural network for recognizing ship types based on location series

Question

I am building a neural network for recognizing ship types based on a 1000-long series of location data (latitude-longitude, normalized to account for different km/longitude° metrics, so that vector difference yields a consistent distance). The dataset I use consists of around 100 000 distinct day-ship pairs, classified by a 10-valued labeling. The cardinality of the classes are: 30115, 26327, 12798, 10940, 5859, 4211, 4176, 3639, 3521, 2834

I tried two different approaches:

1. A recurrent network (using LSTM): Dense[relu] -> LSTM -> Dense[relu] -> Dropout -> Dense[softmax]
2. A 1D convolutional network: Dense[relu] -> Conv1D -> Dense[relu] -> Dropout -> Dense[softmax]

I experimented with the hyperparameters of the above networks, but they all converge to a 40% accuracy, where the model classifies all inputs as class 0 or 1 (choosing the most likely class of the output layer).

I could accept that the data is not well-defined and this kind of prediction is impossible, but the strange thing is that even if I give the same data as training and validation, the model stops getting better at the 40% accuracy mark. Shouldn't it go further, and "memorize" the classes in this case, resulting in ~100% accuracy on the training data?

Answer 1

There are a lot of questions to be asked about your test setup, data preprocessing, and model architecture. RNNs, or in your case LSTMs, can be tricky when it comes to implementation. I suspect your problem lies somewhere with the LSTM or preprocessing.

Are all lat/lon data points in one class the same? If not one thing you might want to do is to somehow round or perform some kind of grouping for the lat and longitude within specific classes. This could help simplify the problem for the network.

Perhaps It's also possible you don't have enough features in your model, i.e. too shallow/ not enough layers.

Have you also considered this problem without a RNN? Is it really needed in this case?