I am currently working on a public project for the National Weather Model. We are experimenting with using a recurrent neural network to replace the output of a quadratic formula that is in use. The aim of the experiment is to get a speedup in the computation by using a neural network to essentially mimic the output of the quadratic formula. We have achieved an accuracy of about +-.02 but would like to see that improve to +-.001 or so in order to make the outputs indiscernible from a usage standpoint. Despite changing or increasing the training data size, validation data size, number of layers, size of layers, optimizer, batch size, epoch number, normalizations, etc. we cannot seem to move past this level of accuracy. We have changed and tested every standard metric we can find on how to improve the model, but nothing improves the accuracy beyond that threshold.
The main question we have is whether or not Keras is rounding at some point between each layer or has some limiting factor on the backend limiting the model's significant figures in the output. The training data resolution should allow for a finer level of accuracy, but as stated before, any changes made the model cannot improve past what has been achieved. Any insight on what is holding the model back would be greatly appreciated and could help with applying this method elsewhere. The Github has a readme file explaining what is occurring in each file and how to run the model as this is still a work in progress. I would be happy to dive deeper into any aspect of the model as well.