Why isn't my Neural Network based calculator working?

Question

I am playing around with neural networks in Tensorflow and I figured an interesting test would be whether I can write a calculator using a Tensorflow Neural Network.

I started with simple addition and it kinda worked (so given 2, 4 it would get around 5.9 or 6.1).

Then I wanted to add the ability to calculate using "+", "-", and "*".

Here is the code I came up with in the end:

import numpy as np
import tensorflow as tf
from random import randrange

def generate_input(size):
    nn_input = []
    for i in range(0,size):
        symbol = float(randrange(3))
        nn_input.append([
                float(randrange(1000)),
                float(randrange(1000)),
                1 if symbol == 0 else 0,
                1 if symbol == 1 else 0,
                1 if symbol == 2 else 0,
                ])
    return nn_input

def generate_output(input_data):
    return [[generate_single_output(i)] for i in input_data]

def generate_single_output(input_data):
    plus = input_data[2]
    minus = input_data[3]
    multiplication = input_data[4]

    if (plus):
        return input_data[0] + input_data[1]

    if (minus):
        return input_data[0] - input_data[1]

    if (multiplication):
        return input_data[0] * input_data[1]

def user_input_to_nn_input(user_input):
    symbol = user_input[1]
    return np.array([[
            float(user_input[0]),
            float(user_input[2]),
            1 if symbol == '+' else 0,
            1 if symbol == '-' else 0,
            1 if symbol == '*' else 0,
            ]])


if __name__ == '__main__':
    model = tf.keras.models.Sequential([
        tf.keras.layers.Dense(64, activation='relu', input_shape=(5,)),
        tf.keras.layers.Dense(64, activation='relu'),
        tf.keras.layers.Dense(1),
        ])

    model.compile(tf.keras.optimizers.RMSprop(0.001), loss=tf.keras.losses.MeanSquaredError())


    input_data = np.array(generate_input(10000))
    output_data = np.array(generate_output(input_data))

    model.fit(input_data, output_data, epochs=20)

    while True:
        user_calculation = input("Enter expression (e.g. 2 + 3):")
        user_input = user_calculation.split()
        nn_input = user_input_to_nn_input(user_input)
        print(model.predict(nn_input)[0][0])

The idea is built on this tutorial: https://www.tensorflow.org/tutorials/keras/basic_regression

The input is 5 fields: number 1, number 2, plus?, minus?, multiplication?

Where the last 3 are simply 1 or 0 depending on whether that is the calculation I am trying to do.

As an output for say [1,4,1,0,0] I would expect 1 + 4 = 5 for [1,4,0,1,0] I would expect 1 - 4 = -3 etc.

For some reason though the numbers I am getting are completely off and seem random.

Basically I am trying to understand what I went wrong? The data being input to the NN seems correct and the model is based on the model used in the tutorial I quoted (and the problems seem fairly similar so I expect if one would work the other would too).

Answer 1

A neural network is not good at selecting a function based on those 3 input parameters, because of the way a neuron is setup.

What you should do is either make a neural network for each operation, or use different input neurons for each operation. E.g. 2 input neurons for the addition operation, 2 for the multiplication, and 2 for the minus. 6 inputs in total of which 4 will always be 0.

This will make it easier for the neural network to calculate the result.