3
$\begingroup$

I am playing around with neural networks in Tensorflow and I figured an interesting test would be whether I can write a calculator using a Tensorflow Neural Network.

I started with simple addition and it kinda worked (so given 2, 4 it would get around 5.9 or 6.1).

Then I wanted to add the ability to calculate using "+", "-", and "*".

Here is the code I came up with in the end:

import numpy as np
import tensorflow as tf
from random import randrange

def generate_input(size):
    nn_input = []
    for i in range(0,size):
        symbol = float(randrange(3))
        nn_input.append([
                float(randrange(1000)),
                float(randrange(1000)),
                1 if symbol == 0 else 0,
                1 if symbol == 1 else 0,
                1 if symbol == 2 else 0,
                ])
    return nn_input

def generate_output(input_data):
    return [[generate_single_output(i)] for i in input_data]

def generate_single_output(input_data):
    plus = input_data[2]
    minus = input_data[3]
    multiplication = input_data[4]

    if (plus):
        return input_data[0] + input_data[1]

    if (minus):
        return input_data[0] - input_data[1]

    if (multiplication):
        return input_data[0] * input_data[1]

def user_input_to_nn_input(user_input):
    symbol = user_input[1]
    return np.array([[
            float(user_input[0]),
            float(user_input[2]),
            1 if symbol == '+' else 0,
            1 if symbol == '-' else 0,
            1 if symbol == '*' else 0,
            ]])


if __name__ == '__main__':
    model = tf.keras.models.Sequential([
        tf.keras.layers.Dense(64, activation='relu', input_shape=(5,)),
        tf.keras.layers.Dense(64, activation='relu'),
        tf.keras.layers.Dense(1),
        ])

    model.compile(tf.keras.optimizers.RMSprop(0.001), loss=tf.keras.losses.MeanSquaredError())


    input_data = np.array(generate_input(10000))
    output_data = np.array(generate_output(input_data))

    model.fit(input_data, output_data, epochs=20)

    while True:
        user_calculation = input("Enter expression (e.g. 2 + 3):")
        user_input = user_calculation.split()
        nn_input = user_input_to_nn_input(user_input)
        print(model.predict(nn_input)[0][0])

The idea is built on this tutorial: https://www.tensorflow.org/tutorials/keras/basic_regression

The input is 5 fields: number 1, number 2, plus?, minus?, multiplication?

Where the last 3 are simply 1 or 0 depending on whether that is the calculation I am trying to do.

As an output for say [1,4,1,0,0] I would expect 1 + 4 = 5 for [1,4,0,1,0] I would expect 1 - 4 = -3 etc.

For some reason though the numbers I am getting are completely off and seem random.

Basically I am trying to understand what I went wrong? The data being input to the NN seems correct and the model is based on the model used in the tutorial I quoted (and the problems seem fairly similar so I expect if one would work the other would too).

$\endgroup$
4
$\begingroup$

A neural network is not good at selecting a function based on those 3 input parameters, because of the way a neuron is setup.

What you should do is either make a neural network for each operation, or use different input neurons for each operation. E.g. 2 input neurons for the addition operation, 2 for the multiplication, and 2 for the minus. 6 inputs in total of which 4 will always be 0.

This will make it easier for the neural network to calculate the result.

$\endgroup$
  • 1
    $\begingroup$ Thanks :) That seems to work to an extent. Now addition and subtraction work but multiplication doesn't work at all. Why exactly isn't a neural network not good at selecting a function the way I had it? Also any idea why multiplication wouldn't be working here? $\endgroup$ – KNejad Sep 13 at 22:52
  • 1
    $\begingroup$ I looked into it and it seems a simple multiplication function is very hard to approximate for a neural network with certain activation funtions (such as relu). And this is only learning multiplication, not trying to do the other operations in the same network. link:sololearn.com/Discuss/1467062/… $\endgroup$ – Lustwelpintje Sep 16 at 8:14
  • 1
    $\begingroup$ Thanks Lustwelpintje, that's what I've discovered too. I'll continue looking into it and see if I can figure out a way to do it $\endgroup$ – KNejad Sep 16 at 11:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.