0
$\begingroup$

I am trying to use a artifical neural network to produce a single output, which in my mind should be an index into a list of data (or close to it). All of the results I get are 0.9999+ and very close to each other. I don't know if my whole way of thinking here is off, or if I am just missing an approach, or if perhaps my network code is just broken.

I am trying to make use of the simple neural network from Microsoft here: https://social.technet.microsoft.com/wiki/contents/articles/36428.basis-of-neural-networks-in-c.aspx

I have tried this with a significantly more complex data set, but I've also tried using a very simple data set.

Here is the simple training data I'm trying to use:

eat poo bad
eat dirt bad
eat cookies okay
eat fruit good
study poo okay
study dirt okay
study cookies okay
study fruit okay
dispose poo good
dispose dirt okay
dispose cookies bad
dispose fruit bad

The basic idea is that the network has two input neurons and a single output neuron. I assigned a unique number to each distinct word such that I can train the network with two inputs (verb and object), and expect a single output (good, bad, or okay).

Example training:

input: 1 (for eat) 10 (for dirt) output: 15 (for bad)
input: 1 (for eat) 11 (for cookies) output: 16 (for good)

I would expect that after training, I would see the output numbers close to 15, 16, etc, but all I get are numbers like 0.999997333313168, etc.

Example run:

input: 1 (for eat) 10 (for dirt), output is 0.999997333313168 (instead of ~15 expected)

What do these outputs mean, or what am I missing in how I should be thinking about making a basic classification system (given inputs, get a meaningful output)?

The C# code I am using, if it is helpful:

using NeuralNet.NeuralNet;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace NeuralNet
{
    internal class TestSmallSampleNetwork
    {
        internal static void Run()
        {
            var data = @"eat poo bad
                        eat dirt bad
                        eat cookies okay
                        eat fruit good
                        sell poo bad
                        sell dirt okay
                        sell cookies okay
                        sell fruit okay
                        study poo okay
                        study dirt okay
                        study cookies okay
                        study fruit okay
                        dispose poo good
                        dispose dirt okay
                        dispose cookies bad
                        dispose fruit bad";
            var simples = data.Split(new[] { "\r\n" }, StringSplitOptions.None ).Select(_ => new Simple(_)).ToList();

            var verbs = simples.Select(_ => _.Verb).Distinct().Select(_ => new NetworkValue(_)).ToList();
            var objects = simples.Select(_ => _.Object).Distinct().Select(_ => new NetworkValue(_)).ToList();
            var judgments = simples.Select(_ => _.Good).Distinct().Select(_ => new NetworkValue(_)).ToList();
            var values = verbs.Concat(objects).Concat(judgments).ToDictionary(_ => _.Term, _ => _);

            // Create a network with 2 inputs, 2 neurons on a single hidden layer, and 1 neuron output.
            var net = new NeuralNetwork(0.9, new int[] { 2, 2, 1 });

            for (int iTrain = 0; iTrain < 1000; iTrain++)
            {
                for (int iSimple = 0; iSimple < simples.Count; iSimple++)
                {
                    net.Train(MakeInputs(simples[iSimple], values), MakeOutputs(simples[iSimple], values));
                }
            }

            foreach (var value in values.Values)
            {
                Console.WriteLine(value);
            }

            Console.WriteLine();

            // Run samples and get results back from the network
            Run("study", "poo", values, net);
            Run("eat", "poo", values, net);
            Run("dispose", "fruit", values, net);
            Run("sell", "dirt", values, net);
        }

        private static void Run(string verb, string obj, Dictionary<string, NetworkValue> values, NeuralNetwork net)
        {
            var result = net.Run(new List<double> {
                values[verb].Value,
                values[obj].Value,
            }).Single();

            var good = "xxx";

            Console.WriteLine($"{verb} {obj} {good} ({result})");
        }

        private static List<double> MakeInputs(Simple simple, Dictionary<string, NetworkValue> values)
        {
            return new List<double>() {
                values[simple.Verb].Value,
                values[simple.Object].Value
            };
        }

        private static List<double> MakeOutputs(Simple simple, Dictionary<string, NetworkValue> values)
        {
            return new List<double> { values[simple.Good].Value };
        }

        public class Simple
        {
            public string Verb { get; set; }
            public string Object { get; set; }
            public string Good { get; set; }

            public Simple(string line)
            {
                var words = line.Trim().Split(" ".ToCharArray(), 3, StringSplitOptions.None);
                Verb = words[0];
                Object = words[1];
                Good = words[2];
            }

            public override string ToString()
            {
                return $"{Verb} {Object} {Good}";
            }
        }

        public class NetworkValue
        {
            private static int Next = 1;

            public string Term { get; set; }
            public double Value { get; set; }

            public NetworkValue(string term)
            {
                Term = term;
                Value = Next++;
            }

            public override string ToString()
            {
                return $"{Value}. {Term}";
            }
        }
    }
}
$\endgroup$
1
$\begingroup$

I've discovered Doc2Vec which does something similar to what I am trying to accomplish. This doesn't exactly answer my question of why the network I was trying to build doesn't work, but at least it shows how indexed outputs can be pulled from a network, with open source to show how it is built.

https://datascience.stackexchange.com/questions/23969/sentence-similarity-prediction

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.