I am trying to use a artifical neural network to produce a single output, which in my mind should be an index into a list of data (or close to it). All of the results I get are 0.9999+ and very close to each other. I don't know if my whole way of thinking here is off, or if I am just missing an approach, or if perhaps my network code is just broken.
I am trying to make use of the simple neural network from Microsoft here: https://social.technet.microsoft.com/wiki/contents/articles/36428.basis-of-neural-networks-in-c.aspx
I have tried this with a significantly more complex data set, but I've also tried using a very simple data set.
Here is the simple training data I'm trying to use:
eat poo bad
eat dirt bad
eat cookies okay
eat fruit good
study poo okay
study dirt okay
study cookies okay
study fruit okay
dispose poo good
dispose dirt okay
dispose cookies bad
dispose fruit bad
The basic idea is that the network has two input neurons and a single output neuron. I assigned a unique number to each distinct word such that I can train the network with two inputs (verb and object), and expect a single output (good, bad, or okay).
Example training:
input: 1 (for eat) 10 (for dirt) output: 15 (for bad)
input: 1 (for eat) 11 (for cookies) output: 16 (for good)
I would expect that after training, I would see the output numbers close to 15, 16, etc, but all I get are numbers like 0.999997333313168, etc.
Example run:
input: 1 (for eat) 10 (for dirt), output is 0.999997333313168 (instead of ~15 expected)
What do these outputs mean, or what am I missing in how I should be thinking about making a basic classification system (given inputs, get a meaningful output)?
The C# code I am using, if it is helpful:
using NeuralNet.NeuralNet;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace NeuralNet
{
internal class TestSmallSampleNetwork
{
internal static void Run()
{
var data = @"eat poo bad
eat dirt bad
eat cookies okay
eat fruit good
sell poo bad
sell dirt okay
sell cookies okay
sell fruit okay
study poo okay
study dirt okay
study cookies okay
study fruit okay
dispose poo good
dispose dirt okay
dispose cookies bad
dispose fruit bad";
var simples = data.Split(new[] { "\r\n" }, StringSplitOptions.None ).Select(_ => new Simple(_)).ToList();
var verbs = simples.Select(_ => _.Verb).Distinct().Select(_ => new NetworkValue(_)).ToList();
var objects = simples.Select(_ => _.Object).Distinct().Select(_ => new NetworkValue(_)).ToList();
var judgments = simples.Select(_ => _.Good).Distinct().Select(_ => new NetworkValue(_)).ToList();
var values = verbs.Concat(objects).Concat(judgments).ToDictionary(_ => _.Term, _ => _);
// Create a network with 2 inputs, 2 neurons on a single hidden layer, and 1 neuron output.
var net = new NeuralNetwork(0.9, new int[] { 2, 2, 1 });
for (int iTrain = 0; iTrain < 1000; iTrain++)
{
for (int iSimple = 0; iSimple < simples.Count; iSimple++)
{
net.Train(MakeInputs(simples[iSimple], values), MakeOutputs(simples[iSimple], values));
}
}
foreach (var value in values.Values)
{
Console.WriteLine(value);
}
Console.WriteLine();
// Run samples and get results back from the network
Run("study", "poo", values, net);
Run("eat", "poo", values, net);
Run("dispose", "fruit", values, net);
Run("sell", "dirt", values, net);
}
private static void Run(string verb, string obj, Dictionary<string, NetworkValue> values, NeuralNetwork net)
{
var result = net.Run(new List<double> {
values[verb].Value,
values[obj].Value,
}).Single();
var good = "xxx";
Console.WriteLine($"{verb} {obj} {good} ({result})");
}
private static List<double> MakeInputs(Simple simple, Dictionary<string, NetworkValue> values)
{
return new List<double>() {
values[simple.Verb].Value,
values[simple.Object].Value
};
}
private static List<double> MakeOutputs(Simple simple, Dictionary<string, NetworkValue> values)
{
return new List<double> { values[simple.Good].Value };
}
public class Simple
{
public string Verb { get; set; }
public string Object { get; set; }
public string Good { get; set; }
public Simple(string line)
{
var words = line.Trim().Split(" ".ToCharArray(), 3, StringSplitOptions.None);
Verb = words[0];
Object = words[1];
Good = words[2];
}
public override string ToString()
{
return $"{Verb} {Object} {Good}";
}
}
public class NetworkValue
{
private static int Next = 1;
public string Term { get; set; }
public double Value { get; set; }
public NetworkValue(string term)
{
Term = term;
Value = Next++;
}
public override string ToString()
{
return $"{Value}. {Term}";
}
}
}
}
