This question is in relation to a previous doubt of mine :
I have made a bit of progress from there, refurbished my code, and got things ready.
What I intend to make is 'Insane Mind', a model which forms random linear neural networks from a set of nodes at random times ( I made out the 'linear neural network' part from a bit of Google searches).
The basic process involved is :
- The system forms nodes of random weights . These nodes also have the Sigmoid function (the logistic fuction : $f(x) = \frac{1}{1 + e^{-x}}$ ) , and I termed these 'Gravitons' (because of the usage of the word 'weights' in them - sorry if my terminology work seems ambiguous...😅)
- The input enters the system via one of the gravitons.
- The node processes it and either passes the output to the next node or to itself .
- Step 3 is repeated a certain number of times as the number of gravitons made for use.
- The output of the final graviton is given as the output of the whole system.
One thing I'm sure of this model is that this model can transform an input vector into an output vector.
I am not sure whether this is ambiguous or similar to previously discovered model. Plus, I'd like to know if this will be effective in any situation (I believe it will be of help in classification problems).
Note : I made this out of my imagination , which means this may be useless one way or the other, but still it seemed to work.
Here's the training algorithm I made for this model :
- In my Python implementation of this model, I had added a provision in the 'Graviton' class to store the derivative of the output of the graviton. Using this, the gravitons are ordered in the increasing order of the derivatives of their outputs.
- The first graviton is taken, and its weight is modified by the error in the output.
- The error is modified by the product of the graviton's output derivative and its weight after editing.
- Steps 2 through 3 are done for the other gravitons as well. The final error (given by the
error
variable ) will be the product of the derivatives, the edited weights and the error in the output.- The set of gravitons thus formed is the next set subjected to this training.
For extra reference, here's the code:
- Insane_Mind.py :
from math import *
from random import *
class MachineError(Exception):
'''standard exception in the API'''
def __init__(self, stmt):
self.stmt = stmt
def sig(x):
'''Sigmoid function'''
try :
return exp(x)/(exp(x) + 1)
except OverflowError:
if x > 0 :
return 1
elif x < 0:
return
class Graviton:
def __init__(self, weight, marker):
'''Basic unit in 'Insane Mind' algorithm'''
self.weight = weight
self.marker = marker + 1
self.input = 0
self.output = 0
self.derivative = 0
def process(self, input_to_machine):
'''processes the input'''
self.input = input_to_machine
self.output = sig(self.weight * self.input)
self.derivative = self.input * self.output * (1- self.output)
return self.output
def get_derivative_at_input(self):
'''returns the derivative of the output'''
return self.derivative
def correct_self(self, learning_rate, error):
'''edits the weight'''
self.weight += -1 * error * learning_rate * self.get_derivative_at_input() * self.weight
class Insane_Mind_Base:
'''Insane_Mind base class - this is what we're gonna use to build the actual machine'''
def __init__(self, number_of_nodes):
'''initialiser for Insane_Mind_Base class.
arguments : number_of_nodes : the number of nodes you want'''
self.system = [Graviton(random(),i) for i in range(number_of_nodes)] # the actual system
self.system_size = number_of_nodes # number of nodes , or 'system size'
def output_sys(self, input_to_sys):
'''system output'''
self.output = input_to_sys
for i in range(self.system_size):
self.output = self.system[randint(0,self.system_size - 1 )].process(self.output)
return self.output
def train(self, learning_rate, wanted):
'''trains the system'''
self.cloned = []
order = []
temp = {}
for graviton in self.system:
temp.update({str(graviton.derivative): self.system.index(graviton)})
order = sorted(temp)
i = 0
error = wanted - self.output
for value in order:
self.cloned.append(self.system[temp[value]])
self.cloned[i].correct_self(learning_rate, error)
error *= self.cloned[i].derivative * self.cloned[i].weight
i += 1
self.system = self.cloned
def details(self):
'''gets the weights of each graviton'''
for graviton in self.system:
print("Node : {0}, weight : {1}".format(graviton.marker , graviton.weight))
class Insane_Mind:
'''Actaul Insane_Mind class'''
def __init__(self, number_of_gravitons):
'''initialiser'''
self.model = Insane_Mind_Base(number_of_gravitons)
self.size = number_of_gravitons
def get(self, input):
'''processes the input'''
return self.model.output_sys(input)
def train_model(self, lrate, inputs, outputs, epoch):
'''train the model'''
if len(inputs) != len(outputs):
raise MachineError("Unequal sizes for training input and output vectors")
epoch = str(epoch)
if epoch.lower() == 'sys_size':
epoch = int(self.model.system_size)
else:
epoch = int(epoch)
for k in range(epoch):
for j in range(len(inputs)):
val = self.model.output_sys(inputs[j])
self.model.train(1/val if str(lrate).lower() == 'output' else lrate, outputs[j])
def details(self):
'''details of the machine'''
self.model.details()
- Insane_Mind_Test.py :
from Insane_Mind import *
from statistics import *
input_data = [3,4,3,5,4,4,3,6,5,4] # list of forces using which the coin is tossed
output_data = [1,0,0,1,1,0,0,0,1,1] # head or tails in binary form (0 = tail (= not head), 1 = head)
wanteds = output_data.copy()
model = Insane_Mind(2) # Insane Mind model
print("Before Training:")
print("----------------")
model.details() # fetches you weights of the model
def normalize(x):
cloned = x.copy()
meanx = mean(x)
stdevx = stdev(x)
for i in range(len(x)):
cloned[i] = (cloned[i] - meanx)/stdevx
return cloned
def random_catch(range_of_catches, sample_length):
# sample data generator. I named it random catch as part of using it in testing whether my model
# ' catches the correct guess'. :)
return [randint(range_of_catches[0], range_of_catches[1]) for i in range(sample_length)]
input_data = normalize(input_data)
output_data = normalize(output_data)
model.train_model('output', input_data, output_data, 'sys_size')
# the argument 'output' for the argument 'lrate' (learning rate) was to specify that the learning rate at # each step is the inverse of the output, and the use of 'sys_size' for the number of times to be trained
# is used to tell the machine that the required number of epochs is equal to the size of the system or
# the number of nodes in it.
print("After Training:")
print("----------------")
model.details() # fetches you weights of the model
predictions = [model.get(i) for i in input_data]
threshold = mean(predictions)
predictions = [1 if i >= threshold else 0 for i in predictions]
print("Predicted : {0}".format(predictions))
print("Actual:{0}".format(wanteds))
mse_array = [(wanteds[j] - predictions[j])**2 for j in range(len(input_data))]
print("Mean squared error:{0}".format(mean(mse_array)))
accuracy = 0
for i in range(len(predictions)):
if predictions[i] == wanteds[i]:
accuracy += 1
print("Accuracy:{0}({1} out of {2} predictions correct)".format(accuracy/len(wanteds), accuracy, len(predictions)))
print("______________________________________________")
print("Random catch test")
print("-----------------")
times = int(input("No. of tests required : "))
catches = int(input("No. of catches per test"))
mse = {}
for m in range(times):
wanted = random_catch([0,1] , catches)
forces = random_catch([1,10], catches)
predictions = [model.get(k) for k in forces]
threshold = mean(predictions)
predictions = [1 if value >= threshold else 0 for value in predictions]
mse_array = [(wanted[j] - predictions[j])**2 for j in range(len(predictions))]
print("Mean squared error:{0}".format(mean(mse_array)))
mse.update({(m + 1):mean(mse_array)})
accuracy = 0
for i in range(len(predictions)):
if predictions[i] == wanted[i]:
accuracy += 1
print("Accuracy:{0}({1} out of {2} predictions correct)".format(accuracy/len(wanteds), accuracy, len(predictions)))
I tried running 'Insane_Mind_Test.py', and the results I got are :
The formula I used from MSE is (please correct me if I was wrong): $$ MSE = \frac{\sum_{i = 1}^n (x_i - x'_i)^2}{n}$$
where,
$$ x_i = \text{Intended output}$$ $$ x'_i = \text{Output predicted}$$ $$ n = \text{Number of outputs}$$ My main intention was to make a guess system.
Note : Here, I had to think differently. I decided to classify the forces as those yielding a head and those that yield a tail (unlike what I say in the comments in the program).
Thanks for all help in advance.
Edit: Here's the training data :
Forces Head(1) or not head(0)[rather call it tail]
_______ ______________________
3 1
4 0
3 0
5 1
4 1
4 0
3 0
6 0
5 1
4 1