4
$\begingroup$

I would like to take in some input values for $n$ variables, say $R$, $B$, and $G$. Let $Y$ denote the response variable of these $n$ inputs (in this example, we have $3$ inputs). Other than these, I would like to use a reference/target value to compare the results.

Now, suppose the relation between the inputs ($R$, $B$ and $G$) with the output $Y$ is (let's say):

$$Y = R + B + G$$

But the system/machine has no knowledge of this relation. It can only read its inputs, $R$, $B$ and $G$, and the output, $Y$. Also, the system is provided with the reference value, say, $\text{REF} = 30$ (suppose).

The aim of the machine is to find this relation between its inputs and output(s). For this, I have come across some quite useful material online like this forum query and Approximation by Superpositions of a Sigmoidal Function by G. Cybenko and felt that it were possible. Also, I doubt that Polynomial Regression may be helpful as suggested Here.

One vague approach that comes to my mind is to use a truth table like approach to somehow deduce the effect of the inputs on the output and hence, get a function for it. But neither am I sure how to proceed with it, nor do I trust its credibility.

Is there any alternative/already existing method to accomplish this?

Improve this question
$\endgroup$
5
  • $\begingroup$ This seems a very standard ML problem. I would suggest reading more about Machine Learning. $\endgroup$ – DuttaA Aug 31 '19 at 18:03
  • $\begingroup$ I would like a direction for that. I know it is related to machine learning and artificial intelligence. Had I had the liberty to start from the very basic of ML, I would love it. But time bounds me. $\endgroup$ – Pe Dro Aug 31 '19 at 18:11
  • $\begingroup$ @DuttaA Actually this isn't a standard ML problem. If $Y$ can be any arbitrary function of the inputs, ML is usually used to find a black box model to predict $Y$ and not care about the question "what is the actual function?" The OP chose an unfortunate example where the question "what is the function?" can be answered exactly using linear regression. But what if it was something like $Y = \sin(x)/x$? A neural net can approximate it well within a certain range. But how can we actually learn that $Y = \sin(x)/x$? This is not a standard ML task. $\endgroup$ – Bridgeburners Sep 4 '19 at 13:44
  • $\begingroup$ @Bridgeburners As you say that the problem quoted here can be solved using linear regression, you would be surprised to know that my immediate need is exactly solving this exact equation.i.e. R,G,B and Y. But inevitably, I will need to solve more complicated equations like the Sine function etc. $\endgroup$ – Pe Dro Sep 4 '19 at 14:22
  • $\begingroup$ @Bridgeburners seems like you also chose a function which theoretically can be approximated sin(x)/x using a Neural net. Also ML is not confined to a Neural Net and theoretically (as per what I consider to be ML) it is possible to approximate any function using ML. $\endgroup$ – DuttaA Sep 4 '19 at 14:41
0
$\begingroup$

There are always a large number of possible functions that can produce a given set of input-output values. The challenge is to find a simplest function (according to whatever criteria you choose) to produce those values.

One approach is to write a general function of the input variables, comprising terms of all order in R, G, and B, with a coefficient for each term, then search for values of the coefficients that A) reproduce the known input-output values accurately and B) leave the largest number of coefficients equal to zero.

Several different algorithms can be used to do the search efficiently. My choice would be a genetic algorithm to seek the minimum of the RMS difference between the produced and known I-O values, summed with the product of a gradually increasing parameter and the number of nonzero coefficients.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I have brushed over Genetic Algorithm and found it to be really novel for me and very useful. But, I need to deploy the trained model onto a MCU and hence it will be too bulky for the device. $\endgroup$ – Pe Dro Sep 3 '19 at 15:17
0
$\begingroup$

As I see your explanation, what you're looking for can be stated via another way saying

Y = aX + b

where Y is output vector, X input vector and a & b are the coefficients you want to find.

Why so? And what is happening?

First thing first: I recommend to look the video [4] about how matrices and vectors work together and form after multiplication very familiar equations:

Y = a1 * x1 + a2 * x2 + a3 * x3

Now you see, you do not only get R + G + B but also some constants supplied for each of the variables.

About polynomial solutions I found [2] and [3] but reading through [3] you'll soon notice it is about completely different approach:

Y = a1 * x + a2 * x^2 + a3 * x^3

which you don't want.

So, you find something called linear regression and a so called Deep Neural Network for example to solve it [1].

I would summarize the source [1] in these steps:

  • You have to find some training data. That is examples of correct Y values, when X values are known.

  • Then you build in Python note book some code that has: a Neural Network with hidden layer(s), Activation Function, Back Propagation and Objective Function.

  • Many many iterations with the data, called Training.

There are plenty of samples and courses online about these in detail, but with correct tools and tutorials some dozen lines and no more are needed.

Process is ended with validation phase with some more known data items and results. It can tell you how well the estimated model works.

Final Notes:

As you may observe, the solution includes quite a many funny terms you have to learn somehow before mastering the task. For example Udemy has great online courses on this topic, also free tutorials are available on another sites. Your plans sound quite ambitious compared to the knowledge you have so far, so I really do recommend you learn little bit more to be able to fine tune the already given examples online. For example tutorial [5] includes one. It is at first quite complicated code, you'd need quite a lot practice to master it line by line.

In short:

Find your favorite tutorial, study neural networks a little bit (basics) and pick code sample to start experimenting. It is a long way but it is worth it.

Source:

[1] https://lightsapplications.wordpress.com/linear-regression-and-deep-learning/

[2] https://www.ritchieng.com/machine-learning-polynomial-regression/

[3] https://arachnoid.com/polysolve/

[4] https://www.youtube.com/watch?v=F2lJ7oSwcyY

[5] https://missinglink.ai/guides/neural-network-concepts/backpropagation-neural-networks-process-examples-code-minus-math/

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Thank Your for the detailed reply. I have begun learning Neural Networks and have also found some papers related to my application. The links provided by you are awesome. $\endgroup$ – Pe Dro Sep 3 '19 at 15:17
0
$\begingroup$

I found a solution to this some time back. I have been studying function approximation (within linear regression) for some time. Here's how I did it:

Neural Networks have been proved to be universal function approximators. So, even a single hidden layer would be sufficient to approximate a function simple as addition (Even somewhat complex functions like the Sine and any random CONTINUOUS wiggly function have been approximated)

First, I used a high level API like TensorFlow and Keras and implemented it here

The model was trained on the data (input-output pairs)

R    = np.array([-4, -10,  -2,  8, 5, 22,  3],  dtype=float)
B    = np.array([4, -10,  0,  0, 15, 5,  1],  dtype=float)
G    = np.array([0, 10,  5,  8, 1, 2,  38],  dtype=float)

Y    = np.array([0, -10, 3, 16, 21, 29, 42],  dtype=float)

And trained as follows:

#Create a hidden layer with 2 neurons
hidden = tf.keras.layers.Dense(units=2, input_shape=[3])

#Create the output (final) layer which symbolises value of **Y**
output = tf.keras.layers.Dense(units=1)

#Combine layers to form the neural network and compile it
model = tf.keras.Sequential([hidden, output])
model.compile(loss='mean_squared_error', optimizer=tf.keras.optimizers.Adam(0.1))
history = model.fit(RBG,Y, epochs=500, verbose=False)

The model converges in about 50 epochs

enter image description here

Also, I have implemented the same using only C/C++ and used GNU plot to visualize the results.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.