# How can we teach a neural net to make arbitrary data associations?

Let's say I have pairs of keys and values of the form $$(x_1, y_1), \dots, (x_N, y_N)$$. Then I give a neural net a key and a value, $$(x_i, y_i)$$. For example, $$x_i$$ could be $$4$$ and $$y_i$$ could be $$3$$, but this does not have to be the case.

Is there a way to teach the neural net to output the $$y_i$$ variable every time it receives the corresponding $$x_i$$?

By the way, how do our brains perform this function?

• Why do you need a neural network to perform this extremely simple task of outputting 3 whenever it receives 4? Can you provide more details about your problem? For example, do you have multiple different parts of numbers or just one pair or what? Also, ask only one question per post, i.e. what is your main question? Is that about mapping 4 to 3 or about brains? Moreover, have you ever heard of supervised learning?
– nbro
Aug 20, 2020 at 9:58
• So, I know there are way more simple ways to do this, but I am just curious about how to replicate how our brains memorize things in a neural net. And it wouldn’t be just mapping 4 to 3. It would be any arbitrary association of numbers, like how our brains can memorize anything Aug 20, 2020 at 11:54
• The answer below is correct but doesn't answer the question I feel. Look into hopfield nets and RBMs for the purpose you mention.
– user9947
Aug 20, 2020 at 13:52
• @DuttaA as far as I know, RBMs do unsupervised learning (visible neurons are for the X data only). If I'm wrong, could you developp your point in an answer ? I'd be pleased to read you. Aug 23, 2020 at 13:11
• @16Aghnar the main idea behind associative learning is that some neurons fire together. page.mi.fu-berlin.de/rojas/neural/index.html the chapter on associative learning deals with this. I can't point out the exact algo, but it is probably possible to force 2 neurons to fire simultaneously (which probably is a supervised learning paradigm after all). So RBMs was not a good example but modified hopfield nets probably can be modified to do this. The correct term would be hetero associative nets/memory.
– user9947
Aug 23, 2020 at 13:56

In a nutshell : Memorizing is not Learning

So, first let's just remind the classical use of a neural net, in Supervised Learning :

• You have a set of $$(x_{train}, y_{train}) \in X \times Y$$ pairs, and you want to extract a general mapping law from $$X$$ to $$Y$$
• You use a neural net function $$f_{\theta} : x \rightarrow f_{\theta}(x)$$, with $$\theta$$ the weights (parameters) of your net.
• You optimise $$f_{\theta}$$ by minimizing the prediction error, represented by the loss function.

Can this solve your question ? Well, I don't think so. With this scheme, your neural net will learn an appropriate mapping from the set $$X$$ to the set $$Y$$, but this mapping is appropriate according to your loss function , not to your $$(x_{train}, y_{train})$$ pairs.

Imagine that a small part of the data is wrongly labelled. A properly trained net learns to extract relevant features and thus will predict correctly the label, not like you did. So the net doesn't memorize your pairs, it infers a general law from the data, and this law may not respect each $$(x_{train}, y_{train})$$. So classical Supervised Deep Learning should not memorize $$(x_{train}, y_{train})$$ pairs.

However, you could memorize using a net with too many parameters : it's Overfitting !

• In this case, you set up the net with too many parameters. That gives too much degrees of freedom to your net, and the net will use these DoFs to exactly fit rightly each $$(x_{train}, y_{train})$$ pair you feed during training.
• However, for an input $$x$$ that it never saw during training, $$f_{\theta}(x)$$ would have no meaning. That's why we say an overfitted net did not learn, and overfitting is feared by many DL practitioner.

But as long as you want only to memorize, and not to learn, a overfitted net may be the a solution. An other solution for memorization may be Expert Systems, I don't know them enough to explain them, but you may check that if you want.