Can a neural network work out the concept of distance?

Question

Imagine a game where it is a black screen apart from a red pixel and a blue pixel. Given this game to a human, they will first see that pressing the arrow keys will move the red pixel. The next thing they will try is to move the red pixel onto the blue pixel.

Give this game to an AI, it will randomly move the red pixel until a million tries later it accidentally moves onto the blue pixel to get a reward. If the AI had some concept of distance between the red and blue pixel, it might try to minimize this distance.

Without actually programming in the concept of distance, if we take the pixels of the game can we calculate a number(s), such as "entropy", that would be lower when pixels are far apart than when close together? It should work with other configurations of pixels. Such as a game with three pixels where one is good and one is bad. Just to give the neural network more of a sense of how the screen looks? Then give the NN a goal, such as "try to minimize the entropy of the board as well as try to get rewards".

Is there anything akin to this in current research?

Answer 1

Answer

I'm going to take your question at face value, and go really deep into this topic.

Yes, they can. The typical human mind can. But consider the human mind. Millions, if not billions, of neurons. In fact, one can consider distance as a human concept, simply a theory developed from interactions with the world.

Therefore, given a year or two, with a ton of neurons on your hand, you could replicate this scenario. That is if your computer is as parallel as the human mind. The short explanation is that the human mind is very parallel.

However, it would be simpler to calculate the distance with a program, not an AI, and simply feed the result to the AI that would make the decisions.

Consider the amount of time you have spent looking at a screen. If you can tell the (approximate) distance between two pixels, so can a Neural Network, as you are one. However, add the amount of time you have spent alive and learning into the equation, and it becomes a disaster.

Further reading

The human brain is parallel

This is a result of the fact that all of the neurons in the human brain are independent of each other. They can run true simultaneous actions, thus making the action of interpreting images and such much easier, as blocks of neurons can "think" independent of the operations of the others, limiting what would be "lag" to a minuscule amount.

Answer 2

You can create AI to "see" as a human. As you said, giving the human the keys, he will click randomly. He just needs to know which keys he presses that brings him closer to other objects on the screen. I think the basics of an AI is object recognition. I would try to create a script to map the screen objects of the game. There are legal examples in Python.

I would try to follow a path like this:

• Make the AI ​​understand that by clicking the arrows or the WASD and it is in the context GAME, the object that move pixels according to the direction, represents the main author (the player).

• In parallel: map all boundaries of the region and index different objects within that region to automatically have the coordinate domain and object distance. AI needs to SEE (stream) the game and through images to categorize objects. Do you understand what I mean?

• In parallel: The AI ​​needs to be aware of all texts and information that is on the screen (all mapped, remember?). You need to understand when a text changes or something different happens. For example: whenever he returns to the initial position of each phase, whenever he has a count, what happens when the cout reaches zero or a common number that generates another type of change.

• He needs to understand what is repeated at every "respawn". You also need to understand what "respawn" is. Maybe a certain map position on every map it returns whenever a count on the screen ends. Or when it comes up against a certain type of object (mapped object)

To be honest, if you want to create a super intelligent robot, you can go following all the steps that go through the heads of different humans, or the best humans, or the rules of each game. But sometimes it's easier to build specific bots to perform specific tasks. It depends on what you want to do

Answer 3

What you mention there is the perfect example for path-planning, which is extensively researched in AI.

Please look for A-star algorithm and how to enhance it with neural networks :)

Answer 4

We can break down the problem as follows:

First, if you have two points on a plane and feed the coordinates of those points to a neural network (e.g., a vector $< x_0, y_0, x_1, y_1 >$) and and train it on a label thats the actual distance (e.g., $ \sqrt{(x_0 - y_0)^2 + (x_1-y_1)^2} $), it should be able to learn this relationship with arbitrarily close accuracy.

Next, if you have an image similar to what you describe, and feed that through a different neural network (e.g., a CNN), and as labels you used the the points of the two dots (once again $< x_0, y_0, x_1, y_1 >$), then it should be able to learn that relationship with arbitrarily close accuracy once again.

Of course, there's no reason to do this in two separate neural network, so we can just combine the two end-to-end have a model that takes the image as input and the distance as output.

This model would need to be trained on labeled data, however, so you'd either need to generate the data yourself or label images.

But if you wanted it to learn the notion of closing a distance in a less supervised way, you'd need to use reinforcement learning. In this case, you'd have to setup an environment that incentivises the agent to reduce the distance. This could be as simple as gaining reward if an action reduces the distance.

Another approach would be to incentivise the agent using future reward. That is, it's reward doesn't just come from the results of the next immediet state, but there's also contributions from the next possible state, and the one after that, and so on. This is the idea behind Deep Q-Learning, and I implement a simple example (very similar to what you're describing) in this notebook.

So, now the question is: has this implementation done something other than randomly moving around until it follows a path to success?

In your example, you talk about rewarding the agent when it lands on the goal. But in what I described, it gained reward by moving closer to the goal (either through the Q-Function or directly from the environment). It is able to do so by learning some abstract idea of distance (which can be illustrated in the supervised version).

When a human learns this, it's for the same exact reason: the human is gaining a reward for moving in that direction through a sense of future rewards.

I'd say that, given enough training and data, reinforcement learning could learn this concept with ease. As far as other rewards being present on the board (e.g., "minimise the entropy of the board as well as try to get rewards"), you need to think about what it is you're asking. Would you rather the agent minimize distance or maximize reward? Cause, in general, it can't do both. If you're looking for some balance between the two, then really you're just redefining the reward to also consider the distance.