Are there any strategies that would help me visualize the 'behavior space' and make a novelty function?

Question

In “Abandoning Objectives: Evolution through the Search for Novelty Alone”, it is explained how the novelty search is a function that is domain specific, depending on the differing behaviors that can potentially emerge.

The primary test is a deceptive maze and it seems like they define novelty as a function that is dependent on each actor's ending position as a distance from other actors' ending position.

I am wanting to try implementing this on some tasks. Some simple AI tasks such as playing pong, or recreating MarI/O, or sticking them in an arena as an actor who can move, turn, and shoot (with other actors in the arena with them).

I have a really hard time thinking of how to model the behavior functions for these kinds of instances without making it into an objective. For pong, I imagine I could determine novelty by the AI's point score, but isn't this basically making the score an objective since it can only go up? For MarI/O, I've seen some implementations that look at the list of unique grid locations that Mario visited in what order, but I didn't come up with that myself.

For the arena example, my first impulse is to have a score based on how long the actor survived and how many other actors the AI eliminated; but again, this can only go up and seems to me like it is defining an objective.

Are there any strategies or ways to think about the problems that would help me better visualize the 'behavior space' and make a better novelty function?

Answer 1

I think that the best approach is to "switch point of view" from the general, objective-oriented, Genetic Algorithm's behaviours.

Usually GAs rely on individualism: the best survives. To do this you have to define what 'best' means and this is done through a fitness function that measures something objective, independent from the individual (i.e. a score, classical example).

If you want to measure novelty, collectivism becomes more important, as the whole population (present and past) must be considered. You have to think about measuring how different the individual X did from 'the rest' (i.e. location never reached before).

Another interesting point is that you should think in a more multidimensional way: a score is uni-dimensional, everyone wants to get higher, that is easy. A location in space is more difficult to achieve and that is why novelty becomes interesting.

---

To sum up: stay multidimensional, always treat the population as a multi-organism rather than focusing on the single individual. I hope this helps :)