Skip to main content
Artificial Intelligence

Return to Question

deleted 1 character in body; edited tags
Source Link
nbro
nbro
  • 41.4k
  • 12
  • 114
  • 205

I'm working on an implementation of NEAT, which evolves neural networks with small and sparse topologies.

Evaluating a sparse and possibly recurrent network requires a different approach than the matrix operations of dense networks, and I'm trying to wrap my head around the order atin which nodes should be evaluated.

I've set up a simple example:

  • Every node (except inputs) starts at 0. (t0)
  • Every weight is 1. There's no activation function neitheror biases.

enter image description here

Assuming that the algorithm has the same intuition we humans do: evaluate the nodes connected to the input, then the "aggregation" node, then the output. How should it decide whether to evaluate node [3] before node [4] or vice-versa?

By starting (t0) at [3], the value of [4] is 0. And vice-versa.

The side-effect of this behavior appears when comparing two networks. Say you have the network above, and a copy of it where [3] and [4] are inverted. I feel like both should return the same result for the same input in order tofor evolution to work properly.

Any thoughts?

I'm working on an implementation of NEAT, which evolves neural networks with small and sparse topologies.

Evaluating a sparse and possibly recurrent network requires a different approach than the matrix operations of dense networks, and I'm trying to wrap my head around the order at which nodes should be evaluated.

I've set up a simple example:

  • Every node (except inputs) starts at 0. (t0)
  • Every weight is 1. There's no activation function neither biases.

enter image description here

Assuming that the algorithm has the same intuition we humans do: evaluate the nodes connected to the input, then the "aggregation" node, then the output. How should it decide whether to evaluate node [3] before node [4] or vice-versa?

By starting (t0) at [3], the value of [4] is 0. And vice-versa.

The side-effect of this behavior appears when comparing two networks. Say you have the network above, and a copy of it where [3] and [4] are inverted. I feel like both should return the same result for the same input in order to evolution work properly.

Any thoughts?

I'm working on an implementation of NEAT, which evolves neural networks with small and sparse topologies.

Evaluating a sparse and possibly recurrent network requires a different approach than the matrix operations of dense networks, and I'm trying to wrap my head around the order in which nodes should be evaluated.

I've set up a simple example:

  • Every node (except inputs) starts at 0. (t0)
  • Every weight is 1. There's no activation function or biases.

enter image description here

Assuming that the algorithm has the same intuition we humans do: evaluate the nodes connected to the input, then the "aggregation" node, then the output. How should it decide whether to evaluate node [3] before node [4] or vice-versa?

By starting (t0) at [3], the value of [4] is 0. And vice-versa.

The side-effect of this behavior appears when comparing two networks. Say you have the network above, and a copy of it where [3] and [4] are inverted. I feel like both should return the same result for the same input in order for evolution to work properly.

Any thoughts?

Source Link
Hugo Aboud
Hugo Aboud
  • 121
  • 3

Order of operations on sparse recurrent network alters the output. How to deal with it?

I'm working on an implementation of NEAT, which evolves neural networks with small and sparse topologies.

Evaluating a sparse and possibly recurrent network requires a different approach than the matrix operations of dense networks, and I'm trying to wrap my head around the order at which nodes should be evaluated.

I've set up a simple example:

  • Every node (except inputs) starts at 0. (t0)
  • Every weight is 1. There's no activation function neither biases.

enter image description here

Assuming that the algorithm has the same intuition we humans do: evaluate the nodes connected to the input, then the "aggregation" node, then the output. How should it decide whether to evaluate node [3] before node [4] or vice-versa?

By starting (t0) at [3], the value of [4] is 0. And vice-versa.

The side-effect of this behavior appears when comparing two networks. Say you have the network above, and a copy of it where [3] and [4] are inverted. I feel like both should return the same result for the same input in order to evolution work properly.

Any thoughts?