I want my neural network structure to not have a circular/looping structure something similar like a directed acyclic graph (DAG). How do I do that?
The naive way is to generate connections randomly as you would for a cyclic graph, but then perform a test to reject any connections that form a cycle. This is the current approach in SharpNEAT and there has been some effort directed at improving the performance of the cycle test in the work-in-progress refactor branch.
One alternative would be to track the depth of all nodes, store a list of node IDs sorted by depth, and sample the connection endpoint nodes in such a way that the target node depth is always higher than the source node. Now I think about it that's probably the better method.
I also struggled with this when I was implementing NEAT.
What worked for me was cycle detection using DFS search in this video https://www.youtube.com/watch?v=tg96sZqhXyU
Simply put, I did DFS on all my input nodes recording all the nodes visited if I encounter a node I've already visited then its a cycle thereby I for my neat to discard this and attempt to make another connection.