I am making a machine learning program for time series data analysis and using NEAT could help the work. I started to learn TensorFlow not long ago but it seems that the computational graphs in TensorFlow are usually fixed. Is there tools in TensorFlow to help build a dynamically evolving neural network? Or something like PyTorch would be a better alternative? Thanks.
Check this implementation in TensorFlow Eager: https://github.com/crisbodnar/TensorFlow-NEAT
The Python language dynamically extensible like most other languages, so there is no reason why NEAT functionality could not be flexibly implemented in Python. Since the trend is to move computationally intensive processes to USB-3 or uPCIe acceleration or GPU cores, it is more likely the computational graphs will be uploaded to hardware along with training data to maximize throughput. Even in the absence of hardware acceleration, scripting languages generally delegate high computational burdens to tuned and optimized C/C++ dynamic libraries.
Python frameworks such as TensorFlow or PyTorch and Java frameworks such as DL4J would then call the C++ or ncpp code that handles the device opening, upload, and download of results. All these high level frameworks and the languages they are written in can handle new lower level computational components accessible through new APIs and flexible interface classes that support variable breadths and structure in data streams.
The technical report Evolving Neural Networks through Augmenting Topologies, Stanley & Miikkulainen, U Texas at Austin, 2001 presents an interesting design not yet implemented in VLSI chips or in mappings to GPUs via popular programming languages and libraries. It may never be.
Work such as Multi-criteria Evolution of Neural Network Topologies: Balancing Experience and Performance in Autonomous Systems, Chidambaran et. al., U Buffalo, 2018 shows significant advancement in the idea of topological structure and balance.
It is not clear why NEAT or Multi-criteria Evolution would be necessary for time series data analysis. Statistical methods for fitting time series to a model through regression are available outside the artificial network space. Here are some examples of successful analyses already completed.
- Windowed spectral analysis for speech-to-text and music transcription from performance audio
- Analysis of vibration data to correlate changes in strain along the surface of turbine blades under failure testing with 3-D imaging for the development of higher centrifugal forces for use in tactical jet propulsion
- Predictive modelling from oceanographic, atmospheric, and solar observations, aggregated by NOAA for hurricane preparedness
- Face replacement in the movie industry
- Tracking objects for stablilizing images in frames in recorded video
If existing technology cannot serve the purpose, then you must learn C/C++ to generate new low level algorithms to deploy in parallel ways to either multicore von Neumann processors or DSP arrays in GPUs (such as CUDA cores in NVidia's products). For Python use, the PyTorch library would need to be extended to facilitate the topographical morphing, especially for re-entrant training where topographical mutations are required after training has begun.
The road maps of higher level libraries, such as that of TensorFlow, do not contain plans for real time or mid training morphological or topological change.