11

I think in this case, you'll probably want to use a genetic algorithm to generate a topology rather than working on your own. I personally like NEAT (NeuroEvolution of Augmenting Topologies). The original NEAT paper involves evolving weights for connections, but if you only want a topology, you can use a weighting algorithm instead. You can also mix ...


9

How Artificial Neural Networks (ANNs) are different from the Biological Neural Networks (BNNs) depends on what you are looking for. We all know that the ANNs are inspired by the Biological ones. Structural differences: In general, a neural network consists of four components: neurons topology: the connectivity path between neurons weights learning ...


5

They are not close, not anymore! [Artificial] Neural Nets vaguely inspired by the connections we previously observed between the neurons of a brain. Initially, there probably was an intention to develop ANN to approximate biological brains. However, the modern working ANNs that we see their applications in various tasks are not designed to provide us a ...


5

The other answer mentions NEAT to generate network weights or topologies. The paper NeuroEvolution: The Importance of Transfer Function Evolution and Heterogeneous Networks, which also gives a short summary of neuroevolution techniques, provides an alternative approach to NEAT. It uses Cartesian Genetic Programming to evolve a multiple activation functions.


4

For a finite value to be 'optimal,' typically you need some benefit from more paired up with some cost for more, and eventually the lines cross because the benefit decreases and the cost increases. Most models will have a reduction in error with more training data, that asymptotically approaches the best the model can do. See this image (from here) as an ...


3

The common statement that Artificial Neural Networks are inspired by the neural structure of brains is only partially true. It is true that Norbert Wiener, Claude Shannon, John von Neuman, and others began the path toward practical AI by developing what they then called the electronic brain. It is also true Artificial networks have functions called ...


2

Meaning of Low Level Goals in Data-Efficient Hierarchical Reinforcement Learning * * Ofir Nachum, Shixiang Gu, Honglak Lee, Sergey Levine Before explaining why a move or a push are goals, let's examine a statement from a prior paper. Although Sergy Levine (Berkeley) is well published and a respected contributor, we can see why Ofir Nachum (Google) is ...


2

In general, the larger the training set, the better. See The Unreasonable effectiveness of Data, though this article is quite dated (written in 2009). Xavier Amatriain, a researcher at Netflix has a Quora answer where he discusses that more data can sometimes hurt algorithms. For deep neural networks in particular, it does not seem that we have hit these ...


2

The networks in NEAT are still implicitly layered. There are neurons that need to be evaluated before other neurons can be evaluated and so this gives us our layers. If you don't know the structure of your network then you can use Kahn's algorithm to find an arbitrary (by arbitrary I just mean one of the possible partially ordered sets) ordering of the ...


2

In recent years the focus has been on layers rather than the more biologically inspired individual nodes. As stated in the comment by thecomplexitytheorist you could use a computational graph, although then you have issues with distribution and you're limited to one framework. I created something in my PhD about the same time as the thesis you reference ...


2

Neural Network equivalents that is not (vanilla) feed forward Neural Nets: Neural net structures such as Recurrent Neural Nets (RNNs) and Convolutional Neural Nets (CNNs), and different architectures within those are good examples. Examples of different architectures within RNNs would would be: Long Short Term Memory (LSTM) or Gated Recurrent Unit (GRU). ...


2

There is indeed an investigation in progress, regarding this topic. A first publication from last march noted that modularity has been done, although not explicitly, since some time ago, but somehow training keeps being monolithic. This paper assess some primary questions about the matter and compares training times and performances on modular and heavily ...


2

A benchmark comparison of systems comprised of separately trained networks relative to single deeper networks would not likely reveal a universally applicable best choice.1 We can see in the literature the increase in the number of larger systems where several artificial networks are combined, along with other types of components. It is to be expected. ...


2

Has this been done? Difficult to prove a negative, but I suspect although plenty of research has been done into finding ideal learning rate values (the need for learning rate at all is an annoyance), it has not been done to the level of suggesting a global function worth approximating. The problem is that learning rate tuning, like other hyperparameter ...


2

The simplistic neural networks that have been given away for free after they prove insufficient by themselves in field use consist solely of two orthogonal dimensions. Layer width — the number of ordinal numbers or floating point numbers that represent the signal path through any given layer comprise of an array of layer elements Network depth — ...


2

Can AI provide a more reliable analysis of the gross effects of carbon emissions on extinctions of species ice-cap melting, and other effects? Yes. The work of Judea Pearl and others over the last 20 years began out of a desire to address uncertainty within AI. Eventually, this led Pearl to become fascinated by the need to quantifiably determine when one ...


2

Check this implementation in TensorFlow Eager: https://github.com/crisbodnar/TensorFlow-NEAT


2

Deep Learning is not a generalization of Hopfield networks. Deep Learning is a "generalization" of the neural networks/connectionism field started by Rumelhart and McClelland. There are two kinds of neural networks: Directed (Perceptron, MLP, ConvNets, RNNs, etc.) Undirected (Hopfield Nets, Boltzmann Machines, Energy-based models, etc.) Any of ...


2

In addition to the ways the term topology is itself used generically to describe the "shape" of various aspects of Machine Learning, the term appears in the field Topological Data Analysis: In applied mathematics, topological data analysis (TDA) is an approach to the analysis of datasets using techniques from topology. Extraction of information from ...


1

I spent some time thinking about it, but I'm aware of only two main meanings. There might be more that aren't coming to me right now though... In local search problems or sometimes in optimization for machine learning, the "topology" of a problem corresponds to the change in the function you're optimizing as you move between adjacent states. If the change ...


1

This is an old area of AI called "Plan Recognition", which has about 3.5 million results in Google Scholar. A lot of the modern work is done with classical search techniques coupled with expert domain knowledge, or related reasoning concepts like Hierarchical Task Networks. I'm not aware of or able to find recent research using deep neural networks for ...


1

To answer the title, there are many other machine learning models, but neural networks work particularly well for some difficult problems (image classification, speech recognition) which is one of the reasons they have gained popularity. Two particularly simple models are the decision tree and the perceptron. These are rather simple models, but they both ...


Only top voted, non community-wiki answers of a minimum length are eligible