# Tag Info

### Should neural nets be deeper the more complex the learning problem is?

Deeper models can have advantages (in certain cases) Most people will answer "yes" to your question, see e.g. Why are neural networks becoming deeper, but not wider? and Why do deep neural networks ...
• 37.1k
Accepted

### How can I automate the choice of the architecture of a neural network for an arbitrary problem?

I think in this case, you'll probably want to use a genetic algorithm to generate a topology rather than working on your own. I personally like NEAT (NeuroEvolution of Augmenting Topologies). The ...
• 226
Accepted

### Why is the merged neural network of AlphaGo Zero more efficient than two separate neural networks?

Why has this merge proven beneficial? If you think about the shared Value/Policy network as consisting of a shared component (the Residual Network layers) with a Value and Policy component on top ...
• 396
Accepted

### Are there any learning algorithms as powerful as "deep" architectures?

Have you read the book The Master Algorithm: by Pedro Domingos? He discusses the present day machine learning algorithms... Their strengths, weaknesses and applications... Deep Neural Network ...

### How can I automate the choice of the architecture of a neural network for an arbitrary problem?

The other answer mentions NEAT to generate network weights or topologies. The paper NeuroEvolution: The Importance of Transfer Function Evolution and Heterogeneous Networks, which also gives a short ...
• 7,176
Accepted

### How can we get a differentiable neural network to count things?

Estimating from an observation is a function, but "really counting" is a process. Feed-forward neural networks can learn arbitrary functions from training examples, but they cannot represent ...
• 26.6k

### How to create an AI to solve a word search?

This sounds like a problem that might be solvable with a LSTM-DQN approach, as described in Language Understanding for Text-based Games using Deep Reinforcement Learning by Narasimhan et al., 2015, ...
• 9,037

### Are there well-established ways of mixing different inputs (e.g. image and numbers)?

A more efficient way would be creating a multi input model, with something like this: ...
• 1,725
Accepted

As I understand, I think you wish to directly try out some deep learning stuff and things like library downloading, tools downloading, and managing all these really stop you from even starting to try ...
• 142

### Are there any learning algorithms as powerful as "deep" architectures?

Deep learning is actually pretty useful (relative to other techniques) precisely when there is no simple mapping between input and output, and features from the raw input need to be aggregated and ...
• 7,176
Accepted

### What linear rectifier is better?

I've read all the papers about PReLU, LeakyReLU (...) and all the claims how it improves this and that but the little dirty secret is: most of the time it doesn't matter at all and you can't go much ...
• 341
Accepted

### Are Neural Net architectures accidental discoveries?

Although there is a strong element of "try and see" that has driven successful architectures, the drivers for what to try are often inspired by underlying theory or knowledge from other disciplines. ...
• 26.6k
Accepted

### Why do Transformers have a sequence limit at inference time?

Transformer models have limited sequence length at inference time because of positional embeddings. But there are workarounds. Self-attention in transformer does not distinguish the order of keys/...
Accepted

### How do I design the network for Deep Q-Network?

What is the strategy to get to a better network? There are a few different strategies that you can use to search for good hyperparameters in reinforcement learning RL, but you should be aware that ...
• 26.6k

### Are Modular Neural Networks more effective than large, monolithic networks at any tasks?

There is indeed an investigation in progress, regarding this topic. A first publication from last march noted that modularity has been done, although not explicitly, since some time ago, but somehow ...
• 501
Accepted

### How to properly use Flatten layer?

The Flatten layer is used for collapsing an ND tensor into a 1D tensor. In your case, the inputs appear to be $28\times28$ images, so Flatten will convert that into a tensor with shape $1\times768$. ...
• 391

### How to compute the output of a neural network produced by NEAT?

The networks in NEAT are still implicitly layered. There are neurons that need to be evaluated before other neurons can be evaluated and so this gives us our layers. If you don't know the structure ...

### What type of reinforcement learning can I do restricted to ~200MB on an average smartphone?

I skimmed through your question and understood that the state/action space is finite, so in this case, RL would be a good option for storage. The most basic RL technique will keep track of a matrix Q ...
• 251
Accepted

### How to teach a model-based reflex agent for doing some task using machine learning methods?

The question is really broad---as stated by @thecomplexitytheorist---so difficult to give a meaningful answer. The following is about a clarification about the problem, and some directions. A model-...
• 1,490

In the perceptron design generally used in Artificial Neural Networks, we know precisely what a single neuron is capable of computing. It can compute a function $$f(x) = g(w^{\top} x),$$ where $x$ ...
• 9,804

### Are Modular Neural Networks more effective than large, monolithic networks at any tasks?

A benchmark comparison of systems comprised of separately trained networks relative to single deeper networks would not likely reveal a universally applicable best choice.1 We can see in the ...
• 7,385
Accepted

### How do neural network topologies affect GPU/TPU acceleration?

The topology of a neural network can have a significant impact on the performance of GPU and TPU acceleration. The most ...
• 1,004

### What is the input to AlphaGo's neural network?

The input to the neural network is a $19 × 19 × 17$ image stack comprising $17$ binary feature planes. $8$ feature planes $X_t$ consist of binary values indicating the presence of the current ...
• 2,018

### Should neural nets be deeper the more complex the learning problem is?

Deeper networks have more learning capacity in the sense that they can fit to more complex data. But at the same time, they are also more prone to overfitting the training data and therefore fails to ...

### Should neural nets be deeper the more complex the learning problem is?

My experience from a tactical standpoint is to start out with a smaller simple model first. Train the model and observe the training accuracy and validation loss and validation accuracy. My ...
• 694
Accepted

### Is the size of a neural network directly linked with an increase in its inteligence?

First of all, there is no real 'intelligence' innate to artificial Neural Networks (NNs). All they do is trying to approximate a mathematical function with a certain degree of generalization (...
• 755

### Why does a neuron in a multi-layer network need several input connections?

There's a few reasons I can think of, though I have not read an explicit description of why it is done this way. It's likely that people just started doing it this way because it's most logical, and ...
• 1,316
Accepted

### Are there deep neural networks that have inputs connected with deeper hidden layers?

This type of connections are called skip or residual connections. There are numerous works which employs this type of mechanism, for example: ResNet, SkipRNN. In addition here you can find a paper ...
• 1,108
Accepted

### What is a unified neural network model?

A unified neural network model consists of one neural network as opposed to other models that rely on two or more neural networks. For example, from page two of the YOLO paper: 2. Unified Detection ...
• 1,698

### Propagating gradients through an "Item Selector" network

As soon as you discretize the selection, i.e. make a hard selection (argmax) instead of a soft selection (softmax), you have a biased gradient. This is because the things you didn't select are not ...
• 1,327

Only top scored, non community-wiki answers of a minimum length are eligible