14 votes

Should neural nets be deeper the more complex the learning problem is?

Deeper models can have advantages (in certain cases) Most people will answer "yes" to your question, see e.g. Why are neural networks becoming deeper, but not wider? and Why do deep neural networks ...
user avatar
  • 34.3k
11 votes
Accepted

How can I automate the choice of the architecture of a neural network for an arbitrary problem?

I think in this case, you'll probably want to use a genetic algorithm to generate a topology rather than working on your own. I personally like NEAT (NeuroEvolution of Augmenting Topologies). The ...
user avatar
  • 226
7 votes
Accepted

Are there any learning algorithms as powerful as "deep" architectures?

Have you read the book The Master Algorithm: by Pedro Domingos? He discusses the present day machine learning algorithms... Their strengths, weaknesses and applications... Deep Neural Network ...
user avatar
6 votes
Accepted

Why is the merged neural network of AlphaGo Zero more efficient than two separate neural networks?

Why has this merge proven beneficial? If you think about the shared Value/Policy network as consisting of a shared component (the Residual Network layers) with a Value and Policy component on top ...
user avatar
  • 386
5 votes

How can I automate the choice of the architecture of a neural network for an arbitrary problem?

The other answer mentions NEAT to generate network weights or topologies. The paper NeuroEvolution: The Importance of Transfer Function Evolution and Heterogeneous Networks, which also gives a short ...
user avatar
4 votes

How to create an AI to solve a word search?

This sounds like a problem that might be solvable with a LSTM-DQN approach, as described in Language Understanding for Text-based Games using Deep Reinforcement Learning by Narasimhan et al., 2015, ...
user avatar
4 votes

Are there well-established ways of mixing different inputs (e.g. image and numbers)?

A more efficient way would be creating a multi input model, with something like this: ...
user avatar
  • 1,715
4 votes
Accepted

Are there ways to learn and practice Deep Learning without downloading and installing anything?

As I understand, I think you wish to directly try out some deep learning stuff and things like library downloading, tools downloading, and managing all these really stop you from even starting to try ...
user avatar
  • 142
4 votes

Are there any learning algorithms as powerful as "deep" architectures?

Deep learning is actually pretty useful (relative to other techniques) precisely when there is no simple mapping between input and output, and features from the raw input need to be aggregated and ...
user avatar
4 votes
Accepted

What linear rectifier is better?

I've read all the papers about PReLU, LeakyReLU (...) and all the claims how it improves this and that but the little dirty secret is: most of the time it doesn't matter at all and you can't go much ...
user avatar
4 votes
Accepted

Are Neural Net architectures accidental discoveries?

Although there is a strong element of "try and see" that has driven successful architectures, the drivers for what to try are often inspired by underlying theory or knowledge from other disciplines. ...
user avatar
  • 24.4k
4 votes
Accepted

How can we get a differentiable neural network to count things?

Estimating from an observation is a function, but "really counting" is a process. Feed-forward neural networks can learn arbitrary functions from training examples, but they cannot represent ...
user avatar
  • 24.4k
4 votes
Accepted

How do I design the network for Deep Q-Network?

What is the strategy to get to a better network? There are a few different strategies that you can use to search for good hyperparameters in reinforcement learning RL, but you should be aware that ...
user avatar
  • 24.4k
3 votes

Are Modular Neural Networks more effective than large, monolithic networks at any tasks?

There is indeed an investigation in progress, regarding this topic. A first publication from last march noted that modularity has been done, although not explicitly, since some time ago, but somehow ...
user avatar
  • 481
3 votes
Accepted

How to properly use Flatten layer?

The Flatten layer is used for collapsing an ND tensor into a 1D tensor. In your case, the inputs appear to be $28\times28$ images, so Flatten will convert that into a tensor with shape $1\times768$. ...
user avatar
  • 391
3 votes
Accepted

Why do Transformers have a sequence limit at inference time?

Transformer models have limited sequence length at inference time because of positional embeddings. But there are workarounds. Self-attention in transformer does not distinguish the order of keys/...
user avatar
2 votes

How to compute the output of a neural network produced by NEAT?

The networks in NEAT are still implicitly layered. There are neurons that need to be evaluated before other neurons can be evaluated and so this gives us our layers. If you don't know the structure ...
user avatar
2 votes

What type of reinforcement learning can I do restricted to ~200MB on an average smartphone?

I skimmed through your question and understood that the state/action space is finite, so in this case, RL would be a good option for storage. The most basic RL technique will keep track of a matrix Q ...
user avatar
  • 251
2 votes
Accepted

How to teach a model-based reflex agent for doing some task using machine learning methods?

The question is really broad---as stated by @thecomplexitytheorist---so difficult to give a meaningful answer. The following is about a clarification about the problem, and some directions. A model-...
user avatar
  • 1,480
2 votes

Are artificial networks based on the perceptron design inherently limiting?

In the perceptron design generally used in Artificial Neural Networks, we know precisely what a single neuron is capable of computing. It can compute a function $$f(x) = g(w^{\top} x),$$ where $x$ ...
user avatar
  • 9,419
2 votes

Should neural nets be deeper the more complex the learning problem is?

Deeper networks have more learning capacity in the sense that they can fit to more complex data. But at the same time, they are also more prone to overfitting the training data and therefore fails to ...
user avatar
2 votes

What is the input to AlphaGo's neural network?

The input to the neural network is a $19 × 19 × 17$ image stack comprising $17$ binary feature planes. $8$ feature planes $X_t$ consist of binary values indicating the presence of the current ...
user avatar
2 votes

Should neural nets be deeper the more complex the learning problem is?

My experience from a tactical standpoint is to start out with a smaller simple model first. Train the model and observe the training accuracy and validation loss and validation accuracy. My ...
user avatar
  • 664
2 votes
Accepted

Is the size of a neural network directly linked with an increase in its inteligence?

First of all, there is no real 'intelligence' innate to artificial Neural Networks (NNs). All they do is trying to approximate a mathematical function with a certain degree of generalization (...
user avatar
  • 705
2 votes

Why does a neuron in a multi-layer network need several input connections?

There's a few reasons I can think of, though I have not read an explicit description of why it is done this way. It's likely that people just started doing it this way because it's most logical, and ...
user avatar
  • 1,246
2 votes
Accepted

Are there deep neural networks that have inputs connected with deeper hidden layers?

This type of connections are called skip or residual connections. There are numerous works which employs this type of mechanism, for example: ResNet, SkipRNN. In addition here you can find a paper ...
user avatar
  • 1,098
2 votes
Accepted

What is a unified neural network model?

A unified neural network model consists of one neural network as opposed to other models that rely on two or more neural networks. For example, from page two of the YOLO paper: 2. Unified Detection ...
user avatar
1 vote

Why are denser layers needed in computer vision neural nets?

Convolutional layers are added in order to extract features from the image (like edges, corners, textures). After extracting those features, you feed them to a fully connected neural network to get ...
user avatar
1 vote

RL: What should be the output of the NN for an agent trying to learn how to play a game?

It depends on whether the action is part of the input or output of a neural network estimating the Q-value(state, action). The network on the left has the state as input and outputs one scalar value ...
user avatar
  • 11

Only top scored, non community-wiki answers of a minimum length are eligible