8 votes
Accepted

What is the "dropout" technique?

Dropout means that every individual data point is only used to fit a random subset of the neurons. This is done to make the neural network more like an ensemble model. That is, just as a random ...
user avatar
7 votes

What does deep learning offer with respect to standard machine learning?

Deep learning allows you to solve complex problems without necessarily being able to specify the important "features" or key input variables for the model in advance. To give an example, a problem ...
user avatar
  • 171
6 votes

What is the "dropout" technique?

The original paper1 that proposed neural network dropout is titled: Dropout: A simple way to prevent neural networks from overfitting. That tittle pretty much explains in one sentence what Dropout ...
user avatar
6 votes
Accepted

What size of neural networks can be trained on current consumer grade GPUs? (1060,1070,1080)

Usually the problem is to fit the model into video RAM. If it does not, you cannot train your model at all without big efforts (like training parts of the model separately). If it does, time is your ...
user avatar
5 votes

What does deep learning offer with respect to standard machine learning?

Deep Learning these days mean a lot of things to a lot of people, its quickly becoming a buzz-word. But so far it still retains two very important conceptual properties: Does away with most feature ...
user avatar
5 votes
Accepted

Why isn't the ElliotSig activation function widely used?

I can't speak for individual researchers, but I can guess why the community as a whole hasn't adopted this activation function. ReLU is just so incredibly cheap. This benefit continues to grow as ...
user avatar
5 votes

How do I improve accuracy and know when to stop training?

Is there anything else I could do to improve accuracy for both training and testing? Yes, of course, there are a lot of methods if you want to try to improve your accuracy, some that I can mention: ...
user avatar
  • 2,551
3 votes
Accepted

How to evaluate an RL algorithm when used in a game?

When you want to compare Reinforcement Learning algorithms, you might want to compare the average rewards they generate and how fast and close they get to the optimal policy. However, in the case of ...
user avatar
  • 365
3 votes

How do I improve accuracy and know when to stop training?

One option is not mentioned by malioboro is getting more data. Getting bigger dataset is almost always improve training results. If it's too hard to obtain more labeled data you can use data ...
user avatar
3 votes

What size of neural networks can be trained on current consumer grade GPUs? (1060,1070,1080)

As a caveat, I’d suggest that unless you’re pushing up against fundamental technological limits, computation speed and resources should be secondary to design rationale when developing a neural ...
user avatar
3 votes

What is the "dropout" technique?

There are some great answers here. The simplest explanation I can give for dropout is that it randomly excludes some neurons and their connections from the network, while training, to stop neurons ...
user avatar
3 votes
Accepted

Why is the generation of deep style images so slow and resource-hungry?

Real time style transfer and neural doodle is very much possible and is an active topic I see users working on to improve upon. The basic idea is to do only feed forward propagation at test time and ...
user avatar
2 votes
Accepted

Can ConvNets be used for real-time object recognition from video feed?

We are getting there, with as usual some trade-off between quality and speed. For example Lecture 8: Spatial Localization and Detection lecture shows some benchmarks (mAP = Mean Average Precision, ...
user avatar
2 votes

Why is the generation of deep style images so slow and resource-hungry?

It is a labor-intensive process, but that does sound excessive. If you have a g2.8xlarge, make sure you are using the using the GPU flags for neural-style, which will cut your render time by an order ...
user avatar
2 votes

What is the "dropout" technique?

I'll try to answer your questions using Geoffrey Hinton's ideas in dropout paper and his Coursera class. What purpose does the "dropout" method serve? Deep neural nets with a large number ...
user avatar
2 votes

What does deep learning offer with respect to standard machine learning?

Deep learning allows you to not know the answer in order to ask the program a question. Their main benefit is their finite ability and flexible nature. The problem with procedural programing to ...
user avatar
  • 151
2 votes
Accepted

Does the quality of training images affect the accuracy of the neural network?

For most of the current use cases, where NNs are used in conjunction with images, the image quality (resolution, color depth) can be low. Consider image classification for example. The CNN extracts ...
user avatar
  • 1,644
2 votes

What size of neural networks can be trained on current consumer grade GPUs? (1060,1070,1080)

It depends on what you need. You can train any size of network on any resource. The problem is the time of training. If you want to train Inception on an average CPU it will take months to converge. ...
user avatar
2 votes

How can we compare the intelligence of AI systems?

There are different ways to compare different kinds of AI techniques. As a starting point, be aware that "AI System" can mean an incredibly broad range of things. In popular culture, we usually think ...
user avatar
2 votes

How to estimate the accuracy upper limit of any CNN model over a computer vision classification task

There is no easy rule for this. You can use transfer learning to select a model that works well on image classification. However the accuracy you achieve will be highly dependent on your training set. ...
user avatar
  • 664
2 votes

How can I merge outputs of two separate layers so that the overall performance improves?

I'm not sure it's possible to help much because this is an experimental question. I'm afraid the only answer comes with testing many different options. I see a little thing that might be making your ...
user avatar
2 votes
Accepted

How is the performance of a model affected by adding a ReLU to fully connected layers?

ReLU is piecewise linear function that outputs the received input directly if it's positive, or outputs a zero. i.e., $max(0, x)$ How significant is adding relu to full connected layers? ReLU, being ...
user avatar
  • 517
2 votes
Accepted

How should I change the hyper-parameters of the C51 algorithm, in order to obtain higher reward?

Here is what I discovered empirically, trial and error. Since tuning the parameters are going to be environment specific, I'll lay out mine to give a better understanding of what I found to work for ...
user avatar
2 votes

Why doesn't the high precision of neural network weights improve accuracy?

First, I have not read and do not have that book. That said, I would interpret that statement in the context of the intractability of guaranteeing that the optimization function will find global ...
user avatar
2 votes

What is a 'degenerate run' in evaluating model performance?

The authors explain their use of the term in the paper: Without the bias correction we observe many degenerate runs, where fine-tuned models fail to outperform the random baseline Specifically, as ...
user avatar
  • 24.5k
1 vote

Is it normal to have the root mean squared error greater on the test dataset than on the training dataset?

It is common to have root mean squared error (RMSE) greater on the test dataset than on the training dataset (this is equal to having accuracy/score higher for model in training dataset than test ...
user avatar
1 vote
Accepted

Why is the effective branching factor used for measuring performance of a heuristic function?

I also walked into that trap the first few times. The difference is the following: $N$ is the number of expanded nodes $b^*$ is the effective branching factor $b^*$ depends on the depth $d$ of the ...
user avatar
  • 126
1 vote

Why is the effective branching factor used for measuring performance of a heuristic function?

As you found $N$ is the number of nodes that are expanded. The cost of expansion of each node is equal to the number of children of that node. Hence, we use $b^*$ for each node. In other words, the ...
user avatar
  • 1,663

Only top scored, non community-wiki answers of a minimum length are eligible