7
votes
What does deep learning offer with respect to standard machine learning?
Deep learning allows you to solve complex problems without necessarily being able to specify the important "features" or key input variables for the model in advance.
To give an example, a problem ...
- 171
6
votes
Accepted
Why isn't the ElliotSig activation function widely used?
I can't speak for individual researchers, but I can guess why the community as a whole hasn't adopted this activation function.
ReLU is just so incredibly cheap. This benefit continues to grow as ...
- 2,064
6
votes
Accepted
What size of neural networks can be trained on current consumer grade GPUs? (1060,1070,1080)
Usually the problem is to fit the model into video RAM.
If it does not, you cannot train your model at all without big efforts (like training parts of the model separately).
If it does, time is your ...
- 176
6
votes
What is the "dropout" technique?
The original paper1 that proposed neural network dropout is titled: Dropout: A simple way to prevent neural networks from overfitting. That tittle pretty much explains in one sentence what Dropout ...
- 850
5
votes
What does deep learning offer with respect to standard machine learning?
Deep Learning these days mean a lot of things to a lot of people, its quickly becoming a buzz-word. But so far it still retains two very important conceptual properties:
Does away with most feature ...
- 236
5
votes
How do I improve accuracy and know when to stop training?
Is there anything else I could do to improve accuracy for both training and testing?
Yes, of course, there are a lot of methods if you want to try to improve your accuracy, some that I can mention:
...
- 2,819
3
votes
Accepted
How should I change the hyper-parameters of the C51 algorithm, in order to obtain higher reward?
Here is what I discovered empirically, trial and error. Since tuning the parameters are going to be environment specific, I'll lay out mine to give a better understanding of what I found to work for ...
3
votes
Accepted
How to evaluate an RL algorithm when used in a game?
When you want to compare Reinforcement Learning algorithms, you might want to compare the average rewards they generate and how fast and close they get to the optimal policy. However, in the case of ...
- 375
3
votes
How do I improve accuracy and know when to stop training?
One option is not mentioned by malioboro is getting more data. Getting bigger dataset is almost always improve training results. If it's too hard to obtain more labeled data you can use data ...
- 735
3
votes
What size of neural networks can be trained on current consumer grade GPUs? (1060,1070,1080)
As a caveat, I’d suggest that unless you’re pushing up against fundamental technological limits, computation speed and resources should be secondary to design rationale when developing a neural ...
- 416
3
votes
Accepted
Does the quality of training images affect the accuracy of the neural network?
For most of the current use cases, where NNs are used in conjunction with images, the image quality (resolution, color depth) can be low.
Consider image classification for example. The CNN extracts ...
- 1,684
3
votes
What is the "dropout" technique?
There are some great answers here. The simplest explanation I can give for dropout is that it randomly excludes some neurons and their connections from the network, while training, to stop neurons ...
- 131
2
votes
What is the "dropout" technique?
I'll try to answer your questions using Geoffrey Hinton's ideas in dropout paper and his Coursera class.
What purpose does the "dropout" method serve?
Deep neural nets with a large number ...
- 121
2
votes
What does deep learning offer with respect to standard machine learning?
Deep learning allows you to not know the answer in order to ask the program a question. Their main benefit is their finite ability and flexible nature.
The problem with procedural programing to ...
- 151
2
votes
How can we compare the intelligence of AI systems?
There are different ways to compare different kinds of AI techniques.
As a starting point, be aware that "AI System" can mean an incredibly broad range of things. In popular culture, we usually think ...
- 9,392
2
votes
How do I check that the combination of these models is good?
Goodness is subjective. Reliable knowledge isn't possible with that flimsy a quality objective.
The sturdy objective criteria you gave is 95%, so it is bad by that criteria. (I'm assuming that the ...
- 7,513
2
votes
Accepted
Relative compute time for each type of layer in a neural network
The graph seems to show the relative compute time of each component of the learning process, independent of the layer, using the typical meaning of layer in the context of artificial networks.
The ...
- 7,513
2
votes
What size of neural networks can be trained on current consumer grade GPUs? (1060,1070,1080)
It depends on what you need. You can train any size of network on any resource. The problem is the time of training. If you want to train Inception on an average CPU it will take months to converge. ...
- 366
2
votes
How to estimate the accuracy upper limit of any CNN model over a computer vision classification task
There is no easy rule for this. You can use transfer learning to select a model that works well on image classification. However the accuracy you achieve will be highly dependent on your training set. ...
- 724
2
votes
How can I merge outputs of two separate layers so that the overall performance improves?
I'm not sure it's possible to help much because this is an experimental question. I'm afraid the only answer comes with testing many different options.
I see a little thing that might be making your ...
- 121
2
votes
Accepted
Why is the effective branching factor used for measuring performance of a heuristic function?
I also walked into that trap the first few times. The difference is the following:
$N$ is the number of expanded nodes
$b^*$ is the effective branching factor
$b^*$ depends on the depth $d$ of the ...
- 136
2
votes
Accepted
How is the performance of a model affected by adding a ReLU to fully connected layers?
ReLU is piecewise linear function that outputs the received input directly if it's positive, or outputs a zero. i.e., $max(0, x)$
How significant is adding relu to full connected layers?
ReLU, being ...
- 539
2
votes
Why doesn't the high precision of neural network weights improve accuracy?
First, I have not read and do not have that book.
That said, I would interpret that statement in the context of the intractability of guaranteeing that the optimization function will find global ...
- 961
2
votes
What is a 'degenerate run' in evaluating model performance?
The authors explain their use of the term in the paper:
Without the bias correction we observe many degenerate runs, where fine-tuned models fail to outperform the random baseline
Specifically, as ...
- 33.3k
2
votes
Accepted
Classifier performance if data are deterministic
This is an important question that goes back to the fundamentals of machine learning theory.
The primary objective of using machine learning to solve a problem is as the following:
For a given set of ...
- 441
2
votes
What does that mean if my precision, F1-score are very high, but my ROC AUC score is around 0.5?
If you are certain that your calculations are correct, then there may be class imbalance in your dataset. The metrics precision, ...
- 997
2
votes
Accepted
Batch wise Inference to speed up Muzero's MCTS
Batching: A Good Idea
You're right, batching is a great way to speed up AlphaZero or MuZero self-play! Your proposed solution of running multiple games in parallel is the easiest way to achieve some ...
- 483
1
vote
Accepted
How would the performance of federated learning compare to the performance of centralized machine learning when the data is i.i.d.?
There are some works that do this comparison. Briefly, it's been observed that the performance of models trained via FL drops as data distributions between participating agents differ. When data is ...
- 1,040
1
vote
Can the attention mechanism improve the performance in the case of short sequences?
They shouldn't have any issues with short sequences, as short dependencies are easier to learn. The only difficult cases are long dependencies which is where most of the research is geared at. However,...
- 1,690
1
vote
Is it normal to have the root mean squared error greater on the test dataset than on the training dataset?
It is common to have root mean squared error (RMSE) greater on the test dataset than on the training dataset (this is equal to having accuracy/score higher for model in training dataset than test ...
Only top scored, non community-wiki answers of a minimum length are eligible