13
It's all about Return On Investment. If DL is "worth doing", it's not overkill.
If the cost of using DL (computer cycles, storage, training time) is acceptable, and the data available to train it is plentiful, and if the marginal advantage over alternative algorithms is valuable, then DL is a win.
But, as you suggest, if your problem is amenable to ...
12
When it comes to classification problem in machine learning, the cross entropy and KL divergence are equal. As already stated in the question, the general formula is this:
$$H(p, q) = H(p) + D_{KL}(p||q)$$
Where $p$ a “true” distribution and $q$ is an estimated distribution, $H(p, q)$ is the cross-entropy, $H(p)$ is the entropy and $D$ is the Kullback-...
11
Deep learning is powerful but it is not a superior method than bayesian. They work well in what they are designed to do:
Use deep learning:
Cost for computation is much cheaper than cost of sampling (e.g: natural language processing)
If you have highly non-linear problem
If you want to simplify feature engineering
If you don't have prior distribution (e.g: ...
10
There are many approaches to this kind of problem. The most obvious one is to create new features. The best features I can come up with is to transform the coordinates to spherical coordinates.
I have not found a way to do it in playground, so I just created a few features that should help with this (sin features). After 500 iterations it will saturate and ...
10
There are several advantages:
Some text classification systems are much more accurate than 50%. For example, most spam classification systems are 99.9% accurate, or more. There will be little value to having employees review these labels.
Many text classification systems can output a confidence as well as a label. You can selectively have employees review ...
6
Neural networks are good at classifying. In some situations that comes down to prediction, but not necessarily.
The mathematical reason for the neural networks prowess at classifying is the universal approximation theorem. Which states that a neural network can approximate any continuous real-valued function on a compact subset. The quality of the ...
6
Yes. For instance, the popular softmax regression gives you probability distribution for each class.
Yes. Softmax is a regression over a set of discrete classes.
We can use regression for classification, the most common strategy is to grab the most likely class for the prediction.
5
There is no predefined classifier for any problem. Two main features of a classifier is its cost function and its corresponding weight updation formula. Since, your problem statement requires a huge cost for falsely classifying a particular class one approach will be.
You have to define a cost function which will penalize hugely for missclassifying for that ...
5
Is there anything else I could do to improve accuracy for both training and testing?
Yes, of course, there are a lot of methods if you want to try to improve your accuracy, some that I can mention:
Try to use a more complex model: ResNet, DenseNet, etc.
Try to use other optimizers: Adam, Adadelta, etc.
Tune your hyperparameters (e.g. change your learning ...
5
The mean squared error (MSE), $J(\theta) = \frac{1}{2m}\sum_{i=1}^m(h_\theta(x_i)-y_i)^2$, is not as appropriate as a cost function for classification, given that the MSE makes assumptions about the data that are not appropriate for classification. Though, as an optimization objective, it is still possible to attempt to minimize MSE even in a classification ...
4
Ideally neural networks should be able to find out the function out on it's own without us providing the spherical features. After some experimentation I was able to reach a configuration where we do not need anything except $X_1$ and $X_2$. This net converged after about 1500 epochs which is quite long. So the best way might still be to add additional ...
4
One of the Pinterest's white paper about Human Curation and Convnets powering item-to-item recommendationsarxiv describes implementation of convolutional neural network (CNN) based visual features (VGG2014, Faster R-CNN). This demonstrates the effectiveness of it (such image or object representations) which can improve user engagement. The visual features ...
4
As a general rule of thumb I typically use 10*(# of features) for shallow machine learning models such as Naive Bayes with only 2 classes.
So it all depends on the number of features you will be using. However, the more output classes the more data you will need for proper discrimination. The addition of more classes is not linear but I think you can get ...
4
This will not be that hard of a problem once you have a lot of training data. But, before you have a lot of training data, you will need to get some training data one way or another. You will need a lot of training data for quite a few of the models that will give you a high accuracy. Then, you will probably want to use a Long short term memory recurrent ...
4
The usual parameters to adjust in a k-means:
Number of clusters (recall many clusters can have same label).
Distance definition (euclidean is the most basic, Gauss is an
improvement)
Selection of initial cluster positions.
Data preprocessing (data normalization, ...)
4
Attentive Recurrent Comparators (2017,
Pranav Shyam, Shubham Gupta, Ambedkar Dukkipati) is an interesting paper that helps to answer the question you're wondering, along with a blog post that helps to describe it in easier terms.
The way it's implemented is actually rather intuitive. If you have ever played a "what is different" game with two images ...
3
Suppose you have data:
color height quality
===== ====== =======
green tall good
green short bad
blue tall bad
blue short medium
red tall medium
red short medium
To calculate the entropy for quality in this example:
X = {good, medium, bad}
x1 = {good}, x2 = {bad}, x3 = {medium}
Probability of each x in X:
p1 = 1/6 = 0....
3
It is possible, but is a pretty terrible idea.
There are a few options. One is to not use the GA as a direct classifier, but instead use a GA to learn the parameters of another classification model like a neural network. The basic idea of a GA is that it (very roughly speaking) forms a black-box method for searching an arbitrary space for solutions that ...
3
Text approach:
Use LDA (Latent Dirichlet Allocation). LDA is unsupervised. Feed it in corpuses of text from the various documents (i.e. OCR them and feed LDA the results of OCR). It will then cluster them based on the contents of the text (with or without stop words - at your discretion). If possible, you could do a supervised approach of using a bag-of-...
3
You seem to be wanting some description of the 'style' of an image.
To make that work in general, I'd guess that would actually require quite a lot of pre-processing to present 'texture elements' (rather than pixels) as the basic features.
This is quite speculative, but one approach might be to use Iterated Function Systems as a means of extracting these....
3
The 2015 paper entitled "Applying deep learning to classify pornographic images and videos" applied various types of convnets for detecting pornography. The proposed architecture achieved 94.1% accuracy on the NPDI dataset, which contains 800 videos (400 porn, 200 non-porn "easy" and 200 non-porn "difficult"). More traditional computer vision methods ...
3
Shane Legg and Marcus Hutter proposed one in 2006. The main descriptive quotes (see the paper for the actual formula):
Intelligence measures an agent’s general ability to achieve goals in a wide range of environments
...
It is clear by construction that universal intelligence measures the general ability of an agent to perform well in a very ...
3
One of the challenges of AI is defining Intelligence.
If we could precisely define general intelligence then we could program it into a computer. After all an algorithm is a process so well defined that it can be run on a computer.
Narrow AI can be evaluated on its success at achieving goals in an environment. In domains such as computer vision and speech ...
3
I did a little search and couldn't find any database that has ground truth for aggressiveness. This means that you need to build yourself a database. This might be huge undertaking. Take thousands of messages, and classify them by hand whether they are aggressive or not. This part is quite labor intensive.
Second part is much easier at start but would be ...
3
Data pre-processing and feature extraction are by far the most important part of any machine learning algorithm. It's even more important that the model you choose to do the classification.
Unfortunately, pre-processing and feature extraction are completely different for each type of data. You need to play around with the data yourself to find out what ...
3
My money would be on something much simpler like Naive Bayes.
In my experience for small data NB outperforms the more exotic methods.
Also, if you want to get more value out of your training data, try 10 fold cross validation
3
Yes. Skewed data is one of the biggest problems in AI applications. As you rightly identified, the real world distribution is skewed. Doing a random sampling results in one major issue of an uneven sampling (like in your case). Even worse could happen, all of the samples may fall into a single class and other classes may not even be recognized by your ...
3
I think it depends on you application and what data you have available.
If the prediction of body temperature itself doesn't have to be accurate and classes like COLD, NORMAL, and HOT will suffice, you should stay with a classification. There isn't a cut off but as you increase the number of classes that represent numbers on the same scale, it may become ...
3
@DuttaA has pretty much mentioned the two most appropriate approaches to having this facility. Either the penalty of false positives should be high or the learning rate for the correct class should be high.
I'll give two real-life examples to help you understand it better.
Say you have to teach a teen that substance abuse is injurious to health (eg. ...
3
The usual way to implement this would be to add the new class with data examples.
Some things you need to address:
Sourcing new data for your "other" class.
Ensuring the amount and variation of data in "other" class examples matches how the predictor will be used.
Code examples for this are not necessary, as you would just use the same network design as ...
Only top voted, non community-wiki answers of a minimum length are eligible
