15 votes
Accepted

Why has the cross-entropy become the classification standard loss function and not Kullback-Leibler divergence?

When it comes to a classification problem in machine learning, the cross-entropy and the KL divergence are equal. As already stated in the question, the general formula is this: $$H(p, q) = H(p) + D_{...
user avatar
  • 1,877
14 votes
Accepted

When is deep learning overkill?

It's all about Return On Investment. If DL is "worth doing", it's not overkill. If the cost of using DL (computer cycles, storage, training time) is acceptable, and the data available to train it is ...
user avatar
  • 671
12 votes

When is deep learning overkill?

Deep learning is powerful but it is not a superior method than bayesian. They work well in what they are designed to do: Use deep learning: Cost for computation is much cheaper than cost of sampling ...
user avatar
  • 1,390
11 votes
Accepted

How should the neural network deal with unexpected inputs?

This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and identically distributed, i....
user avatar
  • 33.8k
10 votes
Accepted

How does text classification reduce manpower costs?

There are several advantages: Some text classification systems are much more accurate than 50%. For example, most spam classification systems are 99.9% accurate, or more. There will be little value ...
user avatar
8 votes

What makes neural networks so good at predictions?

Neural networks are good at classifying. In some situations that comes down to prediction, but not necessarily. The mathematical reason for the neural networks prowess at classifying is the ...
user avatar
6 votes

Do I need classification or regression to predict the availability of a user given some features?

Yes. For instance, the popular softmax regression gives you probability distribution for each class. Yes. Softmax is a regression over a set of discrete classes. We can use regression for ...
user avatar
  • 1,390
6 votes
Accepted

Can ML/DL solve my classification problem?

A simple sanity-check on whether an image classifier can perform a task in theory is: Can a human expert, using the same image plus a list of catgeories that they are familiar with, perform the same ...
user avatar
5 votes
Accepted

How to refine K-means clustering on a data set?

The usual parameters to adjust in a k-means: Number of clusters (recall many clusters can have same label). Distance definition (euclidean is the most basic, Gauss is an improvement) Selection of ...
user avatar
5 votes

How do I improve accuracy and know when to stop training?

Is there anything else I could do to improve accuracy for both training and testing? Yes, of course, there are a lot of methods if you want to try to improve your accuracy, some that I can mention: ...
user avatar
  • 2,551
5 votes
Accepted

Can a deep neural network be trained to classify an integer N1 as being divisible by another integer N2?

There is a recent development in research that was looking into effectiveness of neural networks on arithmetic. Interestingly, feed-forward neural networks (MLPs) with various activation functions as ...
user avatar
  • 384
4 votes
Accepted

How does Pinterest decipher what's on unmarked pictures and categorize them?

One of the Pinterest's white paper about Human Curation and Convnets powering item-to-item recommendationsarxiv describes implementation of convolutional neural network (CNN) based visual features (...
user avatar
  • 10k
4 votes
Accepted

How many training example text classifier needs to be trained?

As a general rule of thumb I typically use 10*(# of features) for shallow machine learning models such as Naive Bayes with only 2 classes. So it all depends on the number of features you will be ...
user avatar
  • 450
4 votes

How to determine if an Amazon review is likely to be fake using text classification

This will not be that hard of a problem once you have a lot of training data. But, before you have a lot of training data, you will need to get some training data one way or another. You will need a ...
user avatar
4 votes

How to implement an "unknown" class in multi-class classification with neural networks?

The usual way to implement this would be to add the new class with data examples. Some things you need to address: Sourcing new data for your "other" class. Ensuring the amount and variation of ...
user avatar
4 votes
Accepted

What is the difference between imitation learning and classification done by experts?

Imitation learning is supervised learning applied to the RL setting. In any general RL algorithm (such as Q-learning), the learning is done on the basis of the reward function. However, consider a ...
user avatar
4 votes
Accepted

Can machine learning algorithms be used to differentiate between small differences in details between images?

Attentive Recurrent Comparators (2017) by Pranav Shyam et al. is an interesting paper that helps to answer the question you're wondering, along with a blog post that helps to describe it in easier ...
user avatar
4 votes

Can I train a neural network incrementally given new daily data?

Yes, this is possible. Continuously extending your training data is known as incremental learning. You might also want to take a look at transfer learning, in which you reuse a trained model for a ...
user avatar
  • 156
4 votes
Accepted

Why not use the MSE instead of the current logistic regression?

The mean squared error (MSE), $J(\theta) = \frac{1}{2m}\sum_{i=1}^m(h_\theta(x_i)-y_i)^2$, is not as appropriate as a cost function for classification, given that the MSE makes assumptions about the ...
user avatar
  • 1,016
4 votes
Accepted

What are causative and exploratory attacks in Adversarial Machine Learning?

When someone is able to do a causative attack it means there is a mechanism by which they are able to input data into the network. Maybe a website where people can input their images and it outputs a ...
user avatar
4 votes
Accepted

Why is there more than one way of calculating the accuracy?

In machine learning, the accuracy is usually defined as the number of correct predictions divided by the total number of predictions. The correct predictions are the true positives ($\mathrm {TP}$) ...
user avatar
  • 33.8k
4 votes
Accepted

Which paper introduced the term "softmax"?

The paper that appears to have introduced the term "softmax" is Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters (...
user avatar
  • 33.8k
3 votes

What algorithm should I use to classify documents?

Text approach: Use LDA (Latent Dirichlet Allocation). LDA is unsupervised. Feed it in corpuses of text from the various documents (i.e. OCR them and feed LDA the results of OCR). It will then cluster ...
user avatar
  • 76
3 votes

Is it possible to classify data using a genetic algorithm?

It is possible, but is a pretty terrible idea. There are a few options. One is to not use the GA as a direct classifier, but instead use a GA to learn the parameters of another classification model ...
user avatar
  • 446
3 votes

How to calculate the entropy in the ID3 decision tree algorithm?

Suppose you have data: ...
user avatar
  • 1,335
3 votes

How to make convnets aware what the image actually is, not what is depicted on it?

You seem to be wanting some description of the 'style' of an image. To make that work in general, I'd guess that would actually require quite a lot of pre-processing to present 'texture elements' (...
user avatar
3 votes
Accepted

How successfully can convnets detect NSFW images?

The 2015 paper entitled "Applying deep learning to classify pornographic images and videos" applied various types of convnets for detecting pornography. The proposed architecture achieved 94.1% ...
user avatar
  • 1,999
3 votes

Do specific units exists for measuring the intelligence of a machine?

Shane Legg and Marcus Hutter proposed one in 2006. The main descriptive quotes (see the paper for the actual formula): Intelligence measures an agent’s general ability to achieve goals in a wide ...
user avatar
3 votes

Do specific units exists for measuring the intelligence of a machine?

One of the challenges of AI is defining Intelligence. If we could precisely define general intelligence then we could program it into a computer. After all an algorithm is a process so well defined ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible