13
It's all about Return On Investment. If DL is "worth doing", it's not overkill.
If the cost of using DL (computer cycles, storage, training time) is acceptable, and the data available to train it is plentiful, and if the marginal advantage over alternative algorithms is valuable, then DL is a win.
But, as you suggest, if your problem is amenable to ...
13
When it comes to a classification problem in machine learning, the cross-entropy and the KL divergence are equal.
As already stated in the question, the general formula is this:
$$H(p, q) = H(p) + D_{KL}(p \parallel q),$$
where $p$ is the "true"/target distribution and $q$ is an estimated distribution, $H(p, q)$ is the cross-entropy, $H(p)$ is the ...
12
Deep learning is powerful but it is not a superior method than bayesian. They work well in what they are designed to do:
Use deep learning:
Cost for computation is much cheaper than cost of sampling (e.g: natural language processing)
If you have highly non-linear problem
If you want to simplify feature engineering
If you don't have prior distribution (e.g: ...
11
This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and identically distributed, i.e., you do not expect the data to come from a distribution different than the distribution from which your training data comes, so there's also the implicit ...
10
There are several advantages:
Some text classification systems are much more accurate than 50%. For example, most spam classification systems are 99.9% accurate, or more. There will be little value to having employees review these labels.
Many text classification systems can output a confidence as well as a label. You can selectively have employees review ...
7
Neural networks are good at classifying. In some situations that comes down to prediction, but not necessarily.
The mathematical reason for the neural networks prowess at classifying is the universal approximation theorem. Which states that a neural network can approximate any continuous real-valued function on a compact subset. The quality of the ...
6
Yes. For instance, the popular softmax regression gives you probability distribution for each class.
Yes. Softmax is a regression over a set of discrete classes.
We can use regression for classification, the most common strategy is to grab the most likely class for the prediction.
6
A simple sanity-check on whether an image classifier can perform a task in theory is:
Can a human expert, using the same image plus a list of catgeories that they are familiar with, perform the same task?
It is important you only consider the contents of the image (or in general the data you are prepared to supply to the classifier) and the expert's ...
5
The usual parameters to adjust in a k-means:
Number of clusters (recall many clusters can have same label).
Distance definition (euclidean is the most basic, Gauss is an
improvement)
Selection of initial cluster positions.
Data preprocessing (data normalization, ...)
5
Is there anything else I could do to improve accuracy for both training and testing?
Yes, of course, there are a lot of methods if you want to try to improve your accuracy, some that I can mention:
Try to use a more complex model: ResNet, DenseNet, etc.
Try to use other optimizers: Adam, Adadelta, etc.
Tune your hyperparameters (e.g. change your learning ...
5
There is a recent development in research that was looking into effectiveness of neural networks on arithmetic. Interestingly, feed-forward neural networks (MLPs) with various activation functions as well as LSTMs (RNNs which are Turing-complete) are not able to model simple arithmetic operations (e.g. addition/multiplication), they designed a new logic unit ...
4
One of the Pinterest's white paper about Human Curation and Convnets powering item-to-item recommendationsarxiv describes implementation of convolutional neural network (CNN) based visual features (VGG2014, Faster R-CNN). This demonstrates the effectiveness of it (such image or object representations) which can improve user engagement. The visual features ...
4
As a general rule of thumb I typically use 10*(# of features) for shallow machine learning models such as Naive Bayes with only 2 classes.
So it all depends on the number of features you will be using. However, the more output classes the more data you will need for proper discrimination. The addition of more classes is not linear but I think you can get ...
4
This will not be that hard of a problem once you have a lot of training data. But, before you have a lot of training data, you will need to get some training data one way or another. You will need a lot of training data for quite a few of the models that will give you a high accuracy. Then, you will probably want to use a Long short term memory recurrent ...
4
Imitation learning is supervised learning applied to the RL setting.
In any general RL algorithm (such as Q-learning), the learning is done on the basis of the reward function. However, consider a scenario where you have available the optimal policy in the form of a table, mapping each state to each action. In this scenario you will not care about the ...
4
Attentive Recurrent Comparators (2017) by Pranav Shyam et al. is an interesting paper that helps to answer the question you're wondering, along with a blog post that helps to describe it in easier terms.
The way it's implemented is actually rather intuitive. If you have ever played a "what is different" game with two images usually what you'd do is look ...
4
The mean squared error (MSE), $J(\theta) = \frac{1}{2m}\sum_{i=1}^m(h_\theta(x_i)-y_i)^2$, is not as appropriate as a cost function for classification, given that the MSE makes assumptions about the data that are not appropriate for classification. Though, as an optimization objective, it is still possible to attempt to minimize MSE even in a classification ...
4
When someone is able to do a causative attack it means there is a mechanism by which they are able to input data into the network. Maybe a website where people can input their images and it outputs a guess on what is in the picture and then you click if it got it right or not. If you continue to input images and lie to it it will obviously get worse and ...
4
In machine learning, the accuracy is usually defined as the number of correct predictions divided by the total number of predictions. The correct predictions are the true positives ($\mathrm {TP}$) and true negatives ($\mathrm {TN}$), so the usual formula to calculate the accuracy is the following one (your first one).
\begin{align}
\text{Accuracy}=\frac {\...
4
As far as I know, the sigmoid is often used as the activation function of the output layer mainly because it is a convenient way of producing an output $p \in [0, 1]$, which can be interpreted as a probability, although that can be misleading or even wrong (if you interpret it as an uncertainty too).
You may require the output of the neural network to be a ...
3
Text approach:
Use LDA (Latent Dirichlet Allocation). LDA is unsupervised. Feed it in corpuses of text from the various documents (i.e. OCR them and feed LDA the results of OCR). It will then cluster them based on the contents of the text (with or without stop words - at your discretion). If possible, you could do a supervised approach of using a bag-of-...
3
It is possible, but is a pretty terrible idea.
There are a few options. One is to not use the GA as a direct classifier, but instead use a GA to learn the parameters of another classification model like a neural network. The basic idea of a GA is that it (very roughly speaking) forms a black-box method for searching an arbitrary space for solutions that ...
3
Suppose you have data:
color height quality
===== ====== =======
green tall good
green short bad
blue tall bad
blue short medium
red tall medium
red short medium
To calculate the entropy for quality in this example:
X = {good, medium, bad}
x1 = {good}, x2 = {bad}, x3 = {medium}
Probability of each x in X:
p1 = 1/6 = 0....
3
You seem to be wanting some description of the 'style' of an image.
To make that work in general, I'd guess that would actually require quite a lot of pre-processing to present 'texture elements' (rather than pixels) as the basic features.
This is quite speculative, but one approach might be to use Iterated Function Systems as a means of extracting these....
3
The 2015 paper entitled "Applying deep learning to classify pornographic images and videos" applied various types of convnets for detecting pornography. The proposed architecture achieved 94.1% accuracy on the NPDI dataset, which contains 800 videos (400 porn, 200 non-porn "easy" and 200 non-porn "difficult"). More traditional computer vision methods ...
3
Shane Legg and Marcus Hutter proposed one in 2006. The main descriptive quotes (see the paper for the actual formula):
Intelligence measures an agent’s general ability to achieve goals in a wide range of environments
...
It is clear by construction that universal intelligence measures the general ability of an agent to perform well in a very wide range of ...
3
One of the challenges of AI is defining Intelligence.
If we could precisely define general intelligence then we could program it into a computer. After all an algorithm is a process so well defined that it can be run on a computer.
Narrow AI can be evaluated on its success at achieving goals in an environment. In domains such as computer vision and speech ...
3
I did a little search and couldn't find any database that has ground truth for aggressiveness. This means that you need to build yourself a database. This might be huge undertaking. Take thousands of messages, and classify them by hand whether they are aggressive or not. This part is quite labor intensive.
Second part is much easier at start but would be ...
3
Data pre-processing and feature extraction are by far the most important part of any machine learning algorithm. It's even more important that the model you choose to do the classification.
Unfortunately, pre-processing and feature extraction are completely different for each type of data. You need to play around with the data yourself to find out what ...
Only top voted, non community-wiki answers of a minimum length are eligible