3
$\begingroup$

Why are the terms classification and prediction used as synonyms especially when it comes to deep learning? For example, a CNN predicts the handwritten digit.

To me, a prediction is telling the next step in a sequence, whereas classification is to put labels on (a finite set of) data.

$\endgroup$
3
  • $\begingroup$ What evidence do you have that classification and prediction are used as synonyms in DL? I would say that's not really the case. $\endgroup$ – Dr. Snoopy Jan 31 '20 at 12:06
  • $\begingroup$ It is very clearly the case when using scikit-learn. All inferences are done, for classifiers and regressors, by running something of the form "model.predict(test)". I have heard quite a few people use the term 'predict' for general classification problems in deep learning which are clearly non-temporal in nature. It is an incorrect use of the word. Prediction implies 'predicting the future' and should generally be used for predicting the future of a time series or the next datum in a data sequence. Classification implies 'guessing what something is at a certain point in time'. $\endgroup$ – Brian O'Donnell Jan 31 '20 at 12:28
  • $\begingroup$ I mostly look in tutorial videos for DL content and I often hear phrases like "The NN predicts if it's a dog or a cat" $\endgroup$ – MScott Jan 31 '20 at 18:47
2
$\begingroup$

Many people confuse and misuse the two terms, classification and prediction (or classify and predict). This is because in many cases classification techniques are being used for prediction purposes which creates part of the confusion to others who then use the term ‘prediction’ (or ‘predict’) inappropriately.

Your understanding of the definitions of classification and prediction is mostly correct and you are absolutely correct that there are many people using the terms synonymously, sometimes correctly but I believe mostly erroneously. There are many good articles elaborating on the two and I have added some links and excerpts at the end of this answer. What these articles don't cover is that many forecasting (i.e. prediction) researchers and practitioners will use conventional classifiers to predict the future state of a time series or data sequence. More advanced researchers and practitioners will use time-recurrent models, which learn temporal patterns. These are still called classifiers but the for purpose of prediction.

There are more papers written on this use of classifiers, conventional and time-recurrent type classifiers, for time series than the use of regressor models!

This adds to the confusion in the data science and machine learning community in the usage of the terms ‘classify’ and ‘predict'.

Galit Shmueli sums it up best in his paper, “To Explain or to Predict?”, where he states: “Conflation between explanation and prediction is common, yet the distinction must be understood for progressing scientific knowledge.”

There is also the opposite problem where people will confuse regression models with classification. See the first article below.

Classification vs. Prediction, by Professor Frank Harrell

Excerpt: By not thinking probabilistically, machine learning advocates frequently utilize classifiers instead of using risk prediction models. The situation has gotten acute: many machine learning experts actually label logistic regression as a classification method (it is not). It is important to think about what classification really implies. Classification is in effect a decision. Optimum decisions require making full use of available data, developing predictions, and applying a loss/utility/cost function to make a decision that, for example, minimizes expected loss or maximizes expected utility. Different end users have different utility functions. In risk assessment this leads to their having different risk thresholds for action. Classification assumes that every user has the same utility function and that the utility function implied by the classification system is that utility function.

To Explain or to Predict?, by Galit Shmueli

Abstract. Statistical modeling is a powerful tool for developing and testing theories by way of causal explanation, prediction, and description. In many disciplines there is near-exclusive use of statistical modeling for causal explanation and the assumption that models with high explanatory power are inherently of high predictive power. Conflation between explanation and prediction is common, yet the distinction must be understood for progressing scientific knowledge. While this distinction has been recognized in the philosophy of science, the statistical literature lacks a thorough discussion of the many differences that arise in the process of modeling for an explanatory versus a predictive goal. The purpose of this article is to clarify the distinction between explanatory and predictive modeling, to discuss its sources, and to reveal the practical implications of the distinction to each step in the modeling process.

What is the difference between classification and prediction?, from KDnuggets

If one does a decision tree analysis, what is the result? A classification? A prediction?

Gregory Piatetsky-Shapiro answers: The decision tree is a classification model, applied to existing data. If you apply it to new data, for which the class is unknown, you also get a prediction of the class.

The assumption is that the new data comes from the similar distribution as the data you used to build your decision tree. In many cases this is a correct assumption and that is why you can use the decision tree for building a predictive model.

When Classification and Prediction are not the same?

Gregory Piatetsky-Shapiro answers: It is a matter of definition. If you are trying to classify existing data, e.g. group patients based on their known medical data and treatment outcome, I would call it a classification. If you use a classification model to predict the treatment outcome for a new patient, it would be a prediction.

gabrielac adds In the book "Data Mining Concepts and Techniques", Han and Kamber's view is that predicting class labels is classification, and predicting values (e.g. using regression techniques) is prediction.

Other people prefer to use "estimation" for predicting continuous values.

$\endgroup$
0
$\begingroup$

They aren't synonyms literally, books never interchange those terms, as they represent two different processes. What they are, though are two similar processes.

Classification can be thought of a "process" that uses specific functions for the generation of one or more discrete values, usually using a cross-entropy function.

Prediction, on the other hand can be thought of a "process" that uses, again specific functions for the generation of continuous values, usually using a linear or multiple dependency model with a MSE loss function.

TL:DR; Classification is for discrete valued dependent variable, Prediction is for continuous valued dependent variable. Both are similar processes with different functions used for learning and estimation of the predicate. You can think of classification as a specific form of prediction.

Hope it helped! Do let me know if I have made any mistakes.

$\endgroup$
3
  • 1
    $\begingroup$ Prediction problems are typically not based on processes with continuous values. Many processes, such as housing prices and the securities, Forex, and commodities markets, are not continuous. They have discrete values. $\endgroup$ – Brian O'Donnell Feb 1 '20 at 15:07
  • $\begingroup$ Let's just agree on a common ground. Let's say prediction involves a lot of discrete values, at no defined intervals, so what's the proper term i can use to distinguish a classification on discrete labels which are again treated as values in the learning context and prediction on "large discrete values". It creates an ambiguity. So to better explain I used the term continuous. Lets say pseudo-continuous? $\endgroup$ – Sri Hari Karthick Feb 3 '20 at 15:09
  • $\begingroup$ It sounds like you are referring to an 'irregular time series', also known as uneven or asynchronous time series. The tick data in the stock market is irregular and so are volcanic eruptions, earthquakes, and sunspots. Our sensors may record them in a periodic manner (i.e. sampling) but their raw mannerism is irregular. The geyser 'Old Faithful' is regular and predictable. Examples of continuous time series are the humidity, the weight of a baby kangaroo, and the sound of the ocean. These are called continuous because there is data at any point in time for the life of the signal. $\endgroup$ – Brian O'Donnell Feb 4 '20 at 1:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.