Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
Why are the terms classification and prediction used as synonyms especially when it comes to deep learning? For example, a CNN predicts the handwritten digit.
To me, a prediction is telling the next step in a sequence, whereas classification is to put labels on (a finite set of) data.
Many people confuse and misuse the two terms, classification and prediction (or classify and predict). This is because in many cases classification techniques are being used for prediction purposes which creates part of the confusion to others who then use the term ‘prediction’ (or ‘predict’) inappropriately.
Your understanding of the definitions of classification and prediction is mostly correct and you are absolutely correct that there are many people using the terms synonymously, sometimes correctly but I believe mostly erroneously. There are many good articles elaborating on the two and I have added some links and excerpts at the end of this answer. What these articles don't cover is that many forecasting (i.e. prediction) researchers and practitioners will use conventional classifiers to predict the future state of a time series or data sequence. More advanced researchers and practitioners will use time-recurrent models, which learn temporal patterns. These are still called classifiers but the for purpose of prediction.
There are more papers written on this use of classifiers, conventional and time-recurrent type classifiers, for time series than the use of regressor models!
This adds to the confusion in the data science and machine learning community in the usage of the terms ‘classify’ and ‘predict'.
Galit Shmueli sums it up best in his paper, “To Explain or to Predict?”, where he states: “Conflation between explanation and prediction is common, yet the distinction must be understood for progressing scientific knowledge.”
There is also the opposite problem where people will confuse regression models with classification. See the first article below.
Classification vs. Prediction, by Professor Frank Harrell
Excerpt: By not thinking probabilistically, machine learning advocates frequently utilize classifiers instead of using risk prediction models. The situation has gotten acute: many machine learning experts actually label logistic regression as a classification method (it is not). It is important to think about what classification really implies. Classification is in effect a decision. Optimum decisions require making full use of available data, developing predictions, and applying a loss/utility/cost function to make a decision that, for example, minimizes expected loss or maximizes expected utility. Different end users have different utility functions. In risk assessment this leads to their having different risk thresholds for action. Classification assumes that every user has the same utility function and that the utility function implied by the classification system is that utility function.
To Explain or to Predict?, by Galit Shmueli
Abstract. Statistical modeling is a powerful tool for developing and testing theories by way of causal explanation, prediction, and description. In many disciplines there is near-exclusive use of statistical modeling for causal explanation and the assumption that models with high explanatory power are inherently of high predictive power. Conflation between explanation and prediction is common, yet the distinction must be understood for progressing scientific knowledge. While this distinction has been recognized in the philosophy of science, the statistical literature lacks a thorough discussion of the many differences that arise in the process of modeling for an explanatory versus a predictive goal. The purpose of this article is to clarify the distinction between explanatory and predictive modeling, to discuss its sources, and to reveal the practical implications of the distinction to each step in the modeling process.
What is the difference between classification and prediction?, from KDnuggets
If one does a decision tree analysis, what is the result? A classification? A prediction?
Gregory Piatetsky-Shapiro answers: The decision tree is a classification model, applied to existing data. If you apply it to new data, for which the class is unknown, you also get a prediction of the class.
The assumption is that the new data comes from the similar distribution as the data you used to build your decision tree. In many cases this is a correct assumption and that is why you can use the decision tree for building a predictive model.
When Classification and Prediction are not the same?
Gregory Piatetsky-Shapiro answers: It is a matter of definition. If you are trying to classify existing data, e.g. group patients based on their known medical data and treatment outcome, I would call it a classification. If you use a classification model to predict the treatment outcome for a new patient, it would be a prediction.
gabrielac adds In the book "Data Mining Concepts and Techniques", Han and Kamber's view is that predicting class labels is classification, and predicting values (e.g. using regression techniques) is prediction.
Other people prefer to use "estimation" for predicting continuous values.
They aren't synonyms literally, books never interchange those terms, as they represent two different processes. What they are, though are two similar processes.
Classification can be thought of a "process" that uses specific functions for the generation of one or more discrete values, usually using a cross-entropy function.
Prediction, on the other hand can be thought of a "process" that uses, again specific functions for the generation of continuous values, usually using a linear or multiple dependency model with a MSE loss function.
TL:DR; Classification is for discrete valued dependent variable, Prediction is for continuous valued dependent variable. Both are similar processes with different functions used for learning and estimation of the predicate. You can think of classification as a specific form of prediction.
Hope it helped! Do let me know if I have made any mistakes.
