Is every process (such as data acquisition, splitting the data for validation, data cleaning, or feature engineering) that is done on the data before we train the model always called the pre-processing part? Or are there some processes that are not included?

  • 3
    $\begingroup$ Despite "data acquisition" which is debatable, I think the other steps are generally considered "preprocessing". $\endgroup$
    – Djib2011
    Dec 15 '19 at 16:40
  • $\begingroup$ I never considered the acquisition or splitting of data to be pre-processing. The reasons being acquiring the data is just the physical act of getting it and not doing anything to it. Similarly, splitting for train/valid/test can be done without any modifications to the data. I accept that there are likely many ways of viewing the different processes and I don't feel strong enough to put it in an answer, so a comment it is. Being quite general, perhaps your question could be reworded or redone to focus on one particular task in the modeling pipeline. $\endgroup$ Dec 15 '19 at 21:48

Data preprocessing consists of all those techniques used to generate the final datasets (with an appropriate size, structure, and format) for the machine learning algorithms or models. Data acquisition should not be part of data preprocessing, but the step preceding it, which gathers the raw data (which may e.g. be noisy).

The book Data Preprocessing in Data Mining (2014), by Salvador García et al., which provides a good overview of the data preprocessing techniques and their connection with data mining and machine learning algorithms and models, defines data preprocessing as follows.

Data preprocessing includes data preparation, compounded by integration, cleaning, normalization and transformation of data; and data reduction tasks; such as feature selection, instance selection, discretization, etc. The result expected after a reliable chaining of data preprocessing tasks is a final dataset, which can be considered correct and useful for further data mining algorithms.

From page 10 onwards, there is a description and categorization of the main data preprocessing techniques. I will just list them, so refer to the book for a definition and explanation of each of these techniques.

  • Data Preparation
    • Data Cleaning
    • Data Transformation
    • Data Integration
    • Data Normalization
    • Missing Data Imputation
    • Noise Identification
  • Data Reduction
    • Feature Selection
    • Instance Selection
    • Discretization
    • Feature Extraction/Instance Generation

Here are two screenshots (from the cited book) that illustrate some of the data preparation

enter image description here

and data reduction techniques.

enter image description here

  • $\begingroup$ Thank you for the reference!.. but from your first statement, what do you think about data augmentation? $\endgroup$
    – malioboro
    Dec 17 '19 at 4:05
  • $\begingroup$ I think data augmentation is part of data pre-processing. It should correspond to the data integration part I mentioned above (maybe combined with other steps of the process). $\endgroup$
    – nbro
    Dec 17 '19 at 4:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.