I am trying to add more data points in my (almost) balanced dataset for training my neural network. I have come across techniques such as SMOTE or Random Over Sampling, but they work best for imbalanced data (as they balance the dataset). How can I do this and is it even worth it?
P.S. I know copying the same data points and appending them at the end doesn't add much value, but can we do it, and can it help to increase the prediction accuracy?
Random over sampling creates duplicates of existing examples, so applying this to your training data would be the same as increasing the weight of the oversampled examples. If it's done to all of the examples uniformly then the effects will probably cancel out. SMOTE, on the other hand, creates synthetic examples that are linear combinations of existing examples. Thus it can be thought of as a type of data augmentation, and in some situations this might improve your model's predictions.
I presume you are attempting to solve a classification problem.
IMO, there's no decision-making template you could follow to know whether to use over sampling or not.I would typically compare results (ROC AUC, PRC curves) across datasets (Original vs Undersampled vs Oversampled) to decide.
You can consider some additional variants of SMOTE like SMOTE NC (SMOTE does not work if any of your predictors is categorical , SMOTE NC does), Borderline SMOTE, K-means SMOTE, as well as ADASYN. Alternately, you can also choose to undersample your majority class using techniques such as ENN, Tomek Links, Instance Hardness, CNN, One sided Selection etc. They usually generate better results than random over/under sampling.
Do note that over/under sampling methods are generally used for imbalanced datasets.
If your dataset is almost balanced, I don't think any sampling method would provide a significant improvement in accuracy. You can also experiment with the focal loss function designed to deal with minor class-imbalance, but I'm not sure how much that would help.
I don't think you can artificially increase the dataset size besides doing data augmentation. For example, for computer vision tasks (like classification and object detection) dealing with images, you can randomly translate and rotate the images, so that the neural network is effectively seeing different "representations" of the same images to alleviate overfitting. This is especially important if your dataset size is small, but does not pertain to solving the class imbalance issue.
If your data is well balanced but small, I would recommend using a simpler algorithm to classify your data.
Does your question pertain to general data augmentation? That is already in heavy use- using transformations while training is very common, and over several epochs the network benefits from learning the new representations. The transformations are applied to all classes, with a probability of transformation ( horizontal flip, for example) specified by the user. If you want to make your almost balanced dataset a balanced one, you can look into specific augmentations that you perform to the (almost) minority class before feeding it to the model. You could look into preprocessing methods that libraries like Keras have made open-source.
Gretel is a good tool for processing data. Facets is good for the visualizations.
Is it worth it? most learners will exhibit bias towards the majority class, and in extreme cases, may ignore the minority class altogether.
It really depends on the goal and requirements of your project. Not because it's desirable it's better for your particular project, if your dataset is almost balanced probably you should continue with something else and consider balancing your data later in the project.
Edit: What's new to learn from the same example?
