Why does KNN Model return 99% accuracy on dataset with default parameters? [closed]

Question

I am building a model that predicts if a user will like a stock or not based on different features, such as Market Cap, Current Ratio, Sector, Trailing PE, etc. I am going to implement this model in a website, where the model is able to adapt over time to user preferences. I made an example dataset that generally represents how the data that gets passed to the model will look. I have build a KNN model based on that sample dataset, and I get 99% accuracy on the training split and test split. I also get 98%-99% on my classification report even though I have class imbalance, about 500 of one class, and 10500 of another class. My macro average and weighted average are both 99%. I was planning on optimizing this model by hyper tuning, but does anyone know why I got 99% accuracy immediately? This is how a sample of my data looks like above. Here is some of my code:

X = data.drop(["Stocks", "target"], axis = 1)
y = data["target"]
from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer

categorical_features = ["sector", "quoteType"]
one_hot = OneHotEncoder()
transformer = ColumnTransformer([("one_hot",
                                  one_hot,
                                  categorical_features)],
                               remainder = "passthrough")

X = transformer.fit_transform(X)

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X = scaler.fit_transform(X) 
X = pd.DataFrame(X)
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier

np.random.seed(42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2)
knn = KNeighborsClassifier(metric = "minkowski")
knn.fit(X_train, y_train)
knn.score(X_test, y_test)

y_preds = knn.predict(X_test)

from sklearn.metrics import classification_report
print(classification_report(y_test, y_preds))

This produces:
precision    recall  f1-score   support

           0       0.98      1.00      0.99      1227
           1       1.00      0.98      0.99      1012

    accuracy                           0.99      2239
   macro avg       0.99      0.99      0.99      2239
weighted avg       0.99      0.99      0.99      2239

Answer 1

First assess whether the accuracy is unreasonably high. You as the domain expert are the best arbiter of that. Keep in mind that a high accuracy is not always impossible. For example, 99% accuracy can be seen on MNIST digits dataset. It is also possible that you have data leakage, so assess whether this may be at play. The second thing you can do is perform an n-fold cross-validation. This will provide you with a better estimate (and error bounds) for your model's performance. If your accuracy remains high on all folds, then you can exclude a spurious split on your holdout validation.

Another issue is that you claim that you have an imbalanced dataset. However, your print out shows roughly equal support for 0 and 1 classes.

Finally, you are performing preprocessing of your features on the total dataset. Instead, you should apply fit_transform on your training data, and the just apply the transform on your test data.