0
$\begingroup$

I'm working on a multiclassification task using LSTM algorithm, i generated my roc curve plots but they give scores like 1 , 0.99, 0.97 however i have an accuracy of 0.97, Precision 0.65, Sensitivity/Recall 0.62, and Specificity 0.62. is it normal ? here's my roc curve:

enter image description here

here's my code to generate the ROC curve

from sklearn.metrics import roc_curve, auc
import matplotlib.pyplot as plt
# Predict probabilities for each class
Y_pred_prob = model.predict(X_test_reshaped)

# Calculate ROC curve for each class
fpr = dict()
tpr = dict()
roc_auc = dict()

n_classes = Y_train_onehot.shape[1]  # Number of classes
class_names = {0: 'Benign traffic', 1: 'Brute force', 2: 'DDoS', 3: 'DoS', 
               4: 'Mirai', 5: 'Reconnaissance', 6: 'Spoofing', 7: 'Web-based'} 

for i in range(n_classes):
    fpr[i], tpr[i], _ = roc_curve(Y_test_onehot[:, i], Y_pred_prob[:, i])
    roc_auc[i] = auc(fpr[i], tpr[i])


# Plot ROC curves for each class with labels
plt.figure(figsize=(8, 6))

for i in range(n_classes):
    plt.plot(fpr[i], tpr[i], label=f' {class_names[i]} (AUC = {roc_auc[i]:.2f})')

plt.plot([0, 1], [0, 1], 'k--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC Curve for Multiclass Classification')
plt.legend(loc="lower right")
plt.show()
$\endgroup$
1
  • $\begingroup$ Depends on the # of examples per class. Make sure X_test_reshaped and Y_test_onehot are lined up. Play with threshold to see how sens/spec changes. $\endgroup$
    – rhn89
    Sep 15, 2023 at 18:31

0

You must log in to answer this question.