# Why am I not getting a good accuracy but bad precision and recall for this binary classification problem (with an unbalanced dataset)?

I'm working on a simple MLP classification problem where I need to have the output layer as softmax.

The dataset that I've been using needs to pass through a parse that I made to remove NaN and change some text type labels to numeric type (attack = 1; natural = 0). Although, when I ran it, the accuracy, f1-score and recall are always outputting the same result and that's because of TP = TN and FP = FN every time.

The dataset is composed of 128 features and +1 marker label that indicates if it is an attack or a natural event, it's a very unbalanced dataset, ~70% attacks and ~30% naturals. The dataset can be found here Dataset, I'm using the "2-classes".

This is the parse function:

def NormalizeData(data):
# print(f'min={np.min(data)} max={np.max(data)} mean={np.mean(data)}')

if (np.max(data) == np.min(data)):
return data

return (data - np.min(data)) / (np.max(data) - np.min(data))

def parseDataset():
path = '/content/gdrive/MyDrive/Colab Notebooks/binaryAllNaturalPlusNormalVsAttacks/'
# all_files = glob.glob(os.path.join(path, "*.csv"))
# dataset = pd.concat((pd.read_csv(f) for f in all_files))

dataset = dataset.replace([np.inf, -np.inf], np.nan)
dataset = dataset.fillna(dataset.mean()

dataset = dataset.replace(['Attack'], 1)
dataset = dataset.replace(['Natural'], 0)

# dt_attacks = dataset[dataset.marker == 1]
# dt_naturals = dataset[dataset.marker == 0]

csv_x = dataset.iloc[:, :-1].values
csv_y = dataset.iloc[:, 128].values

for i in range(0,128): # 0~127 -> 128 features
csv_x[:,i] = NormalizeData(csv_x[:,i])

return csv_x, csv_y


And the MLP:

# Configuration options
feature_vector_length = 128
num_classes = 2

x, y = parseDataset()

y = to_categorical(y, 2) # [[1. 0.] .... [0. 1.]]

train_x, test_x, train_y, test_y = train_test_split(x, y, test_size=0.30)

train_x = train_x.astype('float32')
test_x = test_x.astype('float32')

#print dataset
print('train_x',train_x.shape, train_x)
print('train_y', train_y.shape, train_y)

print('test_x', test_x.shape, test_x)
print('test_y', test_y.shape, test_y)

input_shape = (128,)

# Create the model
model = Sequential()

# Configure the model and start training
'categorical_accuracy',
Precision(),
keras.metrics.Recall(),
keras.metrics.TruePositives(),
keras.metrics.TrueNegatives(),
keras.metrics.FalsePositives(),
keras.metrics.FalseNegatives()
])

hist = model.fit(train_x,
train_y,
epochs=10,
batch_size=100)

# Test the model after training
test_results = model.evaluate(test_x,
test_y, verbose=0)

print(f'Test results - Loss: {test_results[0]}')
print(f'Accuracy: {test_results[1]}')
print(f'Precision: {test_results[2]}')
print(f'Recall: {test_results[3]}')


Even with this being an unbalanced dataset, I would expect a good accuracy of ~70% but a bad precision and recall and big loss. But I don't really understand why it is happening with this dataset