0
$\begingroup$

I am creating a NN in tensorflow keras. the inputs are all float and the output is a class.

The output currently encoded as a float, but only has 4 values (0,1,2,3).

My model is similar to this:

model = tf.keras.Sequential([
    normalize,
    layers.Dense(128, activation='relu'),
    layers.Dense(254, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(254, activation='relu'),
    layers.Dense(128, activation='relu'),
    layers.Dense(4)
])

model.compile(loss = tf.keras.losses.SparseCategoricalCrossentropy(), metrics=['accuracy'],
                           optimizer = tf.optimizers.Adam())

history=model.fit(data_set_features, data_set_labels,validation_split=0.33, epochs=100)

is the model last layer correct?

what type of activation should I use and loss function?

$\endgroup$
1
  • $\begingroup$ Programming questions are off-topic here. Please, edit your post to clarify how this is not a programming question. $\endgroup$ – nbro Feb 19 at 10:32
1
$\begingroup$

Since you're using categorical cross-entropy loss, the last layer (output layer) should come with softmax activation instead of identity (as being blank in layers.Dense(4)).

model = tf.keras.Sequential([
    normalize,
    ...,
    layers.Dense(4, activation=tf.keras.activations.softmax)
])

And SparseCategoricalCrossentropy is different from CategoricalCrossentropy, for categorical cross-entropy, the shape of output and label should be the same, for example:

When using categorical cross-entropy (example labels in one-hot):

Output: [[.1,.0,.9,.0],...] <-- Prediction of 90% for the 3rd class, sum=100%
Label:  [[ 0, 0, 1, 0],...] <-- Third class holds 100% classification possibility

When using sparse categorical cross-entry (labels are class indices):

Output: [[.1,.0,.9,.0],...] <-- Prediction of 90% for the 3rd class, sum=100%
Label:  [[2]          ,...] <-- Third class index is 2
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.