TensorFlow training evaluation

Question

I'm having trouble with accuracy evaluation at the end of Session.

Training process looks like this:

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    epochs = 10
    batch_size = 128

    for e in range(epochs):
        shuffle_indices = np.random.permutation(np.arange(len(inputx)))
        X_train = inputx[shuffle_indices]
        y_train = inputy[shuffle_indices]

        epoch_loss = 0
        for i in range(int(len(inputx) // batch_size)):
            start = i * batch_size
            batch_x = X_train[start:start + batch_size]
            batch_y = y_train[start:start + batch_size]

            _, c = sess.run([optimizer, cost], feed_dict={x: batch_x, y: batch_y})
            epoch_loss += c

        print(e + 1, '/', epochs, 'loss:', epoch_loss)

At the end of this session I want to evaluate reached accuracy by calculating mean value of percentual differences between prediction and testy for each of 3 output layer neurons.

percentual_differences_for_neuron1 = # insert your advice here
accuracy_for_neuron1 = tf.reduce_mean(percentual_differences_for_neuron1)
print('Accuracy of neuron 1 is', '''another advice''')

What's the best way of doing this in TensorFlow?

Answer 1

Suppose you are dealing with a classification problem you can simply define

def error_rate_classification(p, t):
    return np.mean(p != t)

where p and t stand for predictions and targets resp.

If you deal with a regression problem the you should modify a bit this function. For example :

def error_rate_regression(p, t):
    return np.mean(np.abs(p-t) < 1e-2) # the criteria is not general

Note that for regression problem assessing the accuracy of the prediction is not as simple as it is in classification problem

Edit

In regression case, you can use the R2 regression standard metrics. Suppose yi is the ground truth value of the i-th element of your test (or validation) set and f(xi) is your i-th input prediction. The R2 is then

R2 = 1 - sum( (yi - f(xi))**2 ) / sum( (yi - y_mean)**2 )

where y_mean is the mean of the ground truth values of the test ( or validation) test. You can find other examples in the scikit-learn regression metrics here