0
$\begingroup$

TFJS:

const AdadeltaOptimizer = tf.train.adadelta();

const model = tf.sequential();
model.add(tf.layers.dense({ units: 5, activation: 'sigmoid', inputShape: [1]}));
model.add(tf.layers.dense({ units: 1, activation: 'sigmoid'}));
model.compile({loss: 'meanSquaredError', optimizer: AdadeltaOptimizer, metrics: ['accuracy']});
const xs = tf.tensor2d([[1], [2], [3], [4], [5], [6], [7], [8], [9]]);
const ys = tf.tensor2d([[0], [0], [0], [0], [0], [1], [1], [1], [1]]);

model.fit(xs, ys, {
    epochs: 2000
});

model.predict(xs).print();
tf.losses.meanSquaredError(ys, model.predict(xs)).print();

Keras

import keras
import numpy as np

model = keras.Sequential()
model.add(keras.layers.Dense(5, activation='sigmoid', input_shape=(1,)))
model.add(keras.layers.Dense(1, activation='sigmoid'))
model.compile('adadelta', 'mse', metrics=['accuracy'])

xs = np.array(range(1, 10))
ys = np.array([[0] for _ in range(5)] + [[1] for _ in range(4)])

model.fit(xs, ys, epochs=2000)
print(model.predict(xs))

Outputs

Those are example results of both networks:

Keras:

9/9 [==============================] - 0s 82us/step - loss: 0.0155 - acc: 1.0000
[[0.00141504]
 [0.00251564]
 [0.00749135]
 [0.0397768 ]
 [0.26316088]
 [0.74775267]
 [0.9371815 ]
 [0.9762724 ]
 [0.986349  ]]

The Keras model was run 10 times and displayed correct results each time.

The TFJS model didn't output the right results even once. It was tested 1 times and did usually find the right trend, but the numbers were either too high or low. It is remarkable that the loss for most of the outputs is approximately 0,25. When one plots the loss function for the network when it consists of only one input and one sigmoid output neuron (thus has only one weight and bias) one finds this graph:graph of loss function

When one rotates the graph and zooms in one sees that there is a local minimum with value also approximately 0,25. I don't know whether this is just coincidence or not, but in any event the fact that the tfjs implementation of Adadelta might get stuck in a local minimum and the Keras one not might also explain the performance difference and the loss function usually being approximately 0,25.

So, is my reasoning correct? Does the difference between Tensor1D and NumpyArray also play a role in this problem? If not, what does explain that difference? And what can I do to avoid such problems with tfjs in the future?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.