I am writing a AlphaGo Zero clone, and sometimes in the training the policy head loss and value head loss would both be decreasing, but the total loss is increasing?

screenshot of training progress

How is this possible? I am using Adam, and here's the code:

    input_tensor = Input((self.input_board_size, self.input_board_size, 2))
    x = input_tensor
    x = self.convolution_block(x)
    for _ in range(self.number_of_residual_block):
        x = self.residual_block(x)
    self.model = Model(inputs=input_tensor, outputs=[self.policy_head(x), self.value_head(x)])['categorical_crossentropy', 'mean_squared_error'])
    self.model.compile(Adam(lr=1e-4), ['categorical_crossentropy', 'mean_squared_error'])

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.