Is my C# Adam implementation correct?

Question

I have some doubt because I incurred in different papers proposing different implementations. Also implementations on opensource projects looks different. In example there is a C++ library that computes different Adam members also for Bias vector.

My Implementation:

Of course before running the optimization I do backward pass, where derivates is the derivate of the activation function computed on the weighted sum of inputs.

    public void Backward(IBackwardOutputLayer<double> iLayer, IBackwardInputLayer<double> iPlusOneLayer, double[] target)
    {
        Parallel.For(0, iLayer.Gradients.Length, i =>
        {
            double sum = 0;
            for (int j = 0; j < iPlusOneLayer.Gradients.Length; j++)
                sum += iPlusOneLayer.Gradients[j] * iPlusOneLayer.Weights[j][i];

            iLayer.Gradients[i] = sum * iLayer.Derivates[i];
        });
    }

Here's my Adam algorithm implementation that is runned after th backward pass.

    public void Optimize(IGradientsLayer<double> layer, ILayerAllocatedVariables<double> variables)
    {
        var m = variables.GetArrayVariable(Params.Momentum);
        var v = variables.GetArrayVariable(Params.Velocity);
        var mt = variables.GetArrayVariable(Params.DeBiasedMomentum);
        var vt = variables.GetArrayVariable(Params.DeBiasedVelocity);

        Parallel.For(0, layerSize , i =>
        {
            var grad = layer.Gradients[i];
            m[i] = B1 * m[i] + (1.0 - B1) * grad;
            v[i] = B2 * v[i] + (1.0 - B2) * grad * grad;

            mt[i] = m[i] / (1.0 - Pow(B1, currentStep));
            vt[i] = v[i] / (1.0 - Pow(B2, currentStep));

            layer.Gradients[i] = mt[i] / Sqrt(vt[i] + Epsilon);  
        });
    }

after each Adam step I accumulate the total gradient for the current batch

    private void AccumulateTotalGradients()
    {
        for (int l = 0; l < layers.Length; l++)
            for (int i = 0; i < layers[l].Outputs.Length; i++)
                layers[l].TotalGradients[i] += layers[l].Gradients[i];
    }

and after a small batch finally I do the propagation using the totale gradient average (mean gradient for a whole batch of inputs).

    private void Propagation()
    {
        double scaleFactor = learningRate / batchSize;

        Parallel.For(0, layers.Length, l => 
        {
            for (int i = 0; i < layers[l].Outputs.Length; i++)
            {
                layers[l].Biases[i] -= scaleFactor * layers[l].TotalGradients[i];

                for (int z = 0; z < layers[l].Weights[i].Length; z++)
                {
                    layers[l].Weights[i][z] -= scaleFactor * layers[l].Weights[i][z]* layers[l].TotalGradients[i];
                }
            }
        });
    }

Then I reset the total gradients:

    private void ResetTotalGradients()
    {
        for (int l = 0; l < layers.Length; l++) 
            for (int i = 0; i < layers[l].Outputs.Length; i++)
                layers[l].TotalGradients[i] = 0;
    }