Return to Revisions

Creating custom gradient for tf.abs may solve the problem:

@tf.custom_gradient
def abs_with_grad(x):
  y = tf.abs(x);

  def grad(div): # Derivation intermediate value
    g = 1; # Use 1 to make the chain rule just skip abs
    return div*g;

  return y,grad;

Use 1 as above to skip thru' abs or, use the actual abs grad (Samual K):

g = tf.where(x<0, -1, 1) #now the gradient at 0 would be one. This way u dont have dead  weights.
# With/without:
g = tf.where(x==0, 0, g) #if you realy want the gradient 0 at 0 add  this.