I am trying to implement SDNE, a algorithm uses deep auto encoder to map a graph to latent representation d dimension.

The idea is kind of simple, SDNE uses the adjacency matrix as input and output is embedding layer (also node embedding of graph). The loss function tries to minimize the reconstruction layer and input layer which defined $\hat{X}$ and $X$. To keeping node embedding Y (embedding layer) is close together like the original node in graph input, loss function continue minimizes $||y_i - y_j||*s_{ij}$ with $s_{ij}$ is weight of node $v_i$ and $v_j$, if not link $s_{ij} = 0$.

To sum up, loss function like this: $$ L = \sum_{i=1}^{n}{||(\hat{x_i} -x_i) \odot{b_i}||_2^2} + \sum_{i,j=1}^{n} {s_{i,j}||y_i -y_j||_2^2} $$ $$ \Leftrightarrow L = ||(\hat{X} - X) \odot B||_F^2 + 2tr(Y^TLY) $$ with $L=D-S, D\in \mathbb{R}^{n\times n}$ is diagonal matrix, $D_{i,i} = \sum_{j}{s_{ij}}$. $L$ is call Laplacian matrix.

When I implement code in PyTorch, I got negative loss value although the loss function does not negative value.

This is my code:

def _compute_loss(self, x, x_hat, y, L):
        def loss_1st(Y, L):
            Y_ = torch.matmul(torch.transpose(Y, 0, 1), L)
            return 2 * torch.trace(torch.matmul(Y_, Y))

        def loss_2nd(X_hat, X, beta):
            B = torch.ones_like(X).to(device)
            B[X != 0] = beta
            return torch.sum(torch.square((X_hat - X) * B))

        batch_size = x.shape[0]
        # TODO: check if divide batch_size
        loss_1 = loss_1st(y, L)
        loss_2 = loss_2nd(x_hat, x, self.beta)
        loss = loss_2 + self.alpha * loss_1
        return loss

I trained on all batch dataset which means batch_size = len(Adjancency_matrix) = len(A). Actually, I'm not sure about this algorithm can train on batch_size = 64, 128,.. because I can not calcualte L on mini batch and mini batch also can not represent for the whole graph.

So, can you see in my code what reason why my loss function is negative?


  • $\begingroup$ Hi. Note that programming issues are off-topic. Can you confirm that this is a programming issue? If yes, I will migrate it to Stack Overflow. Please, read our on-topic page: ai.stackexchange.com/help/on-topic. It explains what is on-topic here. $\endgroup$ – nbro Aug 16 '20 at 10:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.