1
$\begingroup$

I am trying to use RealNVP with some data I have (the input size is a 1D vector of size 22). Here is the link to the RealNVP paper and here is a nice, short explanation of it (the paper is pretty long). My code is mainly based on this code from GitHub and below are the main piece that I am using (with slight adjustments). The problem is that the loss is getting negative, which in the definition of my code means that the log-probability of the my data is positive, which in turn means that the probabilities are bigger than 1. This is impossible mathematically, and I see no way how this can happen, from a mathematical point of view. I also couldn't find a mistake in my code. Can someone help me with this? Is there a mistake there? Am I missing something with my understanding of normalizing flows? Thank you!

class NormalizingFlowModel(nn.Module):

    def __init__(self, prior, flows):
        super().__init__()
        self.prior = prior
        self.flows = nn.ModuleList(flows)

    def forward(self, x):
        m, _ = x.shape
        log_det = torch.zeros(m).cuda()
        for flow in self.flows:
            x, ld = flow.forward(x)
            log_det += ld
        z, prior_logprob = x, self.prior.log_prob(x)
        return z, prior_logprob, log_det

    def inverse(self, z):
        m, _ = z.shape
        log_det = torch.zeros(m).cuda()
        for flow in self.flows[::-1]:
            z, ld = flow.inverse(z)
            log_det += ld
        x = z
        return x, log_det

    def sample(self, n_samples):
        z = self.prior.sample((n_samples,))
        x, _ = self.inverse(z)
        return x


class FCNN_for_NVP(nn.Module):
    """
    Simple fully connected neural network to be used for Real NVP
    """
    def __init__(self, in_dim, out_dim):
        super().__init__()
        self.network = nn.Sequential(
            nn.Linear(in_dim, 32),
            nn.Tanh(),
            nn.Linear(32, 32),
            nn.Tanh(),
            nn.Linear(32, 64),
            nn.Tanh(),
            nn.Linear(64, 64),
            nn.Tanh(),
            nn.Linear(64, 32),
            nn.Tanh(),
            nn.Linear(32, 32),
            nn.Tanh(),
            nn.Linear(32, out_dim),
        )

    def forward(self, x):
        return self.network(x)


class RealNVP(nn.Module):
    """
    Non-volume preserving flow.

    [Dinh et. al. 2017]
    """
    def __init__(self, dim, base_network=FCNN_for_NVP):
        super().__init__()
        self.dim = dim
        self.t1 = base_network(dim // 2, dim // 2)
        self.s1 = base_network(dim // 2, dim // 2)
        self.t2 = base_network(dim // 2, dim // 2)
        self.s2 = base_network(dim // 2, dim // 2)

    def forward(self, x):
        lower, upper = x[:,:self.dim // 2], x[:,self.dim // 2:]      
        t1_transformed = self.t1(lower)
        s1_transformed = self.s1(lower)
        upper = t1_transformed + upper * torch.exp(s1_transformed)
        t2_transformed = self.t2(upper)
        s2_transformed = self.s2(upper)
        lower = t2_transformed + lower * torch.exp(s2_transformed)
        z = torch.cat([lower, upper], dim=1)
        log_det = torch.sum(s1_transformed, dim=1) + torch.sum(s2_transformed, dim=1)
        return z, log_det

    def inverse(self, z):
        lower, upper = z[:,:self.dim // 2], z[:,self.dim // 2:]
        t2_transformed = self.t2(upper)
        s2_transformed = self.s2(upper)
        lower = (lower - t2_transformed) * torch.exp(-s2_transformed)
        t1_transformed = self.t1(lower)
        s1_transformed = self.s1(lower)
        upper = (upper - t1_transformed) * torch.exp(-s1_transformed)
        x = torch.cat([lower, upper], dim=1)
        log_det = torch.sum(-s1_transformed, dim=1) + torch.sum(-s2_transformed, dim=1)
        return x, log_det

flow = RealNVP(dim=data.size(1))
flows = [flow for _ in range(1)]
prior = MultivariateNormal(torch.zeros(data.size(1)).cuda(), torch.eye(data.size(1)).cuda())
model = NormalizingFlowModel(prior, flows)
model = model.cuda()

for i in range(10):
    for j, dtt in enumerate(my_dataloader_bkg_only):
        optimizer.zero_grad()
        x = dtt[0].float()
        z, prior_logprob, log_det = model(x)
        logprob = prior_logprob + log_det
        loss = -torch.mean(prior_logprob + log_det)
        loss.backward()
        optimizer.step()
    if i % 1 == 0:
        print("Saved")
        best_loss = logprob.mean().data.cpu().numpy()
        print(logprob.mean().data.cpu().numpy(), prior_logprob.mean().data.cpu().numpy(),
                  log_det.mean().data.cpu().numpy())
$\endgroup$
  • $\begingroup$ You might well have a bug, but in general there's no reason to not have a log-likelihood greater than 0. E.g. consider fitting a single 1-d point with a Gaussian with parameterised mean and variance. The maximum-likelihood solution is an infinitely sharp peak with infinite log likelihood. $\endgroup$ – Chris Cundy Feb 2 at 2:41
  • $\begingroup$ So how should I think about a probability bigger than 1? Assuming I have 2 1-d points (in my case I have 1M), and one of them has a probability of, say, 1.5, how can I make sense of that mathematically? Thank you! $\endgroup$ – BillKet Feb 2 at 3:19
  • $\begingroup$ You are computing a probability density (flow models are density models), so you should interpret it as a probability density. Maybe this will help towardsdatascience.com/… $\endgroup$ – Chris Cundy Feb 2 at 3:54
0
$\begingroup$

I don't know if this comment will be helpful, but shouldn't the sum of the log determinant of the Jacobian (LDJ) have opposite sign in the forward and inverse pass? I'm not talking about the LDJ being sum of the positive scaling function in the forward and sum of the negative of the scaling function in the inverse, I'm talking about the LDJ itself.

For example, from the change of variable formula, $$p_z(z) \ dz = p_x(x) \ dx$$ differentiating w.r.t $\ dz$ gives, $$ p_z(z) = p_x(x) \left|\frac{dx}{dz}\right|$$ taking the log of both sides gives, $$ ln(p_z(z)) = ln(p_x(x)) + ln\left(\left|\frac{dx}{dz}\right|\right)$$ Now, if we repeat this process but differentitating w.r.t $\ dx$, $$ p_x(x) = p_z(z) \left|\frac{dz}{dx}\right|$$ taking the log and rearranging for $ln(p_z(z))$ gives, taking the log of both sides gives, $$ ln(p_z(z)) = ln(p_x(x)) - ln\left(\left|\frac{dz}{dx}\right|\right)$$

A change of sign occurs when the Jacobian is reversed. Could this be your problem? Also, as @chris-cundy said you can have a probability density greater than 1 at a single point. Remember it is the integral over all space that cannot be greater than one, an individual point between greater than 1 is fine.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.