I'm wondering how could I incorporate specific constraints during the training phase of a deep learning model.
In particular, I work for a materials-science related project where I feed to my models chemical compositions seen as long sparse vectors indexed by atomic numbers where at $i$-th atomic number we encode the fraction of the specific element. I have recently trained and implemented a conditional variational autoencoder (CVAE) aimed at learning a multimodal probability distribution where we attach to the latent sample, a vector of desirable properties that we might want to encode. The problem looking at reconstructed vectors though, is that there are many elements different than zero and one might find difficult to distinguish between important small but relevant fractions of elements from noise.
I've tried something else like projecting the reconstructed vectors to the $D$- dimensional simplex, in order to obtain the closest compositional vector, but still too many values are different than $0$.
What I'm actually thinking about is to add some chemically-based constraint to be taken into account during training, to make sure that the reconstructed vectors have more chemical meaningfulness.
So do you have anything to suggest, in terms of references, maybe modifications on the loss function, or anything else about even general constraining techniques?