3
$\begingroup$

I have a lot of empty values in my dataset, so I want to let my neural network 'learn more' on the rows that have no empty values because these rows are of higher importance.

Is there a way to do this?

$\endgroup$
3
$\begingroup$

Yes, you can weight the loss function for each example, so that instead of your cost function being

$$J = \sum_i \mathcal{L}(y_i, \hat{y}_i)$$

It will be

$$J = \sum_i w_i\mathcal{L}(y_i, \hat{y}_i)$$

Where $i$ iterates over your data set, $\mathcal{L}$ is the loss function you are using, $y_i$ is ground truth for each example and $\hat{y}_i$ is prediction for each example.

You can set your relevance weighting according to any criteria you like based on the example, and your dataset/goals. So for instance you could set it to $1.0$ for complete examples and $0.1$ for incomplete examples. Depending on your NN framework, it may already offer per-example weighting, and even if it does not, auto-differentiation means that typically you only need to implement the forward logic into the cost function for each minibatch, and the weighting will be applied correctly to gradients with no more work required. If you do need to implement gradient calculations yourself, you simply multiply the initial gradient of each example by the example's weight $w_i$.

Once you make a change like this, you need to take care on how you set up and interpret your test set. When your model is used in production, if there are still inputs with missing details, you probably don't want to weight the metrics (e.g. accuracy rating) in the same way, but to report it as-is against a correctly sampled dataset of unseen examples from the population of expected inputs.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.