2
$\begingroup$

I have a neural network that connects $N$ input variables to $M$ output variables (qoi). By default, neural networks just give out point estimations.

Now, I want to plot some of the quantity of interests and produce also a prediction interval. To calculate the model uncertainty, I use the bootstrap method.

$$\sigma_{model}^2=\frac{1}{B-1}\sum_{i=1}^B(\hat{y}_i^b-\hat{y}_i)^2\qquad \text{with}\quad\hat{y}_i = \frac{1}{B}\sum_{b=1}^B\hat{y}_i^b$$ $B$ training datasets are resampled from original dataset with replacement. $\hat{y}_i^b$ is the preditcion of the $i$ sample generated by the $b$th bootstrap model.

If I understood it correctly, the model uncertainty (or epistemic uncertainty) is enough to create a confidence interval.

But for the PI I also need the irreducible error $\sigma_{noise,\epsilon}^2$. $$\sigma_y^2= \sigma_{model}^2+\sigma_{noise,\epsilon}^2$$

The aleatoric uncertainty is explained in the following picture: enter image description here

Is there a procedure to calculate this aleatoric uncertainty?

I read the paper High-Quality Prediction Intervals for Deep Learning and watched the corresponding YouTube video. And I read the paper Neural Network-Based Prediction Intervals.

EDIT I suggest the following algorithm to estimate the noise variance, but I am not sure if this makes sense:

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.