I have a neural network that connects $N$ input variables to $M$ output variables (qoi). By default, neural networks just give out point estimations.

Now, I want to plot some of the quantity of interests and produce also a prediction interval. To calculate the model uncertainty, I use the bootstrap method.

$$\sigma_{model}^2=\frac{1}{B-1}\sum_{i=1}^B(\hat{y}_i^b-\hat{y}_i)^2\qquad \text{with}\quad\hat{y}_i = \frac{1}{B}\sum_{b=1}^B\hat{y}_i^b$$ $B$ training datasets are resampled from original dataset with replacement. $\hat{y}_i^b$ is the preditcion of the $i$ sample generated by the $b$th bootstrap model.

If I understood it correctly, the model uncertainty (or epistemic uncertainty) is enough to create a confidence interval.

But for the PI I also need the irreducible error $\sigma_{noise,\epsilon}^2$. $$\sigma_y^2= \sigma_{model}^2+\sigma_{noise,\epsilon}^2$$

The aleatoric uncertainty is explained in the following picture: enter image description here

Is there a procedure to calculate this aleatoric uncertainty?

I read the paper High-Quality Prediction Intervals for Deep Learning and watched the corresponding YouTube video. And I read the paper Neural Network-Based Prediction Intervals.

EDIT I suggest the following algorithm to estimate the noise variance, but I am not sure if this makes sense:

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.