In Bayesian statistics, as opposed to frequentist statistics, you can model the parameters as random variables. Bayesian machine learning is the application of Bayesian statistics in the context of machine learning. The specific application of Bayesian statistics to learning in neural networks is denoted as Bayesian deep learning.
Even just in the simple example of maximum a posteriori (MAP) estimation, you model the weights as random variables. For example, you can view weight decay in neural networks as MAP estimation when you put a prior on the weights, i.e. you assume that the weights are random variables or samples from a distribution. Here you can find more details about MAP estimation and how it's different from MLE.
In MAP, we only find a point estimate of the weights. However, we could also learn a probability distribution over the weights. If you do this with neural networks, we have Bayesian neural networks. Two of the first people that worked on this were David J. C. MacKay and Radford M. Neal, but only very recently did Bayesian neural networks really become popular, with Alex Graves and Blundel et al. Anyway, there are different approaches to Bayesian learning in neural networks. There are techniques based on Monte Carlo methods (e.g. Neal) or variational inference (e.g. Graves and Blundell).
So, what are the advantages of viewing weights as random variables? Well, you can derive MAP and Bayesian neural networks. Basically, all the advantages of Bayesian statistics.