Consider the following from Numerical Computation chapter of Deep Learning book
Machine learning algorithms usually require a high amount of numerical computation. This typically refers to algorithms that solve mathematical problems by methods that update estimates of the solution via an iterative process, rather than analytically deriving a formula to provide a symbolic expression for the correct solution. Common operations include optimization (finding the value of an argument that minimizes or maximizes a function) and solving systems of linear equations. Even just evaluating a mathematical function on a digital computer can be difficult when the function involves real numbers, which cannot be represented precisely using a finite amount of memory.
The paragraph clearly mentions that solving system of linear equations is a common operation in machine learning. I just know that solving system of linear equations is useful in reinforcement learning and some basic algorithms of machine learning including regression.
Is solving system of linear equations useful anywhere in deep learning?
I think that we use them nowhere since optimization is the only algorithm generally used in deep learning.