# Questions tagged [linear-algebra]

For questions about the use/aspects/implementation/intuition/mathematical proofs of various Linear Algebra methods used in Machine Learning and AI algorithms.

28 questions
Filter by
Sorted by
Tagged with
77 views

### Is there any difference between affine transformation and linear transformation?

Consider the following statements from A Simple Custom Module of PyTorch's documentation To get started, let’s look at a simpler, custom version of PyTorch’s Linear module. This module applies an ...
193 views

### How can we find the value function by solving a system of linear equations?

I am following the book "Reinforcement Learning: An Introduction" by Richard Sutton and Andrew Barto, and they give an example of a problem for which the value function can be computed ...
92 views

### What do we mean by 'principal angle between subspaces'?

I came across the term 'principal angle between subspaces' as a tool for comparing objects in images. All material that I found on the internet seems to deal with this idea in a highly mathematical ...
30 views

### How can the gradient of the weight be calculated in the viewpoint of matrix calculus?

Let $\sigma(x)$ be sigmoid function. Consider the case where $\text{out}=\sigma(\vec{x} \times W + \vec{b})$, and we want to compute $\frac{\partial{\text{out}}}{\partial{w} }.$ Set the dimension as ...
31 views

### What's the difference between a 1d tensor and a 2d tensor with 1 dimension?

I'm doing a TensorFlow tutorial, where they convert an array of the numbers [1,2,3] to a tensor like this: ...
88 views

### Which linear algebra book should I read to understand vectorized operations?

I am reading Goodfellow's book about neural networks, but I am stuck in the mathematical calculus of the back-propagation algorithm. I understood the principle, and some Youtube videos explaining this ...
352 views

### How to express a fully connected neural network succintly using linear algebra?

I'm currently reading the paper Federated Learning with Matched Averaging (2020), where the authors claim: A basic fully connected (FC) NN can be formulated as: $\hat{y} = \sigma(xW_1)W_2$ [...] ...
32 views

### Backpropagation implementation not applicable for other cases

I saw this implementation of backpropagation in MATLAB, where the loss function used is MSE, and the last layer's activation function was sigmoid. I denoted the portions of the formula for what I ...
75 views

### Why is the derivative of the softmax layer shaped differently than the derivative of other neurons?

If the derivative is supposed to give the rate of change of a function at that point, then why is the derivative of the softmax layer (a vector) the Jacobian matrix, which has a different shape than ...
124 views

29 views

### How to find distance between 2 points when dimensions are all of different nature?

I have a dataset with four features: the x coordinate the y coordinate the velocity magnitude angle Now, I want to measure the distance between two points in the dataset, taking into account the ...
63 views

### Is there any way to apply linear transformations on a vector other than matrix multiplication?

I am trying to optimize the cost function calculation in regression analysis using a non-matrix multiplication based approach. More specifically, I have a point $x = (1, 1, 2, 3)$, to which I want to ...
21 views

### How can I calibrate 3 cameras and track the object using only synchronized cameras feeds from all the cameras?

I have camera feed (in the form of RGB images) from 3 cameras with partially overlapping Field-of-view i.e. for the time stamp 0 to 100, I have total 300 frames or say synchronized 100 RGB frames for ...
29 views

### How can I calibrate 3 cameras without knowing global pose of the object & camera locations? How can I find the pose of each camera wrt the first one?

I have camera feed (in the form of RGB images) from 3 cameras with overlapping FOV e.g. for the time stamp 0 to 100, I have synchronized RGB frames for each camera. The object (Robot) is moving from ...
15 views

### How to decompose a non-positive definite matrix in the same manner as Cholseky decomposition?

I want to make a covariance matrix that incorporates my belief of how correlated the various dimensions are. The reason why I want to incorporate my belief is that in my modelling, the dimensions are ...
19 views

### Human intuition behind SVD in case of recommendation system

This does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to connect the dots. So, I started to see all the ...
238 views

### Using reinforcement learning to find a preconditioner for linear systems of the form Ax = b

Sparse linear systems are normally solved by using solvers like MINRES, Conjugate gradient, GMRES. Efficient preconditioning, i.e., finding a matrix P such that PAx = Pb is easier to solve than the ...
44 views

### Simplifying Log Loss

I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
761 views

### How do you find the homography matrix given 4 points in both images?

I want to understand the process of finding a homography matrix given 4 points in both images. I am able to do that in python OpenCV, but I wonder how it works behind the scenes. Suppose I have ...
58 views

### Why MLP cannot approximate a closed shape function?

[TL;DR] I generated two classes Red and Blue on a 2D space. Red are points on Unit Circle and Blue are points on a Circle Ring with radius limits (3,4). I tried to train a Multi Layer Perceptron ...
29 views

### CSP Formulation of an algebraic problem

Is anyone able to explain how to do this? I'm not looking for the complete answer, I would settle for a "how to for dummies" explanation of how this is supposed to be solved. I understand ...
220 views

### What does it mean to do multi-dimensional processing with tensors in tensor cores?

In some tweets about NeurIPS 2018, this video from NVIDIA appeared. At around 0.37, she says: If you think about the current computations in our deep learning systems, they are all based on Linear ...
23 views

### Estimating Baselines using ALS

I am trying to figure out how ALS works when minimizing the following formula: $\\ \\$ \$\text{min}_{\lbrace b_u,b_i \rbrace} \sum_{(u,i)\in \mathcal{K}} (r_{ui} - \bar{r} - b_u - b_i )^2 + \lambda_{...