# Questions tagged [linear-algebra]

For questions about the use/aspects/implementation/intuition/mathematical proofs of various Linear Algebra methods used in Machine Learning and AI algorithms.

26 questions
Filter by
Sorted by
Tagged with
1answer
24 views

### What's the difference between a 1d tensor and a 2d tensor with 1 dimension?

I'm doing a TensorFlow tutorial, where they convert an array of the numbers [1,2,3] to a tensor like this: ...
1answer
30 views

### How can the gradient of the weight be calculated in the viewpoint of matrix calculus?

Let $\sigma(x)$ be sigmoid function. Consider the case where $\text{out}=\sigma(\vec{x} \times W + \vec{b})$, and we want to compute $\frac{\partial{\text{out}}}{\partial{w} }.$ Set the dimension as ...
0answers
32 views

### Backpropagation implementation not applicable for other cases

I saw this implementation of backpropagation in MATLAB, where the loss function used is MSE, and the last layer's activation function was sigmoid. I denoted the portions of the formula for what I ...
1answer
349 views

### How to express a fully connected neural network succintly using linear algebra?

I'm currently reading the paper Federated Learning with Matched Averaging (2020), where the authors claim: A basic fully connected (FC) NN can be formulated as: $\hat{y} = \sigma(xW_1)W_2$ [...] ...
1answer
75 views

### Why is the derivative of the softmax layer shaped differently than the derivative of other neurons?

If the derivative is supposed to give the rate of change of a function at that point, then why is the derivative of the softmax layer (a vector) the Jacobian matrix, which has a different shape than ...
0answers
28 views

### How to interpret the variance calculation in a Guassian process

I answered another question here about the mean prediction of a GP, but I have a hard time coming up with an intuitive explanation of the variance prediction of a GP. Thew specific equation that I am ...
1answer
62 views

### What exactly is the eigenspace of a graph (in spectral clustering)?

When we find the eigenvectors of a graph (say in the context of spectral clustering), what exactly is the vector space involved here? Of what vector space (or eigenspace) are we finding the ...
0answers
31 views

0answers
21 views

### How can I calibrate 3 cameras and track the object using only synchronized cameras feeds from all the cameras?

I have camera feed (in the form of RGB images) from 3 cameras with partially overlapping Field-of-view i.e. for the time stamp 0 to 100, I have total 300 frames or say synchronized 100 RGB frames for ...
0answers
29 views

### How can I calibrate 3 cameras without knowing global pose of the object & camera locations? How can I find the pose of each camera wrt the first one?

I have camera feed (in the form of RGB images) from 3 cameras with overlapping FOV e.g. for the time stamp 0 to 100, I have synchronized RGB frames for each camera. The object (Robot) is moving from ...
0answers
15 views

### How to decompose a non-positive definite matrix in the same manner as Cholseky decomposition?

I want to make a covariance matrix that incorporates my belief of how correlated the various dimensions are. The reason why I want to incorporate my belief is that in my modelling, the dimensions are ...
0answers
17 views

### Human intuition behind SVD in case of recommendation system

This does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to connect the dots. So, I started to see all the ...
1answer
90 views

### What do we mean by 'principal angle between subspaces'?

I came across the term 'principal angle between subspaces' as a tool for comparing objects in images. All material that I found on the internet seems to deal with this idea in a highly mathematical ...
0answers
44 views

### Simplifying Log Loss

I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
1answer
664 views

### How do you find the homography matrix given 4 points in both images?

I want to understand the process of finding a homography matrix given 4 points in both images. I am able to do that in python OpenCV, but I wonder how it works behind the scenes. Suppose I have ...
1answer
63 views

### Is there any way to apply linear transformations on a vector other than matrix multiplication?

I am trying to optimize the cost function calculation in regression analysis using a non-matrix multiplication based approach. More specifically, I have a point $x = (1, 1, 2, 3)$, to which I want to ...
1answer
58 views

### Why MLP cannot approximate a closed shape function?

[TL;DR] I generated two classes Red and Blue on a 2D space. Red are points on Unit Circle and Blue are points on a Circle Ring with radius limits (3,4). I tried to train a Multi Layer Perceptron ...
0answers
29 views

### CSP Formulation of an algebraic problem

Is anyone able to explain how to do this? I'm not looking for the complete answer, I would settle for a "how to for dummies" explanation of how this is supposed to be solved. I understand ...
2answers
84 views

### Which linear algebra book should I read to understand vectorized operations?

I am reading Goodfellow's book about neural networks, but I am stuck in the mathematical calculus of the back-propagation algorithm. I understood the principle, and some Youtube videos explaining this ...
0answers
21 views

### Estimating Baselines using ALS

I am trying to figure out how ALS works when minimizing the following formula: $\\ \\$ \$\text{min}_{\lbrace b_u,b_i \rbrace} \sum_{(u,i)\in \mathcal{K}} (r_{ui} - \bar{r} - b_u - b_i )^2 + \lambda_{...
1answer
218 views

### What does it mean to do multi-dimensional processing with tensors in tensor cores?

In some tweets about NeurIPS 2018, this video from NVIDIA appeared. At around 0.37, she says: If you think about the current computations in our deep learning systems, they are all based on Linear ...
1answer
232 views

### Using reinforcement learning to find a preconditioner for linear systems of the form Ax = b

Sparse linear systems are normally solved by using solvers like MINRES, Conjugate gradient, GMRES. Efficient preconditioning, i.e., finding a matrix P such that PAx = Pb is easier to solve than the ...
1answer
413 views

### Does k consistency always imply (k - 1) consistency?

From Russell-Norvig: A CSP is strongly k-consistent if it is k-consistent and is also (k − 1)-consistent, (k − 2)-consistent, . . . all the way down to 1-consistent. How can a CSP be k-consistent ...
3answers
422 views

### How does neural network classifier classify from just drawing a decision plane?

I understand that a neural network basically distorts(non-linear transformation) and changes the perspective(linear transformations) of input space to draw a plane to classify data. How does the ...