Questions tagged [linear-algebra]

For questions about the use/aspects/implementation/intuition/mathematical proofs of various Linear Algebra methods used in Machine Learning and AI algorithms.

Filter by
Sorted by
Tagged with
-1
votes
0answers
13 views

How does maxpooling function work on pointnet architecture for learn features globally and hierarchically? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers. This question does not appear to be about programming within the scope defined in the help center. ...
0
votes
0answers
28 views

Backpropagation implementation not applicable for other cases

I saw this implementation of backpropagation in MATLAB, where the loss function used is MSE, and the last layer's activation function was sigmoid. I denoted the portions of the formula for what I ...
6
votes
1answer
333 views

How to express a fully connected neural network succintly using linear algebra?

I'm currently reading the paper Federated Learning with Matched Averaging (2020), where the authors claim: A basic fully connected (FC) NN can be formulated as: $\hat{y} = \sigma(xW_1)W_2$ [...] ...
0
votes
1answer
71 views

Why is the derivative of the softmax layer shaped differently than the derivative of other neurons?

If the derivative is supposed to give the rate of change of a function at that point, then why is the derivative of the softmax layer (a vector) the Jacobian matrix, which has a different shape than ...
2
votes
0answers
26 views

How to interpret the variance calculation in a Guassian process

I answered another question here about the mean prediction of a GP, but I have a hard time coming up with an intuitive explanation of the variance prediction of a GP. Thew specific equation that I am ...
2
votes
1answer
55 views

What exactly is the eigenspace of a graph (in spectral clustering)?

When we find the eigenvectors of a graph (say in the context of spectral clustering), what exactly is the vector space involved here? Of what vector space (or eigenspace) are we finding the ...
1
vote
0answers
28 views

Is the 3d convolution associative given that it can be represented as matrix multiplication?

I'm trying to understand if a 3D convolution of the sort performed in a convolutional layer of a CNN is associative. Specifically, is the following true: $$ X \otimes(W \cdot Q)=(X \otimes W) \cdot Q, ...
1
vote
0answers
29 views

How to find distance between 2 points when dimensions are all of different nature?

I have a dataset with four features: the x coordinate the y coordinate the velocity magnitude angle Now, I want to measure the distance between two points in the dataset, taking into account the ...
2
votes
2answers
116 views

How does PCA work when we reduce the original space to 2 or higher-dimensional space?

How does PCA work when we reduce the original space to a 2 or higher-dimensional space? I understand the case when we reduce the dimensionality to $1$, but not this case. $$\begin{array}{ll} \text{...
0
votes
0answers
21 views

How can I calibrate 3 cameras and track the object using only synchronized cameras feeds from all the cameras?

I have camera feed (in the form of RGB images) from 3 cameras with partially overlapping Field-of-view i.e. for the time stamp 0 to 100, I have total 300 frames or say synchronized 100 RGB frames for ...
0
votes
0answers
27 views

How can I calibrate 3 cameras without knowing global pose of the object & camera locations? How can I find the pose of each camera wrt the first one?

I have camera feed (in the form of RGB images) from 3 cameras with overlapping FOV e.g. for the time stamp 0 to 100, I have synchronized RGB frames for each camera. The object (Robot) is moving from ...
0
votes
0answers
14 views

How to decompose a non-positive definite matrix in the same manner as Cholseky decomposition?

I want to make a covariance matrix that incorporates my belief of how correlated the various dimensions are. The reason why I want to incorporate my belief is that in my modelling, the dimensions are ...
1
vote
0answers
16 views

Human intuition behind SVD in case of recommendation system

This does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to connect the dots. So, I started to see all the ...
1
vote
1answer
70 views

What do we mean by 'principal angle between subspaces'?

I came across the term 'principal angle between subspaces' as a tool for comparing objects in images. All material that I found on the internet seems to deal with this idea in a highly mathematical ...
1
vote
0answers
43 views

Simplifying Log Loss

I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
2
votes
1answer
332 views

How do you find the homography matrix given 4 points in both images?

I want to understand the process of finding a homography matrix given 4 points in both images. I am able to do that in python OpenCV, but I wonder how it works behind the scenes. Suppose I have ...
1
vote
1answer
62 views

Is there any way to apply linear transformations on a vector other than matrix multiplication?

I am trying to optimize the cost function calculation in regression analysis using a non-matrix multiplication based approach. More specifically, I have a point $x = (1, 1, 2, 3)$, to which I want to ...
4
votes
1answer
58 views

Why MLP cannot approximate a closed shape function?

[TL;DR] I generated two classes Red and Blue on a 2D space. Red are points on Unit Circle and Blue are points on a Circle Ring with radius limits (3,4). I tried to train a Multi Layer Perceptron ...
1
vote
0answers
29 views

CSP Formulation of an algebraic problem

Is anyone able to explain how to do this? I'm not looking for the complete answer, I would settle for a "how to for dummies" explanation of how this is supposed to be solved. I understand ...
5
votes
2answers
75 views

Which linear algebra book should I read to understand vectorized operations?

I am reading Goodfellow's book about neural networks, but I am stuck in the mathematical calculus of the back-propagation algorithm. I understood the principle, and some Youtube videos explaining this ...
1
vote
0answers
19 views

Estimating Baselines using ALS

I am trying to figure out how ALS works when minimizing the following formula: $\\ \\$ $\text{min}_{\lbrace b_u,b_i \rbrace} \sum_{(u,i)\in \mathcal{K}} (r_{ui} - \bar{r} - b_u - b_i )^2 + \lambda_{...
7
votes
1answer
218 views

What does it mean to do multi-dimensional processing with tensors in tensor cores?

In some tweets about NeurIPS 2018, this video from NVIDIA appeared. At around 0.37, she says: If you think about the current computations in our deep learning systems, they are all based on Linear ...
2
votes
1answer
217 views

Using reinforcement learning to find a preconditioner for linear systems of the form Ax = b

Sparse linear systems are normally solved by using solvers like MINRES, Conjugate gradient, GMRES. Efficient preconditioning, i.e., finding a matrix P such that PAx = Pb is easier to solve than the ...
1
vote
1answer
360 views

Does k consistency always imply (k - 1) consistency?

From Russell-Norvig: A CSP is strongly k-consistent if it is k-consistent and is also (k − 1)-consistent, (k − 2)-consistent, . . . all the way down to 1-consistent. How can a CSP be k-consistent ...
5
votes
3answers
403 views

How does neural network classifier classify from just drawing a decision plane?

I understand that a neural network basically distorts(non-linear transformation) and changes the perspective(linear transformations) of input space to draw a plane to classify data. How does the ...