Skip to main content
Share Your Experience: Take the 2024 Developer Survey
8 votes
Accepted

How do you find the homography matrix given 4 points in both images?

To understand homographies and how to find them, you will need a good dose of projective geometry. I will briefly describe some preliminary concepts that you need to know before trying to find the ...
nbro's user avatar
  • 40.8k
8 votes
Accepted

How to express a fully connected neural network succintly using linear algebra?

The equation $$\hat{y} = \sigma(xW_\color{green}{1})W_\color{blue}{2} \tag{1}\label{1}$$ is the equation of the forward pass of a single-hidden layer fully connected and feedforward neural network, i....
nbro's user avatar
  • 40.8k
7 votes

Is there any difference between affine transformation and linear transformation?

In linear algebra, a linear transformation (aka linear map or linear transform) $f: \mathcal{V} \rightarrow \mathcal{W}$ is a function that satisfies the following two conditions $f(u + v)=f(u)+f(v)$ ...
nbro's user avatar
  • 40.8k
4 votes

Is there any difference between affine transformation and linear transformation?

The fact is you can always express an affine transformation as a linear transformation (more convenient because it is just a matrix/dot product). For instance, given an input $\textbf{x}=[x_1, ..., ...
y-prudent's user avatar
3 votes

How can we find the value function by solving a system of linear equations?

Provided you have a finite number of states and actions, then there will not be an infinite number of terms. Therefore the state and action spaces need to be discrete and finite before the quote from ...
Neil Slater's user avatar
  • 32.5k
3 votes
Accepted

Why is the derivative of the softmax layer shaped differently than the derivative of other neurons?

When you use the softmax activation function is usually as a last layer of your network and to get an output that is a vector. Now your confusion is about shapes, so let's review a bit of calculus. If ...
Uskebasi's user avatar
  • 278
3 votes
Accepted

Why MLP cannot approximate a closed shape function?

In neural networks, the family of functions and the shapes that they can make for decision surfaces is determined by the activation function you use (in your case, ...
Neil Slater's user avatar
  • 32.5k
3 votes
Accepted

Which linear algebra book should I read to understand vectorized operations?

If you already have two years of a bachelor's of mathematics, I recommend part I of the book that you're mentioning. That part of the book reviews the main mathematics used in the optimization of ...
2 votes

Does k consistency always imply (k - 1) consistency?

Define P as a CSP where X, Y are the variables, domain of both is {1,2,3,4} and conditions in normal form are: node-condition X<4 arc-condition X=Y P is 2-consistent (arc consistent) because for ...
pasaba por aqui's user avatar
2 votes

What does it mean to do multi-dimensional processing with tensors in tensor cores?

Both PR and Theoretically Valid Although Anima Anandkumar's presentation is a puff piece for NVidia, her representation is not contrary to theory. ... next level [above NVidia's GPU success] ... ...
Douglas Daseeco's user avatar
2 votes

How does PCA work when we reduce the original space to 2 or higher-dimensional space?

You might want to have a look at the wikipedia article of PCA, where it says: "The $k$th component can be found by subtracting the first $k − 1$ principal components from $\mathbf{X}$:" $$\...
Tinu's user avatar
  • 628
2 votes

How does neural network classifier classify from just drawing a decision plane?

If you are working with supervised learning, each training example has a label. That label is your classification of the provided input. Just like linear or logistic regression, if your problem only ...
Paul's user avatar
  • 121
1 vote

Is it possible to simplify max(max(a,b),c)?

We are looking for coefficients $\alpha_1, \alpha_2$ and $\alpha_3$ such that $$ \max(\max(a,b), c)=\alpha_1 \max(a, b) + \alpha_2 \max(a, c) +\alpha_3 \max(b, c) $$ for all $a,b,c \in \mathbb R$. ...
hff1's user avatar
  • 111
1 vote

How are sentences turned into a vector in LLM

Aggregating embeddings from pretrained models If you want a single vector representation from a vanilla language model (i.e., one not specially trained for producing sentence embeddings) like GPT you'...
Alexander Wan's user avatar
1 vote

Is the multi-headed projection matrix in self-attention redundant?

The technical reason is the residual connection around the self-attention block (first line in your code: x += self_attention(x)). The transformation $W_O$ is ...
Chillston's user avatar
  • 1,748
1 vote
Accepted

What are the formulas for the MAE , MSE when output is a vector?

You'd just sum the losses over each of the dimensions. If your model outputs a vector of size $d$, it's functionally the same as if you had a model that outputs $d$ different outputs, with $d$ ...
Alexander Wan's user avatar
1 vote

Why my best fit line is not having a single straight line | Multiple Linear Regression

If you want to plot some fitted line, you have to project the hyperplane (your overall fitted model) onto different slices (2D planes), which will indeed produce lines. To find the equation of the ...
Lelouch's user avatar
  • 201
1 vote

Why my best fit line is not having a single straight line | Multiple Linear Regression

Isn't the best fit line supposed to be a single line in between the data points Not here. The linear relationship is between prediction ...
lpounng's user avatar
  • 383
1 vote

How Does The Scaled Dot Product's Dimensions Work Out In Mult-Head Attention?

Say you have 3-dimensional tensors Q & K of shape (B, T, C) (for Batch, Time, and Channels). K is transposed. 'Transposing' a 3-d tensor in this case means ...
Robin van Hoorn's user avatar
1 vote

Encoding data in which the order of features does not matter

Is there maybe a better way [to combine them into one vector in such a way that their order does not matter]? There are many ways to perform feature fusion (see ref). The way you described is one way ...
Snehal Patel's user avatar
1 vote

Does $(\langle w, x \rangle + b) = ||x - x'||$ hold?

I think that your missing the fact that $r$ is not a random constant, but precisely the minimum distance we want to ensure between each point and the separation hyperplane (called margin). And indeed ...
Edoardo Guerriero's user avatar
1 vote

Do solving system of linear equations required anywhere in contemporarty deep learning?

I guess a first distinction should be made between deep learning as a whole or deep learning as architecture. I think the paragraph you quote refers to solving systems of linear equations as a simple ...
Edoardo Guerriero's user avatar
1 vote
Accepted

How can we find the value function by solving a system of linear equations?

First of all, we assume that we have a finite MDP, i.e. the set of states $\mathcal{S}$, the set of actions $\mathcal{A}$ and the set of rewards $\mathcal{R}$ all have a finite number of elements (I ...
nbro's user avatar
  • 40.8k
1 vote
Accepted

What's the difference between a 1d tensor and a 2d tensor with 1 dimension?

The required shape of the tensor $T$ depends on the shape of other tensors that are involved in the same operations of that same tensor $T$ and the required/desired shape of the resulting tensor, in ...
nbro's user avatar
  • 40.8k
1 vote
Accepted

What exactly is the eigenspace of a graph (in spectral clustering)?

In spectral clustering we not find the eigenvectors of a graph (a graph is not a matrix) but the eigenvalues/eigenvectors of the Laplacian matrix related to the adjacency matrix of the graph: graph =&...
pasaba por aqui's user avatar
1 vote

What do we mean by 'principal angle between subspaces'?

Let's consider the case where you have two photos, one base photo and one other photo which is a scaled version of the base photo. Consider then that you could create a 'mapping' from the base photo ...
Nate Diamond's user avatar
1 vote

Is there any way to apply linear transformations on a vector other than matrix multiplication?

You can sometimes exploit the structure of your matrix to perform faster matrix multiplication. For example, if your matrix is sparse (or dense), there are algorithms that exploit this fact. In your ...
nbro's user avatar
  • 40.8k
1 vote

Which linear algebra book should I read to understand vectorized operations?

Linear Algebra Done Right by Axler seems to be the best book on linear algebra, with a brisk and modern approach.
k.c. sayz 'k.c sayz''s user avatar
1 vote

Using reinforcement learning to find a preconditioner for linear systems of the form Ax = b

Reinforcement Learning is a method for learning to perform beneficial actions in an environment. One way this is accomplished is by learning to predict useful actions as a function of the observed ...
John Doucette's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible