8

The equation $$\hat{y} = \sigma(xW_\color{green}{1})W_\color{blue}{2} \tag{1}\label{1}$$ is the equation of the forward pass of a single-hidden layer fully connected and feedforward neural network, i.e. a neural network with 3 layers, 1 input layer, 1 hidden layer, and 1 output layer, where the input layer is connected to the hidden layer (all scalar inputs ...


4

To understand homographies and how to find them, you will need a good dose of projective geometry. I will briefly describe some preliminary concepts that you need to know before trying to find the homography, but don't expect to understand all these concepts with one reading iteration and only by reading this answer, if you are not familiar with them, ...


3

Provided you have a finite number of states and actions, then there will not be an infinite number of terms. Therefore the state and action spaces need to be discrete and finite before the quote from the book applies. I am having a hard time understanding how one could solve this system of equations. There are a few techniques for solving simulteneous ...


3

When you use the softmax activation function is usually as a last layer of your network and to get an output that is a vector. Now your confusion is about shapes, so let's review a bit of calculus. If you have a function $$f:\mathbb{R}\rightarrow\mathbb{R}$$ the derivative is a function on its own and you have $$f':\mathbb{R}\rightarrow\mathbb{R}.$$ If you ...


3

In neural networks, the family of functions and the shapes that they can make for decision surfaces is determined by the activation function you use (in your case, tanh or hyperbolic tangent). Assuming at least one hidden layer, then the universal approximation theorem applies. How closely you can approximate any given function is limited by the number of ...


3

If you already have two years of a bachelor's of mathematics, I recommend part I of the book that you're mentioning. That part of the book reviews the main mathematics used in the optimization of neural nets (in part 1), and then actually goes through the various models in detail in the later parts. The review is done at a level that is suitable for someone ...


2

Both PR and Theoretically Valid Although Anima Anandkumar's presentation is a puff piece for NVidia, her representation is not contrary to theory. ... next level [above NVidia's GPU success] ... means new algorithmic research. So, if you think about the current computations in our deep learning systems, they are all based on linear algebra. Can we come ...


2

Reinforcement Learning is a method for learning to perform beneficial actions in an environment. One way this is accomplished is by learning to predict useful actions as a function of the observed state of the environment. Another is by learning to predict the expected utility gain of doing an action in a particular observed state. Usually the fact that the ...


2

You might want to have a look at the wikipedia article of PCA, where it says: "The $k$th component can be found by subtracting the first $k − 1$ principal components from $\mathbf{X}$:" $$\hat{\mathbf{X}}_k = \mathbf{X} - \sum_{s=1}^{k-1}\mathbf{X}\mathbf{w}_s\mathbf{w}_s^T$$ Then you repeat the process to find the next component: $$\mathbf{w}_k = \...


2

The fact is you can always express an affine transformation as a linear transformation (more convenient because it is just a matrix/dot product). For instance, given an input $\textbf{x}=[x_1, ..., x_n]$, some weights $\textbf{a} = [a_1, a_2, ..., a_n]$ and a bias $b \in \mathbb{R}$, you can express the affine operation $y = \textbf{a}\cdot \textbf{x} + b$ ...


2

In linear algebra, a linear transformation (aka linear map or linear transform) $f: \mathcal{V} \rightarrow \mathcal{W}$ is a function that satisfies the following two conditions $f(u + v)=f(u)+f(v)$ (additivity) $f(\alpha u) = \alpha f(u)$ (scalar multiplication), where $u$ and $v$ vectors (i.e. elements of a vector space, which can also be $\mathbb{R}$ [...


2

If you are working with supervised learning, each training example has a label. That label is your classification of the provided input. Just like linear or logistic regression, if your problem only has 2 classes (e.g. determine whether a tumor is malignant or not), your network will have a single output. An output value of 1.0 could represent one class and ...


2

Welcome! Your query seems to be 3 part on maths, books and programming languages Maths: you can learn for free here Linear Algebra Calculus Probability Theory Discrete Maths Statistics Books : most ideal for beginners Python for Data science for dummies [John Paul Mueller & Luca Massaron] Machine Learning for dummies [John Paul Mueller & Luca ...


1

I guess a first distinction should be made between deep learning as a whole or deep learning as architecture. I think the paragraph you quote refers to solving systems of linear equations as a simple operation involved in deep learning generically. And this is definitely the case, when training a deep model we're always solving systems of linear equations, ...


1

First of all, we assume that we have a finite MDP, i.e. the set of states $\mathcal{S}$, the set of actions $\mathcal{A}$ and the set of rewards $\mathcal{R}$ all have a finite number of elements (I didn't think about how the explanations below would extend to other cases, but I suspect you will need differential equations). For simplicity, let's only ...


1

The required shape of the tensor $T$ depends on the shape of other tensors that are involved in the same operations of that same tensor $T$ and the required/desired shape of the resulting tensor, in the same way that the number of columns of the matrix $M \in \mathbb{R}^{n \times m}$ needs to match the number of rows of the matrix $M' \in \mathbb{R}^{n' \...


1

In spectral clustering we not find the eigenvectors of a graph (a graph is not a matrix) but the eigenvalues/eigenvectors of the Laplacian matrix related to the adjacency matrix of the graph: graph => adjacency matrix => Laplacian matrix => eigenvalues (spectrum). The adjacency matrix describes the "similarity" between two graph vertexs. ...


1

Let's consider the case where you have two photos, one base photo and one other photo which is a scaled version of the base photo. Consider then that you could create a 'mapping' from the base photo to the scaled photo as defined by a set of vector changes for each pixel in the base photo. That is to say, if you had a pixel at point (0,0) in the base photo ...


1

You can sometimes exploit the structure of your matrix to perform faster matrix multiplication. For example, if your matrix is sparse (or dense), there are algorithms that exploit this fact. In your case, you can actually compute $A^n$ in less time than $\mathcal{O}(n^3$). For example, have a look at this question at CS SE and this one at Stack Overflow (...


1

Linear Algebra Done Right by Axler seems to be the best book on linear algebra, with a brisk and modern approach.


1

Define P as a CSP where X, Y are the variables, domain of both is {1,2,3,4} and conditions in normal form are: node-condition X<4 arc-condition X=Y P is 2-consistent (arc consistent) because for any X value it is possible to find a Y value that fulfills the arc-condition X=Y. However, P is not 1-consistent (node-consistent) because exist a X value (X=4)...


1

For other who were wondering the same questions as me, I'll answer it. My view above was inconsistent. Ultimately the last layer of simple feed-forward networks don't have any special properties previous layers exhibit. NNs are just glorified mathematical functions. It distorts space with linear(matrix multiplies) and non-linear functions. Theres no '...


Only top voted, non community-wiki answers of a minimum length are eligible