Consider the following paragraph from section 5.4 Gradients fo Matrices of the chapter Vector Calculus from the textbook titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.

Since matrices represent linear mappings, we can exploit the fact that there is a vector-space isomorphism (linear, invertible mapping) between the space $\mathbb{R}^{m \times n}$ of $m \times n$ matrices and the space $\mathbb{R}^{mn}$ of mn vectors. Therefore, we can re-shape our matrices into vectors of lengths $mn$ and $pq$, respectively. The gradient using these $mn$ vectors results in a Jacobian Matrices can be of size $mn \times pq$. .... In practical applications, it is often desirable to re-shape the matrix into a vector and continue working with this Jacobian matrix: The chain rule... boils down to simple matrix multiplication, whereas in the case of a Jacobian tensor, we will need to pay more attention to what dimensions we need to sum out.

What I understood from the paragraph is: There is always a one-one mapping(?) between $\mathbb{R}^{m \times n}$ and $\mathbb{R}^{mn}$. So, we use this property to replace any element in $\mathbb{R}^{m \times n}$ (matrix) to an element in $\mathbb{R}^{mn}$.

I have doubt on how the property allows us to replace the matrix by vector without any discrepancies?

  • $\begingroup$ Can you please provide more details about your doubt? Why do you think that we cannot map $m \times n$ matrices to $m*n$ vectors without "any discrepancies"? What discrepancies or problems do you potentially see here? I am asking this question because I don't see any problem, but I am not thinking about all problems now. So, maybe, there's a problem or this has side-effects/consequences for certain operations. $\endgroup$
    – nbro
    Dec 3, 2021 at 11:35
  • $\begingroup$ @nbro The paragraph is saying vector-space isomorphism as the only requisite for reshaping. So, I asked with the intention of how it can assure compatibility with all tasks. $\endgroup$
    – hanugm
    Dec 3, 2021 at 22:24

2 Answers 2


An isomorphism $T$ between vector spaces $V$ and $W$ over the same field $K$ (for example, $K = \mathbb{R}$) is defined as a bijective (i.e. 1-to-1 and onto, which makes a bijection an invertible function) transformation (a function) that preserves the 2 main properties

  1. scalar multiplication: $T(c\mathbf {u} )=cT(\mathbf {u} )$, where $c \in K$ and $\mathbf {u} \in V$
  2. vector addition: $T(\mathbf {u} +\mathbf {v} )=T(\mathbf {u} )+T(\mathbf {v})$, where $\mathbf {u}, \mathbf {v} \in V$

Linear maps preserve these 2 properties, but not all linear maps are bijective, so not all linear maps are isomorphisms. In fact, linear maps that are bijective are called linear isomorphisms. You can denote an isomorphism between $V$ and $W$ as follows (to emphasize that it's not just a linear map): $T: V {\overset {\sim }{\to }} W$.

The fact that an isomorphism is invertible means that there exists a function $T^{-1}$ that transforms any element of $W$ to any element of $V$.

So, if there's an isomorphism $T$ between $V = \mathbb{R}^{n \times m}$ and $W = \mathbb{R}^{nm}$ (note that I am not proving it here, but it's probably not complicated to prove), then it implies that, for each $\mathbf {v}_W \in \mathbb{R}^{nm}$, you can use $T^{-1}: \mathbb{R}^{nm} {\overset {\sim }{\to }} \mathbb{R}^{n \times m}$ to retrieve the corresponding vector $\mathbf {v}_V \in \mathbb{R}^{n \times m}$ (not that, in the context of vector spaces, matrices are also vectors, which is just the name used to denote objects in vector spaces).

This should prove why, if reshaping is an isomorphism, then you can perform operations first on $\mathbf {v}_W$ and then go back to $\mathbf {v}_V$.

  • 1
    $\begingroup$ To prove reshaping is an isomorphism, you need to prove it's bijective and show that it preserves scalar and vector addition. This shouldn't be too difficult (I think, at least if you had one course in linear algebra), but if you have problems, let me know and I can try to help. $\endgroup$
    – nbro
    Dec 3, 2021 at 12:17

I think the textbook is making it unnecessarily complicated. $\mathbb{R}^{n\times m}$ and $\mathbb{R}^{nm}$ both represent the same vector space, a real vector space of dimension $nm$. (So clearly they are isomorphic because they are the same.)

The interpretation of them being either a space of vectors or a space of matrices is an artificial distinction in this context.

  • $\begingroup$ $\mathbb{R}^{n\times m}$ and $\mathbb{R}^{nm}$ are different spaces: matrices and vectors. Each real value in a matrix has a row and a column. Each value in a vector has just an index. There are two standard mappings between them: row-major and column-major, plus many uncommon mappings. Therefore they are not "the same" space (although they are isomorphic.) You must specify which mapping you want to use before moving between them. The author is teaching math, not engineering, and therefore must make this distinction. $\endgroup$
    – Eponymous
    Sep 30, 2022 at 15:25
  • $\begingroup$ @Eponymous Both of them describes a vector space of dimension $nm$. In this case, multiple automorphisms exist for this object, two of which you gave as example. An automorphism always exists for any object (the identity mapping), and that is what the OP wanted to show. $\endgroup$
    – PeaBrane
    Sep 30, 2022 at 18:15
  • $\begingroup$ @Eponymous "the author is teaching maths, not engineering" absolute savage hahahaha $\endgroup$
    – David
    May 28 at 21:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .