How the vector-space isomorphism between $\mathbb{R}^{m \times n}$ and $\mathbb{R}^{mn}$ guarantees reshaping matrices to vectors?

Question

Consider the following paragraph from section 5.4 Gradients fo Matrices of the chapter Vector Calculus from the textbook titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.

Since matrices represent linear mappings, we can exploit the fact that there is a vector-space isomorphism (linear, invertible mapping) between the space $\mathbb{R}^{m \times n}$ of $m \times n$ matrices and the space $\mathbb{R}^{mn}$ of mn vectors. Therefore, we can re-shape our matrices into vectors of lengths $mn$ and $pq$, respectively. The gradient using these $mn$ vectors results in a Jacobian Matrices can be of size $mn \times pq$. .... In practical applications, it is often desirable to re-shape the matrix into a vector and continue working with this Jacobian matrix: The chain rule... boils down to simple matrix multiplication, whereas in the case of a Jacobian tensor, we will need to pay more attention to what dimensions we need to sum out.

What I understood from the paragraph is: There is always a one-one mapping(?) between $\mathbb{R}^{m \times n}$ and $\mathbb{R}^{mn}$. So, we use this property to replace any element in $\mathbb{R}^{m \times n}$ (matrix) to an element in $\mathbb{R}^{mn}$.

I have doubt on how the property allows us to replace the matrix by vector without any discrepancies?

Answer 1

An isomorphism $T$ between vector spaces $V$ and $W$ over the same field $K$ (for example, $K = \mathbb{R}$) is defined as a bijective (i.e. 1-to-1 and onto, which makes a bijection an invertible function) transformation (a function) that preserves the 2 main properties

1. scalar multiplication: $T(c\mathbf {u} )=cT(\mathbf {u} )$, where $c \in K$ and $\mathbf {u} \in V$
2. vector addition: $T(\mathbf {u} +\mathbf {v} )=T(\mathbf {u} )+T(\mathbf {v})$, where $\mathbf {u}, \mathbf {v} \in V$

Linear maps preserve these 2 properties, but not all linear maps are bijective, so not all linear maps are isomorphisms. In fact, linear maps that are bijective are called linear isomorphisms. You can denote an isomorphism between $V$ and $W$ as follows (to emphasize that it's not just a linear map): $T: V {\overset {\sim }{\to }} W$.

The fact that an isomorphism is invertible means that there exists a function $T^{-1}$ that transforms any element of $W$ to any element of $V$.

So, if there's an isomorphism $T$ between $V = \mathbb{R}^{n \times m}$ and $W = \mathbb{R}^{nm}$ (note that I am not proving it here, but it's probably not complicated to prove), then it implies that, for each $\mathbf {v}_W \in \mathbb{R}^{nm}$, you can use $T^{-1}: \mathbb{R}^{nm} {\overset {\sim }{\to }} \mathbb{R}^{n \times m}$ to retrieve the corresponding vector $\mathbf {v}_V \in \mathbb{R}^{n \times m}$ (not that, in the context of vector spaces, matrices are also vectors, which is just the name used to denote objects in vector spaces).

This should prove why, if reshaping is an isomorphism, then you can perform operations first on $\mathbf {v}_W$ and then go back to $\mathbf {v}_V$.

Answer 2

I think the textbook is making it unnecessarily complicated. $\mathbb{R}^{n\times m}$ and $\mathbb{R}^{nm}$ both represent the same vector space, a real vector space of dimension $nm$. (So clearly they are isomorphic because they are the same.)

The interpretation of them being either a space of vectors or a space of matrices is an artificial distinction in this context.