Do the rows of the design matrix refer to the observations or predictors?

Question

I attempt to understand the formulation of dictionary learning for this paper:

1. Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution
2. Multimodal Task-Driven Dictionary Learning for Image Classification

Both papers used the exact formulation in two different domains.

Part 1: Clarification on math notations

Based on my understanding, in common machine learning, we formulate our matrices, from vectors, as rows to be observations, columns to be predictors.

Given a matrix, $A$:

         $p_1$ $p_2$ $p_3$ $p_4$ $p_5$ label
$o_1$      1     2     3     4     1     1
$o_2$      2     3     4     5     2     1
$o_3$      3     4     5     6     2     0
$o_4$      4     5     6     7     3     0

So using math notation and excluding label, I can define this matrix, $A = [o_1, o_2, o_3, o_4] ∈ R^{4×5}$, as $A = [{(1, 2, 3, 4, 1), (2, 3, 4, 5, 2), (3, 4, 5, 6, 2), (4, 5, 6, 7, 3)}]$, and in numpy:

import numpy as np

A = np.array([[1, 2, 3, 4, 1],
              [2, 3, 4, 5, 2],
              [3, 4, 5, 6, 2],
              [4, 5, 6, 7, 3]])

A.shape
# (4, 5)

Am I right?

Answer 1

Based on my understanding, in common machine learning, we formulate our matrices, from vectors, as rows to be observations, columns to be predictors.

The rows (or, in general, the first dimension of your tensor) are typically the observations. For example, in TensorFlow, the first dimension of the input tensor typically refers to the batch size, i.e. the number of observations. If you are using Pandas (a Python library to manipulate data), the rows are typically the observations and the columns are the predictors.

However, in general, it does not really matter which convention you use, provided that you use one of the conventions consistently in your implementation (i.e. you choose one of the conventions and you stick with it throughout all your code, to avoid complexity), and make it clear in the documentation. So, you can have a matrix where either the rows or columns are observations and, consequently, the columns or, respectively, rows are the features (aka predictors or independent variables).

Anyway, it is probably a good idea to be consistent with existing literature and implementations/libraries, so you should probably use the rows for the observations.