I attempt to understand the formulation of dictionary learning for this paper:

  1. Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution
  2. Multimodal Task-Driven Dictionary Learning for Image Classification

Part 2: Confusion on math notation definitions (Conflict with my understanding)

(Part 1 is in the other post)

From both papers that were published in reputable venues, they defined the dictionary learning as follow:

Given the original feature representation $X = [x_1, ..., x_N] ∈ R^{M×N}$ , dictionary learning aims to learn a set of latent concepts or feature patterns, $D = [d_1,..., d_D] ∈ R^{M×D}$ and a latent sparse representation $A = [α_1, ..., α_N] ∈ R^{D×N}$, with the following empirical cost:

$$ \min_D \frac1N \sum_{n=1}^N \ell(x_n, D), s.t., ||d_k||_{\ell_{2}} \le 1, \forall k = 1, ... , D $$

And in (2), they defined:

Let $X = [x_1, x_2, . . . , x_N ] ∈ Rn×N$ be the collection of $N$ (normalized) training samples that are assumed to be statistically independent. Using the example in part 1, $N$ here is the observation which means the matrix looks like this:

         $o_1$ $o_2$ $o_3$ $o_4$ 
$p_1$      1     2     3     4     
$p_2$      2     3     4     5     
$p_3$      3     4     5     6    
$p_4$      4     5     6     7    
$p_5$      1     2     2     3 
label      1     1     0     0

But since this is the dictionary learning problem, instead of a conventional machine learning one, it is okay to represent the matrix in any form. So basically, if I want it to be my conventional way, I can just transpose the matrix to the one I familiar, from $R^{5×4}$ transpose to $R^{4×5}$, right?

Their formulation:

$$X_{M\times N} = D_{M \times D} \times A_{D \times N}$$

To replicate their experiments in my own setting, the formulation becomes:

$$X_{N \times M} = A_{N \times D} \times D_{D \times M}$$

Given that:

$$X^T = (DA)^T = A^T \times D^T$$

$X = [x_1, ..., x_N] ∈ R^{N×M}$, with $N$ represents the number of observations, $M$ represents the number of features / predictors.


  • $\begingroup$ Can please put your main question in the title? $\endgroup$ – nbro Sep 8 '20 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.