1
$\begingroup$

For a project I am doing, I found the paper Face Alignment in Full Pose Range: A 3D Total Solution.

It is using a cascaded convolutional neural network, but I wasn't able to find the original paper explaining what that is.

In layman's terms and intuitively, how does a cascaded CNN work? What does it solve?

$\endgroup$
2
$\begingroup$

The paper you are citing is the paper that introduced the cascaded convolution neural network. In fact, in this paper, the authors say

To realize 3DDFA, we propose to combine two achievements in recent years, namely, Cascaded Regression and the Convolutional Neural Network (CNN). This combination requires the introduction of a new input feature which fulfills the "cascade manner" and "convolution manner" simultaneously (see Sec. 3.2) and a new cost function which can model the priority of 3DMM parameters (see Sec. 3.4)

where 3DDFA stands for 3D Dense Face Alignment, the framework proposed in this paper for face alignment, in which a dense 3D Morphable Model (3DMM) is fitted to the image via cascaded CNNs (the regressor), where the term dense refers to the number of points of the face that will be modeled. See figure 1 of this paper, which should provide some intuition behind the purpose of this framework.

In section 3 (page 3), they also say

In this section, we introduce how to combine Cascaded Regression and CNNs to realize 3DDFA. By applying a CNN as the regressor in Eqn. 1, Cascaded CNN can be formulated as:

\begin{align} \mathbf{p}^{k+1} = \mathbf{p}^{k} + \text{Net}^{k} (\text{Fea}(\mathbf{I}, \mathbf{p}^k)) \tag{1}\label{1} \end{align}

where

  • $k$ is the iteration number
  • $\mathbf{p}$ is the regression objective
  • $\text{Net}$ is the CNN structure
  • $\text{Fea}$ contains the two constructed image features
    • Pose Adaptive Feature (PAF) (section 3.2.1)
    • Projected Normalized Coordinate Code (PNCC) (section 3.2.2)
  • $\mathbf{I}$ is the image

The expression cascaded CNN apparently refers to the fact that equation \ref{1} is used iteratively, so there will be multiple CNNs, one for each iteration $k$. In fact, in the paper, they say

Unlike existing CNN methods that apply different network structures for different fitting stages, 3DDFA employs a unified network structure across the cascade. In general, at iteration $k$ ($k = 0, 1, \dots, K$), given an initial parameter $\mathbf{p}^k$, we construct PNCC and PAF with $\mathbf{p}^k$ and train a two-stream CNN $\text{Net}^k$ to conduct fitting. The output features from two streams are merged to predict the parameter update $\Delta \mathbf{p}^k$

$$ \Delta \mathbf{p}^k = \text{Net}^k(\text{PAF}(\mathbf{p}^k, \mathbf{I}), \text{PNCC}(\mathbf{p}^k, \mathbf{I})) $$

Afterwards, a better intermediate parameter $\mathbf{p}^{k+1} = \mathbf{p}^k + \Delta \mathbf{p}^k$ becomes the input of the next network $\text{Net}^k$ which has the same structure but different weights with $\text{Net}^k$.

In figure 2 of the paper (page 4), the structure of this two-stream CNN, $\text{Net}^k$, at iteration $k$, is shown.

| improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ I wasn't familiar with this cascade CNN. I only quickly read certain parts of the paper and I tried to provide some intuition behind it. Maybe later I will refine this answer to provide even more intuition. $\endgroup$ – nbro Jan 10 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.