1. As defined in the CPO paper, $d$ is the discounted future state distribution, roughly representing the weighted state distribution in the future. Consider infinite horizon with policy $\pi$,
$$
d_\pi(s)=(1-\gamma)\sum_{k=0}^\infty \gamma^kP(s_k=s|\pi,s_0)\mu(s_0)\tag{1}
$$
In the hierarchical case, we only consider the first $c$ steps, therefore $(1)$ needs to be truncated as
$$
d_\pi(s)=(1-\gamma)\sum_{k=0}^c \gamma^kP(s_k=s|\pi,s_0)\mu(s_0)\tag{2}
$$
In the function form, it is
$$
d_\pi=(1-\gamma)A_\pi\mu\tag{3}
$$
But after truncation the sum of $d_\pi(s)$ over all states $s$ is not 1, therefore the normalizing term $(I-\gamma^cP_\pi^c)^{-1}$ needs to be multiplied, resulting in the finite horizon future state distribution
$$
d_\pi=(1-\gamma)A_\pi(I-\gamma^cP_\pi^c)^{-1}\mu\tag{4}
$$
 2. The "every-$c$-step discounted state frequency" builds on $(4)$, but it lumps the $c$ steps into one "high level" step where the discount factor is $\gamma^c$ and the transition matrix is $P_\pi^c$. Starting with $(4)$, replace $c$ with 1, we get the "every one step future state distribution"
$$
d_\pi=(1-\gamma)(I-\gamma P_\pi)^{-1}\mu\tag{5}
$$
Then replace $\gamma$ and $P_\pi$ in $(5)$ with $\gamma^c$ and $P_\pi^c$, we get the "every-$c$-step discounted state frequency", or "every-$c$-step future state distribution"
$$
d_\pi^c=(1-\gamma^c)(I-\gamma^c P_\pi^c)^{-1}\mu\tag{6}
$$

By the way, I read your blogpost on this paper. It's very helpful for me, thank you!

**Update of the 1st part with respect to @Sherwin Chen's comment**

Derivation of the normalizing term $(I-\gamma^cP_\pi^c)^{-1}$: Starting from $(3)$ (the function form),
\begin{align}
d_\pi&=(1-\gamma)A_\pi\mu\\
&=(1-\gamma)(\sum_{k=0}^{c-1}\gamma^kP_\pi^k)\mu\\
&=(1-\gamma)(1-\gamma P_\pi)^{-1}(I-\gamma^cP_\pi^c)\mu\\
&=(1-\gamma)(\sum_{k=0}^{\infty}\gamma^kP_\pi^k)(I-\gamma^cP_\pi^c)\mu
\end{align}
If the normalizing term is not multiplied, it gets us nowhere; however, if we multiply the normalizing term, use $\tilde d_\pi$ to denote the normalized $d_\pi$, then
\begin{align}
\tilde d_\pi&=(1-\gamma)(\sum_{k=0}^{\infty}\gamma^kP_\pi^k)(I-\gamma^cP_\pi^c)(I-\gamma^cP_\pi^c)^{-1}\mu\\
&=(1-\gamma)(\sum_{k=0}^{\infty}\gamma^kP_\pi^k)\mu
\end{align}
Explicitly write out the dependent variables:
\begin{align}
\tilde d_\pi(s)&=(1-\gamma)\bigg[\sum_{k=0}^{\infty}\gamma^kP_\pi^k(s|s_0)\bigg]\mu(s_0)
\end{align}
\begin{align}
\sum_{s\in S}\tilde d_\pi(s)&=(1-\gamma)\bigg[\sum_{k=0}^{\infty}\gamma^k\sum_{s\in S}P_\pi^k(s|s_0)\bigg]\mu(s_0)\\
&=(1-\gamma)\bigg[\sum_{k=0}^{\infty}\gamma^k\bigg]\mu(s_0)\\
&=\mu(s_0)=1
\end{align}
as $\mu()$ is defined to be the Dirac delta centered at $s_0$.