I am designing a deep autoencoder for graph embedding (exactly node embedding) following this paper SDNE. In the original paper, they used the sigmoid activation for all hidden layers in the autoencoder model, even for the embedding layer.
However, I think the embedding layer should use the tanh activation and the reconstruction layer should be used ReLU activation. Because, embedding is in the range $[-1, 1]$ and reconstruction layer is in the range $[0, x]$, which generates better results due to a larger range for representation and directed graph. Instead of in the range $[0,1]$ from sigmoid will lead to a lack of embedding information.
So, what is the best activation function for deep autoencoders to capture good information about the structure of graph?