It is well-known that deep feedforward networks can approximate any continuous function from $\mathbb{R}^k$ to $\mathbb{R}^l$, (uniformly on compacts).
However, in practice feature maps are typically used to improve the learning quality and likewise, readout maps are used to make neural networks suited for specific learning tasks.
For example:
- Classification: networks are composed with the softmax (readout) function so they take values in $(0,1)^l$.
What are examples of commonly used feature and readout maps?