Actaully, the hierarchical learning explanation given by mindcrime is not that acceptable anymore because there are networks with 150 layers or more, and this explanation is not sufficient for such a network.  However, we can think of it as solving the knots of high dimensional manifolds i.e. we transform the input into high dimensional space, and this helps us to find better represatation of the data. 

Geometric interpretation was explained as such in the book *Deep Learning with Python* by François Chollet:

> ...you can interpret a neural network as a very complex geometric transformation
    in a high-dimensional space, implemented via a long series of simple steps... 
    
> Imagine two sheets of colored
    paper: one red and one blue. Put one on top of the other. Now crumple them
    together into a small ball. That crumpled paper ball is your input data, and each sheet
    of paper is a class of data in a classification problem. What a neural network (or any
    other machine-learning model) is meant to do is figure out a transformation of the
    paper ball that would uncrumple it, so as to make the two classes cleanly separable
    again. With deep learning, this would be implemented as a series of simple transformations
    of the 3D space, such as those you could apply on the paper ball with your fingers,
    one movement at a time.
    Uncrumpling paper balls is what machine learning is about: finding neat representations
    for complex, highly folded data manifolds. At this point, you should have a
    pretty good intuition as to why deep learning excels at this: it takes the approach of
    incrementally decomposing a complicated geometric transformation into a long
    chain of elementary ones, which is pretty much the strategy a human would follow to
    uncrumple a paper ball. Each layer in a deep network applies a transformation that
    disentangles the data a little—and a deep stack of layers makes tractable an extremely
    complicated disentanglement process.



I suggest you to read [this brilliant blog post][1] to learn about topological interpretation of deep learning

Also, [this][2] toy interactive code may help you

Edit: In the context of machine learning, the concept of manifold can be illustrated as in the figure. [![swiss][3]][3]

In the first part, data are 3-dimensional. However, we can find a transformation to get the second image, which shows that, data is actually artificially high dimensional i.e. it is 2-dimensonal manifold in 3-D space. This example may be thought as classification problem, and colours may represent classes, and we can find trivial representation of the data for classification.

Another example could be following figures from the blog I mentioned. In here this classification problem cannot be solved without having a layer that has 3 or more hidden units, regardless of depth. So the notion of high dimensonal transformation is important.
[![1st image][4]][4]

We can map this data to 3-D, and find a plane to separate them.

[![2nd iamge][5]][5]




  [1]: https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/
  [2]: https://cs.stanford.edu/people/karpathy/convnetjs/demo/classify2d.html
  [3]: https://i.sstatic.net/bpS1y.png
  [4]: https://i.sstatic.net/NfkSD.jpg
  [5]: https://i.sstatic.net/tyRcb.jpg