Actaully, the hierarchical learning explanation given by mindcrime is not that acceptable anymore because there are networks with 150 layers or more, and this explanation is not sufficient for such a network.  However, we can think of it as solving the knots of high dimensional manifolds i.e. we transform the input into high dimensional space, and this helps us to find better represatation of the data. 

Geometric interpretation was explained as such in the book *Deep Learning with Python* by François Chollet:

> ...you can interpret a neural network as a very complex geometric transformation
    in a high-dimensional space, implemented via a long series of simple steps... 
    
> Imagine two sheets of colored
    paper: one red and one blue. Put one on top of the other. Now crumple them
    together into a small ball. That crumpled paper ball is your input data, and each sheet
    of paper is a class of data in a classification problem. What a neural network (or any
    other machine-learning model) is meant to do is figure out a transformation of the
    paper ball that would uncrumple it, so as to make the two classes cleanly separable
    again. With deep learning, this would be implemented as a series of simple transformations
    of the 3D space, such as those you could apply on the paper ball with your fingers,
    one movement at a time.
    Uncrumpling paper balls is what machine learning is about: finding neat representations
    for complex, highly folded data manifolds. At this point, you should have a
    pretty good intuition as to why deep learning excels at this: it takes the approach of
    incrementally decomposing a complicated geometric transformation into a long
    chain of elementary ones, which is pretty much the strategy a human would follow to
    uncrumple a paper ball. Each layer in a deep network applies a transformation that
    disentangles the data a little—and a deep stack of layers makes tractable an extremely
    complicated disentanglement process.



I suggest you to read [this brilliant blog post][1] to learn about topological interpretation of deep learning

Also, [this][2] toy interactive code may help you


  [1]: https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/
  [2]: https://cs.stanford.edu/people/karpathy/convnetjs/demo/classify2d.html