Actaully, the hierarchical learning explanation given by mindcrime is not that acceptable anymore because there are networks with 150 layers or more, and this explanation is not sufficient for such a network. However, we can think of it as solving the knots of high dimensional manifolds i.e. we transform the input into high dimensional space, and this helps us to find better represatation of the data.
Geometric interpretation was explained as such in the book Deep Learning with Python by François Chollet:
...you can interpret a neural network as a very complex geometric transformation
in a high-dimensional space, implemented via a long series of simple steps...
Imagine two sheets of colored
paper: one red and one blue. Put one on top of the other. Now crumple them
together into a small ball. That crumpled paper ball is your input data, and each sheet
of paper is a class of data in a classification problem. What a neural network (or any
other machine-learning model) is meant to do is figure out a transformation of the
paper ball that would uncrumple it, so as to make the two classes cleanly separable
again. With deep learning, this would be implemented as a series of simple transformations
of the 3D space, such as those you could apply on the paper ball with your fingers,
one movement at a time.
Uncrumpling paper balls is what machine learning is about: finding neat representations
for complex, highly folded data manifolds. At this point, you should have a
pretty good intuition as to why deep learning excels at this: it takes the approach of
incrementally decomposing a complicated geometric transformation into a long
chain of elementary ones, which is pretty much the strategy a human would follow to
uncrumple a paper ball. Each layer in a deep network applies a transformation that
disentangles the data a little—and a deep stack of layers makes tractable an extremely
complicated disentanglement process.
I suggest you to read this artical to learn about topological interpretation of deep learning
Also, this toy interactive code may help you