Detect street and sidewalk surface in aerial imagery (neural network)

Question

I would like to detect street and sidewalk surface in a very detailed (0.075m/pix) USGS High Resolution Orthoimagery which basically means image segmentation with two classes. Places in question are residential areas similar to this one. I will download uncompressed raw imagery in GeoTIFF from USGS for the detection.

I read that neural networks can perform very good in image segmentation and I would like to try them. I am a developer by day so I can code but am a beginner to neural networks only knowing the basic principles about architecture, weighting and backpropagation etc. Is it possible to jump right in into my task or do I need to start with something simpler? I would prefer jumping right in if it can save time.

I skimmed though few papers dealing with similar thing and they seem quite complicated. Is there some simple way I can get started? I mean maybe an open source project in neural networks that deals with image segmentation that is similar to my task and I could make use of it?

I see neural networks need to be trained first and I am prepared to do manual segmentation first to have data for training. However, I have no idea about neural network design/architecture, how to design the layers, how many layers do I need etc. I also would like to use the fact that the network would learn some basics on how streets and sidewalks are built - that they are (not sure if my term is correct) "linear structures" which usually run many meters in length and may not even end in the image, also that sidewalks usually run alongside streets, streets have intersections etc.

Answer 1

Yes, in fact neural networks (NNs) are very efficient at segmentation and it seems to me that your problem matches the capabilities of neural networks very well.

I think it best for you to truly understand what a NN is before using it. First, let's start with the architecture. A NN has 3 regions, the input layer, the hidden layers and the output layer. The input layer depends on the number of features in your dataset. The hidden layers, you can have multiple layers all of different breadth (number of nodes per layer). The output layer depends on the number of classes in your dataset.

An easy example is applying a NN to the MNIST datatset. This is a dataset which contains handwritten digits between 0-9. Let's assume each of these images is 16*16=256 pixels. Thus, you will need 256 input nodes. And you will need 10 output nodes, one for each possible output. The hidden layers can be set in any way you can creatively imagine. There are however ways to optimize your hidden layer to get the best performance possible while not spending too much computational power.

This is always how a NN works. The beauty of a NN is that you only need to code it once and it can learn any function. All you need to do is change your architecture, but the underlining principles will always be the same.

In your case, you want to do segmentation. This is often done using a window around the pixel you want to classify. Popular choices are 3*3 or 5*5 pixels. The choice of your considered window will determine the number of nodes in your input layer. Then you want to classify them as one of two classes, thus you need 2 output nodes. You can also use just 1, but I don't recommend it, I can expand on this if you care.

One caveat of NN is that you will need quite a bit of training data. So get ready to classify a lot of pixels manually.

How to know how many layers in hidden layer? How to know how many nodes per layer in the hidden layer?

In general, for simple operations like the one you are trying to learn you do not need to have multiple layers. One hidden layer should be enough, at most 2. But, how do you determine the number of nodes you should use? You need to use some model validation techniques to do this. One way is through grid search and cross-validation. Train, and re-train your model with multiple number of nodes and then compare their performances to identify the optimal number of nodes. To get good results this does require a large dataset.

Rule of thumb: 1 hidden layer for NN! Don't get dragged into deep models if you don't need them.