In regards to the question you mention (in the comments of the OP), these searches are related to optimization. I'm not sure of your background, so let me describe it from scratch, briefly:
Remember the derivative? The base idea is to talk about how the function changes in regards to changes in input. So now, we're out of high school and we're building neural nets. We've done the basic coding, and want to look at how our model is working. Back from our statistics class, we remember we use a certain measure of error (e.g. least squares) to determine the efficacy of the models from that class, so we decide to use that here. We get this error, and it's a bit too big for our liking, so we decide to fiddle with our model and adjust the weights to get that error down. But how?
This is where the 'search' comes into play. It's really a search for the best weights to put on the edges of our net to optimize it. We use the derivative (in some fancy ways, using the 'stochasitc' (think random sampling) and other ways the question mentions) to search for which way is 'down' in the high dimensional space of our weights. In other words, what we are searching for is minima or maxima to optimize our neural net, and we 'search' for it by doing a derivative which tells us which way to go, moving a bit in that direction, then doing that again and again iteratively to find (hopefully) the best weights.
This video here goes into all the detail you'd want, and I recommend the entire series as a robust but understandable intro to neural nets: Demystifying Neural Networks
Go and look up 'gradient descent' to get any related material. (Note, the gradient here is equivalent to multidimensional derivative direction to go in, and descent is just searching for the minima)