Short answers
Is back-propagation applied immediately after getting the output for each input or after getting the output for all inputs in a batch?
You can perform back-propagation using (or after) only one training input (also known as data point, example, sample or observation) or multiple ones (a batch). However, the loss function to train the neural network is slightly different in both cases.
Is back-propagation applied $n$ times till the neural network gives a satisfactory result for a single data point before going to work on the next data point?
If we use only one example, we usually do not wait until the neural network gives satisfactory results for a single input-label pair $(x_i, y_i)$, but we keep feeding it with several input-label pairs, one after the other (and each time updating the parameters of the neural network using back-propagation), without usually caring whether the neural network already produces a satisfactory output for an input-label pair before feeding it with the next (although you could also do that).
Long answer (to the 1st question)
In case you want to know more about the first question, keep reading!
What is back-propagation?
Back-propagation is the (automatic) process of differentiating the loss function, which we use to train the neural network, $\mathcal{L}$, with respect to all of the parameters (or weights) of the same neural network. If you collect the $N$ parameters of the neural network in a vector
$$\boldsymbol{\theta} =
\begin{bmatrix}
\theta_1\\
\vdots \\
\theta_N
\end{bmatrix},
$$
then the derivative of the loss function $\mathcal{L}$ with respect to $\boldsymbol{\theta}$ is called the gradient, which is a vector that contains the partial derivatives of the loss function with respect to each single scalar parameter of the network, $\theta_i$, for $i=1, \dots, N$, that is, the gradient looks something like this
$$
\nabla \mathcal{L} =
\begin{bmatrix}
\frac{\partial \mathcal{L}}{ \partial \theta_1}\\
\vdots \\
\frac{\partial \mathcal{L}}{ \partial \theta_N}
\end{bmatrix},
$$
where the symbol $\nabla$ denotes the gradient of the function $\mathcal{L}$.
Loss functions
The specific loss function $\mathcal{L}$ that is used to train the neural network depends on the problem that we need to solve. For example, if we need to solve a regression problem (i.e. predict a real number), then the mean squared error (MSE) can be used. If we need to solve a classification problem (i.e. predict a class), the cross-entropy (CE) (aka negative log-likelihood) may be used instead.
Example
Let us assume that we need to solve a regression problem, so we choose the mean squared error (MSE) as the loss function. For simplicity, let's also assume that the neural network, denoted by $f_{\boldsymbol{\theta}}$, contains only one output neuron and contains no biases.
Stochastic gradient descent
Given an input-label pair $(x_i, y_i) \in D$ (where $D$ is a dataset of input-label pairs), the squared error function (not the mean squared error yet!) is defined as
$$\mathcal{L}_i(\boldsymbol{\theta}) = \frac{1}{2} (f_{\boldsymbol{\theta}}(x_i) - y_i)^2,$$
where $f_{\boldsymbol{\theta}}(x_i) = \hat{y}_i$ is the output of the neural network for the the data point $x_i$ (which depends on the specific values of $\boldsymbol{\theta}$) and $y_i$ is the corresponding ground-truth label.
As the name suggests, $\mathcal{L}_i$ (where the subscript $_i$ is only used to refer to the specific input-label pair $(x_i, y_i)$) measures the squared error (i.e. some notion of distance) between the current prediction (or output) of the neural network, $\hat{y}_i$, and the expected output for the given input $x_i$, i.e. $y_i$.
We can differentiate $\mathcal{L}_i(\boldsymbol{\theta})$ with respect to the parameters of the neural network, $\boldsymbol{\theta}$. However, given that the details of back-propagation can easily become tedious, I will not describe them here. You can find more details here.
So, let me assume that we have a computer program that is able to compute $\nabla \mathcal{L}_i$. At that point, we can perform one step of the gradient descent algorithm
$$
\boldsymbol{\theta} \leftarrow \boldsymbol{\theta} - \gamma \nabla \mathcal{L}_i, \label{1}\tag{1}
$$
where $\gamma$ is the learning rate and $\leftarrow$ is the assignment operator. Note that $\boldsymbol{\theta}$ and $\nabla \mathcal{L}_i$ have the same dimensions, $N$.
So, I have just shown you that you can update the parameters of the neural network using only one input-label pair, $(x_i, y_i)$. This way of performing GD with only one input-label pair is known as stochastic gradient descent (SGD).
Batch (or mini-batch) gradient descent
In practice, for several reasons (including learning instability and inefficiency), it is rarely the case that you update the parameters using only one input-label pair $(x_i, y_i)$. Instead, you use multiple input-label pairs, which are collected in a so-called batch
$$B = \{(x_1, y_i), \dots, (x_M, y_M) \},$$
where $M$ is the size of batch $B$, which is also known as mini-batch when $M$ is smaller than the total number of input-label pairs in the training dataset, i.e. when $|B| = M < |D|$. If you use mini-batches, typical values for $M$ are $32$, $64$, $128$, and $256$ (yes, powers of 2: can you guess why?). See this question for other details.
In this case, the loss function $\mathcal{L}_M(\boldsymbol{\theta})$ is defined as as the mean (or average) of the squared errors for single input-label pairs, $\mathcal{L}_i(\boldsymbol{\theta})$, i.e.
\begin{align}
\mathcal{L}_M(\boldsymbol{\theta})
&= \frac{1}{M} \sum_{i=1}^M \mathcal{L}_i(\boldsymbol{\theta}) \\
&= \frac{1}{M} \sum_{i=1}^M \frac{1}{2} (f_{\boldsymbol{\theta}}(x_i)-y_i)^2 \\
&= \frac{1}{M} \frac{1}{2} \sum_{i=1}^M (f_{\boldsymbol{\theta}}(x_i)-y_i)^2.
\end{align}
The normalisation factor, $\frac{1}{M}$, can be thought of as averaging out the losses of all input-label pairs. Note also that we can take the $\frac{1}{2}$ out of the summation because it is a constant with respect to the variable of the summation, $i$.
In this case, let me also assume that we are able to compute (using back-propagation) the gradient of $\mathcal{L}_M$, so that we can perform a gradient descent (GD) update (using a batch of examples)
$$
\boldsymbol{\theta} \leftarrow \boldsymbol{\theta} - \gamma \nabla \mathcal{L}_M \label{2} \tag{2}
$$
The only thing that changes with respect to the GD update in equation \ref{1} is the loss function, which is now $\mathcal{L}_M$ rather than $\mathcal{L}_i$.
This is known as mini-batch (or batch) gradient descent. In the case $M = |D|$, this is simply known as gradient descent, which is a term that can also be used to refer to any of its variants (including SGD or mini-batch GD).
Further reading
You may also be interested in this answer, which provides more details about mini-batch gradient descent.
