These answers are based on my personal understanding of Bert from both the paper and official_implementation, hope it will help:
What do they mean by "maximum scoring span is used as the prediction"?
As you know in SQuAD the input sequence is divided to 2 parts: Question and Document (from which we extract the answer if possible).
Sometimes the input length exceeds the max_seq_length parameter, in this case the document is truncated to as many parts as needed and we end up having more than one input for the same question/document. Mention that the question is replicated in all the resulting inputs (see line_350 for details).
So in such cases, in order to determine the predicted span among all the generated-inputs we use the maximum scoring i.e (max_start + max_end) / 2 = ( max(Softmax(S*Ti)) + max(Softmax(E*Ti)) ) / 2 of all the inputs related to the same question.
"The training objective is the loglikelihood of the correct start and end positions"
The loss is the average of the start_position_loss start_loss and the end_postition_loss end_loss. Each loss is computed in the same way: after applying the Softmax to the final output (i.e S*Ti or E*Ti) we use the real start/end postitions to compute the loss (see code below).
From run_squad.py in Bert_repo:
def compute_loss(logits, positions):
one_hot_positions = tf.one_hot(positions, depth=seq_length, dtype=tf.float32)
log_probs = tf.nn.log_softmax(logits, axis=-1)
loss = -tf.reduce_mean(tf.reduce_sum(one_hot_positions * log_probs, axis=-1))
return loss
start_positions = features["start_positions"]
end_positions = features["end_positions"]
start_loss = compute_loss(start_logits, start_positions)
end_loss = compute_loss(end_logits, end_positions)
total_loss = (start_loss + end_loss) / 2.0