Of course, it does create more parameters to train, but that seems like a small sacrifice to me.

It is not a "small sacrifice". For the very deep networks that skip connections are applied to, to get the same benefits when concatenating, you would end up witha significant multiplier on the number of parameters.

To get the same passthrough effect on gradient signals (and allow later layers to learn modifications to the identity function), each layer's output would need to be copied to all the following layers. This scales poorly.

Let's take an example of a 10-layer fully-connected network, with 100 neurons per layer in the hidden layers where we want to apply skip connections. In the simple version of this network (ignoring bias to keep the maths simpler), there are 100x100=10,000 parameters for each added layer, making 90,000 parameters overall. If you use addition-based skip connections, the total number of parameters remains the same, at 90,000. If you use concatenation, the layers connect as 100x100, (100+100)x100, (100+100+100)x100 etc, so you end up with 450,000 parameters, five times as many. This is not a "small sacrifice", this is a scaling problem.

The technique of concatenating layers into later parts of the network is known and used. As is adding "early output" or head layers to generate gradient signals at different points in the network. These are valid approaches, and can help with vanishing gradient problems in a similar way. However, the additive skip connections in a residual network scale much better into very deep networks.

Concatenated copies of layers, or additional network heads with the same target functions and loss, are still used, but more sparingly.

This avoids the strange choice of addition, which has a questionable benefit

It may avoid a specific addition mechanism that you seem concerned about. However there are still very similar additions occurring when the concatenated layer feeds forward to the next layer. In fact I suspect that you could set those weights in a specific way (I think a copy of the weights of the new layer that it is concatenating with for just two combined layers) when the layer sizes are the same, and the concatenation and addition approaches would be identical - even down to the number of free parameters because the weights would be forced.

Of course, it does create more parameters to train, but that seems like a small sacrifice to me.

It is not a "small sacrifice". For the very deep networks that skip connections are applied to, to get the same benefits when concatenating, you would end up witha significant multiplier on the number of parameters.

To get the same passthrough effect on gradient signals (and allow later layers to learn modifications to the identity function), each layer's output would need to be copied to all the following layers. This scales poorly.

Let's take an example of a 10-layer fully-connected network, with 100 neurons per layer in the hidden layers where we want to apply skip connections. In the simple version of this network (ignoring bias to keep the maths simpler), there are 100x100=10,000 parameters for each added layer, making 90,000 parameters overall. If you use addition-based skip connections, the total number of parameters remains the same, at 90,000. If you use concatenation, the layers connect as 100x100, (100+100)x100, (100+100+100)x100 etc, so you end up with 450,000 parameters, five times as many. This is not a "small sacrifice", this is a scaling problem.

The technique of concatenating layers into later parts of the network is known and used. As is adding "early output" or head layers to generate gradient signals at different points in the network. These are valid approaches, and can help with vanishing gradient problems in a similar way. However, the additive skip connections in a residual network scale much better into very deep networks.

Concatenated copies of layers, or additional network heads with the same target functions and loss, are still used, but more sparingly.

This avoids the strange choice of addition, which has a questionable benefit

It may avoid a specific addition mechanism that you seem concerned about. However there are still very similar additions occurring when the concatenated layer feeds forward to the next layer. You could set those weights in a specific way (a copy of the weights of the new layer that it is concatenating with for just two combined layers) when the layer sizes are the same, and the concatenation and addition approaches would very similar.