If I understood things correctly: You have a task which you need to estimate two values, gender and age. Your question revolves about the difference between networks which share layers for both inputs, whether the shared layers should be followed independent linear layers.
Firstly, using shared layers in the networks of two related tasks may be useful to create more general latent representations in the network hidden layers. It also can speed up training, the shared layer will learn useful features more quickly than if there were two separate networks for each task.
Some examples which demonstrate the potential benefit of shared networks can be found in the papers for two RL algorithms, A3C and PPG (PPG adds some extra tricks to the shared layers it):
http://proceedings.mlr.press/v48/mniha16.html
https://arxiv.org/abs/2009.04416v1
Whether the shared layers should be followed by many separate linear layers or a single one, at least for me, isn't something easy to deduce. Intuitively, having a single linear mapping after the shared layers will help prevent over-fitting because the shared layers induce more general features in those layers. While having many separate layers may be useful if there is some complex non linear mapping between the final output and the latent features from the shared layers.
I think the best way to find out is just to experiment with it and see which gives the best results.
A little bit anecdotal, but, an example from my experience:
Shared layers are commonly used in RL for actor critic algorithms. A network takes an image as input and outputs an action and a value (the output for the actor and the critic, receptively). Generally a single linear mapping from the shared layers works just fine, even better than more complex networks.
=========== Edit
In pseudo-code, this is the networks that came to my mind:
# Network 1
shared_layer = Linear(input_dim, latent_dim)
output_layer = Linear(latent_dim, m + n)
# Network 2
shared_layer = Linear(input_dim, latent_dim)
output1_layer = Linear(latent_dim, m)
output2_layer = Linear(latent_dim, n)
And, in the question you mention about using BCE with Network 1 and changing to CE for one of the outputs of the Network 2.
The networks themselves are equal to one another. One implementation might be more practical than the other, but they are the same. Depending on the framework you use, you can use either BCE or CE loss in Network 1 and 2. In Network 1 this would mean taking the output of the last layer and slicing the outputs for age and gender into two variables and applying a loss function in each of them.
That said, I would expect to see a difference between using CE or BCE for age classification. When training with BCE it's possible that one estimator turn to be more 'optimist' or 'pessimist' and gives overall high/low probabilities (this will depend on factors such as whether there is class unbalance, if those are taken into account in the training procedure, etc...). And, that will mean that when you take the maximum probability of the age outputs there will be some bias.
CE seems to me to be a more appropriate choice for age classification. Using CE will not prevent bias if there is class unbalance in your data, but with it is more straightforward to handle these issues.
