I am trying to design a deep network that works on signals. The network should include multiple stacked tasks, but each task would work on a different window size of the signal.
For example, the first task should chunk the input into 3 seconds window frames then the second task should take the outputs for 7 frames (21 seconds input), etc.
I was thinking of adding 7 similar modules for the first task in parallel and merge their output, but in this case each of these modules will be trained on 1/7 of the data.
I also need to add feedback from the next layers, so pretraining the lower layers might not work.
Any idea how to design that?