First of all the scope of the question is as follows - we have Sequence2Sequence architecture with:
Decoder: Bidirectional LSTM
Encoder: regular (single directional) LSTM
What I know:
When you pass Encoder's output - it's just concatenation of
- so the output size i doubled
What I don't know:
How to pass Encoder's state to initialize Decoder?
I would concatenate Encoder's:
But in that case the
unit_size of the Decoder must double the
unit_size of the Encoder.
Other alternatives which may preserve original
unit_size of the Decoder are as follows:
pass just the
forward_stateto the Decoder
sum forward and backward states before passing to the Decoder
Could you please comment on that matter? Because I'm not sure what would be the best practice here.
I would be very glad for your help :)