First of all the scope of the question is as follows - we have Sequence2Sequence architecture with:

  • Decoder: Bidirectional LSTM

  • Encoder: regular (single directional) LSTM

What I know:

  • When you pass Encoder's output - it's just concatenation of [forward_output, backward_output]

    • so the output size i doubled

What I don't know:

  • How to pass Encoder's state to initialize Decoder?

  • I would concatenate Encoder's: [forward_state, backward_state]

But in that case the unit_size of the Decoder must double the unit_size of the Encoder.

Other alternatives which may preserve original unit_size of the Decoder are as follows:

  • pass just the forward_state to the Decoder

  • sum forward and backward states before passing to the Decoder

Could you please comment on that matter? Because I'm not sure what would be the best practice here.

I would be very glad for your help :)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.