Understanding Seq2Seq Neural Networks – Part 5: Decoding the Context Vector

In the previous article, we stopped at the concept of the context vector . In this article, we will start by decoding the context vector . Connecting the Decoder The first thing we need to do is connect the long-term and short-term memories (the cell states and hidden states ) that form the context vector to a new set of LSTMs . Just like the encoder , the decoder will also have two layers , and each layer will have two LSTM cells . The LSTMs in the decoder are different from the ones in the encoder and have their own separate weights and biases . Using the Context Vector The context vector is used to initialize the long-term and short-term memories (the cell states and hidden states ) in the LSTMs of the decoder . This is important because it allows the decoder to start with the information learned from the input sentence . Goal of the Decoder The ultimate goal of the decoder is to convert the context vector into the output sentence . In simple terms, the encoder understands the input