Understanding LSTMs – Part 2: The Long-Term and Short-Term Memory Paths

In the previous article we just began with the concept of LSTMS and its diagrams. In this article we will slowly start understanding the LSTM structure, lets begin. In this article, we will begin exploring the details of LSTM. First, let us take a look at the green line. The long-term memory can be modified through multiplication and later through addition. You can notice that there are no weights and biases directly modifying it. This lack of direct weights allows the long-term memory to flow through a series of unrolled units without causing the gradient to explode or vanish. This line is called the hidden state and represents the short-term memory. The short-term memory is connected to weights that can modify it. To better understand how long-term and short-term memories interact and produce predictions, we can run some numbers through this unit. We will explore this in the next article. Looking for an easier way to install tools, libraries, or entire repositories? Try Installerpedi