Understanding Word2Vec – Part 3: Predicting the Next Word with Neural Networks

In the previous article , we explored how words can be converted into neural network inputs and represented numerically. Now, we will look further into how the network uses these representations to make predictions. In order to perform backpropagation , we first need the network to make predictions . We will use the input word to predict the next word in the phrase . For example, consider the sentence: “The Incredibles is great!” If the input word is “The Incredibles” , we indicate that by putting a 1 in the The Incredibles input and 0s in all other inputs . In this case, we want the next word “is” to have the highest output value . Similarly, if the input word is “is” , which means the input for is is 1 and all the other inputs are 0 , then we want the output corresponding to the next word “great!” to have the largest value . And if the input word is “Despicable Me” , then we want the output corresponding to the next word “is” to have the largest value . To make these predictions, we