Understanding Word2Vec – Part 2: Converting Words into Neural Network Inputs

In the previous article , we just got started with the concept Word2Vec and assigning numbers to words. We will now see how to represent it in the form of a neural network. To show how we can get a neural network to figure out what numbers should go with different words, let’s imagine we have two phrases: The Incredibles is great! Despicable Me is great! To create a neural network that figures out what numbers we should associate with each word, the first thing we do is create inputs for each unique word . For simplicity, assume the movie name is treated as a single word . In this case, we have 4 unique words in the training data, so we create 4 input nodes . Next, each input is connected to at least one activation function . These activation functions use the identity function , which means the input value is exactly the same as the output value . The number of activation functions corresponds to how many numbers we want to associate with each word . The weights on these connections a