Sequential data & Recurrent neural network - μμ°¨ λ°μ΄ν°μ μν μ κ²½λ§
What is sequential data?
Any data for which an order is relevant: time series is an exemple of data for which the order (chronological order) is important.
What is a Recurrent Neural Network (RNN)?
Googleβs assistant and Appleβs Siri both use RNNs.
Recurrent Neural Network (RNN) is a Deep learning algorithm that is specialized for processing sequential data. What makes RNN special is that it maintains internal memory. It is the reason why RNNs are so efficient at solving machine learning problems that involve sequential data. The internal memory allows for back propagation, meaning the data doesnβt go a one way direction (feed-forward), but also travel back and forth between layers.
Through backpropagation, at each pass back, the weights are adjusted to obtain a more accurate model. This is how nodels learn.
By default, the memory is short term, though with the addition of LSTM they can have long term memory.
We can take a closer look at the looping process by unrolling the network:
Inputs all go through the same cell A, then passes through it again with the additional data from the new input⦠etc. How many of the past data points should be kept for the next loop is called a timestep.
β in analysing a sentence of 5 words, set the timestep to 5 so the whole sentence is taken into account by the RNN. If you set it to 4, when analysing the fifth word, the first one will not be taken into account: βWhat time is it now ?β
The hidden state is the result output of the pass, that will go on to the next timestep. It kind of is a draft-output. When the whole sequence has passed through the cell, the last hidden state becomes the final output.
