Recurrent Neural Networks

RNNs are used when we have sequential data. Some examples of problems that use sequence data include:

• Speech Recognition

• Music Generation

• Sentiment Classification

• DNA Sequence Analysis

• Machine Translation

• Video Activity Recognition

• Name Entity Recognition etc

An RNN learns how to map an input sequence to an output sequence.

The input sequence is denoted by $X^{\lt 1\gt(i)}, X^{\lt 2\gt(i)}, ..., X^{\lt T_x^{(i)}\gt(i)}$where i denotes training examples and $T_x^{(i)}$ is the length of the input sequence of the $i^{th}$ training example.

Similarly, the output sequence is denoted by $Y^{\lt 1\gt(i)}, Y^{\lt 2\gt(i)}, ..., Y^{\lt T_y^{(i)}\gt(i)}$where i denotes training examples and $T_y^{(i)}$ is the length of the output sequence of the $i^{th}$ training example.

The input and output sequences can have different lengths. This is one of the main reasons why we can't use a standard neural network to learn the mapping from the input to the output sequences. Another reason is because standard neural networks don't share features learned across different positions of a sequence.