Deep Learning Specialization - Coursera
1.0.0
1.0.0
  • Introduction
  • Neural Networks and Deep Learning
    • Introduction to Deep Learning
    • Logistic Regression as a Neural Network (Neural Network Basics)
    • Shallow Neural Network
    • Deep Neural Network
  • Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
    • Practical Aspects of Deep Learning
    • Optimization Algorithms
    • Hyperparameter Tuning, Batch Normalization and Programming Frameworks
  • Structuring Machine Learning Projects
    • Introduction to ML Strategy
    • Setting Up Your Goal
    • Comparing to Human-Level Performance
    • Error Analysis
    • Mismatched Training and Dev/Test Set
    • Learning from Multiple Tasks
    • End-to-End Deep Learning
  • Convolutional Neural Networks
    • Foundations of Convolutional Neural Networks
    • Deep Convolutional Models: Case Studies
      • Classic Networks
      • ResNets
      • Inception
    • Advice for Using CNNs
    • Object Detection
      • Object Localization
      • Landmark Detection
      • Sliding Window Detection
      • The YOLO Algorithm
      • Intersection over Union
      • Non-Max Suppression
      • Anchor Boxes
      • Region Proposals
    • Face Recognition
      • One-Shot Learning
      • Siamese Network
      • Face Recognition as Binary Classification
    • Neural Style Transfer
  • Sequence Models
    • Recurrent Neural Networks
      • RNN Structure
      • Types of RNNs
      • Language Modeling
      • Vanishing Gradient Problem in RNNs
      • Gated Recurrent Units (GRUs)
      • Long Short-Term Memory Network (LSTM)
      • Bidirectional RNNs
    • Natural Language Processing & Word Embeddings
      • Introduction to Word Embeddings
      • Learning Word Embeddings: Word2Vec and GloVe
      • Applications using Word Embeddings
      • De-Biasing Word Embeddings
    • Sequence Models & Attention Mechanisms
      • Sequence to Sequence Architectures
        • Basic Models
        • Beam Search
        • Bleu Score
        • Attention Model
      • Speech Recognition
Powered by GitBook
On this page

Was this helpful?

  1. Sequence Models

Recurrent Neural Networks

PreviousSequence ModelsNextRNN Structure

Last updated 5 years ago

Was this helpful?

RNNs are used when we have sequential data. Some examples of problems that use sequence data include:

  • Speech Recognition

  • Music Generation

  • Sentiment Classification

  • DNA Sequence Analysis

  • Machine Translation

  • Video Activity Recognition

  • Name Entity Recognition etc

An RNN learns how to map an input sequence to an output sequence.

The input sequence is denoted by X<1>(i),X<2>(i),...,X<Tx(i)>(i)X^{\lt 1\gt(i)}, X^{\lt 2\gt(i)}, ..., X^{\lt T_x^{(i)}\gt(i)}X<1>(i),X<2>(i),...,X<Tx(i)​>(i)where i denotes training examples and Tx(i)T_x^{(i)}Tx(i)​ is the length of the input sequence of the ithi^{th}ith training example.

Similarly, the output sequence is denoted by Y<1>(i),Y<2>(i),...,Y<Ty(i)>(i)Y^{\lt 1\gt(i)}, Y^{\lt 2\gt(i)}, ..., Y^{\lt T_y^{(i)}\gt(i)}Y<1>(i),Y<2>(i),...,Y<Ty(i)​>(i)where i denotes training examples and Ty(i)T_y^{(i)}Ty(i)​ is the length of the output sequence of the ithi^{th}ith training example.

The input and output sequences can have different lengths. This is one of the main reasons why we can't use a standard neural network to learn the mapping from the input to the output sequences. Another reason is because standard neural networks don't share features learned across different positions of a sequence.