Deep Learning Specialization - Coursera
main
main
  • Introduction
  • Neural Networks and Deep Learning
    • Introduction to Deep Learning
    • Logistic Regression as a Neural Network (Neural Network Basics)
    • Shallow Neural Network
    • Deep Neural Network
  • Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
    • Practical Aspects of Deep Learning
    • Optimization Algorithms
    • Hyperparameter Tuning, Batch Normalization and Programming Frameworks
  • Structuring Machine Learning Projects
    • Introduction to ML Strategy
    • Setting Up Your Goal
    • Comparing to Human-Level Performance
    • Error Analysis
    • Mismatched Training and Dev/Test Set
    • Learning from Multiple Tasks
    • End-to-End Deep Learning
  • Convolutional Neural Networks
    • Foundations of Convolutional Neural Networks
    • Deep Convolutional Models: Case Studies
      • Classic Networks
      • ResNets
      • Inception
    • Advice for Using CNNs
    • Object Detection
      • Object Localization
      • Landmark Detection
      • Sliding Window Detection
      • The YOLO Algorithm
      • Intersection over Union
      • Non-Max Suppression
      • Anchor Boxes
      • Region Proposals
    • Face Recognition
      • One-Shot Learning
      • Siamese Network
      • Face Recognition as Binary Classification
    • Neural Style Transfer
  • Sequence Models
    • Recurrent Neural Networks
      • RNN Structure
      • Types of RNNs
      • Language Modeling
      • Vanishing Gradient Problem in RNNs
      • Gated Recurrent Units (GRUs)
      • Long Short-Term Memory Network (LSTM)
      • Bidirectional RNNs
    • Natural Language Processing & Word Embeddings
      • Introduction to Word Embeddings
      • Learning Word Embeddings: Word2Vec and GloVe
      • Applications using Word Embeddings
      • De-Biasing Word Embeddings
    • Sequence Models & Attention Mechanisms
      • Sequence to Sequence Architectures
        • Basic Models
        • Beam Search
        • Bleu Score
        • Attention Model
      • Speech Recognition
Powered by GitBook
On this page

Was this helpful?

  1. Sequence Models

Recurrent Neural Networks

RNNs are used when we have sequential data. Some examples of problems that use sequence data include:

  • Speech Recognition

  • Music Generation

  • Sentiment Classification

  • DNA Sequence Analysis

  • Machine Translation

  • Video Activity Recognition

  • Name Entity Recognition etc

An RNN learns how to map an input sequence to an output sequence.

The input sequence is denoted by X<1>(i),X<2>(i),...,X<Tx(i)>(i)X^{\lt 1\gt(i)}, X^{\lt 2\gt(i)}, ..., X^{\lt T_x^{(i)}\gt(i)}X<1>(i),X<2>(i),...,X<Tx(i)​>(i)where i denotes training examples and Tx(i)T_x^{(i)}Tx(i)​ is the length of the input sequence of the ithi^{th}ith training example.

Similarly, the output sequence is denoted by Y<1>(i),Y<2>(i),...,Y<Ty(i)>(i)Y^{\lt 1\gt(i)}, Y^{\lt 2\gt(i)}, ..., Y^{\lt T_y^{(i)}\gt(i)}Y<1>(i),Y<2>(i),...,Y<Ty(i)​>(i)where i denotes training examples and Ty(i)T_y^{(i)}Ty(i)​ is the length of the output sequence of the ithi^{th}ith training example.

The input and output sequences can have different lengths. This is one of the main reasons why we can't use a standard neural network to learn the mapping from the input to the output sequences. Another reason is because standard neural networks don't share features learned across different positions of a sequence.

PreviousSequence ModelsNextRNN Structure

Last updated 4 years ago

Was this helpful?