Deep Learning Specialization - Coursera
main
main
  • Introduction
  • Neural Networks and Deep Learning
    • Introduction to Deep Learning
    • Logistic Regression as a Neural Network (Neural Network Basics)
    • Shallow Neural Network
    • Deep Neural Network
  • Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
    • Practical Aspects of Deep Learning
    • Optimization Algorithms
    • Hyperparameter Tuning, Batch Normalization and Programming Frameworks
  • Structuring Machine Learning Projects
    • Introduction to ML Strategy
    • Setting Up Your Goal
    • Comparing to Human-Level Performance
    • Error Analysis
    • Mismatched Training and Dev/Test Set
    • Learning from Multiple Tasks
    • End-to-End Deep Learning
  • Convolutional Neural Networks
    • Foundations of Convolutional Neural Networks
    • Deep Convolutional Models: Case Studies
      • Classic Networks
      • ResNets
      • Inception
    • Advice for Using CNNs
    • Object Detection
      • Object Localization
      • Landmark Detection
      • Sliding Window Detection
      • The YOLO Algorithm
      • Intersection over Union
      • Non-Max Suppression
      • Anchor Boxes
      • Region Proposals
    • Face Recognition
      • One-Shot Learning
      • Siamese Network
      • Face Recognition as Binary Classification
    • Neural Style Transfer
  • Sequence Models
    • Recurrent Neural Networks
      • RNN Structure
      • Types of RNNs
      • Language Modeling
      • Vanishing Gradient Problem in RNNs
      • Gated Recurrent Units (GRUs)
      • Long Short-Term Memory Network (LSTM)
      • Bidirectional RNNs
    • Natural Language Processing & Word Embeddings
      • Introduction to Word Embeddings
      • Learning Word Embeddings: Word2Vec and GloVe
      • Applications using Word Embeddings
      • De-Biasing Word Embeddings
    • Sequence Models & Attention Mechanisms
      • Sequence to Sequence Architectures
        • Basic Models
        • Beam Search
        • Bleu Score
        • Attention Model
      • Speech Recognition
Powered by GitBook
On this page
  • Use an Open-Source Implementation
  • Transfer Learning
  • Data Augmentation
  • Tips for Doing Well in Competitions

Was this helpful?

  1. Convolutional Neural Networks

Advice for Using CNNs

Use an Open-Source Implementation

As discussed earlier, several parameters need to be tuned while training deep neural networks. It is, therefore, advised to use an existing open-source implementation and tweak it to suit our needs rather than starting afresh.

Transfer Learning

Instead of learning weights from scratch, it is much faster to start with the weights learned by an existing model for a similar task.

For example, a model trained on ImageNet can be used for another object recognition task by altering the softmax layer and updating the weights accordingly. The model would have already learned weights to represent low level features like edges, and this knowledge is transferable to the new task.

This is especially useful when we have limited data and computational resources for the new task.

Data Augmentation

As discussed, data augmentation is the process of increasing the amount of training data by performing operations such as mirroring, flipping, rotating, transforming the existing images, since deep learning models need large amounts of data.

Some commonly used techniques involve:

  • mirroring

  • random cropping

Some other techniques include rotating, shearing, local warping etc.

Another commonly type of data augmentation is color shifting. This involves slightly modifying the R, G, B values of the images to generate new images that have slightly different color schemes i.e. we distort the color channels. This makes the model more robust to colors in the images i.e. it should be able to identify a cat regardless of changes in color, background illumination etc. One way to implement this is PCA Color Augmentation.

Tips for Doing Well in Competitions

When we have less data, hand engineering is the way to go.

Techniques that can be used to do well in competitions include:

  • Ensembling Train multiple neural networks independently and average their outputs

  • Multi-crop at Test Time Run the classifier on multiple versions (mirrored, cropped, rotated etc) of the test images and average the outputs

PreviousInceptionNextObject Detection

Last updated 4 years ago

Was this helpful?