Advice for Using CNNs

Use an Open-Source Implementation

As discussed earlier, several parameters need to be tuned while training deep neural networks. It is, therefore, advised to use an existing open-source implementation and tweak it to suit our needs rather than starting afresh.

Transfer Learning

Instead of learning weights from scratch, it is much faster to start with the weights learned by an existing model for a similar task.

For example, a model trained on ImageNet can be used for another object recognition task by altering the softmax layer and updating the weights accordingly. The model would have already learned weights to represent low level features like edges, and this knowledge is transferable to the new task.

This is especially useful when we have limited data and computational resources for the new task.

Data Augmentation

As discussed, data augmentation is the process of increasing the amount of training data by performing operations such as mirroring, flipping, rotating, transforming the existing images, since deep learning models need large amounts of data.

Some commonly used techniques involve:

• mirroring

• random cropping

Some other techniques include rotating, shearing, local warping etc.

Another commonly type of data augmentation is color shifting. This involves slightly modifying the R, G, B values of the images to generate new images that have slightly different color schemes i.e. we distort the color channels. This makes the model more robust to colors in the images i.e. it should be able to identify a cat regardless of changes in color, background illumination etc. One way to implement this is PCA Color Augmentation.

Tips for Doing Well in Competitions

When we have less data, hand engineering is the way to go.

Techniques that can be used to do well in competitions include:

• Ensembling Train multiple neural networks independently and average their outputs

• Multi-crop at Test Time Run the classifier on multiple versions (mirrored, cropped, rotated etc) of the test images and average the outputs