# Neural Style Transfer

Neural Style Transfer is the process of using a convolutional neural network to transfer the style of one image (style image S) to another image (content image C) and generate a new image G.

It is used by several popular apps such as Prisma.

![](https://3668624528-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M5-0RGo4dZhdwCIHrC1%2F-M5-0TEuN77zAmTpQjIr%2F-M5-0WQQWJNjW1kakmwQ%2FNeural%20Style%20Transfer.JPG?generation=1586990829448779\&alt=media)

Neurons in the layers of a CNN learn to detect different patterns. Neurons in deeper layers learn to identify more sophisticated patterns than those in shallower layers.

The following cost function is used for neural style transfer:

$$J(G) = \alpha J\_{content}(C, G) + \beta J\_{style}(S, G)$$

The first term (content cost) determines how similar the content of G is to that of C and the second term (style cost) determines how similar the style of G is to that of S.

To generate G, first we create G and initialize random values for its pixels. Then, we perform gradient descent on J(G), thereby updating the pixel values of G to get the required G after style transfer.

## The Content Cost Function

Say we use a hidden layer l (of a pre-trained CNN) to compute the content cost. (l is usually one of the middle hidden layers; neither too shallow nor too deep).

If $$a^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/C)}$$ and $$a^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/G)}$$ denote the activations of layer l for the images C and G respectively, we conclude that the images C and G have similar content if $$a^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/C)}$$ and $$a^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/G)}$$are similar.

The content cost function is given by:

$$J\_{cost}^{\[l]}(C, G) = \frac{1}{2} ||a^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/C)}-a^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/G)}||^2$$

## The Style Cost Function

Say we use hidden layer l's activation to measure style. But what exactly is style?

We define style as the correlation between activations across channels.

Let $$a\_{i, j, k}^{\[l]}$$ be the activation at (i, j, k). We compute a $$n\_c^{\[L]}\times n\_c^{\[L]}$$ style matrix $$G^{\[L]}$$ for each image (S and G) with elements denoting the correlations of activations across channels. (k, k'=1, 2, ..., $$n\_c^{\[l]}$$).

$$G\_{kk'}^{\[l]} = \sum\_{i=1}^{n\_h^{\[l]}}\sum\_{j=1}^{n\_w^{\[l]}} a\_{i,j,k}^{\[l]} a\_{i,j,k'}^{\[l]}$$

Now, we will have two style matrices $$G^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/S)}$$and $$G^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/G)}$$. The style cost function for layer l is given by:

$$J\_{style}^{\[l]}(S, G) = \frac{1}{(2n\_h^{\[l]}n\_w^{\[l]}n\_c^{\[l]})^2}||G^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/S)} - G^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/G)}||*F^2 = \frac{1}{(2n\_h^{\[l]}n\_w^{\[l]}n\_c^{\[l]})^2}\sum*{k=1}^{n\_c^{\[l]}}\sum\_{k'=1}^{n\_c^{\[l]}}(G\_{kk'}^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/S)} - G\_{kk'}^{[l](https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/G)})^2$$

The overall style cost function is as follows:

$$J\_{style}(S, G) = \sum\_l \lambda^{\[l]} J\_{style}^{\[l]}(S, G)$$

where $$\lambda^{\[l]}$$ is a hyperparameter for layer l.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://vikram-bajaj.gitbook.io/deep-learning-specialization-coursera/convolutional-neural-networks/neural-style-transfer.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.