> For the complete documentation index, see [llms.txt](https://vikram-bajaj.gitbook.io/cs-gy-6923-machine-learning/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://vikram-bajaj.gitbook.io/cs-gy-6923-machine-learning/types-of-machine-learning/supervised-learning/neural-networks.md).

# Neural Networks

These models aim to mimic the human brain, or are at least inspired by it.

The basic unit in a neural network is a **neuron**. A neuron computes a weighted sum of its inputs and then an activation is computed.

![](https://3080175707-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M5-0RGi-3PgC2puufMH%2F-M5-0Ri_FPdoeNpowHRz%2F-M5-0U8_PTj4KxWKyhl7%2Fneuron.png?generation=1586990817790079\&alt=media)

$$X=\begin{bmatrix}x\_0\x\_1\\.\\.\\.\\.\x\_d\end{bmatrix}$$, $$W=\begin{bmatrix}w\_0\w\_1\\.\\.\\.\\.\w\_d\end{bmatrix}$$and $$x\_0 = 1$$.

The **sigmoid activation function** $$\frac{1}{1+e^{-W^TX}}$$ gives a probability as an output. The **threshold/step activation function** outputs 1 if $$W^TX > 0$$ and 0 otherwise. The **linear activation function** simply outputs $$W^TX$$.

Some common activation functions:

![](https://3080175707-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M5-0RGi-3PgC2puufMH%2F-M5-0Ri_FPdoeNpowHRz%2F-M5-0U8etCm5dOBbdU-6%2Factivation%20functions.png?generation=1586990817914844\&alt=media)

A neural network (also known as a **Multi Layer Perceptron (MLP)**) has multiple layers of neurons. The most common problem faced in neural networks is the **credit assignment problem**: it is difficult to determine which neurons are to be given credit/blame for an increase/decrease in accuracy.