# Gradient Descent

So we have our hypothesis function and we have a way of measuring how accurate it is. Now what we need is a way to automatically improve our hypothesis function. That's where gradient descent comes in.

To get the most accurate hypothesis, we must **minimize the cost function**. (This means that we have to find values for $$θ\_0, θ\_1$$ such that the cost function would have a minimum value).

To do so, we take the derivative (the line tangent to a function) of our cost function. The slope of the tangent is the derivative at that point and it will give us a direction to move towards. We make steps down that derivative by the parameter $$α$$, called the **learning rate**.

The gradient descent equation is:

repeat until convergence:{

$$θ\_j:= θ\_j − α ∂/∂θ\_j J(θ\_0,θ\_1)$$

}

for j=0 and j=1

* If $$α$$ is too small, gradient descent is slow, i.e. it takes longer to reach the minimum value of J
* If $$α$$ is too large, we may overshoot the minimum, i.e. it may fail to converge or diverge

The equation for gradient descent for *Linear Regression in One Variable* can be obtained by substituting the hypothesis function and the cost function in the gradient descent formula above. We get:

repeat until convergence:{

$$θ\_0:=θ\_0 − α (1/m) ∑\_{i=1}^{m}(h\_θ(x^{(i)})−y^{(i)})$$

$$θ\_1:=θ\_1 − α (1/m) ∑\_{i=1}^{m}((h\_θ(x^{(i)})−y^{(i)})x^{(i)})$$

}

The Gradient Descent used here is also called *Batch Gradient Descent* because it uses all the training examples in each step.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://vikram-bajaj.gitbook.io/machine-learning-stanford-coursera/supervised-learning/linear-regression/linear-regression-in-one-variable/gradient-descent.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.