Parametric Estimation for Simple Polynomial Regression

In cases where the data cannot be fit using a linear decision boundary, we may want to use polynomial regression.

Say we want to use a degree 2 polynomial. The equation can be given by:

$g(x|w_2,w_1,w_0) = w_2x^2 + w_1x + w_0$

Our aim is to find values for $w_0,w_1,w_2$ that minimize the squared error $\sum_t (r^t-g(x^t))^2$

Note: Given a dataset $\{x^t,r^t\}_{t=1}^N$ where $x^t\in R$ i.e. where $x^t=[x_1^t]$ (1 dimension), to find the polynomial of degree 2 $g(x|w_2,w_1,w_0) = w_2x^2 + w_1x + w_0$ that minimizes the squared error, we can construct a related dataset with inputs in $R^2$ (2 dimensions) with the second dimension $x_2^t=(x_1^t)^2$, and then use simple linear regression on this new dataset to obtain $w_2,w_1,w_0$ that minimize the squared error, and finally output $g(x|w_2,w_1,w_0) = w_2x^2 + w_1x + w_0$ with these $w_2,w_1,w_0$ values as the best 2 degree polynomial that fits the original dataset.

This can be extended to higher degree polynomials as well.