Parametric Estimation for Multivariate Linear Regression

$x=\begin{bmatrix}x_1\\.\\.\\.\\x_d\end{bmatrix}$

We need to find the parameters $W=\begin{bmatrix}w_0\\.\\.\\.\\w_d\end{bmatrix}$

so that the linear function $g(x|w_d,w_{d-1},...,w_1,w_0) = w_dx_d + w_{d-1}x_{d-1}+...+w_1x_1+w_0$

minimizes the square error on the dataset $\{x^t,r^t\}_{t=1}^N$ where $x^t=\begin{bmatrix}x_1^t\\x_2^t\\.\\.\\.\\x_d^t\end{bmatrix}$

Let $D=\begin{bmatrix}1 & x_1^1&x_2^1&...&x_d^1\\1 & x_1^2&x_2^2&...&x_d^2\\.&.&.&...&.\\.&.&.&...&.\\.&.&.&...&.\\1 & x_1^N&x_2^N&...&x_d^N\\\end{bmatrix}_{N \times (d+1)}$ and $r=\begin{bmatrix}r_1\\r_2\\.\\.\\.\\r_N\end{bmatrix}_{N\times 1}$

Then, $W=(D^TD)^{-1}D^Tr$

Sometimes, the inverse doesn't exist. This usually happens when the number of dimensions is too less.