Regression with Multiple Outputs

The aim here is to predict multiple values, instead of just a single value. This is analogous to multi-label classification where we attempt to predict multiple classses at once.

Reasons to do so:

• less training time

• less number of weights, therefore, less prone to overfitting

• possible relation between values being predicted can be learned

The Error Function is as follows:

$E = \sum_{i=1}^k \sum_{t=1}^N (r_i^t-y_i^t)^2$

i.e. the mean squared error. (k is the number of values to be predicted)

• can use stochastic gradient descent

• can use mini-batches in the gradient descent