Notations

The training set is denoted by X.

It has N training examples.

Each example is denoted by $x^t, r^t$ where $x^t$ is the feature set of the $t^{th}$ training example and $r^t$ is the corresponding label.

$x^t=\begin{bmatrix}x_1^t\\x_2^t\\.\\.\\.\\x_d^t\end{bmatrix}$

The training set is denoted as $X = \{x^t, r^t\}_{t=1}^{N}$

h(x) is the hypothesis that assigns a label r to x. For example, if we have a task of classifying cars as family cars/not family cars, based on two features $X_1, X_2$, based on the below feature space, we could hypothesize that:

$h(x) = \begin{cases}1; P_1\leq X_1\leq P_2 \,\,\&\,\, e_1\leq X_2\leq e_2\\0; otherwise\end{cases}$

This hypothesis, however, may or may not be correct.

The error of the hypothesis h on X is given by:

$E(h|X)$ or $Err(h|X) = |\{x^t \in X | h(x^t) \neq r^t \}|$, basically the number of misclassified examples.

Say we know the correct hypothesis and we compare our current hypothesis with the correct hypothesis:

The current hypothesis labels everything inside the orange box as + and everything outside as -.

False positives are examples that are mistakenly labeled by our current hypothesis as positive. False negatives are examples that are mistakenly labeled by our current hypothesis as negative.

Based on the task at hand, we must focus on reducing either false positives or false negatives.