The Learning Problem

Lecture

Use machine learning when

• a pattern exists
• we cannot pin it down mathematically
• we have data on it

Components of learning, formalization:

• input $x \in X$ (ex: $x$ is a d-dimensional vector)
• output $y \in Y$ (ex: $y$ is binary 1/0)
• $f$ target function such that $f: X \rightarrow Y$. $f$ is unknown.
• data ($x_1$, $y_1$) … ($x_n$, $y_n$)
• $g$ hypothesis, such that $g: X \rightarrow Y$. $g$ is an approximation of $f$ built from data.

Goal: make $g$ the best approximation of $f$.

Q & A

Q. How does the convergence of the perceptron learning algorithm change with dimensionality of the data? A. Badly. This algorithm is shown because of its simplicity, don't focus on it we will see better algo by the end of the course.

Q. How does statistics compare with machine learning?
A. Statistics use assumptions to have a proof, whereas machine learning make few assumptions to cover a broader territory.

Q. Bottleneck of machine learning?
A. Two types: computational bottleneck (time) and generalization (learn instead of memorize), the later is the hardest.