Classification trees and binary linear classification

In this lecture we will finish talking about classification trees and we will talk about binary linear classification.

Autograd demo

For assignment 3 you will need to use Autograd package. You can install it using

pip install autograd

You must import NumPy from the autograd for the package to work properly.

import autograd, autograd.misc
from autograd import numpy ad np

You can run full demo using Jupyter Notebook provided below

autograd_demo.ipynb

Classification trees

Let's consider a jar with 6 balls: three red, two blue, and one green. If we draw two balls from it with replacement, what's the probability that we will get different colors?

Let's consider the first ball drawn. There is a $50\%$ probability of it being red color. If the first ball is red, the probability of the second ball not being red is also $50\%$

$$ P(\text{1st is red, 2nd is not red}) = \frac{1}{2} \times \frac{1}{2}. $$

If the first ball is blue, the probability of this is $\frac{1}{3}$ and probability of second ball being not blue is $\frac{2}{3}$

$$ P(\text{first is blue, second is not blue}) = \frac{1}{3} \times \frac{2}{3}. $$

Similarly, probability of first ball being green and second ball not being green is

$$ P(\text{first is green, second is not green}) = \frac{1}{6} \times \frac{5}{6}. $$

Finally, since these are non-overlapping events we add these probabilities to get the final answer

$$ P(\text{balls have different colors})=\frac{1}{2}\times\frac{1}{2}+\frac{1}{3}\times\frac{2}{3}+\frac{1}{6}\times\frac{5}{6} = \sum_vp_v(1-p_v), $$