Logistic Regression

• A mtehod of estimating probaility of an event

$$P(d,x) = \text{Probability of an event (death of drug trial participant,}\\\text{ given a quantity x (age of participant), and a treamtetn d)}$$

So we want to estimate $P(d,x)$

• Experiment: separate randomly the data (participants) into those getting treatment $A$ & those getting treatment $B$.

• Odds: if $p$ is a probability, $h = \displaystyle \frac{p}{1-p}$ is the odds $\in (0,+\infty)$

• Define log odds as $\mu=\log h$ and $\mu \in (-\infty, +\infty)$

  • note: log function crunches the scale

graph of log h

Logistic Regression Model

$$\mu(d,x) = \beta_0 + \beta_1 d_A + \beta_2 x$$

• where: $\beta_0,\beta_1,\beta_2$ are the regression parameters
• $x$ indicates age
• $d_A$ indicates if data belongs to treatment $A$ or not

Data $Y_i$ is assumed: $Y_i \sim Bern(p(d_i,x_i))$

• $\forall \text{participant }i$, we know data $x_i$ (age), andd which group they are in: $d_i=1 \text{ for i in group A}$

Idea

• To estimate $p(d,x)$ we will look for an estimator of $\mu(d,x)$ first.

• We are interested in the so called "odds ratio":

  $$R = \frac{h \text{ for A}}{h \text{ for B}} = \frac{\displaystyle e^{\mu(A,x)}}{\displaystyle e^{\mu(B,x)}} = e^{\mu(A,x) - \mu(B,x)} = e^{\beta_1}$$

  This shows why we want to estimate $\beta_1$.

It turns out that it is possible to define aan MLE for $\hat{\beta}=(\hat{\beta}_0, \hat{\beta}_1, \hat{\beta}_2)$ based on the data $(x_i, Y_i)$ such that $Y - \hat{P}$ is perpendicular to the data.

• Note: $\hat{P} = \text{predicrted value of }p\text{ according to } \hat{\beta}$

  $$\hat{\mu}(d,x) = \hat{\beta}_0 + \hat{\beta}_1d_A + \hat{\beta}_2x$$

• Compute $p$ from $\mu$:

  • $\mu = \log h \displaystyle \Longrightarrow h = e^{\mu}$

  • $h = \displaystyle \frac{p}{1-p} \displaystyle \Longrightarrow p = \displaystyle \frac{h}{1+h}$

  $$\Longrightarrow \hat{p} (d,x) = \frac{\displaystyle e^{\hat{\mu} (d,x)}}{1 + \displaystyle e^{\hat{\mu} (d,x)}}$$

  Of course, as we said, our estimated odds ratio is $\hat{R} = e^{\beta_1}$

Question: why not using a model like $$\mu(d,x) = \beta_0 + \beta_1 d_A +\beta_2 d_B + \beta_3 x$$

• Answer: $d_A$ and $d_B$ are not independnt. In fact you can build $d_B$ from $d_A$ by $\displaystyle d_B = \left(\begin{array}{c}1\\1\\1\\..\\1\end{array}\right) - d_A = \textbf{1} - d_A$

Notes:

• If we have 3 treatments ($A$,$B$,$C$), then develop a model with 2 parameter (like $d_A,d_B$) (for the same reason as question above)

• If you don't have enough data, use Baysian statistics