interlude: try to derive the formula for $\int \Phi(\cdot) \phi(\cdot)$

We want, using a simplified version for now:

$$ I_1 = \int_{-\infty}^\infty \Phi(x)\phi(x)\,dx $$

And we have:

$$ \frac{d}{dx} \Phi(x) = \phi(x) $$

Using integration by parts, with:

  • $u(x) = \Phi(x)$, and
  • $v'(x) = \phi(x)$, so that
  • $v(x) = \int \phi(x) \, dx = \Phi(x)$

So:

$$ I_1 = \int_{-\infty}^\infty \Phi(x)\phi(x)\,dx \\ = [\Phi(x)^2]_{-\infty}^\infty - \int_{-\infty}^\infty \phi(x) \Phi(x) \, dx $$
$$ = 1 - 0 - I_1 = 1 - I_1 $$

Therefore:

$$ 2I_1 = 1 $$

And so:

$$ I_1 = \frac{1}{2} $$

Try now for:

$$ I_2 = \int_{-\infty}^\infty \Phi(\lambda x)\phi(x)\, dx $$

We can try:

  • $u(x) = \Phi(\lambda x)$, and
  • $v'(x) = \phi(x) \, dx$, so that
  • $v(x) = \int \phi(x) \, dx = \Phi(x)$

We need to find the derivative of $u(x)$, ie $\frac{d}{dx}\Phi(\lambda x)$

Looking at wikipedia, https://en.wikipedia.org/wiki/Normal_distribution , we have:

$$ \Phi(x) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^x \exp\left(-\frac{t^2}{2}\right)\,dt $$

We also have the error function $\def\erf{\text{erf}}\erf(x)$, defined as the probability of a random variable with normal distribution of mean 0 and variance $1/2$ falling in the range $[-x, x]$; that is:

$$ \erf(x) = \frac{2}{\sqrt{\pi}} \int_0^x \exp(-t^2)\,dt $$

$\Phi(x)$ and $\erf(x)$ are closely related, namely:

$$ \Phi(x) = \frac{1}{2} \left( 1 + \erf \left( \frac{x}{\sqrt{2}} \right) \right) $$

"For a generic normal distribution with density $f$, mean $\mu$ and deviation $\sigma$, the cumulative distribution function is:

$$ F(x) = \Phi \left( \frac{x - \mu}{\sigma} \right) = \frac{1}{2} \left( 1 + \erf \left( \frac{x - \mu}{\sigma \sqrt{2}} \right) \right) $$

Thinking about this, from the definition of what is the CDF for the Gaussian, we can form the following identity:

$$ \Phi\left( \frac{x - \mu}{\sigma} \right) = \frac{1}{\sqrt{2\pi \sigma^2}} \int_{-\infty}^x \exp\left( -\frac{(t - \mu)^2}{2\sigma^2} \right) \, dt $$

substitue in $r = \frac{ t - \mu}{\sigma}$. Therefore:

$$ t = \sigma r + \mu $$

Therefore:

$$ dt/dr = \sigma $$

For the limits, we have $t_1 = -\infty$, and $t_2 = x$. Applying the subsitution $r = \frac{ t - \mu}{\sigma}$, we have:

$$ r_1 = \frac{t_1 - \mu}{\sigma} = \frac{-\infty - \mu }{\sigma} = -\infty $$

And:

$$ r_2 = \frac{t_2 - \mu}{\sigma} = \frac{x - \mu}{\sigma} $$

So:

$$ \Phi\left(\frac{x-\mu}{\sigma}\right) = \frac{1}{\sqrt{2\pi\sigma^2}} \int_{-\infty}^{\frac{x-\mu}{\sigma}} \exp \left( - \frac{r^2}{2} \right) \sigma \, dr $$
$$ = \sigma \frac{1}{\sqrt{2\pi\sigma^2}} \int_{-\infty}^{\frac{x - \mu}{\sigma}} \exp\left( - \frac{r^2}{2} \right) \, dr $$
$$ = \Phi\left(\frac{x - \mu}{\sigma}\right) $$

... which is at least consistent

We also have, from the wikipedia page:

$$ \mathcal{N}(x \mid \mu, \sigma^2) = \frac{1}{\sigma} \phi \left( \frac{x - \mu}{\sigma} \right) $$

Therefore:

$$ \frac{d}{dx} \Phi\left( \frac{x - \mu}{\sigma} \right) = \frac{1}{\sigma} \phi\left(\frac{x - \mu}{\sigma}\right) $$

Therefore, $u'(x) = \frac{d}{dx}\Phi(\lambda x) = \lambda \phi(\lambda x)$

So we have:

$$ I_2 = \left[ \Phi(\lambda x) \Phi(x) \right]_{-\infty}^\infty - \lambda \int_{-\infty}^\infty \phi(\lambda x) \Phi(x)\,dx $$
$$ = 1 - 0 - \lambda \int_{-\infty}^\infty \Phi(x)\phi(\lambda x)\,dx $$

substitute $u=\lambda x$, therefore $x = u / \lambda$, and $dx = 1/\lambda\, du$, So:

$$ \lambda \int_{-\infty}^\infty \Phi(x)\phi(\lambda x) \, dx $$
$$ = \lambda \int_{-\infty}^\infty \Phi(u/\lambda)\phi(u) 1/\lambda \,du $$
$$ = \int_{-\infty}^\infty \Phi(u/\lambda)\phi(u)\,du $$

That didnt work

Let's try integration by parts again, with:

  • $u = \Phi(x)$,
  • $v'(x) = \phi(\lambda x)\,dx$, and so
  • $v(x) = \frac{1}{\lambda} \Phi(\lambda x)$

Therefore:

$$ I_2 = 1 - \lambda [\Phi(x)\frac{1}{\lambda}\Phi(\lambda x)]^\infty_{-\infty} + \lambda \int_{-\infty}^\infty \phi(x) \frac{1}{\lambda}\Phi(\lambda x)\,dx $$

= 1 - 1 + I_2

...so that doesnt work either

Question: "How to calculate $\int \phi(x) \Phi(\frac{x - b}{a})\,dx$?"

Compared to what we want, we dont need the $-b$ bit, but we will need to generalize from $\phi(x)$ to $\mathcal{N}(x \mid \mu, S)$. However, let's work through the derivation for this particular form first.

"We have $\phi(x) = \frac{1}{\sqrt{2\pi}}\exp\left(-\frac{t^2}{2}\right)$ and $\Phi(x) = \int_{-\infty}^x \phi(t)\,dt$. We try to compute:

$$ I(\mu,\sigma) = \int \phi(x) \Phi\left(\frac{x - \mu}{\sigma} \right)\,dx $$

"Using the dominated convergence theorem, we are allowed to take the derivative with respect to $\mu$ inside the integral. We have:

$$ \partial_\mu I(\mu, \sigma) = \int \phi(x) \left( - \frac{1}{a}\right) \phi \left( \frac{x - \mu}{\sigma} \right) \, dx $$

At this point I get stuck, since it's an indefinite integral. I assume that the answer assumes that it's a definite integral, from $-\infty$ to $\infty$, but it's an assumption. Looking around at other answers:

Start with:

$$ I = \def\A{\mathbb{A}} \def\B{\mathbb{B}} \int_{-\infty}^\infty \Phi\left( \frac{f - \A} {\B} \right)\phi(f)\, df $$

Taking derivative wrt $\A$:

$$ \partial_\A I = \int_{-\infty}^\infty \partial_\A \left( \Phi \left( \frac{f - \A}{\B} \right) \phi(f) \right) \, df $$
$$ =\int_{-\infty}^\infty \left( \frac{-1}{\B} \right) \phi\left( \frac{f - \A}{\B} \right) \phi(f) \, df $$

Looking at $E_1 = \phi\left(\frac{f - \A}{\B}\right) \phi(f)$:

$$ E_1 = \frac{1}{2\pi} \exp \left( - \frac{1}{2} \left( \frac{f^2 - 2f\A + \A^2 + \B^2f^2} {B^2} \right) \right) $$
$$ =\frac{1}{2\pi} \exp\left( - \frac{1}{2\B^2} \left( \left( \sqrt{(1 + \B^2)}f - \A\frac{1}{\sqrt{1 + \B^2}} \right)^2 - \frac{\A^2} {1 + \B^2} + \A^2 \right) \right) $$
$$ - \frac{\A^2}{1+\B^2} + \A^2 $$
$$ = \frac{-\A^2 + \A^2 + \A^2\B^2} {1 + \B^2} $$
$$ = \frac{\A^2\B^2} {1 + \B^2} $$

Therefore $E_1$ is:

$$ \frac{1}{2\pi} \exp \left( -\frac{1}{2\B^2} \frac{\A^2\B^2}{1 + \B^2} \right) \exp \left( - \frac{1}{2\B^2} \left( f\sqrt{1 + \B^2} - \frac{\A}{\sqrt{1 + \B^2}} \right)^2 \right) $$
$$ = \frac{1}{2\pi} \exp \left( -\frac{\A^2}{2(1 + \B^2)} \right) \exp \left( - \frac{1}{2\B^2} \left( f\sqrt{1 + \B^2} - \frac{\A}{\sqrt{1 + \B^2}} \right)^2 \right) $$

Make change of variable:

$$ y = f\frac{\sqrt{1 + \B^2}}{\B} - \frac{A}{\B\sqrt{1 + \B^2}} $$

Therefore:

$$ dy = \frac{\sqrt{1 + \B^2}}{\B}\,df $$

For the limits, we have $f_1 = -\infty$, and $f_2 = \infty$

$\sqrt{1 + \B^2}$ is always positive. $\B$ is always negative. Therefore:

$$ y_1 = +\infty, y_2 = -\infty $$

Therefore:

$$ \partial_\A I = \frac{-1}{\B}\int_{+\infty}^{-\infty} \frac{1}{2\pi} \exp \left( - \frac{\A^2} {2(1+\B^2)} \right) \exp \left( - \frac{1}{2} y^2 \right) \frac{\B}{\sqrt{1 + \B^2}} \, dy $$
$$ = \frac{1}{\sqrt{2\pi}} \frac{1}{\B} \frac{\B}{\sqrt{1 + \B^2}} \exp \left( - \frac{\A^2}{2(1 + \B^2)} \right) \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}} \exp\left( -\frac{1}{2} y^2 \right) \, dy $$
$$ = \frac{1}{\sqrt{2\pi}} \frac{1}{\sqrt{1+\B^2}} \exp \left( - \frac{\A^2}{2(1 + \B^2)} \right) (1) $$
$$ = \frac{1}{\sqrt{1 + \B^2}} \phi\left( \frac{\A}{\sqrt{1 + \B^2}} \right) $$

Now we need to re-integrate back up again, since we currently have $\partial_\A I$, and we need $I$.

Since we dont have limits, we'll need to find at least one known point.

We have the following integral:

$$ I = \frac{1}{\sqrt{1 + \B^2}} \int \phi\left( \frac{\A}{\sqrt{1 + \B^2}} \right) \,d\A $$
$$ = \frac{1}{\sqrt{1 + \B^2}} \sqrt{1 + \B^2} \Phi\left( \frac{\A} {\sqrt{1 + \B^2}} \right) + C $$

... where $C$ is a constant of integration

$$ = \Phi \left( \frac{\A}{\sqrt{1 + \B^2}} \right) + C $$

Looking at the original integral, we had/have:

$$ I = \int_{-\infty}^\infty \Phi\left( \frac{f - \A}{\B} \right) \phi(f) \, df $$

We can see that, given that $\B$ is negative, as $\A \rightarrow \infty$, $\Phi\left(\frac{f-\A}{\B}\right) \rightarrow \Phi(\infty) = 1$.

Therefore, as $\A \rightarrow \infty$, $\int_{-\infty}^\infty \Phi(\cdot)\phi(f)\,df \rightarrow 1$

Meanwhile, looking at the later expression for $I$, ie:

$$ I= \Phi\left( \frac{\A}{\sqrt{1 + \B^2}} \right) + C $$

... as $\A \rightarrow +\infty$, $\Phi\left( \frac{\A}{\sqrt{1 + \B^2}} \right) \rightarrow 1$

But we know that as $\A \rightarrow +\infty$, $I \rightarrow 1$.

Therefore, $C = 0$

Therefore:

$$ I = \Phi \left( \frac{\A} {\sqrt{1 + \B^2}} \right) $$