F4B101A / TP1 / MCMC Methods

D'après le notebook IPython par P. Tandeo et T. Guilment.


In [98]:
using Distributions
using Gadfly

Monte-Carlo approximation of $\pi$


In [99]:
T = 1000; # Number of darts launched (simulated)
R = 1; # Circle radius
theta = linspace(0,pi,100);
x = R*cos(theta);
y1 = R*sin(theta);
y2 = -y1;

# Launch darts
points = [rand(Uniform(-R,R), T) rand(Uniform(-R,R), T)];

plot(
layer(x=x, y=y1, Geom.line),
layer(x=x, y=y2, Geom.line),
layer(x=points[:,1], y=points[:,2], Geom.point)
)


Out[99]:
x -3.5 -3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -2.4 -2.3 -2.2 -2.1 -2.0 -1.9 -1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1 -1.0 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 -4 -2 0 2 4 -3.0 -2.8 -2.6 -2.4 -2.2 -2.0 -1.8 -1.6 -1.4 -1.2 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 -3.5 -3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -2.4 -2.3 -2.2 -2.1 -2.0 -1.9 -1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1 -1.0 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 -4 -2 0 2 4 -3.0 -2.8 -2.6 -2.4 -2.2 -2.0 -1.8 -1.6 -1.4 -1.2 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 y

In [100]:
# Count the number of darts inside the circle to estimate the area
insideCircle(x, y) = ((x^2 + y^2) < R^2) ? 1 : 0
N = reduce(+, mapslices(p -> insideCircle(p[1], p[2]), points, 2))

# Estimate Pi
pi_hat = 4*(N/T)


WARNING: Method definition insideCircle(Any, Any) in module Main at In[88]:2 overwritten at In[100]:2.
Out[100]:
3.084

Monte-Carlo integration

We would like to approximate the integral $I$ defined by:

$$ I=\int_a^b \frac{1}{\sqrt{2\pi}} \exp^{-x^2/2} dx. $$

With $a = -1.96$ and $b = 1.96$ (95% confidence interval).

We rewrite $$ I = \int_a^b g(x)\,f(x)\,dx \\ h(x) = \frac{1}{\sqrt{2\pi}} \exp^{-x^2/2} \\ g(x) = h(x)\,(b-a) \\ f(x) = \frac{\mathbb{1}_{[a,b]}}{b-a} $$

(Théorème de transfert)

$$ I = \mathbb{E}_f[g(X)] = \int_a^b g(x)\,f(x)\,dx = \int_a^b h(x)\,(b-a)\,f(x)\,dx \\ = \frac{(b-a)}{(b-a)} \int_a^b h(x)\,dx = \int_a^b h(x)\,dx $$

$\{x_i\}_{1\lt i\lt N} \sim \mathcal{U}_{[a,b]}$, for large N:

$$ \hat{I} = \frac{1}{N}\sum_{i=1}^{N}g(x_i) $$

In [101]:
T = 100;
a = -1.96;
b = 1.96;
x1 = linspace(a, b, T);
x2 = linspace(-5, 5, T);

h(x) = pdf(Normal(0, 1), x);
g(x) = h(x)*(b-a);

# Plot the integral to compute (note: only between a and b)
plot(x=x2, y=h(x2), Geom.line)


WARNING: Method definition h(Any) in module Main at In[92]:7 overwritten at In[101]:7.
WARNING: Method definition g(Any) in module Main at In[92]:8 overwritten at In[101]:8.
Out[101]:
x -20 -15 -10 -5 0 5 10 15 20 -15.0 -14.5 -14.0 -13.5 -13.0 -12.5 -12.0 -11.5 -11.0 -10.5 -10.0 -9.5 -9.0 -8.5 -8.0 -7.5 -7.0 -6.5 -6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.0 9.5 10.0 10.5 11.0 11.5 12.0 12.5 13.0 13.5 14.0 14.5 15.0 -20 -10 0 10 20 -15 -14 -13 -12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -0.5 -0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 -0.40 -0.38 -0.36 -0.34 -0.32 -0.30 -0.28 -0.26 -0.24 -0.22 -0.20 -0.18 -0.16 -0.14 -0.12 -0.10 -0.08 -0.06 -0.04 -0.02 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.22 0.24 0.26 0.28 0.30 0.32 0.34 0.36 0.38 0.40 0.42 0.44 0.46 0.48 0.50 0.52 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 0.70 0.72 0.74 0.76 0.78 0.80 0.82 -0.5 0.0 0.5 1.0 -0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 y

In [102]:
X = rand(Uniform(a, b), T);
G = g(X);

# Estimate I by E_f[g(X)]
I_hat = (1/T) * sum(G)


Out[102]:
0.9575386316986803

In [103]:
for T = [100,1000,10000,100000]
    X = rand(Uniform(a, b), T);
    G = g(X);
    I_hat = (1/T) * sum(G)
    println("Estimate of I for T = ", T, ": ", I_hat)
end


Estimate of I for T = 100: 0.9517082598351836
Estimate of I for T = 1000: 0.9723372002996593
Estimate of I for T = 10000: 0.9440884457988756
Estimate of I for T = 100000: 0.9489644213631062

In [ ]: