random variables and modulo division

Random numbers, while impossible to generate algorithmically, can approximately rather closely. While rigorous analysis of exactly how close approximations are and what shortcomings exist are a matter of ongoing research, the current body of knowledge on the subject is quite immense. Take this paper as a starting point.

When using random numbers, one often needs to generate numbers within a range, via sampliing a set of integers, real numbers, etc. Extreme caution must be taken when using custom methods not provided in the standard libraries or other well-vetted ones. Hacky solutions can often cause otherwise random sets of number to exhibit non-random properties.

Take this common scenario:

Suppose you want to generate a set of random numbers whose elements are between 0 and 1000, and your RNG outputs an integer between 0 and 34767. You come up with a clever hack:

\begin{equation} x_{random} = RANDOM \mod 1000 \end{equation}

Now, the space of $0\leq x \leq32768$ gets mapped down to $0 \leq x \leq 1000$. See the results:


In [15]:
srand(1234) # for reproducibility

modulus = 1000
rand_data = rand(1:32768, 1000000);
modulo_data = rand_data % modulus;

In [16]:
using Gadfly

plot(x=modulo_data, Geom.histogram(bincount=modulus))


Out[16]:
x -1500 -1000 -500 0 500 1000 1500 2000 2500 -1000 -950 -900 -850 -800 -750 -700 -650 -600 -550 -500 -450 -400 -350 -300 -250 -200 -150 -100 -50 0 50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 950 1000 1050 1100 1150 1200 1250 1300 1350 1400 1450 1500 1550 1600 1650 1700 1750 1800 1850 1900 1950 2000 -1000 0 1000 2000 -1000 -900 -800 -700 -600 -500 -400 -300 -200 -100 0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 -2000 -1500 -1000 -500 0 500 1000 1500 2000 2500 3000 3500 -1500 -1450 -1400 -1350 -1300 -1250 -1200 -1150 -1100 -1050 -1000 -950 -900 -850 -800 -750 -700 -650 -600 -550 -500 -450 -400 -350 -300 -250 -200 -150 -100 -50 0 50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 950 1000 1050 1100 1150 1200 1250 1300 1350 1400 1450 1500 1550 1600 1650 1700 1750 1800 1850 1900 1950 2000 2050 2100 2150 2200 2250 2300 2350 2400 2450 2500 2550 2600 2650 2700 2750 2800 2850 2900 2950 3000 -2000 0 2000 4000 -1500 -1400 -1300 -1200 -1100 -1000 -900 -800 -700 -600 -500 -400 -300 -200 -100 0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100 2200 2300 2400 2500 2600 2700 2800 2900 3000

Seems random at first glance. Now, see what happens with a higher modulus:


In [25]:
srand(1234)

big_modulus = 10000
rand_data = rand(1:32768, 1000000);
big_modulo_data = rand_data % big_modulus;

In [26]:
using Gadfly

plot(x=big_modulo_data, Geom.histogram(bincount=big_modulus))


Out[26]:
x -1.5×10⁴ -1.0×10⁴ -5.0×10³ 0 5.0×10³ 1.0×10⁴ 1.5×10⁴ 2.0×10⁴ 2.5×10⁴ -1.00×10⁴ -9.50×10³ -9.00×10³ -8.50×10³ -8.00×10³ -7.50×10³ -7.00×10³ -6.50×10³ -6.00×10³ -5.50×10³ -5.00×10³ -4.50×10³ -4.00×10³ -3.50×10³ -3.00×10³ -2.50×10³ -2.00×10³ -1.50×10³ -1.00×10³ -5.00×10² 0 5.00×10² 1.00×10³ 1.50×10³ 2.00×10³ 2.50×10³ 3.00×10³ 3.50×10³ 4.00×10³ 4.50×10³ 5.00×10³ 5.50×10³ 6.00×10³ 6.50×10³ 7.00×10³ 7.50×10³ 8.00×10³ 8.50×10³ 9.00×10³ 9.50×10³ 1.00×10⁴ 1.05×10⁴ 1.10×10⁴ 1.15×10⁴ 1.20×10⁴ 1.25×10⁴ 1.30×10⁴ 1.35×10⁴ 1.40×10⁴ 1.45×10⁴ 1.50×10⁴ 1.55×10⁴ 1.60×10⁴ 1.65×10⁴ 1.70×10⁴ 1.75×10⁴ 1.80×10⁴ 1.85×10⁴ 1.90×10⁴ 1.95×10⁴ 2.00×10⁴ -1×10⁴ 0 1×10⁴ 2×10⁴ -1.0×10⁴ -9.0×10³ -8.0×10³ -7.0×10³ -6.0×10³ -5.0×10³ -4.0×10³ -3.0×10³ -2.0×10³ -1.0×10³ 0 1.0×10³ 2.0×10³ 3.0×10³ 4.0×10³ 5.0×10³ 6.0×10³ 7.0×10³ 8.0×10³ 9.0×10³ 1.0×10⁴ 1.1×10⁴ 1.2×10⁴ 1.3×10⁴ 1.4×10⁴ 1.5×10⁴ 1.6×10⁴ 1.7×10⁴ 1.8×10⁴ 1.9×10⁴ 2.0×10⁴ -250 -200 -150 -100 -50 0 50 100 150 200 250 300 350 400 450 -200 -190 -180 -170 -160 -150 -140 -130 -120 -110 -100 -90 -80 -70 -60 -50 -40 -30 -20 -10 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 270 280 290 300 310 320 330 340 350 360 370 380 390 400 -200 0 200 400 -200 -180 -160 -140 -120 -100 -80 -60 -40 -20 0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400

This result is decidedly non-random. The reason should be rather obvious at this point. Mapping a random uniform distrubution to a smaller space using a modulus that is not a factor of the original space will cause some 'bins' to be more likely to get mapped than others. The result - a non-uniform random distrubution.

This isn't a complicated issue to deal with. RNG libraries of any respecable programming account for this already. Just use library functions.

In the event that such functions don't exist, always remember that quick, hacky solutions can produce unintended effects that break the functionality you were trying to achieve.

In other words, profile you bash scripts.


In [29]:
scaled_big_modulo_data = ( rand_data / 32768  * big_modulus );
plot(x=scaled_big_modulo_data, Geom.histogram(bincount=big_modulus))


Out[29]:
x -2.0×10⁴ -1.5×10⁴ -1.0×10⁴ -5.0×10³ 0 5.0×10³ 1.0×10⁴ 1.5×10⁴ 2.0×10⁴ 2.5×10⁴ 3.0×10⁴ 3.5×10⁴ -1.50×10⁴ -1.45×10⁴ -1.40×10⁴ -1.35×10⁴ -1.30×10⁴ -1.25×10⁴ -1.20×10⁴ -1.15×10⁴ -1.10×10⁴ -1.05×10⁴ -1.00×10⁴ -9.50×10³ -9.00×10³ -8.50×10³ -8.00×10³ -7.50×10³ -7.00×10³ -6.50×10³ -6.00×10³ -5.50×10³ -5.00×10³ -4.50×10³ -4.00×10³ -3.50×10³ -3.00×10³ -2.50×10³ -2.00×10³ -1.50×10³ -1.00×10³ -5.00×10² 0 5.00×10² 1.00×10³ 1.50×10³ 2.00×10³ 2.50×10³ 3.00×10³ 3.50×10³ 4.00×10³ 4.50×10³ 5.00×10³ 5.50×10³ 6.00×10³ 6.50×10³ 7.00×10³ 7.50×10³ 8.00×10³ 8.50×10³ 9.00×10³ 9.50×10³ 1.00×10⁴ 1.05×10⁴ 1.10×10⁴ 1.15×10⁴ 1.20×10⁴ 1.25×10⁴ 1.30×10⁴ 1.35×10⁴ 1.40×10⁴ 1.45×10⁴ 1.50×10⁴ 1.55×10⁴ 1.60×10⁴ 1.65×10⁴ 1.70×10⁴ 1.75×10⁴ 1.80×10⁴ 1.85×10⁴ 1.90×10⁴ 1.95×10⁴ 2.00×10⁴ 2.05×10⁴ 2.10×10⁴ 2.15×10⁴ 2.20×10⁴ 2.25×10⁴ 2.30×10⁴ 2.35×10⁴ 2.40×10⁴ 2.45×10⁴ 2.50×10⁴ 2.55×10⁴ 2.60×10⁴ 2.65×10⁴ 2.70×10⁴ 2.75×10⁴ 2.80×10⁴ 2.85×10⁴ 2.90×10⁴ 2.95×10⁴ 3.00×10⁴ -2×10⁴ 0 2×10⁴ 4×10⁴ -1.5×10⁴ -1.4×10⁴ -1.3×10⁴ -1.2×10⁴ -1.1×10⁴ -1.0×10⁴ -9.0×10³ -8.0×10³ -7.0×10³ -6.0×10³ -5.0×10³ -4.0×10³ -3.0×10³ -2.0×10³ -1.0×10³ 0 1.0×10³ 2.0×10³ 3.0×10³ 4.0×10³ 5.0×10³ 6.0×10³ 7.0×10³ 8.0×10³ 9.0×10³ 1.0×10⁴ 1.1×10⁴ 1.2×10⁴ 1.3×10⁴ 1.4×10⁴ 1.5×10⁴ 1.6×10⁴ 1.7×10⁴ 1.8×10⁴ 1.9×10⁴ 2.0×10⁴ 2.1×10⁴ 2.2×10⁴ 2.3×10⁴ 2.4×10⁴ 2.5×10⁴ 2.6×10⁴ 2.7×10⁴ 2.8×10⁴ 2.9×10⁴ 3.0×10⁴ -250 -200 -150 -100 -50 0 50 100 150 200 250 300 350 400 450 -200 -190 -180 -170 -160 -150 -140 -130 -120 -110 -100 -90 -80 -70 -60 -50 -40 -30 -20 -10 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 270 280 290 300 310 320 330 340 350 360 370 380 390 400 -200 0 200 400 -200 -180 -160 -140 -120 -100 -80 -60 -40 -20 0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400