Going beyond the limits of native integer types

The methods and techniques used to provide privacy rely (in some cases) on working with very big integers. However, the platforms we are working with (PyTorch, TensorFlow, DL4J) all have limitations with the maximum size they can handle.

To overcome this limitation, we are providing a new type, LargePrecisionTensor that uses an internal structure to represent these big numbers.


In [1]:
import torch
from syft.frameworks.torch.tensors.interpreters.large_precision import LargePrecisionTensor

In [2]:
import syft
from syft import TorchHook
hook = TorchHook(torch)

In [3]:
tensor = torch.tensor([3.0])

In [4]:
x = LargePrecisionTensor(tensor)

In [5]:
x.on(tensor)


Out[5]:
(Wrapper)>LargePrecisionTensor>tensor([3.])

We can fix the precision of this tensor with regular values. This in PyTorch can be done with precisions below 64 bits.


In [6]:
tensor.fix_prec()


Out[6]:
(Wrapper)>FixedPrecisionTensor>tensor([3000])

Let's make this tensor larger. precision_fractional is the desired precision. It'll be a very big number, so big that PyTorch cannot hold it in any of its types.

We also tell the function fix_prec() which internal type we'll use to represent these large numbers from thoses available in the platform.


In [7]:
large_tensor = tensor.fix_prec(internal_type=torch.int16, precision_fractional=256, verbose=True)
large_tensor


Adding number 30000000000000000903829797004216275086714596193240853864963239397098224001329386984633370688074372907047315290515183484539008024100591320340654985059867159958835993685232641400781191169686886680672647790808531134864424057208559359749288237392529192038432768 for item 3.0

Out[7]:
(Wrapper)>LargePrecisionTensor>tensor([[ 15, 251, 230,  31, 150, 207, 109,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0]], dtype=torch.int16)

And here we have our tensor[3.] represented as an integer number of 256 bits. Internally we have built a new Torch tensor that can restore the original data by using the precision and virtual precision. What is more, we can use these internal representations to make operations!

Restoring the original value

We can restore the original number by applying float_precision() to our very large tensor


In [8]:
original = large_tensor.float_precision()
original


Out[8]:
tensor([3.])

Operating with large precision tensors


In [20]:
precision = torch.int16
virtual_prec = 256

x1 = torch.tensor([100000.])
x2 = torch.tensor([20.])
lpt1 = x1.fix_prec(internal_type=precision, precision_fractional=virtual_prec, verbose=True)
lpt2 = x2.fix_prec(internal_type=precision, precision_fractional=virtual_prec, verbose=True)


Adding number 999999999999999928773840520366757536876739320811576612231781480701470095354527494007746341441138276442474389769547563525432293116501122567178714359381222777104854460745804679379644497043208267383631647167377861948545889974808961869943571076775428108923489484800 for item 100000.0


Adding number 200000000000000006025531980028108500578097307954939025766421595980654826675529246564222471253829152713648768603434556563593386827337275468937699900399114399725573291234884276005207941131245911204484318605390207565762827048057062398328588249283527946922885120 for item 20.0

Let's add them. We should have a result tensor containing 100020.


In [21]:
result_lpt = lpt1 + lpt2
result_lpt


Out[21]:
(Wrapper)>LargePrecisionTensor>tensor([[  8,  33, 169, 229, 249, 252, 173, 188,  44,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0]],
       dtype=torch.int16)

In [11]:
result_lpt.float_precision()


Out[11]:
tensor([100020.])

We have added numbers with a precision of 10^256!

What about multiplication?


In [18]:
precision = torch.int16
virtual_prec = 256

x1 = torch.tensor([10.])
x2 = torch.tensor([20.5])
lpt1 = x1.fix_prec(internal_type=precision, precision_fractional=virtual_prec)
lpt2 = x2.fix_prec(internal_type=precision, precision_fractional=virtual_prec)

result_lpt = lpt1 * lpt2
result_lpt


Out[18]:
(Wrapper)>LargePrecisionTensor>tensor([[  4,  68,  61,  23, 195, 236,   8, 192,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
           0,   0,   0,   0,   0,   0,   0,   0,   0,   0]], dtype=torch.int16)

In [19]:
result_lpt.float_precision()


Out[19]:
tensor([205.])

We have multiplied numbers represented as 10^256!

Congratulations!!! - Time to Join the Community!

Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the movement toward privacy preserving, decentralized ownership of AI and the AI supply chain (data), you can do so in the following ways!

Star PySyft on GitHub

The easiest way to help our community is just by starring the Repos! This helps raise awareness of the cool tools we're building.

Join our Slack!

The best way to keep up to date on the latest advancements is to join our community! You can do so by filling out the form at http://slack.openmined.org

Join a Code Project!

The best way to contribute to our community is to become a code contributor! At any time you can go to PySyft GitHub Issues page and filter for "Projects". This will show you all the top level Tickets giving an overview of what projects you can join! If you don't want to join a project, but you would like to do a bit of coding, you can also look for more "one off" mini-projects by searching for GitHub issues marked "good first issue".

If you don't have time to contribute to our codebase, but would still like to lend support, you can also become a Backer on our Open Collective. All donations go toward our web hosting and other community expenses such as hackathons and meetups!

OpenMined's Open Collective Page