Part 3: Advanced Remote Execution Tools

In the last section we train a toy model using Federated Learning. We do am by calling .send() and .get() on our model, we send am to the location of trainig data, updating it, and we come bring am back. At the end of the example We discover sey we fit go further on how to protect people privacy. The way we go do am, we go average the gradients before calling .get(). If we do am like this, we no go see people exact gradient (this one dey good for protecting privacy!!!)

So that we go fit do am, we go need some pieces:

  • use a pointer to send a Tensor directly to another worker

Another tin wey you go learn na advanced tensor operations and which one go helep us with the example and some in the future!

Person wey write am:

Person wey translate am:


In [ ]:
import torch
import syft as sy
hook = sy.TorchHook(torch)

Section 3.1 - Pointers to Pointers

As you know, PointerTensor objects just dey like normal tensors. In fact, they are so much like tensors wey we fit have pointers to the pointers. Check am out!


In [ ]:
bob = sy.VirtualWorker(hook, id='bob')
alice = sy.VirtualWorker(hook, id='alice')

In [ ]:
# this one na local tensor
x = torch.tensor([1,2,3,4])
x

In [ ]:
# this one go send local tensor to Bob
x_ptr = x.send(bob)

# this one na the pointer
x_ptr

In [ ]:
# we go send the pointer to alice!!!
pointer_to_x_ptr = x_ptr.send(alice)

pointer_to_x_ptr

Wetin happen?

For the previous example, we create tensor called x and we send am to Bob, we come create pointer on top our local machine (x_ptr).

We go call x_ptr.send(alice) wey go sent the pointer to Alice.

Look am wella, Data no change level! Instead, pointer move to data!!


In [ ]:
# As you dey see the above, Bob still get the actual data (data dey always dey stored in a LocalTensor type). 
bob._objects

In [ ]:
# Alice, on the other hand, get x_ptr!! (notice sey he dey point at Bob)
alice._objects

In [ ]:
# we fit use .get() to get x_ptr back from Alice

x_ptr = pointer_to_x_ptr.get()
x_ptr

In [ ]:
# we fit use x_ptr to get x back from Bob!

x = x_ptr.get()
x

Arithmetic on Pointer -> Pointer -> Data Object

Just like with normal pointers, we fit perform arbitrary Pytorch operations across these tensors


In [ ]:
bob._objects

In [ ]:
alice._objects

In [ ]:
p2p2x = torch.tensor([1,2,3,4,5]).send(bob).send(alice)

y = p2p2x + p2p2x

In [ ]:
bob._objects

In [ ]:
alice._objects

In [ ]:
y.get().get()

In [ ]:
bob._objects

In [ ]:
alice._objects

In [ ]:
p2p2x.get().get()

In [ ]:
bob._objects

In [ ]:
alice._objects

Section 3.2 - Pointer Chain Operations

In the last section whenever we call a .send() or a .get() operation, we call that operation directly on the tensor on our local machine. If you get a chain of pointers, sometimes we wan dey able to call operations like .get() or .send() on the last pointer in the chain (such as sending data directly from one worker to another). To accomplish this, we go use function wey dey specially design for this privacy preserving operation.

These operations are:

  • my_pointer2pointer.move(another_worker)

In [ ]:
# xis now a pointer to the data wey dey on top Bob's machine.
x = torch.tensor([1,2,3,4,5]).send(bob)

In [ ]:
print('  bob:', bob._objects)
print('alice:',alice._objects)

In [ ]:
x = x.move(alice)

In [ ]:
print('  bob:', bob._objects)
print('alice:',alice._objects)

In [ ]:
x

Excellent! Now we are equiped with the tools wey we fit use to perfom remote gradient averaging using a trusted aggregator!

Congratulations!!! - Oya Join the Community!

Clap for una sef as you don finish this notebook tutorial! If you enjoy am and you wan join the movement towards privacy preserving, decentralized ownership of AI and the AI supply chain (data), follow the steps wey dey below.

Star PySyft on GitHub

The easiset way to helep our community na to star the GitHub repos! This go helep raise awareness of the tools we dey build.

Join our Slack!

To follow up bumper to bumper on how latest advancements, join our community! You can do so by filling out the form at http://slack.openmined.org

Join a Code Project!

The best way to contribute to our community na to become code contributor! You fit go to PySyft GitHub Issues page and filter for "Projects". E go show you all the top level Tickets giving an overview of what projects you fit join! If you no wan join any project, but you wan code small, you fit look for more "one off" mini-projects by searching for GitHub issues marked "good first issue"

If you no get time to contribute to our codebase, but still like to lend support, you fit be a Backer on our Open Collective. All donations wey we get na for our web hosting and other community expenses such as hackathons and meetups! meetups!

OpenMined's Open Collective Page


In [ ]: