In the last section we trained a toy model using Federated Learning. We did this by calling .send() and .get() on our model, sending it to the location of training data, updating it, and then bringing it back. However, at the end of the example we realized that we needed to go a bit further to protect people privacy. Namely, we want to average the gradients before calling .get(). That way, we won't ever see anyone's exact gradient (thus better protecting their privacy!!!)
But, in order to do this, we need a few more pieces:
And in addition, while we're here, we're going to learn about a few more advanced tensor operations as well which will help us both with this example and a few in the future!
Authors:
In [ ]:
import torch
import syft as sy
hook = sy.TorchHook(torch)
In [ ]:
bob = sy.VirtualWorker(hook, id='bob')
alice = sy.VirtualWorker(hook, id='alice')
In [ ]:
# this is a local tensor
x = torch.tensor([1,2,3,4])
x
In [ ]:
# this sends the local tensor to Bob
x_ptr = x.send(bob)
# this is now a pointer
x_ptr
In [ ]:
# now we can SEND THE POINTER to alice!!!
pointer_to_x_ptr = x_ptr.send(alice)
pointer_to_x_ptr
In [ ]:
# As you can see above, Bob still has the actual data (data is always stored in a LocalTensor type).
bob._objects
In [ ]:
# Alice, on the other hand, has x_ptr!! (notice how it points at bob)
alice._objects
In [ ]:
# and we can use .get() to get x_ptr back from Alice
x_ptr = pointer_to_x_ptr.get()
x_ptr
In [ ]:
# and then we can use x_ptr to get x back from Bob!
x = x_ptr.get()
x
In [ ]:
bob._objects
In [ ]:
alice._objects
In [ ]:
p2p2x = torch.tensor([1,2,3,4,5]).send(bob).send(alice)
y = p2p2x + p2p2x
In [ ]:
bob._objects
In [ ]:
alice._objects
In [ ]:
y.get().get()
In [ ]:
bob._objects
In [ ]:
alice._objects
In [ ]:
p2p2x.get().get()
In [ ]:
bob._objects
In [ ]:
alice._objects
So in the last section whenever we called a .send() or a .get() operation, it called that operation directly on the tensor on our local machine. However, if you have a chain of pointers, sometimes you want to call operations like .get() or .send() on the last pointer in the chain (such as sending data directly from one worker to another). To accomplish this, you want to use functions which are especially designed for this privacy preserving operation.
These operations are:
my_pointer2pointer.move(another_worker)
In [ ]:
# x is now a pointer to the data which lives on Bob's machine
x = torch.tensor([1,2,3,4,5]).send(bob)
In [ ]:
print(' bob:', bob._objects)
print('alice:',alice._objects)
In [ ]:
x = x.move(alice)
In [ ]:
print(' bob:', bob._objects)
print('alice:',alice._objects)
In [ ]:
x
Excellent! Now we're equiped with the tools to perform remote gradient averaging using a trusted aggregator!
Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the movement toward privacy preserving, decentralized ownership of AI and the AI supply chain (data), you can do so in the following ways!
The easiest way to help our community is just by starring the Repos! This helps raise awareness of the cool tools we're building.
The best way to keep up to date on the latest advancements is to join our community! You can do so by filling out the form at http://slack.openmined.org
The best way to contribute to our community is to become a code contributor! At any time you can go to PySyft GitHub Issues page and filter for "Projects". This will show you all the top level Tickets giving an overview of what projects you can join! If you don't want to join a project, but you would like to do a bit of coding, you can also look for more "one off" mini-projects by searching for GitHub issues marked "good first issue".
If you don't have time to contribute to our codebase, but would still like to lend support, you can also become a Backer on our Open Collective. All donations go toward our web hosting and other community expenses such as hackathons and meetups!