CrypTen

A framework for Privacy Preserving Machine Learning

1,518

274

1,518

View on GitHub

Top Related Projects

PySyft

9,429

Perform data science on data that remains in someone else's server

tf-encrypted

1,202

A Framework for Encrypted Machine Learning in TensorFlow

SEAL

3,519

Microsoft SEAL is an easy-to-use and powerful homomorphic encryption library.

Quick Overview

CrypTen is a framework for secure and private machine learning, developed by Facebook Research. It provides a PyTorch-like interface for training and evaluating machine learning models on encrypted data, enabling privacy-preserving computations.

Pros

Privacy-Preserving: CrypTen allows for training and inference on encrypted data, ensuring the privacy of sensitive information.
PyTorch Integration: CrypTen is built on top of PyTorch, allowing developers to leverage their existing PyTorch knowledge and code.
Flexible Cryptographic Backends: CrypTen supports multiple cryptographic backends, including Pysyft, SEAL, and Crypten's own backend, providing flexibility in deployment.
Active Development: The project is actively maintained and developed by Facebook Research, ensuring ongoing improvements and bug fixes.

Cons

Complexity: Implementing secure and private machine learning can be complex, and CrypTen may have a steeper learning curve for developers new to the field.
Performance Overhead: Performing computations on encrypted data can introduce performance overhead, which may be a concern for some use cases.
Limited Documentation: While the project has good documentation, some areas may be less comprehensive, especially for more advanced use cases.
Dependency on External Libraries: CrypTen relies on external cryptographic libraries, which may introduce additional dependencies and potential compatibility issues.

Code Examples

Here are a few code examples demonstrating the usage of CrypTen:

Encrypted Tensor Operations:

import crypten

# Create encrypted tensors
x = crypten.encrypt(torch.tensor([1, 2, 3]))
y = crypten.encrypt(torch.tensor([4, 5, 6]))

# Perform encrypted operations
z = x + y
print(crypten.decrypted_tensor(z))  # Output: [5, 7, 9]

Encrypted Linear Regression:

import crypten
import torch.nn as nn

# Create encrypted data and labels
X = crypten.encrypt(torch.randn(100, 10))
y = crypten.encrypt(torch.randn(100))

# Define and train the model
model = nn.Linear(10, 1)
model = crypten.nn.from_pytorch(model)
optimizer = crypten.optim.SGD(model.parameters(), lr=0.01)

for epoch in range(100):
    optimizer.zero_grad()
    output = model(X)
    loss = (output - y).pow(2).mean()
    loss.backward()
    optimizer.step()

print(crypten.decrypted_tensor(model.weight), crypten.decrypted_tensor(model.bias))

Encrypted Logistic Regression:

import crypten
import torch.nn as nn

# Create encrypted data and labels
X = crypten.encrypt(torch.randn(100, 10))
y = crypten.encrypt((torch.rand(100) > 0.5).long())

# Define and train the model
model = nn.Linear(10, 1)
model = crypten.nn.from_pytorch(model)
optimizer = crypten.optim.SGD(model.parameters(), lr=0.01)

for epoch in range(100):
    optimizer.zero_grad()
    output = model(X).sigmoid()
    loss = nn.BCELoss()(output, y)
    loss.backward()
    optimizer.step()

print(crypten.decrypted_tensor(model.weight), crypten.decrypted_tensor(model.bias))

Getting Started

To get started with CrypTen, follow these steps:

Install CrypTen and its dependencies:

pip install crypten

Import the necessary modules and create encrypted tensors:

import crypten

# Create encrypted tensors
x = crypten.encrypt(torch.tensor([1, 2, 3]))
y = crypten.encrypt(torch.tensor([4, 5, 6]))

Perform encrypted operations on the tensors:

# Perform encrypted operations
z = x + y
print(crypten.decrypted_

Competitor Comparisons

PySyft

9,429

Perform data science on data that remains in someone else's server

Pros of PySyft

PySyft provides a comprehensive set of tools for secure and private machine learning, including support for federated learning, differential privacy, and homomorphic encryption.
PySyft has a large and active community, with regular updates and a wide range of tutorials and documentation.
PySyft is highly extensible, with support for a variety of deep learning frameworks and the ability to integrate with other privacy-preserving technologies.

Cons of PySyft

PySyft can be more complex to set up and configure than CrypTen, especially for users who are new to privacy-preserving machine learning.
PySyft may have a steeper learning curve than CrypTen, as it covers a broader range of privacy-preserving techniques.

Code Comparison

CrypTen (5 lines):

import crypten
import torch

# Initialize CrypTen
crypten.init()

# Encrypt a tensor
encrypted_tensor = crypten.encrypt(torch.tensor([1, 2, 3]))

PySyft (5 lines):

import torch
import syft as sy

# Create a private tensor
private_tensor = sy.tensor([1, 2, 3]).private()

# Perform a secure computation
result = private_tensor.add(private_tensor)

tf-encrypted

1,202

A Framework for Encrypted Machine Learning in TensorFlow

Pros of tf-encrypted/tf-encrypted

Supports Multiple Frameworks: tf-encrypted supports both TensorFlow and PyTorch, allowing for greater flexibility in the choice of machine learning framework.
Active Community: The tf-encrypted project has a more active community with regular updates and a larger number of contributors.
Extensive Documentation: The tf-encrypted project provides more comprehensive documentation, including detailed tutorials and examples.

Cons of tf-encrypted/tf-encrypted

Limited Functionality: While tf-encrypted supports a wider range of frameworks, it may have a more limited set of features and functionality compared to CrypTen.
Steeper Learning Curve: The tf-encrypted project may have a steeper learning curve, especially for users who are more familiar with the CrypTen ecosystem.
Potential Performance Overhead: The support for multiple frameworks in tf-encrypted may come with a slight performance overhead compared to the more specialized CrypTen.

Code Comparison

CrypTen:

import crypten
import torch

# Initialize a tensor
x = torch.tensor([1, 2, 3])
x_encrypted = crypten.encrypt(x)

# Perform secure computation
y_encrypted = x_encrypted * 2
y = crypten.get_plain_text(y_encrypted)

print(y)  # Output: tensor([2, 4, 6])

tf-encrypted:

import tf_encrypted as tfe

# Initialize a tensor
x = tfe.define_private_variable([1, 2, 3])

# Perform secure computation
y = x * 2

# Reveal the result
with tfe.Session() as sess:
    sess.run(tfe.global_initialization_op())
    result = sess.run(y.reveal())

print(result)  # Output: [2, 4, 6]

SEAL

3,519

Microsoft SEAL is an easy-to-use and powerful homomorphic encryption library.

Pros of SEAL

SEAL is a mature and well-documented library, with a strong focus on performance and efficiency.
SEAL provides a wide range of homomorphic encryption schemes, including BFV, CKKS, and BGV, allowing for greater flexibility in application development.
SEAL has a strong community and is actively maintained by Microsoft Research.

Cons of SEAL

SEAL has a steeper learning curve compared to CrypTen, as it requires a deeper understanding of homomorphic encryption concepts.
SEAL is primarily written in C++, which may be less accessible to developers who are more comfortable with Python.

Code Comparison

CrypTen (Python):

import crypten
import torch

# Initialize a tensor
x = torch.tensor([1, 2, 3])
encrypted_x = crypten.encrypt(x)

# Perform homomorphic addition
encrypted_y = encrypted_x + encrypted_x
y = crypten.decrypt(encrypted_y)
print(y)  # Output: tensor([2, 4, 6])

SEAL (C++):

#include <seal/seal.h>

int main() {
    seal::EncryptionParameters params(seal::scheme_type::BFV);
    // Set up parameters and keys
    seal::SecretKey secret_key;
    seal::PublicKey public_key;
    // Encrypt and perform homomorphic addition
    seal::Ciphertext encrypted_x, encrypted_y;
    // ...
    return 0;
}

fully-homomorphic-encryption

3,502

An FHE compiler for C++

Pros of google/fully-homomorphic-encryption

Comprehensive documentation and examples for understanding and implementing fully homomorphic encryption (FHE)
Supports multiple FHE schemes, including CKKS, BFV, and BGV
Actively maintained with regular updates and bug fixes

Cons of google/fully-homomorphic-encryption

Steep learning curve for those new to FHE
Limited support for practical applications beyond academic use
Performance may be slower compared to other cryptographic libraries

Code Comparison

Google/fully-homomorphic-encryption:

import tensorflow as tf
import tfhe

# Generate a random secret key
secret_key = tfhe.generate_secret_key()

# Encrypt a plaintext
plaintext = tf.constant([1, 2, 3])
ciphertext = tfhe.encrypt(secret_key, plaintext)

# Perform homomorphic addition
result = tfhe.add(secret_key, ciphertext, ciphertext)

CrypTen:

import crypten

# Initialize a secure computation environment
crypten.init()

# Create encrypted tensors
x = crypten.encrypt(torch.tensor([1, 2, 3]))
y = crypten.encrypt(torch.tensor([4, 5, 6]))

# Perform homomorphic addition
result = x + y

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

CrypTen logo

CrypTen is a framework for Privacy Preserving Machine Learning built on PyTorch. Its goal is to make secure computing techniques accessible to Machine Learning practitioners. It currently implements Secure Multiparty Computation as its secure computing backend and offers three main benefits to ML researchers:

It is machine learning first. The framework presents the protocols via a CrypTensor object that looks and feels exactly like a PyTorch Tensor. This allows the user to use automatic differentiation and neural network modules akin to those in PyTorch.
CrypTen is library-based. It implements a tensor library just as PyTorch does. This makes it easier for practitioners to debug, experiment on, and explore ML models.
The framework is built with real-world challenges in mind. CrypTen does not scale back or oversimplify the implementation of the secure protocols.

Here is a bit of CrypTen code that encrypts and decrypts tensors and adds them

import torch
import crypten

crypten.init()

x = torch.tensor([1.0, 2.0, 3.0])
x_enc = crypten.cryptensor(x) # encrypt

x_dec = x_enc.get_plain_text() # decrypt

y_enc = crypten.cryptensor([2.0, 3.0, 4.0])
sum_xy = x_enc + y_enc # add encrypted tensors
sum_xy_dec = sum_xy.get_plain_text() # decrypt sum

It is currently not production ready and its main use is as a research framework.

Installing CrypTen

CrypTen currently runs on Linux and Mac with Python 3.7. We also support computation on GPUs. Windows is not supported.

For Linux or Mac

pip install crypten

If you want to run the examples in the examples directory, you should also do the following

pip install -r requirements.examples.txt

Examples

To run the examples in the examples directory, you additionally need to clone the repo and

pip install -r requirements.examples.txt

We provide examples covering a range of models in the examples directory

The linear SVM example, mpc_linear_svm, generates random data and trains a SVM classifier on encrypted data.
The LeNet example, mpc_cifar, trains an adaptation of LeNet on CIFAR in cleartext and encrypts the model and data for inference.
The TFE benchmark example, tfe_benchmarks, trains three different network architectures on MNIST in cleartext, and encrypts the trained model and data for inference.
The bandits example, bandits, trains a contextual bandits model on encrypted data (MNIST).
The imagenet example, mpc_imagenet, performs inference on pretrained models from torchvision.

For examples that train in cleartext, we also provide pre-trained models in cleartext in the model subdirectory of each example subdirectory.

You can check all example specific command line options by doing the following; shown here for tfe_benchmarks:

python examples/tfe_benchmarks/launcher.py --help

How CrypTen works

We have a set of tutorials in the tutorials directory to show how CrypTen works. These are presented as Jupyter notebooks so please install the following in your conda environment

conda install ipython jupyter
pip install -r requirements.examples.txt

Introduction.ipynb - an introduction to Secure Multiparty Compute; CrypTen's underlying secure computing protocol; use cases we are trying to solve and the threat model we assume.
Tutorial_1_Basics_of_CrypTen_Tensors.ipynb - introduces CrypTensor, CrypTen's encrypted tensor object, and shows how to use it to do various operations on this object.
Tutorial_2_Inside_CrypTensors.ipynb â delves deeper into CrypTensor to show the inner workings; specifically how CrypTensor uses MPCTensor for its backend and the two different kind of sharings, arithmetic and binary, are used for two different kind of functions. It also shows CrypTen's MPI-inspired programming model.
Tutorial_3_Introduction_to_Access_Control.ipynb - shows how to train a linear model using CrypTen and shows various scenarios of data labeling, feature aggregation, dataset augmentation and model hiding where this is applicable.
Tutorial_4_Classification_with_Encrypted_Neural_Networks.ipynb â shows how CrypTen can load a pre-trained PyTorch model, encrypt it and then do inference on encrypted data.
Tutorial_5_Under_the_hood_of_Encrypted_Networks.ipynb - examines how CrypTen loads PyTorch models, how they are encrypted and how data moves through a multilayer network.
Tutorial_6_CrypTen_on_AWS_instances.ipynb - shows how to use scrips/aws_launcher.py to launch our examples on AWS. It can also work with your code written in CrypTen.
Tutorial_7_Training_an_Encrypted_Neural_Network.ipynb - introduces the automatic differentiation functionality of CrypTensor. This functionality makes it easy to train neural networks in CrypTen.

Documentation and citing

CrypTen is documented here.

The protocols and design protocols implemented in CrypTen are described in this paper. If you want to cite CrypTen in your papers (much appreciated!), you can cite it as follows:

@inproceedings{crypten2020,
  author={B. Knott and S. Venkataraman and A.Y. Hannun and S. Sengupta and M. Ibrahim and L.J.P. van der Maaten},
  title={CrypTen: Secure Multi-Party Computation Meets Machine Learning},
  booktitle={arXiv 2109.00984},
  year={2021},
}

Join the CrypTen community

Please contact us to join the CrypTen community on Slack

See the CONTRIBUTING file for how to help out.

License

CrypTen is MIT licensed, as found in the LICENSE file.

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot