cleverhans
An adversarial example library for constructing attacks, building defenses, and benchmarking both
Top Related Projects
An adversarial example library for constructing attacks, building defenses, and benchmarking both
A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
A Toolbox for Adversarial Robustness Research
Quick Overview
CleverHans is an open-source Python library designed to benchmark machine learning systems' vulnerability to adversarial examples. It provides implementations of various adversarial attack methods and defenses, allowing researchers and practitioners to evaluate the robustness of their models against different types of attacks.
Pros
- Comprehensive collection of state-of-the-art adversarial attacks and defenses
- Easy integration with popular deep learning frameworks like TensorFlow and PyTorch
- Well-documented and actively maintained by the research community
- Supports both targeted and untargeted attacks
Cons
- Steep learning curve for users unfamiliar with adversarial machine learning concepts
- Some implementations may not be optimized for large-scale or production use
- Limited support for certain specialized domains or architectures
- Requires careful interpretation of results, as performance can vary across different datasets and models
Code Examples
- Creating a basic FGSM attack:
import tensorflow as tf
from cleverhans.tf2.attacks.fast_gradient_method import fast_gradient_method
def attack_fgsm(model, x, eps=0.3):
return fast_gradient_method(model, x, eps, np.inf, targeted=False)
- Implementing adversarial training:
from cleverhans.tf2.attacks.projected_gradient_descent import projected_gradient_descent
def adversarial_train(model, x, y, eps=0.3, epochs=10):
for epoch in range(epochs):
x_adv = projected_gradient_descent(model, x, eps, 0.01, 40, np.inf)
model.fit(x_adv, y, epochs=1)
- Evaluating model robustness:
from cleverhans.tf2.utils import compute_accuracy
def evaluate_robustness(model, x_test, y_test, attack_fn):
x_adv = attack_fn(model, x_test)
clean_acc = compute_accuracy(model, x_test, y_test)
adv_acc = compute_accuracy(model, x_adv, y_test)
return clean_acc, adv_acc
Getting Started
To get started with CleverHans, follow these steps:
- Install the library:
pip install cleverhans
- Import the necessary modules:
import tensorflow as tf
from cleverhans.tf2.attacks import fast_gradient_method
from cleverhans.tf2.utils import compute_accuracy
- Load your model and data, then create and evaluate an attack:
model = tf.keras.models.load_model('your_model.h5')
x_test, y_test = load_your_data()
x_adv = fast_gradient_method(model, x_test, eps=0.3, clip_min=0, clip_max=1)
adv_acc = compute_accuracy(model, x_adv, y_test)
print(f"Adversarial accuracy: {adv_acc}")
Competitor Comparisons
An adversarial example library for constructing attacks, building defenses, and benchmarking both
Pros of cleverhans
- More established and widely used library for adversarial machine learning
- Extensive documentation and examples available
- Supports multiple deep learning frameworks (TensorFlow, PyTorch, JAX)
Cons of cleverhans
- Larger codebase, potentially more complex to navigate
- May include deprecated or outdated methods
- Slower development cycle and less frequent updates
Code Comparison
cleverhans:
from cleverhans.future.torch.attacks import fast_gradient_method
perturbations = fast_gradient_method(model_fn, x, eps=0.3, norm=np.inf)
adv_x = x + perturbations
cleverhans>:
from cleverhans.torch.attacks import fast_gradient_method
perturbations = fast_gradient_method(model, x, eps=0.3, norm=np.inf)
adv_x = x + perturbations
The code structure is similar, with the main difference being the import path. cleverhans> uses a simplified import structure, potentially indicating a more streamlined codebase. However, both repositories provide similar functionality for generating adversarial examples using the Fast Gradient Method.
A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX
Pros of Foolbox
- More user-friendly API with a focus on simplicity and ease of use
- Supports a wider range of deep learning frameworks (PyTorch, TensorFlow, JAX, etc.)
- Actively maintained with frequent updates and new features
Cons of Foolbox
- Less comprehensive documentation compared to CleverHans
- Fewer pre-implemented attack algorithms
Code Comparison
Foolbox example:
import foolbox as fb
model = fb.PyTorchModel(net, bounds=(0, 1))
attack = fb.attacks.FGSM()
epsilons = [0.0, 0.001, 0.01, 0.03, 0.1, 0.3, 0.5, 1.0]
_, advs, success = attack(model, images, labels, epsilons=epsilons)
CleverHans example:
from cleverhans.future.tf2.attacks import fast_gradient_method
logits = model(x)
loss_object = tf.keras.losses.CategoricalCrossentropy(from_logits=True)
adv_x = fast_gradient_method(model, x, eps, np.inf, targeted=False,
loss_fn=loss_object, y=y)
Both libraries offer efficient implementations of adversarial attacks, but Foolbox's API is generally more intuitive and flexible. CleverHans provides more detailed documentation and a larger collection of attack algorithms, making it potentially better suited for research purposes. Foolbox's multi-framework support gives it an edge in terms of versatility, while CleverHans focuses primarily on TensorFlow.
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
Pros of adversarial-robustness-toolbox
- Broader scope, covering various ML frameworks (TensorFlow, Keras, PyTorch, MXNet, scikit-learn)
- More comprehensive set of attacks, defenses, and robustness metrics
- Active development and regular updates
Cons of adversarial-robustness-toolbox
- Steeper learning curve due to its extensive features
- Potentially slower execution for simpler tasks compared to cleverhans
Code Comparison
cleverhans:
from cleverhans.future.tf2.attacks import fast_gradient_method
adv_x = fast_gradient_method(model, x, eps=0.3, norm=np.inf)
adversarial-robustness-toolbox:
from art.attacks.evasion import FastGradientMethod
attack = FastGradientMethod(estimator=classifier, eps=0.3)
adv_x = attack.generate(x=x)
Both libraries offer similar functionality for common attacks, but adversarial-robustness-toolbox provides a more object-oriented approach with additional customization options.
A Toolbox for Adversarial Robustness Research
Pros of advertorch
- More focused on PyTorch, offering better integration for PyTorch users
- Includes a wider range of adversarial attacks and defenses
- Provides more comprehensive documentation and examples
Cons of advertorch
- Less established community compared to cleverhans
- Fewer contributions and updates in recent years
- Limited support for frameworks other than PyTorch
Code Comparison
advertorch:
from advertorch.attacks import PGDAttack
adversary = PGDAttack(model, loss_fn=nn.CrossEntropyLoss(), eps=0.3,
nb_iter=40, eps_iter=0.01, rand_init=True)
adv_examples = adversary.perturb(images, labels)
cleverhans:
from cleverhans.future.torch.attacks import projected_gradient_descent
adv_examples = projected_gradient_descent(model_fn, images, eps=0.3,
eps_iter=0.01, nb_iter=40,
norm=np.inf, clip_min=0.0,
clip_max=1.0, y=labels)
Both libraries offer similar functionality for implementing adversarial attacks, but advertorch's API is more PyTorch-centric, while cleverhans provides a more framework-agnostic approach. advertorch's documentation and examples are generally more comprehensive, making it easier for newcomers to get started with adversarial machine learning in PyTorch.
Convert 
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
CleverHans (latest release: v4.0.0)
This repository contains the source code for CleverHans, a Python library to benchmark machine learning systems' vulnerability to adversarial examples. You can learn more about such vulnerabilities on the accompanying blog.
The CleverHans library is under continual development, always welcoming contributions of the latest attacks and defenses. In particular, we always welcome help towards resolving the issues currently open.
Since v4.0.0, CleverHans supports 3 frameworks: JAX, PyTorch, and TF2. We are currently prioritizing implementing
attacks in PyTorch, but we very much welcome contributions for all 3 frameworks. In versions v3.1.0 and prior,
CleverHans supported TF1; the code for v3.1.0 can be found under cleverhans_v3.1.0/ or by checking
out a prior Github release.
The library focuses on providing reference implementation of attacks against machine learning models to help with benchmarking models against adversarial examples.
The directory structure is as follows:
cleverhans/ contain attack implementations, tutorials/ contain scripts demonstrating the features
of CleverHans, and defenses/ contains defense implementations. Each framework has its own subdirectory
within these folders, e.g. cleverhans/jax.
Setting up CleverHans
Dependencies
This library uses Jax, PyTorch or TensorFlow 2 to accelerate graph computations performed by many machine learning models. Therefore, installing one of these libraries is a pre-requisite.
Installation
Once dependencies have been taken care of, you can install CleverHans using
pip or by cloning this Github repository.
pip installation
If you are installing CleverHans using pip, run the following command:
pip install cleverhans
This will install the last version uploaded to Pypi. If you'd instead like to install the bleeding edge version, use:
pip install git+https://github.com/cleverhans-lab/cleverhans.git#egg=cleverhans
Installation for development
If you want to make an editable installation of CleverHans so that you can develop the library and contribute changes back, first fork the repository on GitHub and then clone your fork into a directory of your choice:
git clone https://github.com/<your-org>/cleverhans
You can then install the local package in "editable" mode in order to add it to
your PYTHONPATH:
cd cleverhans
pip install -e .
Currently supported setups
Although CleverHans is likely to work on many other machine configurations, we currently test it with Python 3.6, Jax 0.2, PyTorch 1.7, and Tensorflow 2.4 on Ubuntu 18.04 LTS (Bionic Beaver).
Getting support
If you have a request for support, please ask a question on StackOverflow rather than opening an issue in the GitHub tracker. The GitHub issue tracker should only be used to report bugs or make feature requests.
Contributing
Contributions are welcomed! To speed the code review process, we ask that:
- New efforts and features be coordinated on the discussion board.
- When making code contributions to CleverHans, you should follow the
Blackcoding style in your pull requests. - We do not accept pull requests that add git submodules because of the problems that arise when maintaining git submodules.
Bug fixes can be initiated through Github pull requests.
Tutorials: tutorials directory
To help you get started with the functionalities provided by this library, the
tutorials/ folder comes with the following tutorials:
- MNIST with FGSM and PGD (jax, tf2: this tutorial covers how to train an MNIST model and craft adversarial examples using the fast gradient sign method and projected gradient descent.
- CIFAR10 with FGSM and PGD (pytorch, tf2): this tutorial covers how to train a CIFAR10 model and craft adversarial examples using the fast gradient sign method and projected gradient descent.
NOTE: the tutorials are maintained carefully, in the sense that we use continuous integration to make sure they continue working. They are not considered part of the API and they can change at any time without warning. You should not write 3rd party code that imports the tutorials and expect that the interface will not break. Only the main library is subject to our six month interface deprecation warning rule.
NOTE: please start a thread on the discussion board before writing a new tutorial. Because each new tutorial involves a large amount of duplicated code relative to the existing tutorials, and because every line of code requires ongoing testing and maintenance indefinitely, we generally prefer not to add new tutorials. Each tutorial should showcase an extremely different way of using the library. Just calling a different attack, model, or dataset is not enough to justify maintaining a parallel tutorial.
Examples : examples directory
The examples/ folder contains additional scripts to showcase different uses
of the CleverHans library or get you started competing in different adversarial
example contests. We do not offer nearly as much ongoing maintenance or support
for this directory as the rest of the library, and if code in here gets broken
we may just delete it without warning.
Since we recently discontinued support for TF1, the examples/ folder is currently
empty, but you are welcome to submit your uses via a pull request :)
Old examples for CleverHans v3.1.0 and prior can be found under cleverhans_v3.1.0/examples/.
Reporting benchmarks
When reporting benchmarks, please:
- Use a versioned release of CleverHans. You can find a list of released versions here.
- Either use the latest version, or, if comparing to an earlier publication, use the same version as the earlier publication.
- Report which attack method was used.
- Report any configuration variables used to determine the behavior of the attack.
For example, you might report "We benchmarked the robustness of our method to
adversarial attack using v4.0.0 of CleverHans. On a test set modified by the
FastGradientMethod with a max-norm eps of 0.3, we obtained a test set accuracy of 71.3%."
Citing this work
If you use CleverHans for academic research, you are highly encouraged (though not required) to cite the following paper:
@article{papernot2018cleverhans,
title={Technical Report on the CleverHans v2.1.0 Adversarial Examples Library},
author={Nicolas Papernot and Fartash Faghri and Nicholas Carlini and
Ian Goodfellow and Reuben Feinman and Alexey Kurakin and Cihang Xie and
Yash Sharma and Tom Brown and Aurko Roy and Alexander Matyasko and
Vahid Behzadan and Karen Hambardzumyan and Zhishuai Zhang and
Yi-Lin Juang and Zhi Li and Ryan Sheatsley and Abhibhav Garg and
Jonathan Uesato and Willi Gierke and Yinpeng Dong and David Berthelot and
Paul Hendricks and Jonas Rauber and Rujun Long},
journal={arXiv preprint arXiv:1610.00768},
year={2018}
}
About the name
The name CleverHans is a reference to a presentation by Bob Sturm titled âClever Hans, Clever Algorithms: Are Your Machine Learnings Learning What You Think?" and the corresponding publication, "A Simple Method to Determine if a Music Information Retrieval System is a 'Horse'." Clever Hans was a horse that appeared to have learned to answer arithmetic questions, but had in fact only learned to read social cues that enabled him to give the correct answer. In controlled settings where he could not see people's faces or receive other feedback, he was unable to answer the same questions. The story of Clever Hans is a metaphor for machine learning systems that may achieve very high accuracy on a test set drawn from the same distribution as the training data, but that do not actually understand the underlying task and perform poorly on other inputs.
Authors
This library is collectively maintained by the CleverHans Lab at the University of Toronto. The current point of contact is Jonas Guan. It was previously maintained by Ian Goodfellow and Nicolas Papernot.
The following authors contributed 100 lines or more (ordered according to the GitHub contributors page):
- Ian Goodfellow (Google Brain)
- Nicolas Papernot (Google Brain)
- Nicholas Carlini (Google Brain)
- Fartash Faghri (University of Toronto)
- Tzu-Wei Sung (National Taiwan University)
- Alexey Kurakin (Google Brain)
- Reuben Feinman (New York University)
- Shiyu Duan (University of Florida)
- Phani Krishna (Video Analytics Lab)
- David Berthelot (Google Brain)
- Tom Brown (Google Brain)
- Cihang Xie (Johns Hopkins)
- Yash Sharma (The Cooper Union)
- Aashish Kumar (HARMAN X)
- Aurko Roy (Google Brain)
- Alexander Matyasko (Nanyang Technological University)
- Anshuman Suri (University of Virginia)
- Yen-Chen Lin (MIT)
- Vahid Behzadan (Kansas State)
- Jonathan Uesato (DeepMind)
- Florian Tramèr (Stanford University)
- Haojie Yuan (University of Science & Technology of China)
- Zhishuai Zhang (Johns Hopkins)
- Karen Hambardzumyan (YerevaNN)
- Jianbo Chen (UC Berkeley)
- Catherine Olsson (Google Brain)
- Aidan Gomez (University of Oxford)
- Zhi Li (University of Toronto)
- Yi-Lin Juang (NTUEE)
- Pratyush Sahay (formerly HARMAN X)
- Abhibhav Garg (IIT Delhi)
- Aditi Raghunathan (Stanford University)
- Yang Song (Stanford University)
- Riccardo Volpi (Italian Institute of Technology)
- Angus Galloway (University of Guelph)
- Yinpeng Dong (Tsinghua University)
- Willi Gierke (Hasso Plattner Institute)
- Bruno López
- Jonas Rauber (IMPRS)
- Paul Hendricks (NVIDIA)
- Ryan Sheatsley (Pennsylvania State University)
- Rujun Long (0101.AI)
- Bogdan Kulynych (EPFL)
- Erfan Noury (UMBC)
- Robert Wagner (Case Western Reserve University)
- Erh-Chung Chen (National Tsing Hua University)
- Joel Frank (Ruhr-University Bochum)
Copyright
Copyright 2021 - Google Inc., OpenAI, Pennsylvania State University, University of Toronto.
Top Related Projects
An adversarial example library for constructing attacks, building defenses, and benchmarking both
A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
A Toolbox for Adversarial Robustness Research
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot