Top Related Projects
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX
PyTorch implementation of adversarial attacks [torchattacks]
Quick Overview
AdvertorchAI is an open-source Python library for adversarial machine learning research. It provides a collection of attack and defense algorithms, as well as utility functions for evaluating the robustness of machine learning models, particularly in the context of deep learning and computer vision.
Pros
- Comprehensive collection of adversarial attack and defense algorithms
- Easy integration with PyTorch models and datasets
- Well-documented with examples and tutorials
- Actively maintained and regularly updated
Cons
- Primarily focused on computer vision tasks, limiting its applicability to other domains
- Steeper learning curve for users not familiar with adversarial machine learning concepts
- Some advanced features may require in-depth understanding of the underlying algorithms
Code Examples
- Generating an adversarial example using the Fast Gradient Sign Method (FGSM):
import torch
from advertorch.attacks import GradientSignAttack
from advertorch.utils import NormalizeByChannelMeanStd
# Assume 'model' is a pre-trained PyTorch model
normalize = NormalizeByChannelMeanStd(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
model = torch.nn.Sequential(normalize, model)
attack = GradientSignAttack(model, loss_fn=torch.nn.CrossEntropyLoss(), eps=0.3)
adv_image = attack.perturb(image, target)
- Implementing adversarial training:
from advertorch.attacks import PGDAttack
from advertorch.utils import Attack, CarliniWagnerLoss
def adv_train(model, x, y, optimizer):
model.train()
optimizer.zero_grad()
attack = PGDAttack(model, loss_fn=CarliniWagnerLoss(), eps=0.3, nb_iter=40)
adv_x = attack.perturb(x, y)
outputs = model(adv_x)
loss = torch.nn.CrossEntropyLoss()(outputs, y)
loss.backward()
optimizer.step()
- Evaluating model robustness:
from advertorch.attacks import CarliniWagnerL2Attack
from advertorch.utils import evaluate_accuracy
def evaluate_robustness(model, test_loader, device):
attack = CarliniWagnerL2Attack(model, num_classes=10, confidence=0, targeted=False)
clean_acc = evaluate_accuracy(model, test_loader, device)
adv_acc = evaluate_accuracy(model, test_loader, device, attack=attack)
print(f"Clean accuracy: {clean_acc:.2f}%")
print(f"Adversarial accuracy: {adv_acc:.2f}%")
Getting Started
To get started with AdvertorchAI, follow these steps:
- Install the library:
pip install advertorch
- Import the necessary modules:
import torch
from advertorch.attacks import PGDAttack
from advertorch.utils import NormalizeByChannelMeanStd
- Load your model and create an attack:
model = YourModel()
normalize = NormalizeByChannelMeanStd(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
model = torch.nn.Sequential(normalize, model)
attack = PGDAttack(model, loss_fn=torch.nn.CrossEntropyLoss(), eps=0.3, nb_iter=40)
- Generate adversarial examples:
adv_images = attack.perturb(images, labels)
Competitor Comparisons
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
Pros of adversarial-robustness-toolbox
- Broader scope, covering various ML frameworks (TensorFlow, Keras, PyTorch, MXNet, scikit-learn)
- More comprehensive, including a wider range of attacks, defenses, and robustness metrics
- Active development and maintenance with regular updates
Cons of adversarial-robustness-toolbox
- Steeper learning curve due to its extensive features and APIs
- Potentially slower execution for some operations compared to advertorch
Code Comparison
advertorch:
from advertorch.attacks import PGDAttack
adversary = PGDAttack(model, loss_fn=nn.CrossEntropyLoss(), eps=0.3, nb_iter=40)
adv_images = adversary.perturb(images, labels)
adversarial-robustness-toolbox:
from art.attacks.evasion import ProjectedGradientDescent
pgd = ProjectedGradientDescent(classifier, eps=0.3, max_iter=40)
adv_images = pgd.generate(x=images)
Both toolboxes offer similar functionality for implementing adversarial attacks, but adversarial-robustness-toolbox provides a more extensive set of options and supports multiple frameworks. advertorch focuses primarily on PyTorch and offers a more streamlined API for certain tasks.
A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX
Pros of Foolbox
- More extensive collection of attacks and benchmarks
- Better documentation and tutorials
- Supports multiple deep learning frameworks (PyTorch, TensorFlow, JAX)
Cons of Foolbox
- Slightly steeper learning curve for beginners
- Less focus on adversarial training techniques
Code Comparison
Foolbox example:
import foolbox as fb
model = fb.PyTorchModel(net, bounds=(0, 1))
attack = fb.attacks.FGSM()
epsilons = [0.0, 0.001, 0.01, 0.03, 0.1, 0.3, 0.5, 1.0]
_, advs, success = attack(model, images, labels, epsilons=epsilons)
Advertorch example:
from advertorch.attacks import GradientSignAttack
adversary = GradientSignAttack(model, loss_fn=nn.CrossEntropyLoss(), eps=0.3)
adv_untargeted = adversary.perturb(data, target)
Both libraries offer similar functionality for implementing adversarial attacks, but Foolbox provides a more flexible API with support for multiple epsilon values in a single call. Advertorch's API is more straightforward for beginners but may require additional code for more complex scenarios.
PyTorch implementation of adversarial attacks [torchattacks]
Pros of adversarial-attacks-pytorch
- More user-friendly with simpler API and better documentation
- Actively maintained with frequent updates and bug fixes
- Supports a wider range of PyTorch versions
Cons of adversarial-attacks-pytorch
- Fewer attack methods implemented compared to advertorch
- Less focus on defensive techniques and robustness evaluation
- Limited support for other deep learning frameworks
Code Comparison
advertorch:
from advertorch.attacks import PGDAttack
adversary = PGDAttack(model, loss_fn=nn.CrossEntropyLoss(), eps=0.3,
nb_iter=40, eps_iter=0.01, rand_init=True)
adv_examples = adversary.perturb(x, y)
adversarial-attacks-pytorch:
from torchattacks import PGD
atk = PGD(model, eps=8/255, alpha=2/255, steps=4)
adv_images = atk(images, labels)
Both libraries offer similar functionality for implementing adversarial attacks, but adversarial-attacks-pytorch provides a more streamlined API. advertorch offers more advanced options and customization, while adversarial-attacks-pytorch focuses on simplicity and ease of use.
Convert 
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
is a Python toolbox for adversarial robustness research. The primary functionalities are implemented in PyTorch. Specifically, AdverTorch contains modules for generating adversarial perturbations and defending against adversarial examples, also scripts for adversarial training.
Latest version (v0.2)
Installation
Installing AdverTorch itself
We developed AdverTorch under Python 3.6 and PyTorch 1.0.0 & 0.4.1. To install AdverTorch, simply run
pip install advertorch
or clone the repo and run
python setup.py install
To install the package in "editable" mode:
pip install -e .
Setting up the testing environments
Some attacks are tested against implementations in Foolbox or CleverHans to ensure correctness. Currently, they are tested under the following versions of related libraries.
conda install -c anaconda tensorflow-gpu==1.11.0
pip install git+https://github.com/tensorflow/cleverhans.git@336b9f4ed95dccc7f0d12d338c2038c53786ab70
pip install Keras==2.2.2
pip install foolbox==1.3.2
Examples
# prepare your pytorch model as "model"
# prepare a batch of data and label as "cln_data" and "true_label"
# ...
from advertorch.attacks import LinfPGDAttack
adversary = LinfPGDAttack(
model, loss_fn=nn.CrossEntropyLoss(reduction="sum"), eps=0.3,
nb_iter=40, eps_iter=0.01, rand_init=True, clip_min=0.0, clip_max=1.0,
targeted=False)
adv_untargeted = adversary.perturb(cln_data, true_label)
target = torch.ones_like(true_label) * 3
adversary.targeted = True
adv_targeted = adversary.perturb(cln_data, target)
For runnable examples see advertorch_examples/tutorial_attack_defense_bpda_mnist.ipynb for how to attack and defend; see advertorch_examples/tutorial_train_mnist.py for how to adversarially train a robust model on MNIST.
Documentation
The documentation webpage is on readthedocs https://advertorch.readthedocs.io.
Coming Soon
AdverTorch is still under active development. We will add the following features/items down the road:
- more examples
- support for other machine learning frameworks, e.g. TensorFlow
- more attacks, defenses and other related functionalities
- support for other Python versions and future PyTorch versions
- contributing guidelines
- ...
Known issues
FastFeatureAttack and JacobianSaliencyMapAttack do not pass the tests against the version of CleverHans used. (They use to pass tests on a previous version of CleverHans.) This issue is being investigated. In the file test_attacks_on_cleverhans.py, they are marked as "skipped" in pytest tests.
License
This project is licensed under the LGPL. The terms and conditions can be found in the LICENSE and LICENSE.GPL files.
Citation
If you use AdverTorch in your research, we kindly ask that you cite the following technical report:
@article{ding2019advertorch,
title={{AdverTorch} v0.1: An Adversarial Robustness Toolbox based on PyTorch},
author={Ding, Gavin Weiguang and Wang, Luyu and Jin, Xiaomeng},
journal={arXiv preprint arXiv:1902.07623},
year={2019}
}
Contributors
- Gavin Weiguang Ding
- Luyu Wang
- Xiaomeng Jin
- Laurent Meunier
- Alexandre Araujo
- Jérôme Rony
- Ben Feinstein
- Francesco Croce
- Taro Kiritani
Top Related Projects
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX
PyTorch implementation of adversarial attacks [torchattacks]
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot