Software and Tools

As a Network of Excellence and a European Lighthouse, ELSA is committed to transparently sharing the network’s research results. Foundational AI and ML research and its results are key to increasing the safety of AI in Europe.

On this page, we share ELSA-affiliated software and tools. You can find out more about ELSA-affiliated research on our publications landing page.

Tools, repositories, Plugins and more

Below, we provide a collection of software, data sets, papers, code and models for AI and ML auditing which were both funded by ELSA and originate form the broader ELSA network.

AI Attacks

• SecurityNets
  • SecurityNets is an “observatory” of AI/ML vulnerabilities that consists of a large dataset of trained models that are audited for vulnerabilities.
  • https://github.com/SecurityNet-Research/SecurityNet

Auditing Code Generation for Vulnerabilities

• CodeLMSec Benchmark
  • Code repository containing data for “CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models”. The paper presents a method to systematically study the security issues of code language models to assess their susceptibility to generating vulnerable code.
  • https://github.com/codelmsec/codelmsec
• (SVEN) Large Language Models for Code: Security Hardening and Adversarial Testing
  • Code repository containing data for the paper “Large Language Models for Code: Security Hardening and Adversarial Testing”
  • https://github.com/eth-sri/sven

Auditing Explainability

• b-cos explainability
  • Code repository for the paper “B-cos Networks: Alignment is all we need for Interpretability”, that presents a new direction for increasing the interpretability of deep neural networks (DNNs) by promoting weight-input alignment during training.
  • https://github.com/moboehle/B-cos

Auditing Machine Learning

• AdvMLPhish
  • AdvMLPhish is an open-source tool for evaluating the robustness of machine-learning phishing webpage detectors. It includes a set of functionality- and rendering-preserving adversarial manipulations, and a black-box optimization algorithm inspired to mutation-based fuzzing to optimally select which manipulations should be applied to evade the target detector.
  • https://github.com/advmlphish/raze_to_the_ground_aisec23
• Fast Minimum-Norm Adversarial Attacks
  • Minimum-norm gradient-based adversarial attack that works with multiple norms.
  • https://github.com/pralab/Fast-Minimum-Norm-FMN-Attack
• Indicators of Attack Failure
  • Indicators of failure is a tool that analyzes failures in the optimization of adversarial attacks, uses indicators to reveal when they happen, and offers a systematic framework to avoid them.
  • https://github.com/pralab/IndicatorsOfAttackFailure
• MLDoctor
  • MLDoctor is a code base for evaluating different attacks.
  • https://github.com/liuyugeng/ML-Doctor
• SecML
  • SecML is a python library for Secure and Explainable Machine Learning. It is equipped with evasion and poisoning adversarial machine learning attacks, and it can wrap models and attacks from other different frameworks.
  • https://github.com/pralab/secml
• SecML Malware
  • SecML Malware is a python library for creating adversarial attacks against Windows Malware detectors. Built on top of SecML, SecML Malware includes most of the attack proposed in the state of the art.
  • https://github.com/pralab/secml_malware
• Waf-a-MOLE
  • Waf-a-MOLE is a guided mutation-based fuzzer for ML-based Web Application Firewalls, inspired by AFL and based on the FuzzingBook by Andreas Zeller et al. Given an input SQL injection query, it tries to produce a semantic invariant query that is able to bypass the target WAF. You can use this tool for assessing the robustness of your product by letting WAF-A-MoLE explore the solution space to find dangerous “blind spots” left uncovered by the target classifier.
  • https://github.com/AvalZ/WAF-A-MoLE

Interpretability

• Interpretable-through-prototypes deepfake detection for diffusion models
  • Repository containing materials for the following paper “Interpretable-through-prototypes deepfake detection for diffusion models.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.
  • https://github.com/lira-centre/DeepfakeDetection/tree/main

LLM Deliberation

• LLM deliberation
  • Repository for benchmark paper “Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation” (NeurIPS 2024)
  • https://github.com/S-Abdelnabi/LLM-Deliberation

LLM Vulnerabilities

• LVE Repository
  • Tracking and documentation of vulnerabilities and exposures of large language models (LVEs).
  • https://lve-project.org/

Privacy Auditing

• GanLeaks
  • Code repository containing the implementation for “GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models” (CCS 2020)
  • https://github.com/DingfanChen/GAN-Leaks
• MLLeaks
  • Code repository for the paper “ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models”
  • https://github.com/AhmedSalem2/ML-Leaks

Privacy-Preserving and Collaborative Learning

• CoBo
  • Code for “CoBo: Collaborative Learning via Bilevel Optimization” published at NeurIPS 2024
  • https://github.com/epfml/CoBo
• CoDE
  • Models and Code for “Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities” paper published at ECCV 2024
  • https://github.com/aimagelab/CoDE
• CoMiGS
  • Code for “On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists”
  • https://github.com/epfml/CoMiGS
• DECOR
  • Code for “The Privacy Power of Correlated Noise in Decentralized Learning” Published at ICML 2024
  • https://github.com/elfirdoussilab1/DECOR
• disco
  • DISCO is a code-free and installation-free browser platform that allows any non-technical user to collaboratively train machine learning models (such as LLMs) without sharing any private data.
  • https://github.com/epfml/disco/
• DPConvCNP
  • Code for: “Noise-Aware Differentially Private Regression via Meta-Learning” published at NeurIPS 2024
  • https://github.com/cambridge-mlg/dpconvcnp
• DPFed-KM
  • Code for: “Private and Collaborative Kaplan- Meier Estimators” published at WPES at CCS 2024
  • https://github.com/ShadiRahimian/DPFed-KM?tab=readme-ov-file
• DP-FSL
  • Code repository containing the implementation for “GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models” (CCS 2020)
  • https://github.com/cambridge-mlg/dp-few-shot/
• FedLAP-DP
  • Code for: “FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations“ published at PoPETs’24
  • https://github.com/hui-po-wang/FedLAP-DP
• GS-WGAN
  • Code for: “GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators” published at NeurIPS 2020
  • https://github.com/DingfanChen/GS-WGAN
• Memorization in Federated LoRA
  • Code for “Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs”
  • https://github.com/tuneinsight/federated-llms
• ML-Doctor Code
  • Code for: “ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models” published at USENIX’22
  • https://github.com/liuyugeng/ML-Doctor
• MyTH
  • Code for “MyThisYourThat: Interpretable Identification of Systematic Bias in Federated Learning for Biomedical Images”
  • https://github.com/EPFLiGHT/MyTH
• Personalized Collaborative LLMs
  • Code for “Personalized Collaborative Fine-Tuning for On-Device Large Language Models” Published at CoLM 2024
  • https://github.com/epfml/personalized-collaborative-llms
• PRO-GENE-GEN
  • Code for: “Towards Biologically Plausible and Private Gene Expression Data Generation” published at PoPETs’24
  • https://github.com/MarieOestreich/PRO-GENE-GEN
• PROLIN
  • Code for: “Client-specific Property Inference against Secure Aggregation in Federated Learning” published at WPES at CCS 2023
  • https://github.com/raouf-kerkouche/PROLIN
• PSG
  • Code for: “Private set generation with discriminative information” published at NeurIPS 2022
  • https://github.com/DingfanChen/Private-Set
• individual-accounting-gdp
  • Code for “Individual Privacy Accounting with Gaussian Differential Privacy” published at ICLR 2023
  • https://github.com/DPBayes/individual-accounting-gdp
• Meditron
  • Model and Code for “Meditron-70b: Scaling medical pretraining for large language models”
  • https://github.com/epfLLM/meditron/
• MT_COOL
  • Code for paper multitask online learning: listen to the neighborhood buzz. AISTATS 2024
  • https://github.com/juliette-achddou/MT_COOL
• PFL-DocVQA-Competition
  • Code for: “Privacy-Aware Document Visual Question Answering” published in ICDAR 2024.
  • https://github.com/rubenpt91/PFL-DocVQA-Competition
• SecurityNet Code
  • Code for: “SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models” published at USENIX 2024
  • https://github.com/SecurityNet-Research/SecurityNet
• subsampling-is-not-magic
  • Code for: “Subsampling is not Magic: Why Large Batch Sizes Work for Differentially Private Stochastic Optimisation” (ICML 2024)
  • https://github.com/DPBayes/subsampling-is-not-magic
• Towards Efficient Scalable Training DP DL
  • Code for “Towards Efficient and Scalable Training of Differentially Private Deep Learning” published at Workshop on Advancing Neural Network Training at International Conference on Machine Learning (WANT@ICML 2024)
  • https://github.com/DPBayes/Towards-Efficient-Scalable-Training-DP-DL
• twinify
  • software package for privacy-preserving generation of a synthetic twin to a given sensitive tabular data set.
  • https://github.com/DPBayes/twinify

Technical Robustness and Safety

• Adversarial Pruning Benchmark
  • The Adversarial Pruning Benchmark is a framework implemented to enable a uniform and reliable evaluation of Adversarial Pruning methods.
  • https://github.com/pralab/AdversarialPruningBenchmark
• AdversarialRecovery
  • AdversarialRecovery is a repository for robust adversarial sample recovery, especially for cross-domain samples (unseen datasets, unseen objects, and unseen adversarial algorithms to the training stage). 
  • https://github.com/Yukino-3/AdversarialRecovery
• Adversarial Robustness Certification for Bayesian Neural Networks
  • This repository provides implementations for training Bayesian Neural Networks (BNNs) using various inference methods, along with comprehensive certifications for their probabilistic robustness, decision robustness, and uncertainty quantification.
  • https://github.com/matthewwicker/AdversarialRobustnessCertificationForBNNs
• Automated Design for Linear Bounding Functions for Sigmoidal Nonlinearities in Neural Networks
  • The code implements a robustness verification framework for neural networks with general activation functions (e.g., Sigmoid, Tanh), focusing on enhancing the quality of linear bounds in convex relaxation techniques.
  • [URL currently not available]
• AttackBench
  • The AttackBench framework fairly compares gradient-based attacks based on their results against a set of robust models.
  • https://github.com/attackbench/attackbench
• CoDE: Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities
  • CoDE (Contrastive Deepfake Embeddings) is a novel approach that utilizes contrastive learning and global-local similarities to create an effective embedding space specifically for deepfake detection.
  • https://aimagelab.github.io/CoDE/
• DAGER
  • DAGER is the first algorithm to recover whole batches of input text exactly, recovering full batches of size up to 128 on large language models (LLMs).
  • https://github.com/insait-institute/dager-gradient-inversion
• FAST (FeAture SelecTion)
  • The code implements FAST (FeAture SelecTion), a method to enhance the efficiency and effectiveness of test case prioritization for deep neural networks (DNNs).
  • https://github.com/Testing4AI/FAST
• FullCert
  •  A certification library for computing worst-case bounds on model parameters during training.
  • https://github.com/t-lorenz/FullCert
• GeometricKernels
  • GeometricKernels is a library that implements kernels — most importantly, heat and Matérn kernels — on non-Euclidean spaces such as Riemannian manifolds, graphs and meshes.
  • https://github.com/geometric-kernels/GeometricKernels
• ModSec-AdvLearn
  • ModSec-AdvLearn is a machine-learning-based methodology that improves the detection of SQL injection attacks on Web Application Firewall (WAF) while addressing vulnerabilities to adversarial manipulations.
  • https://github.com/pralab/modsec-advlearn
• Nebula
  • Nebula is a tool to perform dynamic analysis of Windows malware which, by generalizing across different behavioral representations and formats, combines diverse information from dynamic log reports.
  • https://github.com/dtrizna/nebula
• PREMAP: A Unifying PREiMage APproximation Framework for Neural Networks
  • A general and flexible preimage approximation framework designed to generate inputs that satisfy specific target properties.
  • https://github.com/Zhang-Xiyue/PreimageApproxForNNs
• SecML-Torch
  • SecML-Torch (SecMLT) is an open-source Python library designed to facilitate research in the area of Adversarial Machine Learning (AML) and robustness evaluation.
  • https://github.com/pralab/secml-torch
• SecML-Torch Encryption Plugin
  • An open-source Python plugin for the SecML-Torch library that integrates homomorphic encryption techniques within machine learning models.
  • https://github.com/simoneminisi/secml-encryption
• SecML-Torch Fairness Plugin
  • An open-source Python plugin for the SecML-Torch library that introduces a set of methods for analyzing and mitigating discriminatory bias in machine learning models.
  • https://github.com/simoneminisi/secml-fair
• SecML-Torch Interpretability Plugin
  • An open-source Python plugin for the SecML-Torch library. This plugin provides tools for the interpretation and explainability of machine learning models.
  • https://github.com/simoneminisi/secml-interpretability
• Sigma-zero
  • This tool is the official PyTorch implementation of the σ-zero: Gradient-based Optimization of L0-norm Adversarial Examples.
  • https://github.com/sigma0-advx/sigma-zero
• TaskTracker
  • TaskTracker is a novel approach to detect task drift in large language models (LLMs) by analyzing their internal activations.
  • https://github.com/microsoft/TaskTracker
• Uncertainty Adversarial Robustness
  • The Uncertainty Adversarial Robustness repository provides practical tools for performing uncertainty attacks against robust models.
  • https://github.com/pralab/UncertaintyAdversarialRobustness
• Understanding Certified Training with Interval Bound Propagation
  • This project leverages a novel metric for measuring the tightness of IBP bounds.
  • https://github.com/eth-sri/ibp-propagation-tightness

Deliverables

The ELSA Work Package on “Technical Robustness” and the work package “Privacy and Infrastructures” have created comprehensive documents (deliverables) that provide descriptions and references to software, models and related publications created by the ELSA partners related to the respective work packages.

Please find the deliverable here: