In the “ELSA Use Case Updates” series, we share insights into the progress of research within the ELSA Use Cases. We speak with ELSA’s Use Case owners, leading researchers, project managers, and engineers.
How do you know if an image on the internet is real or fake? Not in general, but you personally? Do you still look for an odd number of fingers, a detail like an earring that looks wrong, or anomalies in the background? This might have been a good method a few years ago. But a systematic review of research papers shows that humans are, at best, okay at spotting fake media now.
As more and more image generators, the systems that create deepfakes, flood the internet, and the quality of their work improves, it gets more difficult to detect fake images, especially for humans. Hence, it is increasingly necessary to develop solutions that can automatically tell the fake from the real.
Here is where ELSA network members Leonardo and the University of Modena and Reggio Emilia (UNIMORE) joined forces. They jointly lead the ELSA Use Case “Multimedia” to:
- Investigate novel ways of understanding and detecting fake data.
- Foster the creation of new datasets for misinformation detection.
- Develop a deepfake detecting algorithm.
To learn more, we talked to Lorenzo Baraldi, Associate Professor at UNIMORE, and Stefano Savian, Research Engineer at Leonardo.
Collaboration IS a key to success.
Lorenzo Baraldi: “The two parts of the use case – the development of algorithms and the development of datasets – are highly interconnected. Hence, we started working on both parts simultaneously.
With the team at Leonardo, we began creating a novel dataset for training and evaluating deepfake detection algorithms while developing the algorithms themselves. This parallel approach and close collaboration were great, as we could give each other timely feedback on our respective work in regular meetings. This intense way of cooperation was highly valuable for both parties.”
Stefano Savian: “I think that joint efforts by multiple actors are the best way forward for the development of secure AI. As part of ELSA, we intensified our existing collaboration with UNIMORE. But at events like the CVPR and ELSA workshops, we also met other members of the network and of the community.”
Generating the dataset.
Stefano Savian: “At Leonardo, we are in charge of developing the training dataset. The ELSA Use Case started in September 2022, shortly after the launch of the image generator “Stable Diffusion”. The launch was unrelated to our project, but it marked a new and highly effective way of creating media, with both positive and negative consequences.
We started by taking the best, license-free, and copyright-free images from existing datasets. We fed the pictures and their descriptions into deepfake generators. Beginning with one, we eventually used about 14 different generators for the test set.
Our goal was to develop a content-agnostic database of deepfake images that covers a wide range of content.
Over time, the dataset grew to about 2.3M records and 11.5M images. Each record in the dataset consists of a prompt, a real image, and four images generated by different generators. Prompts and corresponding real images are taken from LAION-400M, while fake images are generated with the same prompt using different text-to-image generators. This large-scale dataset was collected thanks to the Leonardo High Performance Computing (HPC) infrastructure, Davinci-1.”
With the dataset, researchers and developers can train their deepfake detection models. The dataset is fully available on Huggingface, and you can find it on the Leonardo website: Diffusion-generated Deepfake Detection dataset (D3).
Lorenzo Baraldi: “Researchers and developers can also test their algorithm on the dataset via the ELSA Benchmarks Platform. They can submit their algorithms, which then run on an unlabelled dataset. Automatically, they receive their algorithm’s performance results.”
You can find all information on the ELSA Benchmarks platform.
Developing and presenting the algorithm.
Lorenzo Baraldi: “Within the ELSA Use Case, we also developed a novel approach for Deepfake detection – termed CoDE, short for Contrastive Deepfake Embeddings. Unlike previous detectors, CoDE performs Deepfake detection in a shared embedding space trained via contrastive learning by additionally enforcing global-local similarities. The paper was accepted at ECCV 2024, the premier European Conference on Computer Vision.”
The team’s paper is available on arXiv. And the source code, trained models, and the collected dataset are available on GitHub.
“At ECCV, we also had a demo booth where the attendees could try the detector. We invited them to play around with it and to try to detect the fake images themselves. They could also upload their own deepfakes. Our detector was highly successful. It is currently state-of-the-art and performs well on images generated by models developed until 2024.”
Challenging the work and building a community,
Lorenzo Baraldi: “We aimed to create a sense of community around deepfake detection and hosted several workshops and challenges, like a workshop at the CVPR 2024 and the CCV 2024.”
… And the Development never stops.
Lorenzo Baraldi: “A challenge during the development of the algorithm was and is the continuous launch of new generators. Because every generator has its own way of working, it means that you need different ways of detecting the deepfakes.
Deepfake detection lies at the intersection of semantics and low-level features, such as pixel distribution. Good performance at a low level can mean good or bad performance at other levels. The semantic level might become more important over time.
To stop this game of cat and mouse, we work on the algorithm’s robustness. We aim to increase its performance on images generated by an unknown generator by leaving a part of the dataset out during training and then testing its performance on the “unknown” part afterwards.”
Stefano Savian: “Our dataset continues to grow, as we constantly update it with new generators and data.”
Lorenzo and Stefano, thank you for the interviews! What is your take on the distribution of deepfakes on the internet?
Lorenzo Baraldi: “The people have a right to know whether what they see online is fake or not.”
Stefano Savian: “Deepfakes are reviving the internet’s original lesson: don’t blindly believe what you see online.”