“AI in Cybersecurity: Offensive Capabilities, Impacts, and Response Strategies” – ELSA General Assembly 2026 Workshop 1

Machine learning has become a key technology in cybersecurity, enabling automated detection of malware, phishing, and web attacks, yet it remains vulnerable in adversarial environments where attackers actively seek to evade detection. This talk surveys recent advances in adversarial machine learning for security, focusing on evasion attacks against malware classifiers for Android and Windows, where carefully crafted modifications can preserve malicious functionality while bypassing detection. It also discusses approaches to web security that integrate learning into web application firewalls while accounting for adversarial manipulation, as well as phishing webpage detection under adversarial constraints. Finally, it explores emerging challenges in the use of large language models for security applications, including issues related to steering and robustness. The talk concludes by outlining key open problems and research challenges for building resilient and trustworthy machine learning systems in cybersecurity.

Take a trillion-parameter, fully autonomous system, give it access to hacking tools and real-world exploits, and turn it loose. What could possibly go wrong?

This talk is about what can go wrong.

We examine the safety and governance risks of AI-driven offensive security systems, especially when they encounter active adversaries. We look at how these systems can be exploited, what the consequences of those failures can be, and how the same weaknesses can be turned into defensive mechanisms against AI-driven cyberattacks.

After a high-level overview, we turn to technical case studies on sandbox escape, lateral movement, and persistence, drawing lessons from vulnerabilities we found in both open-source and proprietary systems. The goal is simple: to help you avoid repeating the same mistakes when designing robust and reliable AI cybersecurity operators.

Large language models (LLMs) have recently shown unexpectedly strong performance in offensive security tasks. In this talk, I present our work demonstrating their effectiveness in concrete scenarios such as privilege escalation and attacks on enterprise networks (e.g., Active Directory environments). To explain these results, we hypothesize why LLMs perform well in this domain and relate these hypotheses to our empirical study of how human penetration testers reason and operate in practice. Finally, I want to discuss the ethical dimension of automated penetration testing: Dual-use concerns have always been central to security research, but become more acute with increasing levels of autonomy. We examine how these concerns manifest in the context of LLM-based penetration testing, including questions around responsible disclosure, access, and the broader implications of commoditizing offensive capabilities.

As AI reshapes cybersecurity, a key gap remains between research and real-world adoption. This session provides a non-technical perspective based on Alias Robotics’ experience: from its origins in robot cybersecurity to developing proprietary AI systems now deployed in real-world environments.

Drawing from this journey, the session will reflect on how Cybersecurity AI is already being applied in critical and defense contexts, and why the main challenge is no longer technological feasibility but integration into real operations while ensuring control, trust, and sovereignty.

At its core, the session focuses on communication’s role in bridging this gap — translating complex research, models, and cybersecurity capabilities into narratives that enable understanding, trust, and adoption across enterprise and public-sector stakeholders.

While model capabilities are advancing rapidly, our ability to translate those capabilities into concrete estimates of real-world risk remains underdeveloped. This talk presents SaferAI’s emerging methodology for quantitative AI risk modeling, which combines scenario-based decomposition with probabilistic estimation. The approach models risk as a sequence of causal steps, assigns baseline probabilities, and then estimates how AI systems uplifts these probabilities. By integrating benchmarks, expert elicitation, and probabilistic aggregation, the methodology enables decision-relevant outputs such as the likelihood of large-scale economic harm. Beyond introducing the framework, the talk highlights key open questions at the frontier of the field. These include how to reliably translate benchmark performance into real-world risk, how to balance different modeling architectures, and how to validate models.

Most security programs are designed around an assumption that is quietly becoming false: that there is a usable window of time between when a vulnerability becomes known and when it is exploited. For two decades, that window has been compressing. It has now, for a significant and growing class of vulnerabilities, reached zero or gone negative.

It then poses the harder question: if patching can no longer be the primary load-bearing control, what takes its place and are our current architectures, operating models, and regulatory frameworks designed for a world where the exploit arrives before the patch?The session proposes a structured discussion around three areas: how to re-engineer defensive architectures for a zero-time-to-exploit baseline, how AI reshapes both sides of the offense-defense equation simultaneously, and what regulatory and institutional adaptations are needed when compliance timelines are structurally slower than attacker timelines.

Modern warfare has expanded beyond traditional domains (land, sea, air, and space) into the cyber domain, where states increasingly project power through digital means. Today’s armed forces operate sophisticated cyber commands capable of disrupting information systems and critical infrastructure at scale. The ongoing Russo-Ukrainian War has demonstrated the strategic role of cyber operations within hybrid warfare, where kinetic and digital actions are tightly integrated.
Against this backdrop, advances in artificial intelligence are poised to fundamentally reshape the nature of cyber conflict. Emerging AI systems are beginning to demonstrate super-human capabilities in vulnerability discovery, exploit development, and the automation of key stages of the cyber kill chain. As these systems mature, they raise the prospect of highly autonomous, adaptive, and scalable offensive operations, marking a transition toward what can be described as AI warfare.
This talk first examines how contemporary cyber warfare is conducted, drawing on insights from military cyber defence exercises. It then reviews recent developments in AI, including survey and benchmark results that illustrate the growing ability of AI systems to automate discrete offensive cyber tasks. Finally, it outlines a forward-looking perspective on defence, arguing that the protection of critical infrastructure will require multi-agent cyber defence architectures that leverage advanced AI to counter increasingly capable AI-driven attacks.