What does the future of Foundation Models look like? What’s the progress – and the objectives – of research in Europe? How can we design safe and secure foundation models?
These questions, and many more, were addressed in the Theme Development Workshop (TDW) on Foundation Models organised by ELIAS, ELLIOT, and ELSA. The TDW took place at the CERTH in Thessaloniki, Greece, and was open for online participants.
The event consisted of three main sessions, in which experts shared their knowledge and experience.
Session 1: Training Foundation Models
Moderator: Elisa Ricci (UNITN & FBK)
The speakers of this session were Jenia Jitsev (JSC & LAION), Frank Hutter (PriorLabs, ELLIST), and Cees Snoek (UvA).
In the first keynote, Jenia Jitsev shared his knowledge and experience on scaling laws and generalisation in open foundation models. You can access his slides here.
The second keynote by Frank Hutter introduced recent innovations in Tabular Foundation Models. His slides are available here.
Cees Snoek concluded the session with his presentation of NeoBabel, a multilingual foundation model for image generation that natively understands six languages: English, Chinese, Dutch, French, Hindi, and Persian. He shared a paper on Neobabel and his slides here.
Session 2: Ethical & Safe Foundation Models
Moderator: Lorenzo Baraldi (UNIMORE)
The session on Ethical and Safe Foundation Models consisted of the presentation of ELSA Coordinator Mario Fritz.
Mario Fritz explored how foundation models can amplify both risks and opportunities, stressing the need for transparent alignment strategies and automated red teaming. In the case of Foundation Models, safety and security are as important as ever.
His slides are available here.
Also, you can listen to his presentation here:
Session 3: Application of Foundation Models
Moderator: Dimosthenis Karatzas (CVC-CERCA & UAB) and member of the ELSA Executive Board.
The speakers in this session were Marc Pollefeys (Microsoft & ETHZ), Matthieu Cord (VALEO), and Dario Garcia-Casulla (BSC).
Marc Pollefeys provided a comprehensive and thrilling overview of Spatial AI foundation models in 3D environments.
Matthieu Cord (University of Sorbonne & VALEO) presented innovations in generative video pretraining through the VaVIM–VaVAM models. VaVIM and VaVAM are both large-scale generative video models for autonomous driving. VaVIM is an “auto-regressive video model”, whilst VaVAM is a “video-action model”.
His slides are available here.
Dario Garcia-Casulla concluded the session with an in-depth look at training and evaluating LLMs using European HPC resources.
You can access his slides here.
Workshop Recording
What an amazing, fully-packed Theme Development Workshop! We are excited to not only share the slides, but also give you the full recording of the workshop. You can find the full playlist on Youtube.
A big thank you to all the speakers and ELIAS and ELLIOT for organizing the event!