Machine Learning and Computer Vision
Computer Science
1 Career
- 2017 Visiting Professor, Google Inc, Zurich, Switzerland (sabbatical stay)
- Since 2015 Professor, Institute of Science and Technology Austria (ISTA)
- 2010 – 2015 Assistant Professor, Institute of Science and Technology Austria (ISTA)
- 2007 – 2010 Senior Researcher, Max Planck Institute for Biological Cybernetics, Tübingen, Germany
- 2004 – 2007 Senior Research Scientist, German Research Center for Artificial Intelligence, Kaiserlautern, Germany
- 2003 Dr. rer. nat. in Mathematics, University of Bonn, Germany
1.1 Selected Distinctions
- 2019 – 2025 Member of the ERC Starting Grant Evaluation Panel Computer Science (PE6)
- 2015 – 2022 Associate Editor in Chief of IEEE Transactions on Pattern Analysis and Machine
Intelligence (TPAMI) - Since 2013 Action Editor of Journal of Machine Learning Research (JMLR)
- Since 2019 ELLIS Scholar and director of ELLIS Unit Vienna
- 2012 – 2017 ERC Starting Grant (Consolidator Phase)
1.2 Selected Grants
- 2024 – 2029 FWF Cluster of Excellence (10.55776/COE12), Bilateral Artificial Intelligence (BILAI), Coordinator: S. Hochreiter (JKU Linz). Total amount: e 33M, FWF-funded: € 19.8M, award to RG: € 612K.
1.3 Tech Transfer (Patents, Spin-Offs)
N/A
1.4 Selected Publications
- Y. Xian, C. H. Lampert, B. Schiele, and Z. Akata, “Zero-shot learning – a comprehensive evaluation of the good, the bad and the ugly,” IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2018.
- R. Sun and C. H. Lampert, “KS(conf): A light-weight test if a multiclass classifier operates outside of its specifications,” International Journal of Computer Vision (IJCV), 2019.
- N. Konstantinov and C. H. Lampert, “Fairness-aware PAC learning from corrupted data,” Journal of Machine Learning Research (JMLR), 2022.
- N. Kalinin and C. H. Lampert, “Banded square root matrix factorization for differentially private model training,” in Conference on Neural Information Processing Systems (NeurIPS), 2024.
- E. Zverev, S. Abdelnabi, S. Tabesh, M. Fritz, and C. H. Lampert, “Can LLMs separate instructions from data? And what do we even mean by that?” In International Conference on Learning Representations (ICLR), 2025
2 Group Members
- Max Cairney-Leeming, PhD Student since 2024
- Nikita Kalinin, PhD Student since 2023
- Fabian Kresse, PhD Student since 2025 (co-affiliated with Tom Henzinger)
- Peter Sukenik, PhD Student since 2022 (co-affiliated with Marco Mondelli)
- Alexandra Volkova, PhD Student since 2025 (co-affiliated with Dan Alistarh)
- Hossein Zakerinia, PhD Student since 2022
- Egor Zverev, PhD Student since 2023
3 Research
Our research focuses on designing trustworthy machine learning systems grounded in rigorous theory rather than heuristics. We emphasize understanding not only what models predict, but also why they behave as they do, aiming to provide formal guarantees within the framework of statistical learning theory. This includes analyzing the fundamental limits of learning algorithms and characterizing when reliable generalization is possible.
One key direction is transfer learning (multi-task learning, meta-learning, continual learning, domain adaptation), which enables models to generalize across tasks and data regimes, especially in settings with limited or heterogeneous data. Complementing this, we study trustworthy machine learning, which aims at enforcing desirable properties—such as privacy, fairness, robustness, and safety—for systems operating in real-world environments.
Our work further addresses challenges such as learning from biased or corrupted data, adapting to distribution shifts, and enabling collaborative or federated learning without compromising sensitive information. Overall, we aim to bridge theory and practice by developing principled methods that make modern machine learning systems more reliable, interpretable, and deployable in high-stakes applications.
3.1 Current Scientific Projects/Directions
- Generalization properties of multi-task, meta- and continual learning
- Characterization of attention sinks in deep transformer networks
- Efficient and fair differentially private model learning
- Instruction-data separation for large language models
- Training and verification of logic gate networks
- Quantization and hybrid architectures for efficient LLMs
3.2 Research in Focus
Instruction-tuned large language models (LLMs) achieve strong performance across many applications, yet they lack a fundamental safety property common in traditional computing systems: a clear separation between instructions and data. This deficiency makes them vulnerable to manipulations such as indirect prompt injection attacks and limits their suitability for safety-critical use.
In two recent publications, we addressed this gap from two complementary angles. First, in line with our preference for principled rather than heuristic approaches, we introduced a formal definition of instruction-data separation, along with an empirical measure that can be computed from model outputs. Reflecting our emphasis of quantitative rather than qualitative research, we created a new benchmark dataset (SEP) that for the first time allowed a systematic evaluation of instruction-data separation across real-world models. Our experimental results showed that current LLMs consistently exhibit poor separation, and that standard mitigation strategies, including prompt engineering and fine-tuning, fail to significantly improve the situation without degrading model utility. This work was published as (E. Zvevev et al, ICLR 2025).
Building on this analysis, we developed ASIDE, a lightweight architectural modification that enforces separation directly within the model. ASIDE applies an orthogonal transformation to the embeddings of data tokens, creating structurally distinct representations for instructions and data without introducing additional parameters. Empirical evaluations demonstrate that models augmented with ASIDE achieve substantially improved instruction–data separation while maintaining task performance, and show increased robustness to prompt injection attacks even without specialized safety training. Together, these works establish both a rigorous framework for analyzing a key safety limitation of LLMs and a principled architectural solution to mitigate it. This work was published as (E. Zverev et al, ICLR 2026).
