ELLIS Institute Seminar Series
Kilian Weinberger
Seminar Series Kickoff: Beyond Parametric Knowledge and Next Token Prediction
Kilian Weinberger — Professor of Computer Science, Cornell University
- Thursday, 26 November 2026 · 14:00–15:00 CET
- Auditorium MvL1, 6th floor, Cyber Valley Building
We are excited to open our new Seminar Series with this inaugural talk.
Abstract
The dominant paradigm in language modeling—scaling next-token prediction and storing knowledge parametrically—has delivered impressive capabilities, but also fundamental limitations: brittle and opaque factual memory, inefficient use of parameters, and inherently myopic reasoning. In this talk, we argue that progress beyond today’s language models requires a shift from parametric memory to external memory, and from purely autoregressive generation to architectures that reason globally before committing to tokens.
We synthesize two recent directions that together support this claim:
Limited-Memory Language Models show that factual knowledge can be systematically externalized during pre-training, yielding models that are more controllable, verifiable, and parameter-efficient.
Latent diffusion–augmented language models demonstrate how planning in continuous latent space can overcome the foresight and coherence limitations of next-token prediction.
Diffusion-based generation enables trajectory-aware retrieval, where intermediate denoising states dynamically refine access to external knowledge, tightly coupling uncertainty-aware generation with efficient, multi-hop retrieval.
Together, these results point toward a new class of language models that treat parameters as computation rather than storage, integrate external memory as a first-class component, and replace myopic token-by-token generation with deliberative, globally structured reasoning.
Bio
Kilian Q. Weinberger is a Professor in the Department of Computer Science at Cornell University. He received his Ph.D. from the University of Pennsylvania in Machine Learning under the supervision of Lawrence Saul and his undergraduate degree in Mathematics and Computing from the University of Oxford. During his career he has won several best paper awards at ICML (2004), CVPR (2004, 2017), AISTATS (2005) and KDD (2014, runner-up award). In 2011 he was awarded the Outstanding AAAI Senior Program Chair Award and in 2012 he received an NSF CAREER award. He is the recipient of the Daniel M Lazar '29 Excellence in Teaching Award (2016) and the Ann S. Bowers Teaching and Advising Excellence Award (2024). As of 2024 he is an ACM and AAAI fellow and in 2021 became a Blavatnik National Awards Finalists. He was elected co-Program Chair for ICML 2016 and for AAAI 2018 and has been on the ICML board since 2016. He served as the 7th president of ICML from 2023 until 2025. Since 2024 he has been a member of the Sloan Research Fellowships Selection Committee. Kilian Weinberger's research focuses on Machine Learning and its applications. In particular, he has worked on learning under resource constraints, metric learning, AI in Science, computer vision, autonomous vehicles, Gaussian Processes, and deep learning. Before joining Cornell University, he was an Associate Professor at Washington University in St. Louis and before that he worked as a research scientist at Yahoo! Research in Santa Clara.