Thomas Serre

Thomas Serre

Machine and Biological Vision • Computational Neuroscience AI • NeuroAI • XAI

Brown University
Providence, RI 02912



Lab website
Lab github
Brown profile
CV

Dr. Thomas Serre is the Thomas J. Watson, Sr. Professor of Science and Professor of Cognitive & Psychological Sciences and Computer Science at Brown University • He serves as Faculty Director of the Center for Computation & Visualization and Associate Director of the Center for Computational Brain Science, and is also an affiliate of the Carney Institute for Brain Science, the Center for Theoretical Physics and Innovation, and the Data Science Institute. He holds an International Chair in AI at the Artificial and Natural Intelligence Toulouse Institute (France) and is a Fellow of the European Laboratory for Learning and Intelligent Systems • Dr. Serre received his Ph.D. in Neuroscience from MIT (2006) and an M.Sc. in Electrical Engineering and Computer Science from Télécom Bretagne, France (2000) • His research investigates the computational principles of biological vision to understand how the brain works—and to build AI systems that see and reason more like humans. He studies how recurrent and feedback processes support visual reasoning, develops cognitive-psychology-inspired benchmarks to reveal gaps between human and machine vision, and designs brain-inspired computational models that help bridge neuroscience and artificial intelligence • His work has been featured by the BBC, The Economist, New Scientist, and Scientific American • Dr. Serre serves as an area chair for premier conferences such as CVPR, ICML, ICLR, and NeurIPS, and as Neuroscience Section Editor for PLOS Computational Biology • He has received an NSF Early Career Award, DARPA Young Faculty Award, and DARPA Director's Award, and his team's work on human action recognition earned the IEEE PAMI Helmholtz Prize (2021) and Mark Everingham Prize (2022).

Research

My research investigates the computational principles of biological vision—both to understand how the brain works and to build more human-like AI systems. Working at the intersection of neuroscience, cognitive science, and artificial intelligence, my lab tackles a fundamental question: How can we bridge the gap between biological and artificial vision to advance both brain science and AI?

Human-AI Alignment in Vision
We develop methods to quantify and improve the alignment between deep neural networks and human visual processing. Our recent work reveals that despite impressive performance, current vision models process information fundamentally differently from humans—a critical gap for both building robust AI systems and understanding brain mechanisms. Importantly, our harmonization procedure shows that alignment can be dramatically improved without changing network architectures, suggesting the misalignment stems from training procedures rather than structural limitations. This insight has motivated us to adopt a developmental psychology approach: we identify the learning principles and developmental trajectories that shape human vision, then incorporate these principles into AI training to create models that not only perform well but see the world as humans do. Funded by NSF.

Cognitive Benchmarks for AI Visual Reasoning
We develop rigorous cognitive-psychology-inspired benchmarks to evaluate fundamental gaps between human and machine vision. These benchmarks reveal systematic failures in modern AI. For example, our Pathfinder challenge shows that feedforward networks fail at contour integration tasks that humans solve effortlessly—a finding later confirmed by Google DeepMind, who showed that even state-of-the-art transformers fail while our brain-inspired recurrent models succeed. Our compositional reasoning benchmark reveals AI's inability to flexibly combine visual concepts, while our 3D-PC benchmark demonstrates failures in visual perspective-taking—a key signature of theory of mind. Even seemingly simple same-different judgments expose how neural networks struggle with basic visual relationships. Critically, this work not only reveals AI limitations but also helps identify brain mechanisms underlying relational processing. Funded by ONR.

Cortical Feedback and Visual Reasoning
We reverse-engineer how feedback connections in the brain enable complex visual reasoning and mental simulation. Our cognitive benchmarks reveal systematic failures of feedforward networks—from contour integration to relational judgments—pinpointing which computations require recurrent processing. These insights guide our experimental work: our neurophysiology studies show that same-different tasks that challenge feedforward AI engage distinct neural dynamics in primates, while our recent work reveals that both monkeys and recurrent neural networks use internal "mental simulations" to solve challenging visual tasks. By identifying where feedforward processing fails, we pinpoint the computational role of cortical feedback. This work is reshaping our understanding of how biological vision achieves robust reasoning through recurrent processing. Funded by ONR.

Explainable AI for Scientific Discovery
In collaboration with the Artificial and Natural Intelligence Toulouse Institute, we create tools to understand and interpret deep learning models. Our CRAFT framework and MACO approach help researchers open the "black box" of AI. CRAFT provides concept-based explanations revealing both "what" and "where" models look, while MACO enables feature visualization for state-of-the-art deep networks. These methods are implemented in our open-source Xplique toolbox, making explainability accessible to the broader research community. Critically, our tools reveal when AI learns deceptive strategies—for instance, in histopathology, we showed that models claiming superhuman cancer diagnosis actually relied on spurious correlations rather than meaningful biological features. See these tools in action: LENS explains what ImageNet models actually see, and LeafLens reveals how AI identifies plant species from cleared leaves. Building on this work, we are developing methods to identify computational mechanisms learned by foundation models—an effort outlined in our perspective on moving from prediction to understanding in brain science. Funded by ANR and NSF.

Teaching

I teach computational courses at the interface between natural and artificial intelligence, bridging neuroscience, cognitive science, and AI.

CPSY 1291: Computational Methods for Mind, Brain & Behavior
Advanced Undergraduate/Graduate • Fall Semester
A broad introduction to NeuroAI combining lectures with hands-on programming assignments. Students explore computational models of brain and cognition, classical machine learning algorithms, and modern deep learning architectures.

CPSY 1950: Deep Learning in Brains, Minds & Machines
Advanced Undergraduate/Graduate • Spring Semester
A seminar-style exploration of cutting-edge research at the intersection of natural and artificial intelligence. Students engage with recent papers and develop critical perspectives on how biological and artificial systems process information.

Selected Recent Publications

See the lab publications page for a complete list of publications.


Selected Talks

Aligning deep networks with primate vision via self-supervised learning
Simons Foundation Workshop on "Self Supervised Learning" • May 2025

Exploring how self-supervised learning can bridge the gap between artificial neural networks and biological vision systems. This talk presents our latest work on developing training procedures that align deep neural networks with primate visual processing, demonstrating that alignment can be dramatically improved without changing network architectures.

Feedforward and feedback processes in visual reasoning
MindCORE Vision Seminar, University of Pennsylvania • April 2024

Demonstrating how feedforward neural networks struggle with visual reasoning problems that appear simple to humans. This talk presents our computational neuroscience model of feedback circuitry in the visual cortex, showing how it can be transformed into a modern deep recurrent network that addresses weaknesses of current state-of-the-art feedforward networks—providing evidence that neuroscience can offer powerful new concepts for AI.


Active Grants