Quanta Perception as Probabilistic Events - Robotics Institute Carnegie Mellon University

VASC Seminar

May

13
Wed
Varun Sundar PhD Candidate UW–Madison
Wednesday, May 13
3:30 pm to 4:30 pm
3305 Newell-Simon Hall
Quanta Perception as Probabilistic Events

Abstract:  Autonomous systems ultimately rely on extracting information from light, yet remain brittle in extreme environments, from nighttime navigation to high-speed robotics. This limitation stems from a classical imaging abstraction: conventional sensors integrate photon flux over fixed exposure windows, imposing trade-offs between sensitivity, dynamic range, and temporal resolution that degrade perception when photons are scarce or dynamics are rapid. Emerging quanta (single-photon) image sensors overcome these limits by detecting individual photons, but they generate photon streams that exceed the compute and latency budgets of real-time systems by orders of magnitude.

Here we introduce probabilistic events, a computational primitive for real-time quanta perception at the limit of individual photons. By computing the posterior distribution over the time since the last abrupt intensity change, we represent photon streams as recursively computed belief states. Rather than the binary, fixed-threshold triggers of event cameras, this recursive Bayesian formulation yields three simultaneous, low-latency signals: motion-adaptive scene flux, high-fidelity activity maps, and an entropy measure quantifying perceptual uncertainty. This representation enables perception in extreme conditions, including detecting and estimating the pose of a running person at ~0.05 lux illumination—without retraining standard vision models. Our approach sustains input throughputs exceeding 50,000 quanta frames per second on commodity GPU hardware—four to five orders of magnitude faster than state-of-the-art quanta reconstruction baselines—yielding kilohertz-scale outputs even for megapixel arrays. By replacing frame reconstruction with direct probabilistic inference over photon streams, this work enables real-time perception at the photon limit and bridges photon-counting quanta sensing with practical robotic vision.
 

Bio:   Varun Sundar is a graduate student at the University of Wisconsin–Madison, pursuing a Ph.D. in computer science. At UW–Madison, he is advised by Prof. Mohit Gupta, where he focuses on single-photon imaging techniques. His work has been published at venues such as CVPR, ICCV, and SIGGRAPH, and has included live demos at ICCP 2023, CVPR 2024 and SIGGRAPH 2024 (which won the best-in-show award in the Emerging Technologies track). In 2026, he was awarded the Ivanisevic Award at UW–Madison, which recognizes outstanding computer science dissertators. He previously received a bachelor’s degree in electrical engineering from the Indian Institute of Technology, Madras in 2020.

Homepage:   https://varun19299.github.io/

Sponsor:

The VASC seminar is generously sponsored by HeyGen, an all-in-one AI-powered video generation platform that leverages advances in computer vision, generative modeling, and multimodal learning to make high-quality video creation both scalable and accessible.