Welcome
Welcome to the homepage of the Learning, Embodied Autonomy, and Forecasting (LEAF) lab, affiliated with the Robotics Institute, Institute for Aerospace Studies, and Department of Computer Science at the University of Toronto. The LEAF lab is led by Prof. Nick Rhinehart.
One of our central aims is general-purpose model-based control: autonomous systems that can be directed to perform a wide range of tasks by combining accurate models of the world with learned objectives. Two capabilities are essential to this vision: forecasting [1,2,3,4,5,6,7,8,9], learning to predict future observations and outcomes from rich sensor data, and reward learning [10,11,12,13,14,15], inferring what humans actually want from demonstrations, preferences, and other feedback. Together, these would allow an agent to simulate what will happen under different actions and select behavior aligned with human intent, without requiring hand-designed rewards or task-specific engineering. Our research draws on imitation learning, reinforcement learning, generative modeling, and information theory, with applications spanning autonomous driving, robot navigation, manipulation, and beyond.
Current research thrusts include: learning transferable world models over high-dimensional sensor data such as LiDAR and occupancy [1,3]; using world models to enable realistic large-scale simulation and efficient planning [1,2]; and learning reward and objective functions from human feedback [10,11] so that autonomous systems can perform complex tasks in alignment with human intent.
Our team
Recent publications
UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning.
arXiv:2606.19328, (arXiv), 2026.
arXiv:2606.19328, (arXiv), 2026.
Preference-based RL provides an approach to learning reward models from pairwise comparisons of behaviors, bypassing the need for explicit reward design. However, existing methods typically rely on passive data collection and suffer from poor sample efficiency, especially during the early stages of learning. We introduce a model-based approach that actively directs exploration by jointly reasoning over uncertainties in the reward, dynamics, and value functions. Our method, Uncertainty-Balanced Preference Planning (UBP2), uses ensembles of reward, dynamics, and value function models to evaluate candidate trajectories according to a unified score that combines expected reward, terminal value, and epistemic uncertainty. Planning under this objective yields an explicit tradeoff between exploitation and information acquisition without requiring ad hoc exploration heuristics. Under standard regularity assumptions, we establish sublinear regret guarantees for both finite-horizon and infinite-horizon settings. Empirically, experiments on the Meta-World benchmark show UBP2 achieves substantially higher sample efficiency than model-free preference-based methods and non-optimistic model-based baselines.
@misc{nabail2026ubp2,
archiveprefix = {arXiv},
author = {Mohamed Nabail and Leo Cheng and Jingmin Wang and Nicholas Rhinehart},
eprint = {2606.19328},
primaryclass = {cs.LG},
title = {UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning},
url = {https://arxiv.org/abs/2606.19328},
year = {2026}
}
QPILOTS: Efficient Test-Time Q-Steering for Flow Policies.
arXiv:2606.14801, (arXiv), 2026.
arXiv:2606.14801, (arXiv), 2026.
Flow-matching and diffusion policies are expressive action generators, but optimizing them with temporal-difference reinforcement learning (RL) remains difficult. Effective policy extraction requires exploiting the critic’s action gradient, yet directly backpropagating this signal through a multi-step denoising process can be numerically unstable. Existing methods work around this either by discarding gradient information, distilling the policy into a simpler one-step actor, or repeatedly fine-tuning the denoising policy as the critic improves. We propose QPILOTS, a method that leaves the original policy unmodified and steers the denoising process at inference time. At each denoising step, instead of evaluating the critic on the noisy intermediate action where critic predictions are unreliable, we first project that intermediate state to an estimate of the final clean action and compute the critic gradient there. We introduce two variants: QPILOTS-U uses a fast single-point approximation, while QPILOTS-M draws differentiable posterior samples via a learned auxiliary network. On a standard offline-to-online RL benchmark, QPILOTS achieves the best aggregate performance, reaching an average success rate of 90% across 50 tasks. We also apply QPILOTS to steer a large, frozen, pretrained Vision-Language Action (VLA) foundation model, outperforming or matching prior inference-time approaches across six manipulation tasks in simulation.
@article{ruan2026qpilots,
title = {QPILOTS: Efficient Test-Time Q-Steering for Flow Policies},
author = {Ruan, Yifan and Cao, Chenyang and Burger, Andreas and Pesaranghader, Ali and Kamali, Kaveh and Kim, Jaehong and Vijaykumar, Nandita and Aspuru-Guzik, Alan and Gilitschenski, Igor and Rhinehart, Nicholas},
journal = {arXiv preprint arXiv:2606.14801},
year = {2026},
doi = {10.48550/arXiv.2606.14801},
url = {https://arxiv.org/abs/2606.14801}
}
OSCAR: Obstacle Survival Curves for Adaptive Robot Navigation.
arXiv:2606.00990, (arXiv), 2026.
arXiv:2606.00990, (arXiv), 2026.
A mobile robot following a graph of known routes can make costly navigation errors when a temporary obstacle blocks a critical edge: waiting too long behind a parked cart wastes time, but immediately rerouting around a person who would move in a few seconds is also inefficient. Standard reactive obstacle avoidance addresses local motion around obstacles, while fixed wait-or-reroute rules ignore how long different obstacle types tend to persist. We propose OSCAR: an adaptive survival-modeling framework for graph-based navigation with temporary blockages. Assuming obstacle class labels are available at encounter time, the robot learns class-conditioned residual clearance-time distributions from online experience, including right-censored observations when it reroutes before observing clearance. These survival models are integrated into a time-dependent graph planner that maintains obstacle memory and computes a patience threshold at each blocked edge: how long to wait before taking an alternate route. The method continuously updates its clearance estimates across episodes and uses them to balance waiting against rerouting. We evaluate the approach in simulation and on a real mobile robot in a university atrium with obstacles including people, chairs, bins, and tubes. In simulation, the learned policy’s time-to-goal converges to within 1% of an oracle with access to ground-truth clearance distributions after fewer than 20 observations per obstacle class, outperforming all heuristic baselines. Real-world deployment confirms that the policy improves online, adapting its patience thresholds from experience across 50 navigation episodes.
@misc{sahak2026oscar,
archiveprefix = {arXiv},
author = {Hshmat Sahak and Aoran Jiao and Nicholas Rhinehart and Tim Barfoot},
eprint = {2606.00990},
primaryclass = {cs.RO},
title = {OSCAR: Obstacle Survival Curves for Adaptive Robot Navigation},
url = {https://arxiv.org/abs/2606.00990},
year = {2026}
}
AutoWorld: Scaling Multi-Agent Traffic Simulation with Self-Supervised World Models.
arXiv:2603.28963, (arXiv), 2026.
arXiv:2603.28963, (arXiv), 2026.
Multi-agent traffic simulation is central to developing and testing autonomous driving systems. Recent data-driven simulators have achieved promising results, but rely heavily on supervised learning from labeled trajectories or semantic annotations, making it costly to scale their performance. Meanwhile, large amounts of unlabeled sensor data can be collected at scale but remain largely unused by existing traffic simulation frameworks. This raises a key question: How can a method harness unlabeled data to improve traffic simulation performance? In this work, we propose AutoWorld, a traffic simulation framework that employs a world model learned from unlabeled occupancy representations of LiDAR data. Given world model samples, AutoWorld constructs a coarse-to-fine predictive scene context as input to a multi-agent motion generation model. To promote sample diversity, AutoWorld uses a cascaded Determinantal Point Process framework to guide the sampling processes of both the world model and the motion model. Furthermore, we designed a motion-aware latent supervision objective that enhances AutoWorld’s representation of scene dynamics. Experiments on the WOSAC benchmark show that AutoWorld ranks first on the leaderboard according to the primary Realism Meta Metric (RMM). We further show that simulation performance consistently improves with the inclusion of unlabeled LiDAR data, and study the efficacy of each component with ablations. Our method paves the way for scaling traffic simulation realism without additional labeling. Our project page contains additional visualizations and released code.
@article{pourkeshavarz2026autoworld,
title = {AutoWorld: Scaling Multi-Agent Traffic Simulation with Self-Supervised World Models},
author = {Pourkeshavarz, Mozhgan and Liu, Tianran and Rhinehart, Nicholas},
journal = {arXiv preprint arXiv:2603.28963},
year = {2026},
doi = {10.48550/arXiv.2603.28963},
url = {https://arxiv.org/abs/2603.28963}
}
OccSim: Multi-kilometer Simulation with Long-horizon Occupancy World Models.
arXiv:2603.28887, (arXiv), 2026.
arXiv:2603.28887, (arXiv), 2026.
Data-driven autonomous driving simulation has long been constrained by its heavy reliance on pre-recorded driving logs or spatial priors, such as HD maps. This fundamental dependency severely limits scalability, restricting open-ended generation capabilities to the finite scale of existing collected datasets. To break this bottleneck, we present OccSim, the first occupancy world model-driven 3D simulator. OccSim obviates the requirement for continuous logs or HD maps; conditioned only on a single initial frame and a sequence of future ego-actions, it can stably generate over 3,000 continuous frames, enabling the continuous construction of large-scale 3D occupancy maps spanning over 4 kilometers for simulation. This represents an >80x improvement in stable generation length over previous state-of-the-art occupancy world models. OccSim is powered by two modules: W-DiT based static occupancy world model and the Layout Generator. W-DiT handles the ultra-long-horizon generation of static environments by explicitly introducing known rigid transformations in architecture design, while the Layout Generator populates the dynamic foreground with reactive agents based on the synthesized road topology. With these designs, OccSim can synthesize massive, diverse simulation streams. Extensive experiments demonstrate its downstream utility: data collected directly from OccSim can pre-train 4D semantic occupancy forecasting models to achieve up to 67% zero-shot performance on unseen data, outperforming previous asset-based simulator by 11%. When scaling the OccSim dataset to 5x the size, the zero-shot performance increases to about 74%, while the improvement over asset-based simulators expands to 22.1%.
@article{liu2026occsim,
title = {OccSim: Multi-kilometer Simulation with Long-horizon Occupancy World Models},
author = {Liu, Tianran and Zhao, Shengwen and Pourkeshavarz, Mozhgan and Li, Weican and Rhinehart, Nicholas},
journal = {arXiv preprint arXiv:2603.28887},
year = {2026},
doi = {10.48550/arXiv.2603.28887},
url = {https://arxiv.org/abs/2603.28887}
}
Our research interests
Topics:
Applications:
Other interests:
Supported by
