RoboPapers

42:31

Ep#68: DreamZero: World Action Models are Zero-Shot...

RoboPapers ·

2026/03/20

Achieving generalizable manipulation is the north star for robotics learning, and while we’ve in the past seen incredible results on specific tasks using fine-tuned VLAs, this north star has remained elusive.Perhaps what is needed is a different approach. DreamZero proposes World Action models (WAMs), which jointly model both action and video in order to achieve state-of-the-art performance on benchmarks like MolmoSpaces and RoboArena. Seonghyeon Ye of NVIDIA Robotics joins us to talk about building

en

52:46

Ep#67: Asimov - Open Source Humanoid

RoboPapers ·

2026/03/18

Robotics research is moving fast, and being able to modify and improve upon hardware is crucial to maintaining velocity. That’s why Menlo Research has started working on their own open-source humanoid project, Asimov.And they are moving fast. It’s been roughly six months since they started the project, and they already have full humanoid with arms, legs, and a head, which can walk forwards and backwards.Selim and Alejandro of Menlo Research join us to talk about the development of this open-source

en

51:43

Ep#66: Ordered Action Tokenization

RoboPapers ·

2026/03/11

How should we represent robot actions for autoregressive transformers? Most robot policies use diffusion or flow to generate continuous action sequences, but this isn’t how large language models work; they predict output tokens, which has many advantages. But coming up with a set of useful action tokens, so we can skip the slow and expensive diffusion steps, is difficult.Chaoqi Liu says action tokens need three qualities: reasonable compression, universal decodability, and a left-to-right causally

en

63:46

Ep#65: VLM4VLA: Revisiting Vision-Language Models in...

RoboPapers ·

2026/03/05

Pretraining is essential for good performance on a wide variety of robotics tasks, and so most vision-language-action models build off of a vision language model (VLM) trained on a wide variety of image-language data. But how does the choice of VLM translate to downstream robotics performance?Jianke Zhang and Yanjiang Guo join us to talk about this key part of the robot policy, looking at a wide variety of different VLMs and how they perform. Interestingly, they see that performance on auxiliary

en

55:51

Ep#64: Project Instinct

RoboPapers ·

2026/02/26

Human motion is instinctual. We know how to interact with the world around us, almost without thinking about it at all. Ziwen and Shaoting joined us on RoboPapers to talk about their ambitious Project Instinct: which provides the tools, algorithms, and environments necessary to build humanoid whole-body control which can handle contact with the environment.Watch Episode #64 of RoboPapers with Michael Cho and Jiafei Duan now!Abstract:We present a unified framework from algorithm, environment, dataset

en

68:50

Ep#63: NovaFlow: Zero-Shot Manipulation via...

RoboPapers ·

2026/02/19

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by Hongyu Li and Jiahui

en

59:36

Ep#62: PolaRiS: Scalable Real-to-Sim Evaluations for...

RoboPapers ·

2026/02/11

Evaluating robot policies is hard. Ideally, instead of testing every new policy on a real robot, you could test in simulation; but simulations rarely correlate well with real-world performance. In order to make good, useful simulations, you need to spend a great deal of time and effort.That’s where PolaRiS comes in: it’s a toolkit that lets you take a short video of a real scene and turn it into a high-fidelity simulation. It provides what you need to build a good evaluation environment, and it

en

54:52

Ep#61: 1x World Model

RoboPapers ·

2026/02/04

Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot manipulation of a wide range of tasks — which is a real challenge for robotics, since so many cutting-edge approaches require expert fine tuning on a small set of in-domain data.Humanoid company 1X has a solution: world models. The internet is filled with human videos; this has resulted in incredible performance in video models. Why not leverage the semantic and spatial knowledge captured by

en

73:32

Ep#60: Sim-to-Real Manipulation with VIRAL and Doorman

RoboPapers ·

2026/01/28

For robots to be useful, they must be able to interact with a wide variety of environments; and yet, scaling interaction data is difficult, expensive, and time consuming. Instead, much research revolves around sim-to-real manipulation — but mostly this has not been mobile manipulation.Recently, though, this has begun to change. Two recent papers from Tairen He and Haoru Xue show us how to unlock the potential of this technique, building policies which, without any real data at all, can move objects

en

48:57

Ep#59: SAIL: Faster-than-Demonstration Execution of...

RoboPapers ·

2026/01/21

Teleoperating a robot is hard. This means that when performing a robot task via teleoperation — say, to collect examples for training a robot policy — it’s almost unavoidably slower than you would like, below either the capabilities of the human expert on their own or the robot performing the task. Wouldn’t it be great if there was a way to fix this?Unfortunately, it’s harder than it looks. You can’t just execute faster, as this alters the distribution of environment states the policy will

en

71:52

Ep#58: RL-100: Performant Robotic Manipulation with...

RoboPapers ·

2026/01/14

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success.The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing

en

51:05

Ep#57: Learning Dexterity from Human Videos with...

RoboPapers ·

2026/01/06

Teaching robots from human video is an important part of overcoming the “data gap” in robotics, but many of the details still need to be worked out. Homanga Bharadwaj tells us about two recent research papers, Gen2Act and Spider, which go over different aspects of the problem:Gen2Act uses generative video models to create a reference of how a task should be performed given a language prompt; then, it uses a multi-purpose policy that can “translate” from human video to robot motion.However, Gen2Act

en

46:06

Ep#56: GSWorld: Closed-Loop Photo-Realistic...

RoboPapers ·

2025/12/22

It’s long been a dream of roboticists to be able to teach a robot in simulation so as to skip the long and expensive process of collecting large amounts of real-world training data. However, building simulations for robot tasks is extremely hard. Ideally, we could go from real data to a useful simulation.This is exactly what Guangqi Jiang and his co-authors do. they use 3d Gaussian splatting to reconstructed scenes which let them create interactive environments that, when combined with a physcs

en

53:40

Ep#55: Trace Anything: Representing Any Video in 4D...

RoboPapers ·

2025/12/19

Modeling how worlds evolve over time is an important aspect of interacting with them. Video world models have become an exciting area of research in robotics over the past year in part for this reason. What if there was a better way to represent changes over time, though?Trace Anything represents each frame in a video as a trajectory field, i.e. a trajectory through 3d space. This provides a very unique foundation for all kinds of downstream tasks like goal-conditioned manipulation and motion

en

50:41

Ep#54: MemER: Scaling Up Memory for Robot Control via...

RoboPapers ·

2025/12/17

Most robot policies today still largely lack memory: they make all their decisions based on what they can see right now. MemER aims to change that by learning which frames are important; this lets it deal with tasks like object search. Ajay Sridhar, Jenny Pan, and Satvik Sharma tell us about how to achieve this fundamental capability for long-horizon task execution.Watch Episode #54 of RoboPapers with Michael Cho and Chris Paxton to learn more!AbstractHumans rely on memory to perform tasks; our goal

en

65:26

Ep#53: Semantic World Models

RoboPapers ·

2025/12/15

World models — action-conditioned predictive models of the environment — are an exciting are of research for robots that can be useful both for training and for test-time compute. But video-based world models waste a lot of predictive power on reconstructing pixels, which makes model and data requirements much higher and limits how far out into the future their predictions remain viable.Instead, what if we learned a purely semantic world model, one which predicts which properties will be true about

en

46:08

Ep#52: Probe, Learn, Distill: Self-improving...

RoboPapers ·

2025/12/12

On their own, vision-language-action models are powerful tools for general robot skills that show impressive generalization. However, they don’t achieve useful levels of reliability on valuable manipulation tasks.Wenli Xiao teaches us one way to achieve this reliability: Probe, Learn, Distill. By freezing the VLA and learning residual actors, specialized policies which predict actions on top of what the underlying VLA predicts. Rollouts from these residual actors can then be distilled back into the

en

53:50

Ep#51: Humanoid Everyday

RoboPapers ·

2025/12/10

Robotics, as we know, has a data problem. Many workarounds have been proposed, but one of the most important things is just to collect a large amount of real-robot data — something very difficult, especially for mobile humanoids. Enter Humanoid Everyday, which provides a large, diverse dataset of humanoid mobile manipulation examples.With 260 tasks across 7 different categories, this is the largest humanoid robot dataset we’ve ever seen — and, most importantly, the authors have provided clear

en

64:23

Ep#50: EMMA: Scaling Mobile Manipulation via...

RoboPapers ·

2025/12/08

Collecting robot teleoperation data for mobile manipulation is incredibly time consuming, even moreso than collecting teleoperation data for a stationary mobile manipulator. Fortunately, Lawrence and Pranav have a solution: EMMA, or Egocentric Mobile MAnipulation.In short, they find that they can skip mobile teleoperation entirely, just using static arms for manipulation tasks and co-training with egocentric human video. This is enough to show generalization to more complex scenes and tasks.To learn

en

50:16

Ep#49: Learning a Unified Policy for Position and...

RoboPapers ·

2025/12/05

Robots need to be able to apply pressure and make contact with objects as needed in order to accomplish their tasks. From compliance to working safely around humans to whole-body manipulation of heavy objects, combining force and position control can dramatically expand the capabilities of robots. This is especially true for legged robots, which have so much ability to exert forces on the world around them. But how do we train robots which can do this?Baoxiong Jia tells us more in our discussion of

en

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Elon Musk Podcast

Stage Zero

Modem Futura

Sean Leahy, Andrew Maynard

Cisco Podcast Network

Cisco

MSP Business School

MSP Business School