RoboPapers

55:46

Ep#48: VisualMimic: Visual Humanoid Loco-Manipulation...

RoboPapers ·

2025/12/04

Robots must often be able to move around and interact with objects in previously-unseen environments to be useful. And the interaction part is important; to do this, they must be able to perceive and interact with the world using onboard sensing. Enter VisualMimic. Shaofeng Yin and Yanjie Ze show us how to use visual sim-to-real to train diverse loco-manipulation tasks, which can even handle diverse outdoor environments.Learn more in Episode #48 of RoboPapers today, hosted by Michael Cho and Chris

en

48:40

Ep#47: ResMimic: From General Motion Tracking to...

RoboPapers ·

2025/12/02

For robots to be useful, they can’t just dance — they must be able to physically interact with the world around them. Unfortunately, the sorts of motion tracking policies you see performing dancing or martial arts are not really capable of the kind of precise, forceful interaction needed to perform useful interactions with the world. Siheng and Yanjie join us to talk about ResMimic, their new paper which takes a general-purpose human motion-tracking policy and improves it with a residual policy to

en

56:48

Ep#46: ManiFlow: A General Robot Manipulation Policy...

RoboPapers ·

2025/12/01

Improving robot's’ ability to learn from human demonstrations is key to getting better performance from them in a wide variety of tasks. Algorithmic improvements like consistency flow training and a new architecture which can leverage multimodal inputs, allows ManiFlow to substantially improve on prior work while also showing strong generalization to unseen environments and distractors. Ge Yan tells us more about how this works and how we can make imitation learning better.Find out more on

en

61:08

Ep#45: HERMES: Human-to-Robot Embodied Learning From...

RoboPapers ·

2025/11/25

Just collecting manipulation data isn’t enough for robots - they need to be able to move around in the world, which has a whole different set of challenges from pure manipulation. And bringing navigation and manipulation together in a single framework is even more challenging.Enter HERMES, from Zhecheng Yuan and Tianming Wei. This is a four-stage process in which human videos are used to set up an RL sim-to-real training pipeline in order to overcome differences between robot and human kinematics,

en

65:21

Ep#44: From Pixels to Predicates: Learning Symbolic...

RoboPapers ·

2025/11/24

Reasoning over long horizons would allow robots to generalize better to unseen environments and settings zero-shot. One mechanism for this kind of reasoning would be world models, but traditional video world models still tend to struggle with long horizons, and are very data intensive to train. But what if instead of predicting images about the future, we predicted just the symbolic information necessary for reasoning?Nishanth Kumar tells us about Pixels to Predicates, a method for symbol grounding

en

57:45

Ep#043: Attention-based map encoding for learning...

RoboPapers ·

2025/11/20

Walking robots can do all kinds of exciting things like dancing, running, and martial arts — but for them to be useful, they must be able to use their legs to handle terrain, to move over obstacles not just around them. So, how can we train walking policies for legged robots that are useful?Unlike with manipulation, these policies are trained with end-to-end, sim-to-real reinforcement learning, using attention. Turns out maybe “attention is all you need” also applies to locomotion. Chong Zhang joins

en

53:52

Ep#42: General Intuition

RoboPapers ·

2025/11/13

With enough data, robots and AI can learn “world models” that let them predict the results of their actions. These models are a way to learn how embodied AI agents can perform a wide variety of useful tasks — but they require a huge amount of data.The team at General Intuition has a solution: use data from video games! Games teach movement, problem solving, and complex spatial reasoning, and they come in a staggering diversity of forms, covering a wide variety of problems. What’s more, the captured

en

41:04

Ep#41: HITTER: A Humanoid Table Tennis Robot via...

RoboPapers ·

2025/11/05

How can we make a humanoid robot play table tennis? The robot must hit a moving ball and return it over and over again, requiring precise whole-body control over again. Zhi Su tells us about how he developed a hierarchical approach for planning an whole body control that lets people play this game with a humanoid robot.Watch Episode #41 of RoboPapers with Michael Cho and Chris Paxton now!Abstract:Humanoid robots have recently achieved impressive progress in locomotion and whole-body control, yet

en

61:15

Ep#40: Daxo Robotics

RoboPapers ·

2025/11/03

How can we build robotic hands with truly superhuman dexterity? Daxo Robotics is developing a unique tendon-driven soft robot hand, which aims to be tougher and more capable than a traditional humanoid hand. Each finger consists of many different tendons, which act in concert to move or manipulate.This is a special episode of RoboPapers where, instead of talking about a scientific paper, we talk to Tom Zhang, founder of Daxo Robotics, in order to learn both about his background and about how this

en

81:20

Ep#39: MolmoAct: An Action Reasoning Model that...

RoboPapers ·

2025/10/28

Reasoning models have massively expanded what LLMs are capable of, but this hasn’t necessarily applied to robotics. Perhaps this is in part because robots need to reason over space, not just words and symbols; so the robotics version of a reasoning model would need to think in 3D. That’s the idea behind MolmoAct, an “Action Reasoning Model” which generates spatial plans in order to predict precise low-level robot actions. Jason Lee, Haoquan Fang, and Jiafei Duan told us more about their work.Watch

en

51:30

Ep#38: Q Learning is Not Yet Scalable

RoboPapers ·

2025/10/24

Offline reinforcement learning is crucial for robotics, but does it scale? We talk to Seohong, who discusses how for long-horizon manipulation problems the answer may be no — at least not yet. But there are tricks that you can use to make it work effectively.Watch episode #38 of RoboPapers with Michael Cho and Chris Paxton now!Abstract:In this work, we study the scalability of offline reinforcement learning (RL) algorithms. In principle, a truly scalable offline RL algorithm should be able to solve

en

44:57

Ep#37: AMPLIFY: Actionless Motion Priors for Robot...

RoboPapers ·

2025/10/21

Robots has a data problem, in that robotics data is rare. While human video is quite common, it’s not usually directly usable for robots for a variety of reasons, most significantly that it’s missing explicit, accurate robot actions. Instead, Jeremy proposes that we predict keypoint trajectories — basically, how any given point in an object will move as a robot performs a task. This lets us use action-free human video to train robot skills.Learn more by watching Episode #37 of RoboPapers with

en

48:48

Ep#36: Whole-Body Conditioned Egocentric Video...

RoboPapers ·

2025/10/17

Learning a true world model for a human body means taking high-dimensional actions representing the full body pose — the location of hands and feet, for example — and using it to predict the effects of each action. This would allow for an unprecedented level of simulation over the effects of each action on the world, but this level of information is usually not available.But, with a new dataset from Meta, Yutong Bai and co-authors were able to train just such a world model, using detailed 3d

en

62:24

Ep#35: Reinforcement Learning with Action Chunking

RoboPapers ·

2025/10/08

Today, most robot learning from demonstration predicts action chunks, small robot action trajectory. Doing this is crucial for better performance, and has all kinds of advantages. But how can we apply these advantages to reinforcement learning?We talked to Colin Li and Paul Zhou to find out more.Abstract:We present Q-chunking, a simple yet effective recipe for improving reinforcement learning (RL) algorithms for long-horizon, sparse-reward tasks. Our recipe is designed for the offline-to-online RL

en

47:31

Ep#34: RoboArena

RoboPapers ·

2025/10/03

Evaluating robot policies is hard. Every lab has a different robot; reproducible evaluations are really challenging. This makes it hard for us to know which methods for learning robot policies are likely to perform the best in real-world scenarios. Taking a page from LLM evaluations like Chatbot Arena, RoboArena aims to address this problem through crowdsourcing evaluations with a network of different evaluators.Watch Episode #34 of RoboPapers, hosted by Chris Paxton and Michael Cho, now to learn

en

81:36

Ep#33: A Smooth Sea Never Made a Skilled SAILOR:...

RoboPapers ·

2025/09/26

Learning policies via imitation is extremely potent, but making sure those policies will generalize to out of distribution settings is still very hard. SAILOR proposes a solution in learning to search via a learned world model, which outperforms existing imitation approaches. Gokul, Vibhakar, and Arnav tell us about their approach.Watch Episode #33 of RoboPapers, co-hosted by Michael Cho and Chris Paxton, now!Abstract:The fundamental limitation of the behavioral cloning (BC) approach to imitation

en

63:18

Ep #32: GMT: General Motion Tracking for Humanoid...

RoboPapers ·

2025/09/21

We’ve all seen videos of humanoid robots performing single tasks that are very impressive, like dancing or karate. But training humanoid robots to perform a wide range of complex motions is difficult. GMT is a general-purpose policy which can learn a wide range of robot motions.Watch Episode #32 of RoboPapers, with Zixuan Chen, co-hosted by Michael Cho and Chris Paxton, to learn more.Abstract:The ability to track general whole-body motions in the real world is a useful way to build general-purpose

en

56:17

Ep#31: Vision in Action: Learning Active Perception...

RoboPapers ·

2025/09/17

Most robots are fixed in one location, with cameras at the correct location to solve whatever their task is going to be. This makes setting up the camera in the correct location a key part of task setup; it also makes the task unnecessarily difficult. Ideally, robots would move their camera around intelligently in order to gather all the information they need to perform a task.In “Vision in Action,” the authors look at how to use a flexible 6-DoF “neck” to move around and gather the information

en

69:59

Ep#30: R2S2: Unleashing Humanoid Reaching Potential...

RoboPapers ·

2025/09/12

How can we train humanoid mobile manipulators to perform complex manipulation tasks in the real world? Humanoids must be able to coordinate their arms and legs to perform complex tasks, but learning such skills via reinforcement learning is challenging. In R2S2, Li Yi tells us how we can learn a robot action space based on reinforcement learning which can transfer from simulation to the real world and supports manipulation tasks like picking up a box.Watch Episode 30 of RoboPapers to find out

en

82:46

Ep#29: Exploring Low Cost Robots using LLM Agents

RoboPapers ·

2025/09/12

One of the most exciting things about modern robot learning is that it’s so accessible. Ilia Larchenko talked to us about his journey into robotics: using low-cost robots like those from HuggingFace, building agents powered by Claude, and using these to put together robots that he can use to perform real, open-vocabulary tasks in the real world.Unlike most of our podcasts, this one isn’t really about a paper — although Ilia has certainly been doing his research, developing new techniques and ways to

en

All Episodes

Similar Podcasts

Lead With AI

Tamara Nall

Elon Musk Podcast

Stage Zero

Modem Futura

Sean Leahy, Andrew Maynard

Cisco Podcast Network

Cisco

MSP Business School

MSP Business School