🎫 You've Got a Golden Ticket: Improving Generative Robot Policies With A Single Noise Vector

Omkar Patil1,2,* Ondrej Biza1 Thomas Weng1 Karl Schmeckpeper1 Wil Thomason1 Xiaohan Zhang1 Kausik Sivakumar1 Robin Walters1,3 Nakul Gopalan2 Sebastian Castro1 Stephen Hart1 Eric Rosen1
1Robotics and AI Institute (RAI)  ·  2Arizona State University (ASU)  ·  3Northeastern University
*Work done while interning at RAI
📄 Paper 🧪 arxiv 💻 Code 🎥 Video

Video Presentation (🔊 Audio included)

We propose that the performance of a pretrained, frozen diffusion or flow matching policy can be improved by swapping the sampling of initial noise from the prior distribution with a well-chosen, constant initial noise input.

Abstract

Generative robot policies trained on demonstrations using behavior cloning often learn actions that are sub-optimal or misaligned with respect to the downstream task. Policy improvement approaches aim to bridge this gap and improve the cumulative reward with minimal interventions on the pre-trained policy. However, an impediment to their practical deployment is the requirement of cumbersome hyperparameter tuning specific to model or task families. In this work, we present an episodic, derivative-free latent policy improvement approach that works with little to no changes across model scales from MLPs to VLAs. We demonstrate that the performance of a pretrained, frozen diffusion or flow matching policy can be improved with respect to a downstream reward by swapping the sampling of initial noise from the prior distribution (typically isotropic Gaussian) with a well-chosen, constant initial noise input—a golden ticket. We show the prevalence of golden tickets by improving the policy performance of 46 out of 51 tasks across manipulation benchmarks, with absolute improvements in success rate by up to 79% for simulated tasks, and 28% within 60 search episodes for real-world tasks. Further, we find that the versatility of our approach opens up new avenues such as simultaneous policy improvement for multiple downstream rewards.

Methodology

The lottery ticket hypothesis for robot control proposes that the performance of a pretrained, frozen diffusion or flow matching policy can be improved by replacing the sampling of initial noise from a prior distribution with a well-chosen, constant initial noise input, called a golden ticket.

Architecture
Overview of standard diffusion policy inference (left) versus our proposed approach of using golden tickets (right). Given a frozen, pretrained diffusion or flow matching policy, rather than sampling from a Gaussian every time an action needs to be computed, we use a constant, well-chosen initial noise vector, called a golden ticket. We find golden tickets improve policy performance across a range of observation inputs, model architectures, and embodiments.

Given a pretrained policy and a reward function and an environment for a new downstream task, we pose finding golden tickets as a search problem, where candidate initial noise vectors, which we call lottery tickets, are optimized to find the ticket that maximizes cumulative discounted expected rewards on the downstream task.

Our proposed method has several unique benefits:

Architecture
For each ticket, we run the policy in each environment, fixing the initial noise vector fed into the policy. We then calculate the average cumulative discounted rewards, providing an empirical estimate (i.e., Monte-Carlo estimate) of the value function for the ticket. After all tickets have been evaluated, we select and return the best performing ticket.

Hardware Experiments

We use Franka hardware to conduct all real-world experiments. We perform three tasks: picking a banana, pushing a cup, and picking a cube. The first two tasks (banana and cup) use pointcloud-based diffusion policies, whereas for the cup picking task we used an RGB-based diffusion policy. For each task (and corresponding base policy), we first evaluate the base policy using Gaussian noise (🎲). Then, we search for golden tickets (🎫), with the most number of rollouts being 150 episodes, and report the performance of the best performing ticket.

🍌 Banana picking task

For the banana picking pointcloud policy, we initially evaluate the base policy at a single location 10 times, where it successfully picks the banana 30% of the time. After searching for 10 tickets in the same location for 5 episodes each (50 episode rollouts total), we found a ticket that performed 100% in that location. We then evaluated that golden ticket and the base policy at 4 other locations on the table, 10 episodes each.

🎲 Gaussian Noise

Location 1 (❌ Grasp too high)
Location 2 (❌ Wrong orientation)
Location 3 (❌ Drops banana)

🎫 Golden Ticket

Location 1 (✅ Correct height)
Location 2 (✅ Correct orientation)
Location 3 (✅ Holds banana)

🥤 Push Cup

The goal of the push cup task is to push a cup to touch the red x on the table. For the cup pushing pointcloud policy, we evaluated the base policy 10 times and observed an average success rate of 40%. We searched for 10 tickets, with 5 rollouts, and found a ticket that achieved 100% success rate. When evaluated an additional 10 times, we found that it continued to achieve a 100% success rate. This constitutes our largest improvements on hardware, where the average success rate was increased by 60%.

🎲 Gaussian Noise

(❌ Too short)
(✅ Touches x)
(✅ Touches x)
(❌ Too short)
(❌ Too short)
(❌ Too short)
(❌ Too short)
(✅ Touches x)
(❌ Too short)
(✅ Touches x)

🎫 Golden Ticket

(✅ Touches x)
(✅ Touches x)
(✅ Touches x)
(✅ Touches x)
(✅ Touches x)
(✅ Touches x)
(✅ Touches x)
(✅ Touches x)
(✅ Touches x)
(✅ Touches x)

📦 Pick cube

When using standard Gaussian noise, our block-picking policy successfully picks up the cube 80% of the time (40 out of 50). After searching for 6 tickets, each with 25 episode rollouts, 150 episodes in total, we find a golden ticket (Ticket 5) that succeeded 25 out of 25 times, and also a ticket (Ticket 6) that succeeded only 1 out of 25 times). When we then evaluate those two tickets an additional 50 times, Ticket 5 successfully picked the cube 98% of the time (49 out of 50), where Ticket 6 only succeeded 2 out of 50 times. We therefore show we can drive success rate from 80% to 98%, an 18% increase, using just 150 search episodes, or conversely steer the policy to miss with extreme reliability.

🎲 Gaussian Noise (40 out of 50, 80% S.R)

🎫 Golden Ticket 5 (49 out of 50, 98% S.R)

🎫 Golden Ticket 6 (2 out of 50, 4% S.R)

Pareto Frontier for Multiple Reward Functions

We use the frankasim MuJoCo environment to train a single-arm Franka robot to pick a cube, which randomly spawns in a 0.5 square meter region in front of the robot. We train a flow matching policy that takes as input the low-dimensional state of the cube and robot.

pareto
Various tickets (pink) for the frankasim pick policy, evaluated according to success rate and speed (determined by length of successful episodes). Higher is better success rate, left is faster time to success. Because lottery tickets exhibit extreme differences in policy performance, a Pareto frontier is defined by tickets that are further left/up than others (represented with golden ticket icons).
Ticket A: Accurate, but slower
Ticket B: Fast, but less accurate
We see a large variation in ticket performance on metrics measuring picking accuracy and speed. This diversity results in a small subset of golden tickets that define a Pareto frontier, meaning that they outperform all other regular tickets to the right and below them, and represent a balance between the two objectives. This is a unique property of golden tickets compared to other RL methods: rather than designing multiple reward functions that balance these objectives differently, we naturally find tickets that balance them without any reward tuning. This suggests a simple way to alter a robot's behavior to adapt between objectives online: collect the golden tickets on the Pareto frontier, and switch between them as desired.

Citation

@article{patil2026you, title={You've Got a Golden Ticket: Improving Generative Robot Policies With A Single Noise Vector}, author={Patil, Omkar and Biza, Ondrej and Weng, Thomas and Schmeckpeper, Karl and Thomason, Wil and Zhang, Xiaohan and Sivakumar, Kausik and Walters, Robin and Gopalan, Nakul and Castro, Sebastian and Hart, Stephen and Rosen, Eric}, journal={arXiv preprint arXiv:2603.15757}, year={2026} }