# Isaac Lab + PPO for Unitree G1 Bottle Grasping and Lifting Reinforcement Learning Project: Starting to Build an MVP
This page has been translated by machine translation. View original
Introduction
In previous internship work, I developed a robot visual inspection agent for operator advisory use, leveraging camera images from the Unitree Go2 and G1.
Starting this time, as a new Physical AI project, I am working on reinforcement learning for bottle grasping and lifting targeting the Unitree G1 Inspire Hand.
This series is planned to proceed under the following title:
Isaac Lab and PPO-based Unitree G1 Bottle Pick-and-Lift Reinforcement Learning Project (G1 Bottle Pick-and-Lift Reinforcement Learning project)
In this project, PPO-based reinforcement learning is applied to a fixed standing G1 in Isaac Lab, targeting a bottle-shaped object on a table.
Extend hand toward the bottle
↓
Form a grasp pose with the Inspire Hand
↓
Lift from the table
↓
Retain
The ultimate goal includes sim-to-real, but currently we are in the simulation-first stage.
Rather than handling walking, autonomous navigation, handoff to humans, and online reinforcement learning from the start, the policy is to first confirm physically valid grasp and lift conditions in Isaac Lab, then build and evaluate reinforcement learning policies around those conditions.
This article summarizes what was implemented and verified during the first week of the project.
- RL environment in Isaac Lab using the Unitree G1 Inspire Hand
- Reasons for dividing the bottle grasping task into stages
- Scripted arm trajectory and 12D hand residual PPO
- Adjusting wrist / thumb / finger contact poses
- Contact and physics parameter diagnosis using AI as an assistant
- Scripted grasp / lift baseline
- PPO lift / hold showcase with fixed initial state
- Random placement curriculum to be pursued next week
Note that this project is independent from the previous Go2 inspection project.
The Go2's camera, detector, dashboard, and VLM / LLM outputs are not connected to the G1's locomotion, arm, hand, or grasp control.
Goal: Stationary Bottle Pick-and-Lift Reinforcement Learning with G1
The task addressed in this v1 involves a fixed standing G1 grasping, lifting, and holding a lightweight bottle-shaped object on a table.
fixed standing G1
↓
approach toward bottle-like object
↓
grasp formation
↓
lift
↓
retention above the table
The initial scope is limited to the following.
- fixed-base / fixed standing G1
- Inspire five-finger hand
- camera-free state-based observation
- DDS-free simulation environment
- teleoperation-free
- lightweight bottle-like rigid object
- large arm trajectory is scripted
- PPO handles residual control of 12 joints in the right hand
Online RL exploration on real hardware is not performed.
Also, success in simulation does not directly imply real hardware performance or safety.
Why We Did Not Start with 19D End-to-End PPO
Initially, a BottlePickLift task handling both the right arm's 7 degrees of freedom and the right hand's 12 degrees of freedom simultaneously was also prepared.
right arm:
7D
right Inspire Hand:
12D
total:
19D
However, bottle grasping involves the following challenges simultaneously.
arm approach
+ wrist orientation
+ thumb opposition
+ finger contact
+ object retention
+ lift trajectory
Having PPO explore all of these from scratch makes it difficult to isolate the cause when failures occur.
For example, even if the bottle is dropped, it becomes hard to determine whether:
- the arm position was off
- the wrist roll was incorrect
- the thumb closed too early
- finger contact was too weak
- the lift trajectory was too abrupt
For this reason, the task was divided into stages in this project.
Stage 1:
Create a safe pre-grasp pose
Stage 2:
Verify hand contact and retention
around the scripted arm trajectory
Stage 3:
Expand to broader approach + grasp + lift
Stage 4:
Introduce bounded object-position randomization
Currently, the large arm trajectory is fixed as scripted, and PPO is responsible for the 12D residual action of the right Inspire Hand.
The role of PPO is not to explore the entire arm motion from scratch, but to adjust finger contact, anti-slip, and retention around a known contact pose.
What Was Accomplished This Week
During the first week of the project, the following were primarily implemented and verified.
- Built a fixed-base RL task using the Unitree G1 Inspire Hand asset
- Built a scene using dynamic object physics and contact sensors
- Confirmed PPO training, checkpointing, deterministic playback, and MP4 recording
- Built a safe pre-grasp task accounting for self-collision
- Implemented a scripted arm phase controller for safe → ingress → grasp → lift
- Built a PPO task using 12D residual action for the right hand
- Confirmed failure where the thumb closes early and pushes the bottle away
- Introduced delayed hand activation and minimum pre-shape
- Diagnosed contact poses for wrist roll, thumb yaw, and thumb pitch
- Created a bottle-like rigid object based on the exterior of a 600 ml-class bottle
- Implemented a slow contact / squeeze / lift trajectory
- Confirmed formal retention-and-lift success with the scripted baseline
- Built a BottleSlowPPO task centered around the scripted baseline
- Ran a fresh PPO training with 128 parallel environments and 300 iterations
- Recorded a bottle lift / hold showcase with fixed initial state using deterministic PPO playback
Rather than simply running PPO, the major progress of the first week was being able to systematically organize what to learn by confirming the pre-contact pose, hand geometry, actual joint positions, and physical failures before contact was established.
Stage 1: Creating a Safe Final-Approach with the BottlePreGrasp Curriculum
In the first Stage 1, a BottlePreGrasp curriculum was built to safely move the G1's right arm close to the bottle as a precursor to grasping it.
The goal here is not simply to reach a target marker.
The aim is to create a physically realizable final-approach trajectory that maintains the hand orientation and arm pose leading to a subsequent grasp, without unnaturally passing through the robot torso.
First, the position of the green target marker was lowered to a position close to the lower-middle part of the bottle.
This changed the setting from simply extending the hand toward the top of the bottle to aiming for a pre-grasp geometry that leads to future thumb-to-finger opposition.
Initial 7D Arm-Only PPO
In the initial baseline, a 7D arm-only policy controlling the right arm's 7 degrees of freedom with PPO was used.
right arm action:
7D
wrist:
controlled by PPO
With this policy, smooth approach toward the target marker and stopping near it were confirmed.
However, when checking the grasp aperture geometry and performing deterministic playback, it became clear that because the wrist action was free, an unnatural approach from the web-of-thumb side was permitted.
In other words, it was found that reward for approaching the target alone does not necessarily yield a wrist pose suitable for the subsequent grasp.
Failure case 1: Unnatural web-of-thumb approach due to free wrist policy
In response to this result, a 4D FixedWrist curriculum was implemented, removing wrist yaw, roll, and pitch from the PPO action and controlling only the shoulder and elbow.
PPO action:
4D shoulder / elbow
wrist:
fixed
With the FixedWrist task, behavior of approaching the bottle direction with the wrist pose fixed was confirmed.
However, after longer training, deterministic playback revealed another problem.
A trajectory where the arm uses a self-intersection shortcut that unnaturally overlaps with the torso to reach near the target.
When looking only at the target reward it appears that approach succeeded, but in reality it is an unnatural pose that does not lead to subsequent manipulation.
Failure case 2: Self-intersection shortcut passing through the torso
To address this problem, articulation self-collision was enabled.
With self-collision enabled, trajectories that previously crossed through the torso could no longer be executed.
On the other hand, it also became clear that having PPO explore from the default pose alone makes it difficult to find a legitimate trajectory that approaches the bottle while avoiding self-collision.
Therefore, it was first confirmed that a natural approach to the bottle using scripted actions was possible with self-collision enabled.
A safe early arm pose was extracted from that trajectory and set as the reset / default pose for the new self-collision curriculum task.
Ultimately, the Stage 1 task was organized as follows.
task:
BottlePreGraspFixedWristSelfCollision
PPO action:
4D shoulder / elbow
wrist:
fixed open pose
self-collision:
enabled
reset pose:
safe early arm pose extracted from scripted diagnostic
This task emphasizes the arm approaching the bottle from outside the torso.
In the final stochastic rollout of the best run, the following value was confirmed.
pregrasp_success:
0.5625
This result does not mean grasp or lift success.
However, the achievement of Stage 1 is the improvement to an RL curriculum that maintains physically realizable trajectories including self-collision, rather than simply maximizing reward.
Stage 1: Self-collision enabled safe pre-grasp demo
Stage 2: Scripted Arm Trajectory and 12D Hand Residual PPO
In Stage 2, the large arm movements are not delegated to PPO but given as scripted phase controller.
Phase 0:
safe pose
Phase 1:
ingress pose
Phase 2:
grasp pose
Phase 3:
lift / hold pose
What PPO outputs is a 12D residual action for the right Inspire Hand.
state observation:
53D
PPO action:
right hand residual 12D
residual scale:
0.08 rad
Conceptually, the configuration is as follows.
scripted arm trajectory
+ scripted hand target
+ PPO hand residual
↓
contact adaptation / anti-slip correction
With this configuration, the policy does not rediscover broad arm approach movements, but instead learns small adjustments of the fingers around contact during a known grasping trajectory.
First Observed Failure: Thumb Closes Early and Pushes the Bottle Away
In the initial hand-only PPO, a failure was observed where the thumb began closing before ingress was complete, pushing the bottle laterally.
thumb closes too early
↓
object is pushed away
↓
unstable contact / object-out-of-workspace
To address this problem, the hand action was phase-gated.
before grasp phase:
default / open hand target
from grasp phase:
scripted pre-shape + PPO residual
Furthermore, a clamp was added to prevent the PPO residual from canceling out the finger pre-shape and reopening the fingers during the grasp / lift phase, so that it does not fall below the minimum pre-shape.
target:
max(scripted pre-shape, scripted pre-shape + PPO residual)
With this change, object-out-of-workspace due to early thumb push was greatly reduced.
Failure demo: Thumb closes early and pushes the bottle
Contact and Physics Parameter Diagnosis Using AI as an Assistant
The adjustment of physical contact, joint limits, thumb opposition, hand stiffness, and trajectory timing for robot hands is not my area of expertise.
Therefore, this time I proceeded by using AI as a supplementary technical partner to form physical hypotheses and verify them with diagnostic scripts in Isaac Lab.
However, rather than adopting AI proposals as-is, the following was always confirmed.
AI-assisted hypothesis
↓
create scripted diagnostic
↓
confirm action target
↓
confirm actual joint position
↓
view contact geometry in video
↓
confirm termination result
What was particularly important was the realization that having a joint target set and that joint actually moving as expected are separate things.
At the time of contact, some finger intermediate joints and thumb joints did not reach the target, being influenced by contact geometry and collision.
Therefore, rather than simply continuing to increase target angles, the relative positions of the wrist, thumb, and four fingers, and the trajectory up to contact, were reconsidered.
Adjusting Wrist Roll and Thumb Opposition
As a result of confirming contact, a form closer to side-opposition facing the finger bank from the side was more promising than a pose where the thumb presses the bottle from above.
The representative settings used in the scripted diagnostic are as follows.
wrist roll:
-0.25 rad
thumb yaw:
+0.50 rad
thumb pitch:
approximately +0.025 rad
contact close:
four fingers: +0.42 rad
thumb intermediate: +0.32 rad
thumb distal: +0.40 rad
deeper squeeze:
four fingers: +0.70 rad
thumb intermediate: +0.38 rad
thumb distal: +0.46 rad
hand stiffness:
50
These values are not optimal values for real hardware, but values confirmed for the current simulation asset, bottle geometry, and fixed initial state.
Also, since thumb pitch is near the soft joint limit, the diagnostic confirmed not only the requested target but also the effective target after clamping.
Creating a 600 ml-class Bottle-like Object
In the early stages, a smooth cylinder proxy was also used, but I wanted to also confirm contact with a shape having bottle-like shoulders, neck, and cap.
Therefore, a bottle-like object moving as a single rigid body was prepared.
type:
single rigid-body bottle-like object
approximate total height:
0.225 m
main body diameter:
0.068 m
mass:
0.08 kg
The exterior is based on a common round PET bottle of 600 ml-class.
However, this is a simulation proxy with an approximated exterior.
It does not fully reproduce the mass distribution, flexibility, surface friction, or liquid movement of an actual commercially available PET bottle filled with beverage.
Abrupt Close Failure and Slow Contact Trajectory
When the hand target was switched abruptly to the close side, the bottle sometimes toppled at the first contact.
abrupt close
↓
first contact shock
↓
bottle topples / slips
From this failure, it became clear that the temporal process of reaching the final close target, not just the target itself, is important.
Therefore, the scripted grasp sequence was changed as follows.
step 80:
thumb side preposition
step 120–200:
slow contact ramp
step 201–270:
slow squeeze ramp
step 270–300:
squeeze hold
step 300–360:
slow arm lift ramp
step 360 onward:
lift hold
This slow trajectory is the result of verifying in simulation diagnostics the hypothesis—formed with AI as an assistant—that "reducing contact shock and establishing stable contact first" is important.
Failure demo: Bottle topples with abrupt close
Slow contact / squeeze / lift demo
Confirming Formal Retention-and-Lift Success with the Scripted Baseline
Using the slow contact / slow squeeze / slow lift sequence, the scripted baseline was run on the bottle task with a fixed initial state.
In this task, success is not counted merely by the object moving slightly or fingers touching.
The success condition includes conditions such as the following.
lift phase
+ bottle center height exceeds the resting reference by at least 1 cm
+ hand-to-object distance < 0.09 m
+ object uprightness >= 0.85
+ object speed < 0.15 m/s
+ 24 consecutive valid steps
In the scripted diagnostic, the following was confirmed.
CONTACT_TARGET_VERIFICATION:
PASS
SQUEEZE_TARGET_VERIFICATION:
PASS
LIFT_TARGET_VERIFICATION:
PASS
retention_lift_success:
True
object_out_of_workspace:
False
maximum bottle-center lift delta:
+0.019683 m
The maximum bottle center lift delta was approximately 1.97 cm.
In other words, although in a fixed initial state and single simulator configuration, a scripted simulation baseline of lifting the bottle from the table and holding it was confirmed with the current wrist and thumb pose and slow trajectory.
What was confirmed here is a physically valid baseline for subsequent RL.
Scripted bottle grasp / lift baseline demo
BottleSlowPPO Centered Around the Scripted Baseline
After the scripted baseline was established, a task was created to learn a 12D residual hand policy around that trajectory.
task:
BottleGraspRetentionUnderLiftBottleSlowPPO
arm:
scripted safe / ingress / slow lift
hand:
side-open → contact ramp → squeeze ramp
+ 12D PPO residual
object:
same bottle-like rigid object
hand stiffness:
50
PPO is not a policy that finds the bottle and moves the arm broadly.
The current objective is to confirm whether the following effects can be learned around the scripted baseline.
contact adaptation
anti-slip correction
small finger adjustments
retention support during lift
Training was run under the following conditions.
parallel environments:
128
iterations:
300
total timesteps:
921,600
action:
12D right-hand residual
PPO Training and Deterministic Showcase
In the fresh PPO training over 300 iterations, the complete workflow of training, checkpoint saving, deterministic playback, and MP4 recording was confirmed.
In the final stochastic rollout, the following values were obtained.
mean reward:
+1.69
lift_phase_retention:
+0.0586
object_out_of_workspace:
0.0547
retention_lift_success:
0.0000
Formal retention success in stochastic rollout has not yet been confirmed.
On the other hand, when model_299 was played back deterministically with a fixed initial state, behavior of the bottle being lifted was confirmed.
Furthermore, when recorded in a showcase task with only the immediate reset on success disabled for demo purposes, a clip of approximately 10 seconds showing the bottle being grasped, lifted, and held while maintaining dynamic physics and PPO inference was recorded.
The current primary demo is described as follows.
G1 Inspire-Hand Bottle Lift with PPO Residual Control
Isaac Lab Simulation — Fixed Initial Bottle State
Showcase Evaluation Conditions
The current PPO showcase is run under the following fixed conditions.
robot:
fixed-base G1
object:
fixed initial bottle state
arm:
scripted bounded trajectory
policy:
deterministic PPO checkpoint playback
hand control:
12D Inspire Hand residual action
observation:
camera-free state-based observation
This video does not include random placement, camera input, or real hardware evaluation.
Current PPO showcase: Lift / hold with fixed initial bottle state
The following is confirmed in this video.
deterministic PPO inference
+ scripted arm trajectory
+ dynamic bottle physics
+ visible grasp
+ tabletop lift
+ visible hold
On the other hand, it does not currently demonstrate the following.
stochastic formal success
random-placement robustness
arbitrary bottle generalization
real robot performance
sim-to-real transfer
Current Configuration
The current pipeline is conceptually as follows.
G1 Inspire Hand asset
↓
fixed-base Isaac Lab scene
↓
scripted safe / ingress / grasp / lift phases
↓
bottle-like dynamic rigid object
↓
state observation (53D)
↓
12D right-hand PPO residual
↓
retention-and-lift termination
↓
checkpoint / deterministic playback / MP4
In the previous Go2 inspection project, camera input, object detection, ROI geometry, and operator advisory were handled.
In this project, rather than directly controlling the robot with cameras or LLMs, the focus is on the following elements in simulation.
robot hand kinematics
contact geometry
rigid-body dynamics
reward / termination design
PPO training
deterministic evaluation
This Week's Achievements and Current Limitations
As progress during the first week of the project, the following were confirmed.
- Isaac Lab RL environment using G1 Inspire Hand
- Pre-grasp task accounting for self-collision
- Scripted arm phase controller
- 12D hand residual PPO
- Diagnostic of wrist / thumb / finger contact poses
- Slow contact / squeeze / lift trajectory
- Scripted retention-and-lift baseline with bottle-like object
- PPO training / checkpoint / playback / video workflow
- Deterministic PPO lift-and-hold showcase with fixed initial state
On the other hand, the following remain as future challenges.
- Formal retention success in stochastic rollout
- Quantitative comparison of PPO residual against zero-residual scripted baseline
- Success rate when the bottle's initial position is varied
- Evaluation with held-out offsets
- Mass / friction variation
- Perception integration
- Sim-to-real transfer
- Real G1 evaluation
The current achievements do not represent the stage where "G1 can autonomously grasp a bottle at an arbitrary position."
More precisely, we have reached the following stage.
A physically valid scripted grasp / lift baseline for a bottle-like object at a fixed position has been created, and an environment has been built where 12D hand residual PPO can be trained, played back, and visualized around that baseline.
Next Week: Moving to Random-Placement Curriculum
Next week, the plan is to proceed to a random-placement curriculum that slightly varies the bottle position on the table, rather than only using a fixed initial position.
Rather than introducing large randomization from the start, we will begin from a range reachable by the current scripted arm trajectory.
The initial plan is as follows.
bottle x:
nominal position ± 0.015 m
bottle y:
nominal position ± 0.015 m
yaw:
fixed initially
mass / friction:
fixed initially
The current policy controls only the hand residual, and the arm is scripted.
Therefore, what we want to verify at the next stage is not "whether a bottle at an arbitrary position can be retrieved," but rather how far hand contact can accommodate small positional deviations near the nominal grasp geometry.
We also plan to compare the zero-residual scripted baseline with the fixed state against the deterministic PPO policy, to confirm what effect the PPO residual has relative to the scripted prior.
After that, we will progress toward an environment where success rate, termination distribution, and video can be evaluated separately, using held-out offsets.
Summary
This time, a reinforcement learning project for bottle grasping and lifting in Isaac Lab targeting the Unitree G1 Inspire Hand was started.
In the first week of the project, the following workflow was built.
safe pre-grasp
↓
scripted arm phases
↓
AI-assisted contact geometry diagnosis
↓
slow contact / squeeze / lift baseline
↓
formal retention-and-lift validation
↓
12D hand residual PPO
↓
deterministic playback / showcase video
In particular, for robot hand tasks involving contact, I realized the importance of using AI as an assistant to confirm wrist roll, thumb opposition, finger target, stiffness, and trajectory timing one by one through diagnostics, rather than simply running PPO for a long time.
At this point, we have confirmed a scripted baseline in a fixed initial state and a deterministic lift / hold showcase incorporating PPO.
Next week, a small bounded random-placement curriculum will be introduced, gradually expanding to conditions beyond just a fixed position.
Without equating simulation success directly with real hardware performance or safety, we will continue advancing the G1 bottle grasping and lifting reinforcement learning project step by step, separating the scope of tasks, evaluation conditions, and failure cases.