feat: sim2real domain randomization + reward fixes for rotary cartpole
Close the sim2real gap for the Furuta pendulum (swings up but can't balance on hardware). Root causes were (a) no domain randomization, so the policy overfit one deterministic sim instance, and (b) reward design flaws that produced degenerate policies. Domain randomization (runner-level, backend-agnostic): - BaseRunner: domain_rand config; per-env action-delay buffer (latency), Gaussian qpos/qvel sensor noise, per-env dynamics-scale sampling (friction/damping/torque), resampled per episode. Sensor noise per step. - privileged_obs/privileged_dim expose normalized DR factors (mu) for RMA. - step() now uses clean state for reward/termination, noisy state for the observation the policy sees. - MuJoCoRunner: applies per-env friction/damping/torque scales. - robot.py: compute_motor_force gains friction/damping scale args. - Configs: DR blocks for mujoco (full) and mjx (delay+noise); clean defaults for mujoco_single/serial; noise/delay anchored to recordings. Reward fixes (rotary_cartpole): - Shift upright reward to [0,1] (was [-1,1]) + alive_bonus, so surviving always beats ending early (kills the "suicide into the limit" policy). - Add balance_bonus * upright * stillness so reward requires upright AND near-zero pendulum velocity (kills the "spin in full loops" policy). Deploy: - eval.py load_policy reconstructs the history/adaptation encoder (auto-detects its dim from the checkpoint) so DR+embedding policies load. Fixes: - MuJoCoRunner._sim_reset referenced self._env (typo) -> self.env, which was breaking every rotary-cartpole reset. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,23 +1,21 @@
|
||||
# Tuned robot config — generated by src.sysid (2026-03-28)
|
||||
# Unified sysid: cost 0.925 (28 params, 0.2s windows, amplitude 200)
|
||||
# Tuned robot config — generated by src.sysid.optimize
|
||||
|
||||
urdf: rotary_cartpole_tuned.urdf
|
||||
urdf: rotary_cartpole.urdf
|
||||
actuators:
|
||||
- joint: motor_joint
|
||||
type: motor
|
||||
gear: [0.846499, 1.183733] # torque constant [pos, neg]
|
||||
ctrl_range: [-0.686251, 0.686251] # effective control bound
|
||||
deadzone: [0.181097, 0.202072] # L298N min |ctrl| for torque [pos, neg]
|
||||
damping: [0.013165, 0.015452] # viscous damping [pos, neg]
|
||||
frictionloss: [0.014244, 0.001005] # Coulomb friction [pos, neg]
|
||||
filter_tau: 0.096263 # 1st-order actuator filter
|
||||
stribeck_friction_boost: 0.068594 # low-velocity friction boost
|
||||
stribeck_vel: 5.279594 # Stribeck velocity scale (rad/s)
|
||||
action_bias: 0.056566 # asymmetric bias offset
|
||||
gear: [0.424182, 0.425031] # torque constant [pos, neg] (motor sysid)
|
||||
ctrl_range: [-0.592, 0.592] # effective control bound (sysid-tuned)
|
||||
deadzone: [0.141291, 0.078015] # L298N min |ctrl| for torque [pos, neg]
|
||||
damping: [0.002027, 0.014665] # viscous damping [pos, neg]
|
||||
frictionloss: [0.057328, 0.053355] # Coulomb friction [pos, neg]
|
||||
filter_tau: 0.005035 # 1st-order actuator filter (motor sysid)
|
||||
viscous_quadratic: 0.000285 # velocity² drag
|
||||
back_emf_gain: 0.006758 # back-EMF torque reduction
|
||||
joints:
|
||||
motor_joint:
|
||||
armature: 0.001676 # reflected rotor inertia
|
||||
armature: 0.002773 # reflected rotor inertia (motor sysid)
|
||||
frictionloss: 0.0 # handled by motor model via qfrc_applied
|
||||
pendulum_joint:
|
||||
damping: 1.2e-05 # pendulum pivot damping
|
||||
frictionloss: 7.2e-05 # pendulum pivot friction
|
||||
damping: 0.000119
|
||||
frictionloss: 1.0e-05
|
||||
|
||||
Reference in New Issue
Block a user