sim2sim render benchmark — 2 scenes, 1 env

41 shown measurements · generated 2026-07-28T11:01:32 · fps are the no-frame-readback loop (frames stay on the GPU; GL renderers sync via a 1×1 readPixels); rows marked * were not re-measured and still include readback; rows marked † include it inherently (the renderer returns the frame as part of rendering) but it is ≤1% of the frame time.

Scene 1 — Cube drop (single rigid body)

Rasterized (native renderer)

Ray traced — engine's own path tracer

Scene 2 — LeRobot bipedal platform (STL meshes, 12 DOF, PD stance hold)

Rasterized (native renderer)

Ray traced — engine's own path tracer

Scene 3 — LeRobot × 2,048 parallel envs (physics throughput, no rendering)

Physics only — 2,048 parallel envs

Bars show env-steps/s — higher is better (physics only, no rendering; steady-state window with one device sync at the end. Per-engine dt differs — an env-step is one engine step, see the table).

Scene 3b — LeRobot × 4,096 parallel envs (RL-training scale)

Scene 4 — Unitree G1 (29-DOF humanoid) × 4,096 parallel envs

Startup — time to first physics step (LeRobot scene, single env)

Time to first step — single env, warm JIT caches

Wall time from process start to the first completed physics step, single env, warm JIT caches — the cost of every script rerun / config change. Bar segments: ■ imports · ■ engine init · ■ scene build + first step. Where a cold caches figure is shown, that's the same run with the engine's kernel cache emptied — the first-ever-run / post-upgrade cost. Hover a bar for exact segments.

Resource usage — cube scene, 480×368 @ 64 spp

The fps rows say how fast each engine is; they say nothing about what it costs to run. The three engines here are the ones that could actually be rebuilt and run on this machine — LuisaRender is not built and mjlab does not import, so they are absent rather than estimated. The picture is a set of trades, not a ranking: Genesis uses ~3.7× Nexus's host RAM but a third of its VRAM; Isaac costs ~12× Nexus's RAM (5.8 GB — the Kit runtime and USD stage) and works the GPU hardest (79%) while path-tracing slowest. VRAM is the constraint that decides how many instances fit on a card: at 3.4 GB Nexus fits ~9 on this 32 GB GPU, Genesis ~30.

Method: RAM is USS summed over the process tree (not RSS, which double-counts shared pages); VRAM is per-PID from nvidia-smi's compute-apps table; GPU util is device-wide and only sampled with the GPU otherwise idle. Peaks include one-time JIT / shader-cache / stage-load costs, deliberately — peak footprint is what sizes a machine. Reproduce with tools/resource_profile.py. Time to first frame is process start → first path-traced frame in hand, on warm caches: it is what you wait through before seeing anything, and the spread is 16×. It is not the same as the Startup rows further down — those stop at the first physics step and never touch the render pipeline, so they miss the RTX shader cache and the first path-trace accumulation. Cold is far worse: Genesis's Taichi JIT costs ~74 s and Isaac's first-ever launch compiles an RTX shader cache for ~2 min.

Physics vs render, separated

The headline fps folds physics and rendering together — and the rows do not fold them the same way. genesis_rt_native.py and isaac_rt_native.py time only the render call, so their published fps is render-only; nexus_rt_native.py and genesis_nyx_native.py time the whole loop. So the fps column above does not compare like with like, and separating the stages changes the conclusion: on the cube, Nyx and Nexus are within ~7% as path tracers (474 vs 442 render-only fps) rather than the ~1.7× the combined number suggests — and on the robot Nexus's renderer is the faster of the two (347 vs 229), losing only because its physics costs 24.4 ms/frame against Genesis's 3.1. Physics ms is derived from physics_steps_s at each script's steps-per-frame where segments were not recorded.

Cube drop — 480×368 @ 64 spp

LeRobot legs — 640×480 @ 32 spp

Genesis-Nyx — cost vs resolution (cube scene, 64 spp)

The cube ray-traced row above is measured at 480×368 because that is what every other native-RT row uses — but at that size the frame is small enough that fixed per-frame overhead, not ray tracing, sets the cost. Per-sample cost keeps falling as the frame grows, which means the 480×368 row understates Nyx: 28× the pixels costs only ~5× the time, and it still path-traces 2560×1920 at 91 fps. Reproduce with tools/nyx_res_sweep.py.

Scene parity across the ray-traced rows

The native-RT rows above do not light the same scene, so read that column as indicative rather than exact:

How much is the disagreement worth? LuisaRender, Isaac and Nexus cannot be re-measured on this machine (no from-source LuisaRender build, no isaac-venv, no nexus3d installed), so tools/nyx_light_sweep.py sweeps the regimes on Nyx instead. The answer: ~10% at a tracing-bound resolution, inside the noise at the row's 480×368 — and soft vs hard shadows costs nothing measurable. So lighting choice does not explain cross-engine gaps of 24×/58×. Isaac is the exception its own row note already records: its dome light costs ~73 ms/frame of env-sampling rebuild.

480x368 @ 64 spp

1920x1440 @ 64 spp

Videos

Scene 1 — Cube drop (single rigid body)

Rasterized (native renderer)

Ray traced — engine's own path tracer

Scene 2 — LeRobot bipedal platform (STL meshes, 12 DOF, PD stance hold)

Rasterized (native renderer)

Ray traced — engine's own path tracer

Scene 3 — LeRobot × 2,048 parallel envs (physics throughput, no rendering)

Scene 3b — LeRobot × 4,096 parallel envs (RL-training scale)

Scene 4 — Unitree G1 (29-DOF humanoid) × 4,096 parallel envs

Startup — time to first physics step (LeRobot scene, single env)

22 shown measurements · generated 2026-07-09T19:29:24 · fps are the no-frame-readback loop (frames stay on the GPU; GL renderers sync via a 1×1 readPixels); rows marked * were not re-measured and still include readback; rows marked † include it inherently (the renderer returns the frame as part of rendering) but it is ≤1% of the frame time.

Scene 1 — Cube drop (single rigid body)

Rasterized (native renderer)

Ray traced — engine's own path tracer

Scene 2 — LeRobot bipedal platform (STL meshes, 12 DOF, PD stance hold)

Rasterized (native renderer)

Ray traced — engine's own path tracer

Scene 3 — LeRobot × 2,048 parallel envs (physics throughput, no rendering)

Scene 3b — LeRobot × 4,096 parallel envs (RL-training scale)

Scene 4 — Unitree G1 (29-DOF humanoid) × 4,096 parallel envs

Startup — time to first physics step (LeRobot scene, single env)

Videos

Scene 1 — Cube drop (single rigid body)

Rasterized (native renderer)

Ray traced — engine's own path tracer

Scene 2 — LeRobot bipedal platform (STL meshes, 12 DOF, PD stance hold)

Rasterized (native renderer)

Ray traced — engine's own path tracer

Scene 3 — LeRobot × 2,048 parallel envs (physics throughput, no rendering)

Scene 3b — LeRobot × 4,096 parallel envs (RL-training scale)

Scene 4 — Unitree G1 (29-DOF humanoid) × 4,096 parallel envs

Startup — time to first physics step (LeRobot scene, single env)