Skip to content

[Feature] CISPO #3207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: gh/vmoens/162/base
Choose a base branch
from
Open

[Feature] CISPO #3207

wants to merge 6 commits into from
+189 −21

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 16, 2025 •
edited
Loading

@vmoens
Update
1794f98 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 16, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3207

Note: Links to docs will display an error until the docs builds have been completed.

❌ 11 New Failures, 4 Unrelated Failures

As of commit d867669 with merge base 13434eb (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 16, 2025
This was referenced Oct 16, 2025
Merged
Merged
Open
Open
@vmoens
Update
5eab45a 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 23, 2025
Merged
@vmoens
Update
66f6f4d 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 23, 2025
Open
@github-actions
Copy link

github-actions bot commented Oct 23, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 85.2928μs 83.5859μs 11.9637 KOps/s 11.9638 KOps/s $-0.00\%$
test_tensor_to_bytestream_speed[torch.save] 0.1448ms 0.1434ms 6.9757 KOps/s 7.0765 KOps/s $\color{#d91a1a}-1.42\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1189s 0.1184s 8.4434 Ops/s 8.8714 Ops/s $\color{#d91a1a}-4.82\%$
test_tensor_to_bytestream_speed[numpy] 2.7903μs 2.7885μs 358.6200 KOps/s 352.6323 KOps/s $\color{#35bf28}+1.70\%$
test_tensor_to_bytestream_speed[safetensors] 44.6681μs 42.9100μs 23.3046 KOps/s 23.2856 KOps/s $\color{#35bf28}+0.08\%$
test_simple 0.5677s 0.5566s 1.7967 Ops/s 1.7266 Ops/s $\color{#35bf28}+4.06\%$
test_transformed 1.2340s 1.1391s 0.8779 Ops/s 0.8751 Ops/s $\color{#35bf28}+0.32\%$
test_serial 1.6880s 1.6812s 0.5948 Ops/s 0.5852 Ops/s $\color{#35bf28}+1.64\%$
test_parallel 1.1774s 1.0942s 0.9139 Ops/s 0.8923 Ops/s $\color{#35bf28}+2.43\%$
test_step_mdp_speed[True-True-True-True-True] 0.1921ms 46.4921μs 21.5090 KOps/s 21.2543 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[True-True-True-True-False] 0.4404ms 26.3410μs 37.9636 KOps/s 38.2638 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[True-True-True-False-True] 0.4441ms 26.0460μs 38.3937 KOps/s 37.8191 KOps/s $\color{#35bf28}+1.52\%$
test_step_mdp_speed[True-True-True-False-False] 0.4124ms 14.5059μs 68.9373 KOps/s 68.8312 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[True-True-False-True-True] 84.4710μs 49.9911μs 20.0036 KOps/s 19.8714 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[True-True-False-True-False] 0.4244ms 29.1971μs 34.2499 KOps/s 34.7769 KOps/s $\color{#d91a1a}-1.52\%$
test_step_mdp_speed[True-True-False-False-True] 0.4336ms 29.2806μs 34.1523 KOps/s 34.3023 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[True-True-False-False-False] 0.4290ms 17.5984μs 56.8234 KOps/s 57.5510 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[True-False-True-True-True] 99.6620μs 51.6728μs 19.3525 KOps/s 19.1274 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-False-True-True-False] 61.9510μs 31.5622μs 31.6835 KOps/s 31.6520 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[True-False-True-False-True] 52.6610μs 28.7830μs 34.7427 KOps/s 34.1618 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[True-False-True-False-False] 44.1610μs 17.3830μs 57.5276 KOps/s 57.1759 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[True-False-False-True-True] 96.4520μs 54.6464μs 18.2995 KOps/s 17.9940 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[True-False-False-True-False] 66.5410μs 34.9983μs 28.5728 KOps/s 28.8694 KOps/s $\color{#d91a1a}-1.03\%$
test_step_mdp_speed[True-False-False-False-True] 62.4410μs 31.9493μs 31.2996 KOps/s 31.0711 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-False-False-False-False] 48.1510μs 20.2468μs 49.3904 KOps/s 49.5559 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-True-True-True-True] 89.8310μs 52.5656μs 19.0239 KOps/s 19.0376 KOps/s $\color{#d91a1a}-0.07\%$
test_step_mdp_speed[False-True-True-True-False] 54.8910μs 32.5376μs 30.7336 KOps/s 31.3442 KOps/s $\color{#d91a1a}-1.95\%$
test_step_mdp_speed[False-True-True-False-True] 2.3913ms 33.8402μs 29.5507 KOps/s 29.6792 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[False-True-True-False-False] 45.8110μs 19.5915μs 51.0426 KOps/s 51.3516 KOps/s $\color{#d91a1a}-0.60\%$
test_step_mdp_speed[False-True-False-True-True] 81.7820μs 55.6872μs 17.9574 KOps/s 17.9062 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-True-False-True-False] 65.4910μs 34.6913μs 28.8257 KOps/s 28.9918 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[False-True-False-False-True] 70.5120μs 36.4869μs 27.4071 KOps/s 27.6032 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[False-True-False-False-False] 51.8610μs 22.3133μs 44.8163 KOps/s 44.5265 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-False-True-True-True] 88.6010μs 58.5293μs 17.0855 KOps/s 16.8874 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-False-True-True-False] 71.4410μs 37.9249μs 26.3679 KOps/s 26.4472 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[False-False-True-False-True] 0.1147ms 36.1136μs 27.6904 KOps/s 27.4245 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-False-True-False-False] 47.0100μs 22.0798μs 45.2903 KOps/s 45.2087 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[False-False-False-True-True] 0.1119ms 60.6088μs 16.4993 KOps/s 16.3476 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[False-False-False-True-False] 67.4310μs 40.1472μs 24.9083 KOps/s 24.8193 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-False-False-True] 65.9410μs 38.9322μs 25.6857 KOps/s 25.5559 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[False-False-False-False-False] 69.4820μs 24.7419μs 40.4173 KOps/s 40.3092 KOps/s $\color{#35bf28}+0.27\%$
test_values[generalized_advantage_estimate-True-True] 9.9533ms 9.4257ms 106.0934 Ops/s 106.7210 Ops/s $\color{#d91a1a}-0.59\%$
test_values[vec_generalized_advantage_estimate-True-True] 17.7403ms 11.4384ms 87.4249 Ops/s 56.4802 Ops/s $\textbf{\color{#35bf28}+54.79\%}$
test_values[td0_return_estimate-False-False] 0.2425ms 0.1288ms 7.7635 KOps/s 7.6870 KOps/s $\color{#35bf28}+1.00\%$
test_values[td1_return_estimate-False-False] 26.6006ms 25.6933ms 38.9207 Ops/s 39.2075 Ops/s $\color{#d91a1a}-0.73\%$
test_values[vec_td1_return_estimate-False-False] 11.6780ms 11.3210ms 88.3314 Ops/s 56.4626 Ops/s $\textbf{\color{#35bf28}+56.44\%}$
test_values[td_lambda_return_estimate-True-False] 40.1777ms 37.9252ms 26.3677 Ops/s 26.2128 Ops/s $\color{#35bf28}+0.59\%$
test_values[vec_td_lambda_return_estimate-True-False] 17.8304ms 11.4575ms 87.2789 Ops/s 56.2112 Ops/s $\textbf{\color{#35bf28}+55.27\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.7779ms 8.0034ms 124.9474 Ops/s 123.9518 Ops/s $\color{#35bf28}+0.80\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7103ms 1.5444ms 647.5043 Ops/s 655.4020 Ops/s $\color{#d91a1a}-1.21\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8548ms 0.4075ms 2.4539 KOps/s 2.4758 KOps/s $\color{#d91a1a}-0.88\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 30.0923ms 29.4563ms 33.9486 Ops/s 28.8762 Ops/s $\textbf{\color{#35bf28}+17.57\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.1955ms 1.7359ms 576.0586 Ops/s 576.8041 Ops/s $\color{#d91a1a}-0.13\%$
test_dqn_speed[False-None] 6.3417ms 1.4416ms 693.6788 Ops/s 705.4486 Ops/s $\color{#d91a1a}-1.67\%$
test_dqn_speed[False-backward] 1.9532ms 1.8936ms 528.1048 Ops/s 525.3698 Ops/s $\color{#35bf28}+0.52\%$
test_dqn_speed[True-None] 0.9113ms 0.5063ms 1.9750 KOps/s 1.9448 KOps/s $\color{#35bf28}+1.55\%$
test_dqn_speed[True-backward] 1.0059ms 0.9586ms 1.0432 KOps/s 1.0172 KOps/s $\color{#35bf28}+2.56\%$
test_dqn_speed[reduce-overhead-None] 0.9252ms 0.5027ms 1.9892 KOps/s 1.9304 KOps/s $\color{#35bf28}+3.05\%$
test_dqn_speed[reduce-overhead-backward] 1.0034ms 0.9464ms 1.0566 KOps/s 946.7579 Ops/s $\textbf{\color{#35bf28}+11.60\%}$
test_ddpg_speed[False-None] 3.3001ms 2.8766ms 347.6383 Ops/s 343.9472 Ops/s $\color{#35bf28}+1.07\%$
test_ddpg_speed[False-backward] 4.5624ms 4.0900ms 244.5011 Ops/s 246.4688 Ops/s $\color{#d91a1a}-0.80\%$
test_ddpg_speed[True-None] 1.8309ms 1.3911ms 718.8614 Ops/s 723.8399 Ops/s $\color{#d91a1a}-0.69\%$
test_ddpg_speed[True-backward] 2.4704ms 2.3719ms 421.6003 Ops/s 420.2814 Ops/s $\color{#35bf28}+0.31\%$
test_ddpg_speed[reduce-overhead-None] 1.5474ms 1.3757ms 726.9270 Ops/s 734.0594 Ops/s $\color{#d91a1a}-0.97\%$
test_ddpg_speed[reduce-overhead-backward] 2.4034ms 2.3462ms 426.2216 Ops/s 391.2844 Ops/s $\textbf{\color{#35bf28}+8.93\%}$
test_sac_speed[False-None] 8.2589ms 7.8292ms 127.7272 Ops/s 127.8056 Ops/s $\color{#d91a1a}-0.06\%$
test_sac_speed[False-backward] 11.3567ms 10.9679ms 91.1755 Ops/s 90.9559 Ops/s $\color{#35bf28}+0.24\%$
test_sac_speed[True-None] 2.2412ms 2.1153ms 472.7567 Ops/s 468.6294 Ops/s $\color{#35bf28}+0.88\%$
test_sac_speed[True-backward] 4.3242ms 4.0654ms 245.9807 Ops/s 225.3508 Ops/s $\textbf{\color{#35bf28}+9.15\%}$
test_sac_speed[reduce-overhead-None] 2.5307ms 2.1086ms 474.2415 Ops/s 459.9609 Ops/s $\color{#35bf28}+3.10\%$
test_sac_speed[reduce-overhead-backward] 4.1680ms 4.0393ms 247.5699 Ops/s 241.5823 Ops/s $\color{#35bf28}+2.48\%$
test_redq_speed[False-None] 11.1154ms 10.4008ms 96.1461 Ops/s 94.1237 Ops/s $\color{#35bf28}+2.15\%$
test_redq_speed[False-backward] 18.7772ms 17.8273ms 56.0938 Ops/s 54.9019 Ops/s $\color{#35bf28}+2.17\%$
test_redq_speed[True-None] 4.8416ms 4.4478ms 224.8280 Ops/s 222.4757 Ops/s $\color{#35bf28}+1.06\%$
test_redq_speed[True-backward] 10.2879ms 9.8678ms 101.3400 Ops/s 101.6349 Ops/s $\color{#d91a1a}-0.29\%$
test_redq_speed[reduce-overhead-None] 4.8126ms 4.3784ms 228.3956 Ops/s 217.9767 Ops/s $\color{#35bf28}+4.78\%$
test_redq_speed[reduce-overhead-backward] 10.4378ms 10.1426ms 98.5937 Ops/s 99.5992 Ops/s $\color{#d91a1a}-1.01\%$
test_redq_deprec_speed[False-None] 11.4620ms 10.9975ms 90.9297 Ops/s 90.4213 Ops/s $\color{#35bf28}+0.56\%$
test_redq_deprec_speed[False-backward] 16.5008ms 15.9068ms 62.8661 Ops/s 63.0233 Ops/s $\color{#d91a1a}-0.25\%$
test_redq_deprec_speed[True-None] 4.0844ms 3.6377ms 274.8967 Ops/s 266.3176 Ops/s $\color{#35bf28}+3.22\%$
test_redq_deprec_speed[True-backward] 8.3200ms 7.7141ms 129.6327 Ops/s 124.0795 Ops/s $\color{#35bf28}+4.48\%$
test_redq_deprec_speed[reduce-overhead-None] 4.0203ms 3.5987ms 277.8816 Ops/s 271.3037 Ops/s $\color{#35bf28}+2.42\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.9190ms 7.6737ms 130.3152 Ops/s 118.6874 Ops/s $\textbf{\color{#35bf28}+9.80\%}$
test_td3_speed[False-None] 8.1541ms 7.8788ms 126.9234 Ops/s 120.9572 Ops/s $\color{#35bf28}+4.93\%$
test_td3_speed[False-backward] 11.2533ms 10.6897ms 93.5476 Ops/s 93.7738 Ops/s $\color{#d91a1a}-0.24\%$
test_td3_speed[True-None] 1.8547ms 1.8098ms 552.5351 Ops/s 554.5677 Ops/s $\color{#d91a1a}-0.37\%$
test_td3_speed[True-backward] 3.8070ms 3.5619ms 280.7507 Ops/s 248.4738 Ops/s $\textbf{\color{#35bf28}+12.99\%}$
test_td3_speed[reduce-overhead-None] 1.8369ms 1.7681ms 565.5710 Ops/s 554.7821 Ops/s $\color{#35bf28}+1.94\%$
test_td3_speed[reduce-overhead-backward] 3.7287ms 3.5971ms 278.0033 Ops/s 274.0711 Ops/s $\color{#35bf28}+1.43\%$
test_cql_speed[False-None] 29.2076ms 26.1123ms 38.2962 Ops/s 37.4785 Ops/s $\color{#35bf28}+2.18\%$
test_cql_speed[False-backward] 39.3651ms 35.6590ms 28.0434 Ops/s 28.3001 Ops/s $\color{#d91a1a}-0.91\%$
test_cql_speed[True-None] 12.9179ms 12.3085ms 81.2447 Ops/s 80.4438 Ops/s $\color{#35bf28}+1.00\%$
test_cql_speed[True-backward] 18.7940ms 18.2795ms 54.7061 Ops/s 55.8931 Ops/s $\color{#d91a1a}-2.12\%$
test_cql_speed[reduce-overhead-None] 15.7636ms 12.7881ms 78.1978 Ops/s 80.4331 Ops/s $\color{#d91a1a}-2.78\%$
test_cql_speed[reduce-overhead-backward] 19.3388ms 18.7732ms 53.2674 Ops/s 55.2870 Ops/s $\color{#d91a1a}-3.65\%$
test_a2c_speed[False-None] 5.9444ms 5.4700ms 182.8156 Ops/s 181.6983 Ops/s $\color{#35bf28}+0.61\%$
test_a2c_speed[False-backward] 12.1946ms 11.8552ms 84.3513 Ops/s 84.3232 Ops/s $\color{#35bf28}+0.03\%$
test_a2c_speed[True-None] 4.1881ms 3.7485ms 266.7768 Ops/s 265.1275 Ops/s $\color{#35bf28}+0.62\%$
test_a2c_speed[True-backward] 9.2086ms 8.6125ms 116.1108 Ops/s 111.8901 Ops/s $\color{#35bf28}+3.77\%$
test_a2c_speed[reduce-overhead-None] 4.2097ms 3.7335ms 267.8448 Ops/s 269.4518 Ops/s $\color{#d91a1a}-0.60\%$
test_a2c_speed[reduce-overhead-backward] 9.4312ms 8.9211ms 112.0934 Ops/s 111.3907 Ops/s $\color{#35bf28}+0.63\%$
test_ppo_speed[False-None] 6.1322ms 5.8915ms 169.7372 Ops/s 168.7382 Ops/s $\color{#35bf28}+0.59\%$
test_ppo_speed[False-backward] 13.0117ms 12.4921ms 80.0508 Ops/s 80.2396 Ops/s $\color{#d91a1a}-0.24\%$
test_ppo_speed[True-None] 3.8300ms 3.6520ms 273.8241 Ops/s 270.4044 Ops/s $\color{#35bf28}+1.26\%$
test_ppo_speed[True-backward] 9.1547ms 8.5599ms 116.8240 Ops/s 117.7469 Ops/s $\color{#d91a1a}-0.78\%$
test_ppo_speed[reduce-overhead-None] 4.1756ms 3.6653ms 272.8321 Ops/s 271.6588 Ops/s $\color{#35bf28}+0.43\%$
test_ppo_speed[reduce-overhead-backward] 8.9021ms 8.7108ms 114.8000 Ops/s 98.3509 Ops/s $\textbf{\color{#35bf28}+16.72\%}$
test_reinforce_speed[False-None] 5.3065ms 4.6424ms 215.4039 Ops/s 216.7251 Ops/s $\color{#d91a1a}-0.61\%$
test_reinforce_speed[False-backward] 7.9107ms 7.4731ms 133.8138 Ops/s 133.8801 Ops/s $\color{#d91a1a}-0.05\%$
test_reinforce_speed[True-None] 3.4339ms 2.9055ms 344.1792 Ops/s 362.3060 Ops/s $\textbf{\color{#d91a1a}-5.00\%}$
test_reinforce_speed[True-backward] 7.9574ms 7.7184ms 129.5597 Ops/s 119.9115 Ops/s $\textbf{\color{#35bf28}+8.05\%}$
test_reinforce_speed[reduce-overhead-None] 3.0570ms 2.8719ms 348.2011 Ops/s 348.0098 Ops/s $\color{#35bf28}+0.05\%$
test_reinforce_speed[reduce-overhead-backward] 8.1323ms 7.9207ms 126.2515 Ops/s 115.7183 Ops/s $\textbf{\color{#35bf28}+9.10\%}$
test_iql_speed[False-None] 25.2321ms 20.0318ms 49.9206 Ops/s 49.6037 Ops/s $\color{#35bf28}+0.64\%$
test_iql_speed[False-backward] 31.6786ms 30.5012ms 32.7856 Ops/s 32.6688 Ops/s $\color{#35bf28}+0.36\%$
test_iql_speed[True-None] 8.8468ms 8.5441ms 117.0404 Ops/s 116.3004 Ops/s $\color{#35bf28}+0.64\%$
test_iql_speed[True-backward] 17.5854ms 16.8586ms 59.3168 Ops/s 57.2075 Ops/s $\color{#35bf28}+3.69\%$
test_iql_speed[reduce-overhead-None] 9.0925ms 8.5868ms 116.4578 Ops/s 103.6743 Ops/s $\textbf{\color{#35bf28}+12.33\%}$
test_iql_speed[reduce-overhead-backward] 18.0819ms 17.3024ms 57.7955 Ops/s 55.1964 Ops/s $\color{#35bf28}+4.71\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.6673ms 6.2081ms 161.0797 Ops/s 162.5840 Ops/s $\color{#d91a1a}-0.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5911ms 0.3333ms 3.0001 KOps/s 3.5434 KOps/s $\textbf{\color{#d91a1a}-15.33\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5798ms 0.3191ms 3.1343 KOps/s 3.7357 KOps/s $\textbf{\color{#d91a1a}-16.10\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.2708ms 5.8954ms 169.6245 Ops/s 171.5896 Ops/s $\color{#d91a1a}-1.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0018ms 0.2815ms 3.5529 KOps/s 3.6457 KOps/s $\color{#d91a1a}-2.55\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4681ms 0.2560ms 3.9062 KOps/s 3.9007 KOps/s $\color{#35bf28}+0.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6460ms 1.1988ms 834.1484 Ops/s 809.8863 Ops/s $\color{#35bf28}+3.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.3362ms 1.1251ms 888.8191 Ops/s 882.2545 Ops/s $\color{#35bf28}+0.74\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 10.0737ms 6.2257ms 160.6251 Ops/s 167.0996 Ops/s $\color{#d91a1a}-3.87\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8662ms 0.4292ms 2.3301 KOps/s 2.3292 KOps/s $\color{#35bf28}+0.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8588ms 0.4091ms 2.4445 KOps/s 2.4525 KOps/s $\color{#d91a1a}-0.33\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.4283ms 5.9242ms 168.7985 Ops/s 171.6621 Ops/s $\color{#d91a1a}-1.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.3529ms 0.3523ms 2.8385 KOps/s 820.2366 Ops/s $\textbf{\color{#35bf28}+246.06\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5307ms 0.3366ms 2.9709 KOps/s 3.2577 KOps/s $\textbf{\color{#d91a1a}-8.80\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.4222ms 5.8797ms 170.0766 Ops/s 168.7290 Ops/s $\color{#35bf28}+0.80\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7650ms 0.3469ms 2.8827 KOps/s 2.8474 KOps/s $\color{#35bf28}+1.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7809ms 0.3306ms 3.0246 KOps/s 2.8948 KOps/s $\color{#35bf28}+4.48\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.1370ms 6.0432ms 165.4750 Ops/s 163.7113 Ops/s $\color{#35bf28}+1.08\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9860ms 0.4979ms 2.0085 KOps/s 1.9649 KOps/s $\color{#35bf28}+2.22\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6935ms 0.4698ms 2.1287 KOps/s 2.0318 KOps/s $\color{#35bf28}+4.77\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.6389ms 5.1743ms 193.2624 Ops/s 187.7061 Ops/s $\color{#35bf28}+2.96\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.6037ms 2.2978ms 435.1905 Ops/s 426.5685 Ops/s $\color{#35bf28}+2.02\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.2037ms 1.1889ms 841.1304 Ops/s 832.5069 Ops/s $\color{#35bf28}+1.04\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5084s 15.2465ms 65.5889 Ops/s 55.0062 Ops/s $\textbf{\color{#35bf28}+19.24\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 10.1846ms 2.0496ms 487.9046 Ops/s 514.1712 Ops/s $\textbf{\color{#d91a1a}-5.11\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 3.3172ms 1.0850ms 921.6703 Ops/s 855.2995 Ops/s $\textbf{\color{#35bf28}+7.76\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 9.4911ms 5.3639ms 186.4307 Ops/s 179.5696 Ops/s $\color{#35bf28}+3.82\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 6.9063ms 2.1758ms 459.6003 Ops/s 449.4072 Ops/s $\color{#35bf28}+2.27\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 9.2542ms 1.3754ms 727.0552 Ops/s 751.2423 Ops/s $\color{#d91a1a}-3.22\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 35.3958ms 32.8147ms 30.4741 Ops/s 29.8797 Ops/s $\color{#35bf28}+1.99\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.4152ms 17.8663ms 55.9713 Ops/s 57.0072 Ops/s $\color{#d91a1a}-1.82\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 36.5366ms 33.8999ms 29.4986 Ops/s 29.3602 Ops/s $\color{#35bf28}+0.47\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.3543ms 17.8045ms 56.1657 Ops/s 56.2237 Ops/s $\color{#d91a1a}-0.10\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 37.2163ms 35.6091ms 28.0827 Ops/s 28.1595 Ops/s $\color{#d91a1a}-0.27\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.6950ms 19.5629ms 51.1171 Ops/s 52.3295 Ops/s $\color{#d91a1a}-2.32\%$

vmoens added 2 commits October 23, 2025 11:45
@vmoens
Update
2dd8c7d 
[ghstack-poisoned]
@vmoens
Update
e956f33 
[ghstack-poisoned]
@vmoens
Update
d867669 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. llm/feature Objectives

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens