[Feature] Add timing options

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] Documentation #3192
• -> [Feature] Add timing options #3191
• [Feature] PPO Trainer Updates #3190

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3191

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 12 New Failures, 1 Unrelated Failure

As of commit 7a28716 with merge base 3d1748f ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 139.
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 0384188b20b7e1d6876b6217e7407d6ec79e647545b64e1ea498610b1edfeb56 /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t a3a29046b68bf5e1668ecc395bad37c61e87a25e705c7aed2489509a5bcbce7f /exec failed with exit code 1
• Push Binary Nightly / build-wheel-unix (linux, ubuntu-22.04, 3.9, cp39-cp39, cpu, cpu) (gh)
  Process completed with exit code 1.
• Push Binary Nightly / build-wheel-unix (macos, macos-latest, 3.9, cp39-cp39, cpu, cpu) (gh)
• Push Binary Nightly / build-wheel-windows (3.9, 3.9) (gh)
• Push Binary Nightly / test-wheel-unix (linux, ubuntu-22.04, 3.9, cp39-cp39, cpu, cpu) (gh)
  Process completed with exit code 1.
• Push Binary Nightly / test-wheel-unix (macos, macos-latest, 3.9, cp39-cp39, cpu, cpu) (gh)
  Process completed with exit code 1.
• Push Binary Nightly / test-wheel-windows (3.9, 3.9) (gh)
  Process completed with exit code 1.
• SOTA Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 952ea35ac05a22a9bc1e880e8605236bb44a9c09c32361dd02f91b37fb3c960a /exec failed with exit code 1
• Unit-tests on Linux / tests-gpu (3.11, 12.8) / linux-job (gh)
  test/test_collector.py::TestPolicyFactory::test_weight_update
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  test/test_collector.py::TestPolicyFactory::test_weight_update

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Linux / tests-olddeps (3.9, 11.6) / linux-job (gh) (trunk failure)
  test/test_collector.py::TestPolicyFactory::test_weight_update

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}22$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	83.9285μs	82.2983μs	12.1509 KOps/s	12.0082 KOps/s	$\color{#35bf28}+1.19\%$
test_tensor_to_bytestream_speed[torch.save]	0.1477ms	0.1448ms	6.9066 KOps/s	6.9305 KOps/s	$\color{#d91a1a}-0.34\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1174s	0.1172s	8.5327 Ops/s	7.5616 Ops/s	$\textbf{\color{#35bf28}+12.84\%}$
test_tensor_to_bytestream_speed[numpy]	2.8369μs	2.8315μs	353.1709 KOps/s	343.4892 KOps/s	$\color{#35bf28}+2.82\%$
test_tensor_to_bytestream_speed[safetensors]	44.5226μs	42.4520μs	23.5560 KOps/s	23.1044 KOps/s	$\color{#35bf28}+1.95\%$
test_simple	0.5562s	0.5556s	1.8000 Ops/s	1.7094 Ops/s	$\textbf{\color{#35bf28}+5.30\%}$
test_transformed	1.1256s	1.1234s	0.8901 Ops/s	0.8629 Ops/s	$\color{#35bf28}+3.16\%$
test_serial	1.6939s	1.6862s	0.5931 Ops/s	0.5779 Ops/s	$\color{#35bf28}+2.63\%$
test_parallel	1.1754s	1.1270s	0.8873 Ops/s	0.9191 Ops/s	$\color{#d91a1a}-3.46\%$
test_step_mdp_speed[True-True-True-True-True]	0.2475ms	46.4553μs	21.5261 KOps/s	22.1035 KOps/s	$\color{#d91a1a}-2.61\%$
test_step_mdp_speed[True-True-True-True-False]	57.1210μs	25.7257μs	38.8716 KOps/s	39.0670 KOps/s	$\color{#d91a1a}-0.50\%$
test_step_mdp_speed[True-True-True-False-True]	58.4110μs	26.0408μs	38.4012 KOps/s	39.0837 KOps/s	$\color{#d91a1a}-1.75\%$
test_step_mdp_speed[True-True-True-False-False]	40.1310μs	14.3375μs	69.7473 KOps/s	70.5459 KOps/s	$\color{#d91a1a}-1.13\%$
test_step_mdp_speed[True-True-False-True-True]	73.8910μs	49.2696μs	20.2965 KOps/s	20.5178 KOps/s	$\color{#d91a1a}-1.08\%$
test_step_mdp_speed[True-True-False-True-False]	63.6610μs	28.6579μs	34.8944 KOps/s	34.6994 KOps/s	$\color{#35bf28}+0.56\%$
test_step_mdp_speed[True-True-False-False-True]	91.9120μs	28.7679μs	34.7610 KOps/s	34.9129 KOps/s	$\color{#d91a1a}-0.44\%$
test_step_mdp_speed[True-True-False-False-False]	44.4010μs	17.0631μs	58.6059 KOps/s	58.3972 KOps/s	$\color{#35bf28}+0.36\%$
test_step_mdp_speed[True-False-True-True-True]	88.7220μs	52.1007μs	19.1936 KOps/s	19.1843 KOps/s	$\color{#35bf28}+0.05\%$
test_step_mdp_speed[True-False-True-True-False]	63.5310μs	31.3934μs	31.8538 KOps/s	31.6027 KOps/s	$\color{#35bf28}+0.79\%$
test_step_mdp_speed[True-False-True-False-True]	58.2010μs	29.0325μs	34.4441 KOps/s	35.2029 KOps/s	$\color{#d91a1a}-2.16\%$
test_step_mdp_speed[True-False-True-False-False]	54.5810μs	17.0676μs	58.5906 KOps/s	59.1384 KOps/s	$\color{#d91a1a}-0.93\%$
test_step_mdp_speed[True-False-False-True-True]	83.3210μs	54.6601μs	18.2949 KOps/s	18.4702 KOps/s	$\color{#d91a1a}-0.95\%$
test_step_mdp_speed[True-False-False-True-False]	72.1510μs	34.0530μs	29.3660 KOps/s	29.0566 KOps/s	$\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-False-False-False-True]	59.2110μs	31.9461μs	31.3027 KOps/s	31.9108 KOps/s	$\color{#d91a1a}-1.91\%$
test_step_mdp_speed[True-False-False-False-False]	50.7210μs	19.8773μs	50.3086 KOps/s	50.1146 KOps/s	$\color{#35bf28}+0.39\%$
test_step_mdp_speed[False-True-True-True-True]	86.0810μs	52.2261μs	19.1475 KOps/s	19.3421 KOps/s	$\color{#d91a1a}-1.01\%$
test_step_mdp_speed[False-True-True-True-False]	55.3210μs	31.5932μs	31.6523 KOps/s	31.3499 KOps/s	$\color{#35bf28}+0.96\%$
test_step_mdp_speed[False-True-True-False-True]	2.4647ms	33.3771μs	29.9606 KOps/s	31.0856 KOps/s	$\color{#d91a1a}-3.62\%$
test_step_mdp_speed[False-True-True-False-False]	48.2810μs	19.2427μs	51.9678 KOps/s	52.0764 KOps/s	$\color{#d91a1a}-0.21\%$
test_step_mdp_speed[False-True-False-True-True]	87.1220μs	55.1834μs	18.1214 KOps/s	18.4426 KOps/s	$\color{#d91a1a}-1.74\%$
test_step_mdp_speed[False-True-False-True-False]	71.8410μs	34.5278μs	28.9622 KOps/s	29.1211 KOps/s	$\color{#d91a1a}-0.55\%$
test_step_mdp_speed[False-True-False-False-True]	72.2910μs	35.6665μs	28.0375 KOps/s	28.4423 KOps/s	$\color{#d91a1a}-1.42\%$
test_step_mdp_speed[False-True-False-False-False]	54.5910μs	21.6957μs	46.0921 KOps/s	45.8285 KOps/s	$\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-False-True-True-True]	98.4520μs	56.5106μs	17.6958 KOps/s	17.5727 KOps/s	$\color{#35bf28}+0.70\%$
test_step_mdp_speed[False-False-True-True-False]	75.5510μs	37.2061μs	26.8773 KOps/s	26.9331 KOps/s	$\color{#d91a1a}-0.21\%$
test_step_mdp_speed[False-False-True-False-True]	76.6210μs	35.3239μs	28.3094 KOps/s	28.5469 KOps/s	$\color{#d91a1a}-0.83\%$
test_step_mdp_speed[False-False-True-False-False]	53.6910μs	21.6988μs	46.0855 KOps/s	45.8409 KOps/s	$\color{#35bf28}+0.53\%$
test_step_mdp_speed[False-False-False-True-True]	90.0310μs	60.0179μs	16.6617 KOps/s	16.7871 KOps/s	$\color{#d91a1a}-0.75\%$
test_step_mdp_speed[False-False-False-True-False]	0.1083ms	39.1165μs	25.5647 KOps/s	24.9741 KOps/s	$\color{#35bf28}+2.36\%$
test_step_mdp_speed[False-False-False-False-True]	65.8110μs	37.9014μs	26.3843 KOps/s	26.7077 KOps/s	$\color{#d91a1a}-1.21\%$
test_step_mdp_speed[False-False-False-False-False]	54.5610μs	24.1521μs	41.4042 KOps/s	40.8322 KOps/s	$\color{#35bf28}+1.40\%$
test_values[generalized_advantage_estimate-True-True]	10.4827ms	10.2648ms	97.4203 Ops/s	97.5288 Ops/s	$\color{#d91a1a}-0.11\%$
test_values[vec_generalized_advantage_estimate-True-True]	13.5373ms	11.1651ms	89.5647 Ops/s	89.6520 Ops/s	$\color{#d91a1a}-0.10\%$
test_values[td0_return_estimate-False-False]	0.2531ms	0.1327ms	7.5330 KOps/s	7.4745 KOps/s	$\color{#35bf28}+0.78\%$
test_values[td1_return_estimate-False-False]	28.6370ms	27.7397ms	36.0494 Ops/s	35.2580 Ops/s	$\color{#35bf28}+2.24\%$
test_values[vec_td1_return_estimate-False-False]	11.9543ms	11.2620ms	88.7943 Ops/s	88.4732 Ops/s	$\color{#35bf28}+0.36\%$
test_values[td_lambda_return_estimate-True-False]	43.5645ms	41.7931ms	23.9274 Ops/s	23.7736 Ops/s	$\color{#35bf28}+0.65\%$
test_values[vec_td_lambda_return_estimate-True-False]	11.8347ms	11.2408ms	88.9616 Ops/s	88.6103 Ops/s	$\color{#35bf28}+0.40\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.8169ms	8.7253ms	114.6089 Ops/s	111.4573 Ops/s	$\color{#35bf28}+2.83\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.9528ms	1.5378ms	650.2951 Ops/s	671.0648 Ops/s	$\color{#d91a1a}-3.10\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4822ms	0.4178ms	2.3932 KOps/s	2.3391 KOps/s	$\color{#35bf28}+2.32\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	23.7641ms	23.1866ms	43.1284 Ops/s	37.9874 Ops/s	$\textbf{\color{#35bf28}+13.53\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	2.2551ms	1.7682ms	565.5471 Ops/s	578.7037 Ops/s	$\color{#d91a1a}-2.27\%$
test_dqn_speed[False-None]	6.4730ms	1.4408ms	694.0696 Ops/s	687.7528 Ops/s	$\color{#35bf28}+0.92\%$
test_dqn_speed[False-backward]	2.0090ms	1.9410ms	515.1981 Ops/s	516.4193 Ops/s	$\color{#d91a1a}-0.24\%$
test_dqn_speed[True-None]	0.9062ms	0.5114ms	1.9556 KOps/s	1.9067 KOps/s	$\color{#35bf28}+2.56\%$
test_dqn_speed[True-backward]	1.0048ms	0.9688ms	1.0323 KOps/s	850.7924 Ops/s	$\textbf{\color{#35bf28}+21.33\%}$
test_dqn_speed[reduce-overhead-None]	0.8812ms	0.5057ms	1.9774 KOps/s	1.8824 KOps/s	$\textbf{\color{#35bf28}+5.05\%}$
test_dqn_speed[reduce-overhead-backward]	1.0077ms	0.9506ms	1.0520 KOps/s	1.0312 KOps/s	$\color{#35bf28}+2.02\%$
test_ddpg_speed[False-None]	3.1966ms	2.9025ms	344.5279 Ops/s	342.3970 Ops/s	$\color{#35bf28}+0.62\%$
test_ddpg_speed[False-backward]	4.2167ms	4.1212ms	242.6492 Ops/s	242.0704 Ops/s	$\color{#35bf28}+0.24\%$
test_ddpg_speed[True-None]	1.7434ms	1.3794ms	724.9771 Ops/s	714.7922 Ops/s	$\color{#35bf28}+1.42\%$
test_ddpg_speed[True-backward]	2.4118ms	2.3574ms	424.1958 Ops/s	403.8023 Ops/s	$\textbf{\color{#35bf28}+5.05\%}$
test_ddpg_speed[reduce-overhead-None]	1.7383ms	1.3645ms	732.8441 Ops/s	719.9638 Ops/s	$\color{#35bf28}+1.79\%$
test_ddpg_speed[reduce-overhead-backward]	2.3830ms	2.3365ms	427.9917 Ops/s	421.3697 Ops/s	$\color{#35bf28}+1.57\%$
test_sac_speed[False-None]	8.4426ms	7.9087ms	126.4426 Ops/s	125.4776 Ops/s	$\color{#35bf28}+0.77\%$
test_sac_speed[False-backward]	11.8973ms	11.1971ms	89.3088 Ops/s	89.2934 Ops/s	$\color{#35bf28}+0.02\%$
test_sac_speed[True-None]	2.2306ms	2.0977ms	476.7175 Ops/s	474.8822 Ops/s	$\color{#35bf28}+0.39\%$
test_sac_speed[True-backward]	4.1193ms	3.9944ms	250.3512 Ops/s	244.8100 Ops/s	$\color{#35bf28}+2.26\%$
test_sac_speed[reduce-overhead-None]	2.2944ms	2.0830ms	480.0813 Ops/s	469.6263 Ops/s	$\color{#35bf28}+2.23\%$
test_sac_speed[reduce-overhead-backward]	4.1265ms	4.0199ms	248.7629 Ops/s	229.9175 Ops/s	$\textbf{\color{#35bf28}+8.20\%}$
test_redq_speed[False-None]	10.8562ms	10.4738ms	95.4761 Ops/s	95.5181 Ops/s	$\color{#d91a1a}-0.04\%$
test_redq_speed[False-backward]	18.5218ms	17.8313ms	56.0812 Ops/s	56.0020 Ops/s	$\color{#35bf28}+0.14\%$
test_redq_speed[True-None]	4.7631ms	4.3251ms	231.2067 Ops/s	222.4506 Ops/s	$\color{#35bf28}+3.94\%$
test_redq_speed[True-backward]	10.3903ms	10.0066ms	99.9337 Ops/s	101.5713 Ops/s	$\color{#d91a1a}-1.61\%$
test_redq_speed[reduce-overhead-None]	4.5166ms	4.3294ms	230.9775 Ops/s	229.2192 Ops/s	$\color{#35bf28}+0.77\%$
test_redq_speed[reduce-overhead-backward]	10.5311ms	10.0846ms	99.1613 Ops/s	100.7822 Ops/s	$\color{#d91a1a}-1.61\%$
test_redq_deprec_speed[False-None]	11.3194ms	10.9909ms	90.9841 Ops/s	91.3474 Ops/s	$\color{#d91a1a}-0.40\%$
test_redq_deprec_speed[False-backward]	16.2503ms	15.7498ms	63.4929 Ops/s	64.2941 Ops/s	$\color{#d91a1a}-1.25\%$
test_redq_deprec_speed[True-None]	4.0075ms	3.6731ms	272.2533 Ops/s	279.3826 Ops/s	$\color{#d91a1a}-2.55\%$
test_redq_deprec_speed[True-backward]	7.9415ms	7.5878ms	131.7913 Ops/s	118.2952 Ops/s	$\textbf{\color{#35bf28}+11.41\%}$
test_redq_deprec_speed[reduce-overhead-None]	3.9242ms	3.6328ms	275.2682 Ops/s	273.6236 Ops/s	$\color{#35bf28}+0.60\%$
test_redq_deprec_speed[reduce-overhead-backward]	8.1089ms	7.6435ms	130.8293 Ops/s	133.6523 Ops/s	$\color{#d91a1a}-2.11\%$
test_td3_speed[False-None]	8.5933ms	8.0200ms	124.6889 Ops/s	118.6660 Ops/s	$\textbf{\color{#35bf28}+5.08\%}$
test_td3_speed[False-backward]	11.5584ms	10.9071ms	91.6836 Ops/s	90.9471 Ops/s	$\color{#35bf28}+0.81\%$
test_td3_speed[True-None]	1.8114ms	1.7734ms	563.8981 Ops/s	552.9866 Ops/s	$\color{#35bf28}+1.97\%$
test_td3_speed[True-backward]	3.7254ms	3.5684ms	280.2352 Ops/s	270.7082 Ops/s	$\color{#35bf28}+3.52\%$
test_td3_speed[reduce-overhead-None]	1.7829ms	1.7431ms	573.7038 Ops/s	556.1732 Ops/s	$\color{#35bf28}+3.15\%$
test_td3_speed[reduce-overhead-backward]	3.7292ms	3.6267ms	275.7364 Ops/s	242.0222 Ops/s	$\textbf{\color{#35bf28}+13.93\%}$
test_cql_speed[False-None]	28.8253ms	26.0584ms	38.3753 Ops/s	38.5419 Ops/s	$\color{#d91a1a}-0.43\%$
test_cql_speed[False-backward]	38.6426ms	35.6885ms	28.0203 Ops/s	28.6062 Ops/s	$\color{#d91a1a}-2.05\%$
test_cql_speed[True-None]	12.8871ms	12.3395ms	81.0403 Ops/s	82.5347 Ops/s	$\color{#d91a1a}-1.81\%$
test_cql_speed[True-backward]	18.7581ms	18.3932ms	54.3678 Ops/s	54.8520 Ops/s	$\color{#d91a1a}-0.88\%$
test_cql_speed[reduce-overhead-None]	13.3199ms	12.4233ms	80.4939 Ops/s	82.1358 Ops/s	$\color{#d91a1a}-2.00\%$
test_cql_speed[reduce-overhead-backward]	18.7447ms	18.3948ms	54.3631 Ops/s	53.7692 Ops/s	$\color{#35bf28}+1.10\%$
test_a2c_speed[False-None]	5.6499ms	5.3213ms	187.9232 Ops/s	182.9426 Ops/s	$\color{#35bf28}+2.72\%$
test_a2c_speed[False-backward]	12.2299ms	11.8311ms	84.5229 Ops/s	83.4712 Ops/s	$\color{#35bf28}+1.26\%$
test_a2c_speed[True-None]	3.8424ms	3.6520ms	273.8192 Ops/s	259.7906 Ops/s	$\textbf{\color{#35bf28}+5.40\%}$
test_a2c_speed[True-backward]	8.9443ms	8.6438ms	115.6900 Ops/s	114.9448 Ops/s	$\color{#35bf28}+0.65\%$
test_a2c_speed[reduce-overhead-None]	3.8425ms	3.7319ms	267.9625 Ops/s	270.8754 Ops/s	$\color{#d91a1a}-1.08\%$
test_a2c_speed[reduce-overhead-backward]	8.9705ms	8.8135ms	113.4628 Ops/s	112.5355 Ops/s	$\color{#35bf28}+0.82\%$
test_ppo_speed[False-None]	6.3304ms	5.9269ms	168.7231 Ops/s	166.3508 Ops/s	$\color{#35bf28}+1.43\%$
test_ppo_speed[False-backward]	12.9291ms	12.6603ms	78.9874 Ops/s	80.1020 Ops/s	$\color{#d91a1a}-1.39\%$
test_ppo_speed[True-None]	3.9671ms	3.6554ms	273.5699 Ops/s	273.6004 Ops/s	$\color{#d91a1a}-0.01\%$
test_ppo_speed[True-backward]	8.6235ms	8.4550ms	118.2730 Ops/s	104.9809 Ops/s	$\textbf{\color{#35bf28}+12.66\%}$
test_ppo_speed[reduce-overhead-None]	3.9887ms	3.6358ms	275.0436 Ops/s	272.6816 Ops/s	$\color{#35bf28}+0.87\%$
test_ppo_speed[reduce-overhead-backward]	9.0235ms	8.7258ms	114.6028 Ops/s	110.6042 Ops/s	$\color{#35bf28}+3.62\%$
test_reinforce_speed[False-None]	4.9463ms	4.5959ms	217.5852 Ops/s	215.4820 Ops/s	$\color{#35bf28}+0.98\%$
test_reinforce_speed[False-backward]	7.7725ms	7.4417ms	134.3775 Ops/s	131.3465 Ops/s	$\color{#35bf28}+2.31\%$
test_reinforce_speed[True-None]	3.2214ms	2.8429ms	351.7574 Ops/s	342.0726 Ops/s	$\color{#35bf28}+2.83\%$
test_reinforce_speed[True-backward]	8.0542ms	7.7631ms	128.8144 Ops/s	125.7185 Ops/s	$\color{#35bf28}+2.46\%$
test_reinforce_speed[reduce-overhead-None]	3.1976ms	2.8468ms	351.2727 Ops/s	337.7669 Ops/s	$\color{#35bf28}+4.00\%$
test_reinforce_speed[reduce-overhead-backward]	8.3301ms	7.8928ms	126.6977 Ops/s	115.9844 Ops/s	$\textbf{\color{#35bf28}+9.24\%}$
test_iql_speed[False-None]	26.0569ms	20.3655ms	49.1026 Ops/s	48.6689 Ops/s	$\color{#35bf28}+0.89\%$
test_iql_speed[False-backward]	35.7979ms	30.6893ms	32.5846 Ops/s	32.6787 Ops/s	$\color{#d91a1a}-0.29\%$
test_iql_speed[True-None]	8.8629ms	8.4728ms	118.0251 Ops/s	117.7180 Ops/s	$\color{#35bf28}+0.26\%$
test_iql_speed[True-backward]	17.3166ms	16.8305ms	59.4159 Ops/s	58.2054 Ops/s	$\color{#35bf28}+2.08\%$
test_iql_speed[reduce-overhead-None]	8.8297ms	8.5479ms	116.9881 Ops/s	111.5563 Ops/s	$\color{#35bf28}+4.87\%$
test_iql_speed[reduce-overhead-backward]	17.7438ms	17.1260ms	58.3907 Ops/s	57.4894 Ops/s	$\color{#35bf28}+1.57\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.5635ms	6.0840ms	164.3650 Ops/s	165.2057 Ops/s	$\color{#d91a1a}-0.51\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6092ms	0.3752ms	2.6653 KOps/s	3.2379 KOps/s	$\textbf{\color{#d91a1a}-17.68\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6986ms	0.2601ms	3.8442 KOps/s	3.2663 KOps/s	$\textbf{\color{#35bf28}+17.69\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.0219ms	5.7813ms	172.9706 Ops/s	174.9987 Ops/s	$\color{#d91a1a}-1.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.8421ms	0.3213ms	3.1126 KOps/s	3.3143 KOps/s	$\textbf{\color{#d91a1a}-6.09\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6151ms	0.3202ms	3.1235 KOps/s	3.3285 KOps/s	$\textbf{\color{#d91a1a}-6.16\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.7203ms	1.3199ms	757.6275 Ops/s	763.8493 Ops/s	$\color{#d91a1a}-0.81\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.5224ms	1.3051ms	766.2010 Ops/s	769.9813 Ops/s	$\color{#d91a1a}-0.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.1904ms	5.9498ms	168.0727 Ops/s	170.2872 Ops/s	$\color{#d91a1a}-1.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9946ms	0.4812ms	2.0783 KOps/s	2.0334 KOps/s	$\color{#35bf28}+2.21\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7354ms	0.4638ms	2.1561 KOps/s	2.0419 KOps/s	$\textbf{\color{#35bf28}+5.59\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.9710ms	5.8576ms	170.7188 Ops/s	174.3748 Ops/s	$\color{#d91a1a}-2.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	2.0759ms	0.3515ms	2.8448 KOps/s	730.7658 Ops/s	$\textbf{\color{#35bf28}+289.29\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5052ms	0.3104ms	3.2218 KOps/s	3.7719 KOps/s	$\textbf{\color{#d91a1a}-14.58\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.1004ms	5.7453ms	174.0552 Ops/s	171.9927 Ops/s	$\color{#35bf28}+1.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.8187ms	0.3155ms	3.1700 KOps/s	3.4870 KOps/s	$\textbf{\color{#d91a1a}-9.09\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5055ms	0.2982ms	3.3539 KOps/s	3.4791 KOps/s	$\color{#d91a1a}-3.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.0539ms	5.9167ms	169.0124 Ops/s	166.8351 Ops/s	$\color{#35bf28}+1.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0651ms	0.5150ms	1.9417 KOps/s	2.0481 KOps/s	$\textbf{\color{#d91a1a}-5.19\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6965ms	0.4474ms	2.2349 KOps/s	2.1751 KOps/s	$\color{#35bf28}+2.75\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.5494s	15.9673ms	62.6281 Ops/s	195.3235 Ops/s	$\textbf{\color{#d91a1a}-67.94\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	10.5803ms	2.0699ms	483.1150 Ops/s	427.9223 Ops/s	$\textbf{\color{#35bf28}+12.90\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	6.4159ms	1.1336ms	882.1673 Ops/s	825.1436 Ops/s	$\textbf{\color{#35bf28}+6.91\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	6.5586ms	5.0384ms	198.4752 Ops/s	56.0513 Ops/s	$\textbf{\color{#35bf28}+254.10\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	12.0566ms	1.9492ms	513.0363 Ops/s	479.7816 Ops/s	$\textbf{\color{#35bf28}+6.93\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	3.3566ms	1.1015ms	907.8586 Ops/s	838.6350 Ops/s	$\textbf{\color{#35bf28}+8.25\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	8.6790ms	5.2265ms	191.3315 Ops/s	185.9409 Ops/s	$\color{#35bf28}+2.90\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	8.1097ms	2.1764ms	459.4763 Ops/s	446.8160 Ops/s	$\color{#35bf28}+2.83\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	2.0598ms	1.0923ms	915.4850 Ops/s	733.4381 Ops/s	$\textbf{\color{#35bf28}+24.82\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	35.4701ms	32.9688ms	30.3317 Ops/s	29.9384 Ops/s	$\color{#35bf28}+1.31\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	19.2947ms	17.7594ms	56.3083 Ops/s	55.5881 Ops/s	$\color{#35bf28}+1.30\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	35.3571ms	33.4397ms	29.9045 Ops/s	29.1178 Ops/s	$\color{#35bf28}+2.70\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.9026ms	18.0312ms	55.4593 Ops/s	54.9891 Ops/s	$\color{#35bf28}+0.86\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	36.9746ms	35.4381ms	28.2182 Ops/s	27.8767 Ops/s	$\color{#35bf28}+1.23\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.9459ms	19.2831ms	51.8588 Ops/s	50.6973 Ops/s	$\color{#35bf28}+2.29\%$