[Feature] Documentation

[vmoens]

Stack from ghstack (oldest at bottom):

• -> [Feature] Documentation #3192
• [Feature] SAC Trainer #3191
• [Feature] PPO Trainer Updates #3190
• [Feature] Trainer Algorithms - Configuration System #3189
• [Feature] Trainer Infrastructure - Timing and Utilities #3188
• [CI] LLM tests integration #3216
• [Quality] Fix flaky test #3211

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3192

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 21 Pending

As of commit c7a19f7 with merge base 963fdd4 ():

NEW FAILURES - The following jobs have failed:

• Generate documentation / build-docs (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 22ce534ce38230ab120852562ec5d5275ff9868946e8de3178cf3c73e8b52fca /exec failed with exit code 1
• Lint / python-source-and-configs / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}12$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	86.6478μs	84.2739μs	11.8661 KOps/s	12.2439 KOps/s	$\color{#d91a1a}-3.09\%$
test_tensor_to_bytestream_speed[torch.save]	0.1417ms	0.1390ms	7.1931 KOps/s	7.2255 KOps/s	$\color{#d91a1a}-0.45\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1158s	0.1153s	8.6766 Ops/s	8.7796 Ops/s	$\color{#d91a1a}-1.17\%$
test_tensor_to_bytestream_speed[numpy]	2.7644μs	2.7614μs	362.1319 KOps/s	363.6065 KOps/s	$\color{#d91a1a}-0.41\%$
test_tensor_to_bytestream_speed[safetensors]	44.0555μs	43.8016μs	22.8302 KOps/s	21.9051 KOps/s	$\color{#35bf28}+4.22\%$
test_simple	0.6605s	0.5779s	1.7303 Ops/s	1.7488 Ops/s	$\color{#d91a1a}-1.05\%$
test_transformed	1.2185s	1.1256s	0.8884 Ops/s	0.8850 Ops/s	$\color{#35bf28}+0.38\%$
test_serial	1.6780s	1.6626s	0.6015 Ops/s	0.5950 Ops/s	$\color{#35bf28}+1.09\%$
test_parallel	1.1504s	1.0891s	0.9182 Ops/s	0.9123 Ops/s	$\color{#35bf28}+0.65\%$
test_step_mdp_speed[True-True-True-True-True]	0.1855ms	44.6332μs	22.4049 KOps/s	22.2370 KOps/s	$\color{#35bf28}+0.76\%$
test_step_mdp_speed[True-True-True-True-False]	63.3810μs	25.4635μs	39.2719 KOps/s	38.8055 KOps/s	$\color{#35bf28}+1.20\%$
test_step_mdp_speed[True-True-True-False-True]	60.2910μs	25.0592μs	39.9055 KOps/s	39.0832 KOps/s	$\color{#35bf28}+2.10\%$
test_step_mdp_speed[True-True-True-False-False]	45.7010μs	13.9768μs	71.5473 KOps/s	70.4654 KOps/s	$\color{#35bf28}+1.54\%$
test_step_mdp_speed[True-True-False-True-True]	0.1091ms	48.4425μs	20.6430 KOps/s	20.4761 KOps/s	$\color{#35bf28}+0.82\%$
test_step_mdp_speed[True-True-False-True-False]	53.2610μs	28.0453μs	35.6566 KOps/s	34.9327 KOps/s	$\color{#35bf28}+2.07\%$
test_step_mdp_speed[True-True-False-False-True]	55.1010μs	27.9889μs	35.7284 KOps/s	34.9657 KOps/s	$\color{#35bf28}+2.18\%$
test_step_mdp_speed[True-True-False-False-False]	74.7320μs	16.7244μs	59.7929 KOps/s	58.9559 KOps/s	$\color{#35bf28}+1.42\%$
test_step_mdp_speed[True-False-True-True-True]	93.6520μs	51.4264μs	19.4453 KOps/s	19.3964 KOps/s	$\color{#35bf28}+0.25\%$
test_step_mdp_speed[True-False-True-True-False]	71.1920μs	31.1509μs	32.1018 KOps/s	32.0651 KOps/s	$\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-False-True-False-True]	59.9220μs	28.5615μs	35.0121 KOps/s	35.5942 KOps/s	$\color{#d91a1a}-1.64\%$
test_step_mdp_speed[True-False-True-False-False]	50.9510μs	16.9837μs	58.8799 KOps/s	59.1193 KOps/s	$\color{#d91a1a}-0.40\%$
test_step_mdp_speed[True-False-False-True-True]	0.1086ms	54.3318μs	18.4054 KOps/s	18.4152 KOps/s	$\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-False-False-True-False]	0.1108ms	33.6552μs	29.7131 KOps/s	29.6708 KOps/s	$\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-False-False-False-True]	61.3710μs	31.0574μs	32.1984 KOps/s	32.4870 KOps/s	$\color{#d91a1a}-0.89\%$
test_step_mdp_speed[True-False-False-False-False]	54.0710μs	19.4634μs	51.3784 KOps/s	51.0231 KOps/s	$\color{#35bf28}+0.70\%$
test_step_mdp_speed[False-True-True-True-True]	85.6410μs	51.5934μs	19.3823 KOps/s	19.5068 KOps/s	$\color{#d91a1a}-0.64\%$
test_step_mdp_speed[False-True-True-True-False]	61.8610μs	31.3928μs	31.8544 KOps/s	32.5154 KOps/s	$\color{#d91a1a}-2.03\%$
test_step_mdp_speed[False-True-True-False-True]	2.3411ms	32.5237μs	30.7469 KOps/s	30.8651 KOps/s	$\color{#d91a1a}-0.38\%$
test_step_mdp_speed[False-True-True-False-False]	36.3200μs	18.5853μs	53.8061 KOps/s	53.3148 KOps/s	$\color{#35bf28}+0.92\%$
test_step_mdp_speed[False-True-False-True-True]	87.9620μs	53.7427μs	18.6072 KOps/s	18.3838 KOps/s	$\color{#35bf28}+1.22\%$
test_step_mdp_speed[False-True-False-True-False]	71.4310μs	33.9083μs	29.4913 KOps/s	29.6684 KOps/s	$\color{#d91a1a}-0.60\%$
test_step_mdp_speed[False-True-False-False-True]	62.2020μs	34.8011μs	28.7347 KOps/s	28.6817 KOps/s	$\color{#35bf28}+0.18\%$
test_step_mdp_speed[False-True-False-False-False]	53.1310μs	21.4130μs	46.7005 KOps/s	46.5660 KOps/s	$\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-False-True-True-True]	88.8820μs	56.4706μs	17.7083 KOps/s	17.6060 KOps/s	$\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-False-True-True-False]	66.5510μs	36.7268μs	27.2281 KOps/s	27.9652 KOps/s	$\color{#d91a1a}-2.64\%$
test_step_mdp_speed[False-False-True-False-True]	74.8910μs	35.2435μs	28.3740 KOps/s	28.8688 KOps/s	$\color{#d91a1a}-1.71\%$
test_step_mdp_speed[False-False-True-False-False]	51.9710μs	21.2663μs	47.0228 KOps/s	46.7755 KOps/s	$\color{#35bf28}+0.53\%$
test_step_mdp_speed[False-False-False-True-True]	92.0920μs	58.3033μs	17.1517 KOps/s	17.0919 KOps/s	$\color{#35bf28}+0.35\%$
test_step_mdp_speed[False-False-False-True-False]	87.2410μs	39.2914μs	25.4509 KOps/s	25.5457 KOps/s	$\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-False-False-False-True]	77.8120μs	37.6159μs	26.5845 KOps/s	26.7428 KOps/s	$\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-False-False-False-False]	52.7310μs	23.6067μs	42.3609 KOps/s	41.4413 KOps/s	$\color{#35bf28}+2.22\%$
test_values[generalized_advantage_estimate-True-True]	9.9321ms	9.3609ms	106.8269 Ops/s	105.3685 Ops/s	$\color{#35bf28}+1.38\%$
test_values[vec_generalized_advantage_estimate-True-True]	19.0572ms	17.4863ms	57.1875 Ops/s	57.4937 Ops/s	$\color{#d91a1a}-0.53\%$
test_values[td0_return_estimate-False-False]	0.2134ms	0.1245ms	8.0323 KOps/s	7.8177 KOps/s	$\color{#35bf28}+2.74\%$
test_values[td1_return_estimate-False-False]	25.8574ms	25.5654ms	39.1154 Ops/s	39.1238 Ops/s	$\color{#d91a1a}-0.02\%$
test_values[vec_td1_return_estimate-False-False]	18.0015ms	17.5696ms	56.9166 Ops/s	57.1755 Ops/s	$\color{#d91a1a}-0.45\%$
test_values[td_lambda_return_estimate-True-False]	38.8846ms	38.3442ms	26.0795 Ops/s	26.2631 Ops/s	$\color{#d91a1a}-0.70\%$
test_values[vec_td_lambda_return_estimate-True-False]	18.5097ms	17.5676ms	56.9229 Ops/s	57.4008 Ops/s	$\color{#d91a1a}-0.83\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.1099ms	8.0403ms	124.3732 Ops/s	121.7753 Ops/s	$\color{#35bf28}+2.13\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.7239ms	1.5251ms	655.7033 Ops/s	659.0418 Ops/s	$\color{#d91a1a}-0.51\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7267ms	0.3889ms	2.5716 KOps/s	2.4816 KOps/s	$\color{#35bf28}+3.63\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	34.5277ms	34.0326ms	29.3836 Ops/s	33.6005 Ops/s	$\textbf{\color{#d91a1a}-12.55\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.8545ms	1.7281ms	578.6796 Ops/s	592.0556 Ops/s	$\color{#d91a1a}-2.26\%$
test_dqn_speed[False-None]	6.3459ms	1.3970ms	715.8375 Ops/s	714.1222 Ops/s	$\color{#35bf28}+0.24\%$
test_dqn_speed[False-backward]	1.9155ms	1.8626ms	536.8791 Ops/s	530.2604 Ops/s	$\color{#35bf28}+1.25\%$
test_dqn_speed[True-None]	0.8856ms	0.4984ms	2.0064 KOps/s	1.9635 KOps/s	$\color{#35bf28}+2.19\%$
test_dqn_speed[True-backward]	0.9757ms	0.9391ms	1.0648 KOps/s	881.0931 Ops/s	$\textbf{\color{#35bf28}+20.85\%}$
test_dqn_speed[reduce-overhead-None]	0.8829ms	0.4924ms	2.0308 KOps/s	1.9230 KOps/s	$\textbf{\color{#35bf28}+5.60\%}$
test_dqn_speed[reduce-overhead-backward]	0.9930ms	0.9227ms	1.0837 KOps/s	888.0140 Ops/s	$\textbf{\color{#35bf28}+22.04\%}$
test_ddpg_speed[False-None]	3.1322ms	2.8214ms	354.4322 Ops/s	349.8101 Ops/s	$\color{#35bf28}+1.32\%$
test_ddpg_speed[False-backward]	4.1726ms	3.9934ms	250.4155 Ops/s	250.1849 Ops/s	$\color{#35bf28}+0.09\%$
test_ddpg_speed[True-None]	1.7471ms	1.3420ms	745.1491 Ops/s	731.2916 Ops/s	$\color{#35bf28}+1.89\%$
test_ddpg_speed[True-backward]	2.3549ms	2.3034ms	434.1493 Ops/s	426.1657 Ops/s	$\color{#35bf28}+1.87\%$
test_ddpg_speed[reduce-overhead-None]	1.5120ms	1.3385ms	747.1134 Ops/s	735.4415 Ops/s	$\color{#35bf28}+1.59\%$
test_ddpg_speed[reduce-overhead-backward]	2.4496ms	2.3051ms	433.8191 Ops/s	429.4136 Ops/s	$\color{#35bf28}+1.03\%$
test_sac_speed[False-None]	8.4638ms	7.6994ms	129.8797 Ops/s	129.9358 Ops/s	$\color{#d91a1a}-0.04\%$
test_sac_speed[False-backward]	11.2400ms	10.8668ms	92.0235 Ops/s	92.0147 Ops/s	$+0.01\%$
test_sac_speed[True-None]	2.5449ms	2.0897ms	478.5418 Ops/s	477.1287 Ops/s	$\color{#35bf28}+0.30\%$
test_sac_speed[True-backward]	4.0676ms	3.9850ms	250.9403 Ops/s	247.6832 Ops/s	$\color{#35bf28}+1.32\%$
test_sac_speed[reduce-overhead-None]	2.4357ms	2.0626ms	484.8287 Ops/s	462.2117 Ops/s	$\color{#35bf28}+4.89\%$
test_sac_speed[reduce-overhead-backward]	4.0492ms	3.9631ms	252.3251 Ops/s	193.4663 Ops/s	$\textbf{\color{#35bf28}+30.42\%}$
test_redq_speed[False-None]	10.5477ms	10.1167ms	98.8462 Ops/s	96.8678 Ops/s	$\color{#35bf28}+2.04\%$
test_redq_speed[False-backward]	23.5485ms	18.1462ms	55.1080 Ops/s	55.6347 Ops/s	$\color{#d91a1a}-0.95\%$
test_redq_speed[True-None]	4.6697ms	4.3108ms	231.9731 Ops/s	230.3822 Ops/s	$\color{#35bf28}+0.69\%$
test_redq_speed[True-backward]	10.0429ms	9.7503ms	102.5606 Ops/s	103.1922 Ops/s	$\color{#d91a1a}-0.61\%$
test_redq_speed[reduce-overhead-None]	4.6581ms	4.3017ms	232.4686 Ops/s	230.4710 Ops/s	$\color{#35bf28}+0.87\%$
test_redq_speed[reduce-overhead-backward]	10.1253ms	9.8907ms	101.1056 Ops/s	100.1062 Ops/s	$\color{#35bf28}+1.00\%$
test_redq_deprec_speed[False-None]	11.2662ms	10.7208ms	93.2770 Ops/s	92.0375 Ops/s	$\color{#35bf28}+1.35\%$
test_redq_deprec_speed[False-backward]	16.0039ms	15.5438ms	64.3342 Ops/s	64.2922 Ops/s	$\color{#35bf28}+0.07\%$
test_redq_deprec_speed[True-None]	4.0094ms	3.5487ms	281.7946 Ops/s	270.8303 Ops/s	$\color{#35bf28}+4.05\%$
test_redq_deprec_speed[True-backward]	7.8168ms	7.5465ms	132.5111 Ops/s	130.9526 Ops/s	$\color{#35bf28}+1.19\%$
test_redq_deprec_speed[reduce-overhead-None]	3.8589ms	3.4824ms	287.1608 Ops/s	275.9532 Ops/s	$\color{#35bf28}+4.06\%$
test_redq_deprec_speed[reduce-overhead-backward]	7.6609ms	7.5124ms	133.1124 Ops/s	129.6014 Ops/s	$\color{#35bf28}+2.71\%$
test_td3_speed[False-None]	7.9193ms	7.7405ms	129.1914 Ops/s	129.3984 Ops/s	$\color{#d91a1a}-0.16\%$
test_td3_speed[False-backward]	11.0467ms	10.5131ms	95.1196 Ops/s	95.6284 Ops/s	$\color{#d91a1a}-0.53\%$
test_td3_speed[True-None]	1.7952ms	1.7476ms	572.2295 Ops/s	562.0029 Ops/s	$\color{#35bf28}+1.82\%$
test_td3_speed[True-backward]	3.5733ms	3.4627ms	288.7900 Ops/s	240.9971 Ops/s	$\textbf{\color{#35bf28}+19.83\%}$
test_td3_speed[reduce-overhead-None]	1.7726ms	1.7257ms	579.4769 Ops/s	575.8450 Ops/s	$\color{#35bf28}+0.63\%$
test_td3_speed[reduce-overhead-backward]	3.6604ms	3.5353ms	282.8643 Ops/s	230.5957 Ops/s	$\textbf{\color{#35bf28}+22.67\%}$
test_cql_speed[False-None]	28.5341ms	25.4325ms	39.3198 Ops/s	39.3919 Ops/s	$\color{#d91a1a}-0.18\%$
test_cql_speed[False-backward]	39.1593ms	35.1214ms	28.4727 Ops/s	28.7964 Ops/s	$\color{#d91a1a}-1.12\%$
test_cql_speed[True-None]	15.0616ms	12.2299ms	81.7665 Ops/s	83.8473 Ops/s	$\color{#d91a1a}-2.48\%$
test_cql_speed[True-backward]	18.6260ms	18.2640ms	54.7525 Ops/s	60.1335 Ops/s	$\textbf{\color{#d91a1a}-8.95\%}$
test_cql_speed[reduce-overhead-None]	15.3720ms	12.4824ms	80.1131 Ops/s	79.8940 Ops/s	$\color{#35bf28}+0.27\%$
test_cql_speed[reduce-overhead-backward]	22.7666ms	18.2046ms	54.9311 Ops/s	56.0378 Ops/s	$\color{#d91a1a}-1.98\%$
test_a2c_speed[False-None]	5.5859ms	5.3121ms	188.2499 Ops/s	187.4323 Ops/s	$\color{#35bf28}+0.44\%$
test_a2c_speed[False-backward]	12.1827ms	11.6671ms	85.7108 Ops/s	85.3902 Ops/s	$\color{#35bf28}+0.38\%$
test_a2c_speed[True-None]	3.8104ms	3.6913ms	270.9091 Ops/s	274.5185 Ops/s	$\color{#d91a1a}-1.31\%$
test_a2c_speed[True-backward]	8.6492ms	8.4753ms	117.9894 Ops/s	112.6487 Ops/s	$\color{#35bf28}+4.74\%$
test_a2c_speed[reduce-overhead-None]	3.8105ms	3.6737ms	272.2064 Ops/s	269.5806 Ops/s	$\color{#35bf28}+0.97\%$
test_a2c_speed[reduce-overhead-backward]	9.0885ms	8.7149ms	114.7462 Ops/s	114.5785 Ops/s	$\color{#35bf28}+0.15\%$
test_ppo_speed[False-None]	6.5712ms	5.8941ms	169.6613 Ops/s	173.1476 Ops/s	$\color{#d91a1a}-2.01\%$
test_ppo_speed[False-backward]	12.6351ms	12.3092ms	81.2400 Ops/s	81.7229 Ops/s	$\color{#d91a1a}-0.59\%$
test_ppo_speed[True-None]	3.7870ms	3.6246ms	275.8914 Ops/s	271.5502 Ops/s	$\color{#35bf28}+1.60\%$
test_ppo_speed[True-backward]	8.5727ms	8.3847ms	119.2652 Ops/s	119.7357 Ops/s	$\color{#d91a1a}-0.39\%$
test_ppo_speed[reduce-overhead-None]	3.7539ms	3.5954ms	278.1353 Ops/s	268.4281 Ops/s	$\color{#35bf28}+3.62\%$
test_ppo_speed[reduce-overhead-backward]	8.8297ms	8.5974ms	116.3140 Ops/s	108.2278 Ops/s	$\textbf{\color{#35bf28}+7.47\%}$
test_reinforce_speed[False-None]	4.6814ms	4.4989ms	222.2771 Ops/s	218.1429 Ops/s	$\color{#35bf28}+1.90\%$
test_reinforce_speed[False-backward]	7.6181ms	7.3001ms	136.9851 Ops/s	135.9318 Ops/s	$\color{#35bf28}+0.77\%$
test_reinforce_speed[True-None]	3.1183ms	2.8817ms	347.0191 Ops/s	349.5555 Ops/s	$\color{#d91a1a}-0.73\%$
test_reinforce_speed[True-backward]	7.7721ms	7.5574ms	132.3208 Ops/s	131.8459 Ops/s	$\color{#35bf28}+0.36\%$
test_reinforce_speed[reduce-overhead-None]	3.0922ms	2.8207ms	354.5253 Ops/s	352.4471 Ops/s	$\color{#35bf28}+0.59\%$
test_reinforce_speed[reduce-overhead-backward]	7.9334ms	7.7617ms	128.8371 Ops/s	123.7796 Ops/s	$\color{#35bf28}+4.09\%$
test_iql_speed[False-None]	25.0348ms	19.8939ms	50.2666 Ops/s	50.1999 Ops/s	$\color{#35bf28}+0.13\%$
test_iql_speed[False-backward]	30.6626ms	29.9184ms	33.4243 Ops/s	33.2012 Ops/s	$\color{#35bf28}+0.67\%$
test_iql_speed[True-None]	8.6183ms	8.3876ms	119.2234 Ops/s	117.7441 Ops/s	$\color{#35bf28}+1.26\%$
test_iql_speed[True-backward]	16.9123ms	16.5995ms	60.2427 Ops/s	60.3204 Ops/s	$\color{#d91a1a}-0.13\%$
test_iql_speed[reduce-overhead-None]	8.6663ms	8.4439ms	118.4286 Ops/s	112.4395 Ops/s	$\textbf{\color{#35bf28}+5.33\%}$
test_iql_speed[reduce-overhead-backward]	17.1968ms	16.9703ms	58.9264 Ops/s	58.8643 Ops/s	$\color{#35bf28}+0.11\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.5640ms	6.0046ms	166.5387 Ops/s	167.7196 Ops/s	$\color{#d91a1a}-0.70\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6050ms	0.4000ms	2.5002 KOps/s	2.9100 KOps/s	$\textbf{\color{#d91a1a}-14.08\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5795ms	0.3513ms	2.8462 KOps/s	2.7359 KOps/s	$\color{#35bf28}+4.03\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.9262ms	5.6947ms	175.6006 Ops/s	177.8337 Ops/s	$\color{#d91a1a}-1.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.1311ms	0.3569ms	2.8022 KOps/s	3.3414 KOps/s	$\textbf{\color{#d91a1a}-16.14\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6391ms	0.3252ms	3.0751 KOps/s	4.0139 KOps/s	$\textbf{\color{#d91a1a}-23.39\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.6108ms	1.3634ms	733.4605 Ops/s	769.7463 Ops/s	$\color{#d91a1a}-4.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.4379ms	1.2628ms	791.9052 Ops/s	814.8729 Ops/s	$\color{#d91a1a}-2.82\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	12.8674ms	6.0390ms	165.5900 Ops/s	172.5804 Ops/s	$\color{#d91a1a}-4.05\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	2.0063ms	0.4992ms	2.0034 KOps/s	2.2489 KOps/s	$\textbf{\color{#d91a1a}-10.92\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6986ms	0.4412ms	2.2664 KOps/s	2.2131 KOps/s	$\color{#35bf28}+2.40\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.8284ms	5.6948ms	175.5982 Ops/s	177.1213 Ops/s	$\color{#d91a1a}-0.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	2.1319ms	0.3766ms	2.6553 KOps/s	763.2603 Ops/s	$\textbf{\color{#35bf28}+247.89\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5840ms	0.3577ms	2.7956 KOps/s	2.8191 KOps/s	$\color{#d91a1a}-0.83\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.8381ms	5.6362ms	177.4241 Ops/s	174.9290 Ops/s	$\color{#35bf28}+1.43\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.8679ms	0.3345ms	2.9894 KOps/s	3.2632 KOps/s	$\textbf{\color{#d91a1a}-8.39\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5681ms	0.3406ms	2.9356 KOps/s	3.6775 KOps/s	$\textbf{\color{#d91a1a}-20.17\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.3516ms	5.8261ms	171.6400 Ops/s	168.9344 Ops/s	$\color{#35bf28}+1.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8569ms	0.4576ms	2.1855 KOps/s	2.1146 KOps/s	$\color{#35bf28}+3.35\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.5879ms	0.4012ms	2.4924 KOps/s	2.4432 KOps/s	$\color{#35bf28}+2.01\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.5328s	15.5181ms	64.4410 Ops/s	196.2847 Ops/s	$\textbf{\color{#d91a1a}-67.17\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	31.2943ms	2.4549ms	407.3555 Ops/s	409.5616 Ops/s	$\color{#d91a1a}-0.54\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	7.7391ms	1.1568ms	864.4695 Ops/s	935.4686 Ops/s	$\textbf{\color{#d91a1a}-7.59\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	6.8022ms	5.0313ms	198.7569 Ops/s	56.5506 Ops/s	$\textbf{\color{#35bf28}+251.47\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	8.9678ms	2.0165ms	495.9048 Ops/s	581.8821 Ops/s	$\textbf{\color{#d91a1a}-14.78\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	6.9008ms	1.1735ms	852.1729 Ops/s	804.4107 Ops/s	$\textbf{\color{#35bf28}+5.94\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.4631s	14.3878ms	69.5035 Ops/s	187.0155 Ops/s	$\textbf{\color{#d91a1a}-62.84\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	12.1038ms	2.1199ms	471.7110 Ops/s	472.5268 Ops/s	$\color{#d91a1a}-0.17\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.1044ms	0.9688ms	1.0322 KOps/s	744.5534 Ops/s	$\textbf{\color{#35bf28}+38.64\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	34.5011ms	32.7077ms	30.5738 Ops/s	29.8309 Ops/s	$\color{#35bf28}+2.49\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	19.0456ms	17.3685ms	57.5756 Ops/s	59.2023 Ops/s	$\color{#d91a1a}-2.75\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	35.2067ms	33.3358ms	29.9978 Ops/s	29.5092 Ops/s	$\color{#35bf28}+1.66\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.5699ms	17.7626ms	56.2979 Ops/s	57.8096 Ops/s	$\color{#d91a1a}-2.61\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	37.2662ms	35.0535ms	28.5278 Ops/s	27.8986 Ops/s	$\color{#35bf28}+2.26\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.3348ms	18.7896ms	53.2208 Ops/s	52.9582 Ops/s	$\color{#35bf28}+0.50\%$