Skip to content

[Feature] Trainer Algorithms - Configuration System #3189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 25 commits into
base: gh/vmoens/152/base
Choose a base branch
from
Open

[Feature] Trainer Algorithms - Configuration System #3189

wants to merge 25 commits into from

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 14, 2025 •
edited
Loading

@vmoens
Update
40c8b63 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3189

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 9 Unrelated Failures

As of commit 1ff164a with merge base 92c20cd (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Oct 14, 2025
Merged
Merged
Merged
Merged
Merged
Open
Open
Open
Open
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 14, 2025
vmoens added 7 commits October 16, 2025 15:00
@vmoens
Update
74bfb5a 
[ghstack-poisoned]
@vmoens
Update
bfd08f5 
[ghstack-poisoned]
@vmoens
Update
eb85308 
[ghstack-poisoned]
@vmoens
Update
0d1f7ce 
[ghstack-poisoned]
@vmoens
Update
351326c 
[ghstack-poisoned]
@vmoens
Update
bb95ddb 
[ghstack-poisoned]
@vmoens
Update
5fcf578 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 18, 2025
Open
vmoens added 4 commits October 18, 2025 19:00
@vmoens
Update
67659d3 
[ghstack-poisoned]
@vmoens
Update
1f9b4c8 
[ghstack-poisoned]
@vmoens
Update
36e3097 
[ghstack-poisoned]
@vmoens
Update
5b3b689 
[ghstack-poisoned]
@vmoens vmoens added the enhancement New feature or request label Oct 20, 2025
@vmoens
Update
ddb96c1 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
920f08a 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
e5e892f 
[ghstack-poisoned]
@github-actions
Copy link

github-actions bot commented Oct 20, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}15$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 84.0713μs 82.4300μs 12.1315 KOps/s 12.1085 KOps/s $\color{#35bf28}+0.19\%$
test_tensor_to_bytestream_speed[torch.save] 0.1474ms 0.1436ms 6.9653 KOps/s 7.0330 KOps/s $\color{#d91a1a}-0.96\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1251s 0.1244s 8.0363 Ops/s 7.9858 Ops/s $\color{#35bf28}+0.63\%$
test_tensor_to_bytestream_speed[numpy] 2.7901μs 2.7878μs 358.7042 KOps/s 359.6585 KOps/s $\color{#d91a1a}-0.27\%$
test_tensor_to_bytestream_speed[safetensors] 45.3634μs 45.0476μs 22.1987 KOps/s 23.7944 KOps/s $\textbf{\color{#d91a1a}-6.71\%}$
test_simple 0.5678s 0.5584s 1.7909 Ops/s 1.7242 Ops/s $\color{#35bf28}+3.87\%$
test_transformed 1.1262s 1.1247s 0.8891 Ops/s 0.8703 Ops/s $\color{#35bf28}+2.16\%$
test_serial 1.6978s 1.6943s 0.5902 Ops/s 0.5822 Ops/s $\color{#35bf28}+1.37\%$
test_parallel 1.2080s 1.1047s 0.9052 Ops/s 0.8724 Ops/s $\color{#35bf28}+3.77\%$
test_step_mdp_speed[True-True-True-True-True] 0.1248ms 45.1006μs 22.1726 KOps/s 22.2016 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-True-True-True-False] 86.8320μs 25.4276μs 39.3273 KOps/s 38.6982 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-True-True-False-True] 55.5910μs 25.5687μs 39.1103 KOps/s 38.6641 KOps/s $\color{#35bf28}+1.15\%$
test_step_mdp_speed[True-True-True-False-False] 40.2200μs 14.0453μs 71.1980 KOps/s 73.5225 KOps/s $\color{#d91a1a}-3.16\%$
test_step_mdp_speed[True-True-False-True-True] 76.4720μs 48.8492μs 20.4712 KOps/s 20.2931 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[True-True-False-True-False] 54.1410μs 28.1488μs 35.5255 KOps/s 34.9968 KOps/s $\color{#35bf28}+1.51\%$
test_step_mdp_speed[True-True-False-False-True] 52.2110μs 28.1797μs 35.4866 KOps/s 34.6597 KOps/s $\color{#35bf28}+2.39\%$
test_step_mdp_speed[True-True-False-False-False] 83.4420μs 16.7563μs 59.6791 KOps/s 58.6444 KOps/s $\color{#35bf28}+1.76\%$
test_step_mdp_speed[True-False-True-True-True] 84.2420μs 51.8384μs 19.2907 KOps/s 19.3087 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[True-False-True-True-False] 61.6810μs 30.9532μs 32.3068 KOps/s 31.7897 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-False-True-False-True] 62.3810μs 28.5555μs 35.0195 KOps/s 34.8833 KOps/s $\color{#35bf28}+0.39\%$
test_step_mdp_speed[True-False-True-False-False] 49.9610μs 16.8062μs 59.5018 KOps/s 59.0606 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[True-False-False-True-True] 0.1200ms 53.4618μs 18.7050 KOps/s 18.5302 KOps/s $\color{#35bf28}+0.94\%$
test_step_mdp_speed[True-False-False-True-False] 71.3310μs 33.6839μs 29.6878 KOps/s 29.2004 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[True-False-False-False-True] 65.5610μs 30.9070μs 32.3551 KOps/s 32.6168 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[True-False-False-False-False] 49.3510μs 19.4269μs 51.4751 KOps/s 52.1840 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[False-True-True-True-True] 89.3710μs 50.3738μs 19.8516 KOps/s 19.3059 KOps/s $\color{#35bf28}+2.83\%$
test_step_mdp_speed[False-True-True-True-False] 99.4420μs 30.7171μs 32.5551 KOps/s 32.0771 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[False-True-True-False-True] 2.4563ms 32.3896μs 30.8741 KOps/s 30.2580 KOps/s $\color{#35bf28}+2.04\%$
test_step_mdp_speed[False-True-True-False-False] 45.7110μs 18.5958μs 53.7755 KOps/s 51.7726 KOps/s $\color{#35bf28}+3.87\%$
test_step_mdp_speed[False-True-False-True-True] 86.1410μs 53.0926μs 18.8350 KOps/s 18.2760 KOps/s $\color{#35bf28}+3.06\%$
test_step_mdp_speed[False-True-False-True-False] 69.7820μs 33.4108μs 29.9305 KOps/s 28.9557 KOps/s $\color{#35bf28}+3.37\%$
test_step_mdp_speed[False-True-False-False-True] 71.1710μs 34.2534μs 29.1942 KOps/s 28.3006 KOps/s $\color{#35bf28}+3.16\%$
test_step_mdp_speed[False-True-False-False-False] 50.6810μs 21.0987μs 47.3963 KOps/s 45.8964 KOps/s $\color{#35bf28}+3.27\%$
test_step_mdp_speed[False-False-True-True-True] 91.0820μs 56.2452μs 17.7793 KOps/s 17.6076 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-False-True-True-False] 79.6420μs 36.1737μs 27.6444 KOps/s 26.8971 KOps/s $\color{#35bf28}+2.78\%$
test_step_mdp_speed[False-False-True-False-True] 63.8020μs 34.4395μs 29.0364 KOps/s 28.9215 KOps/s $\color{#35bf28}+0.40\%$
test_step_mdp_speed[False-False-True-False-False] 52.8300μs 21.0970μs 47.4001 KOps/s 46.1782 KOps/s $\color{#35bf28}+2.65\%$
test_step_mdp_speed[False-False-False-True-True] 96.9120μs 57.5869μs 17.3650 KOps/s 16.7225 KOps/s $\color{#35bf28}+3.84\%$
test_step_mdp_speed[False-False-False-True-False] 65.9610μs 38.4113μs 26.0340 KOps/s 25.4939 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[False-False-False-False-True] 62.5610μs 37.0701μs 26.9759 KOps/s 26.6491 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[False-False-False-False-False] 52.1310μs 23.6424μs 42.2970 KOps/s 42.0807 KOps/s $\color{#35bf28}+0.51\%$
test_values[generalized_advantage_estimate-True-True] 11.0104ms 10.7220ms 93.2663 Ops/s 93.9239 Ops/s $\color{#d91a1a}-0.70\%$
test_values[vec_generalized_advantage_estimate-True-True] 15.4465ms 11.2269ms 89.0721 Ops/s 87.7092 Ops/s $\color{#35bf28}+1.55\%$
test_values[td0_return_estimate-False-False] 0.2297ms 0.1320ms 7.5781 KOps/s 7.7253 KOps/s $\color{#d91a1a}-1.91\%$
test_values[td1_return_estimate-False-False] 29.5071ms 29.1169ms 34.3444 Ops/s 34.0059 Ops/s $\color{#35bf28}+1.00\%$
test_values[vec_td1_return_estimate-False-False] 11.8242ms 11.2991ms 88.5023 Ops/s 87.1381 Ops/s $\color{#35bf28}+1.57\%$
test_values[td_lambda_return_estimate-True-False] 44.1443ms 43.6266ms 22.9218 Ops/s 22.6514 Ops/s $\color{#35bf28}+1.19\%$
test_values[vec_td_lambda_return_estimate-True-False] 12.2980ms 11.3924ms 87.7777 Ops/s 87.8582 Ops/s $\color{#d91a1a}-0.09\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.4850ms 9.3773ms 106.6400 Ops/s 107.3674 Ops/s $\color{#d91a1a}-0.68\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.0151ms 1.5653ms 638.8383 Ops/s 658.1339 Ops/s $\color{#d91a1a}-2.93\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5724ms 0.4280ms 2.3367 KOps/s 2.3259 KOps/s $\color{#35bf28}+0.46\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 30.7569ms 20.0999ms 49.7515 Ops/s 49.4987 Ops/s $\color{#35bf28}+0.51\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.9580ms 1.7286ms 578.4898 Ops/s 575.7900 Ops/s $\color{#35bf28}+0.47\%$
test_dqn_speed[False-None] 6.5680ms 1.4519ms 688.7484 Ops/s 686.2736 Ops/s $\color{#35bf28}+0.36\%$
test_dqn_speed[False-backward] 2.0151ms 1.9650ms 508.8975 Ops/s 502.9429 Ops/s $\color{#35bf28}+1.18\%$
test_dqn_speed[True-None] 0.9052ms 0.5357ms 1.8666 KOps/s 1.8398 KOps/s $\color{#35bf28}+1.46\%$
test_dqn_speed[True-backward] 1.0509ms 1.0002ms 999.8328 Ops/s 837.0978 Ops/s $\textbf{\color{#35bf28}+19.44\%}$
test_dqn_speed[reduce-overhead-None] 0.9259ms 0.5177ms 1.9315 KOps/s 1.8822 KOps/s $\color{#35bf28}+2.62\%$
test_dqn_speed[reduce-overhead-backward] 1.0289ms 0.9786ms 1.0219 KOps/s 984.2513 Ops/s $\color{#35bf28}+3.82\%$
test_ddpg_speed[False-None] 3.2556ms 2.9316ms 341.1108 Ops/s 339.4482 Ops/s $\color{#35bf28}+0.49\%$
test_ddpg_speed[False-backward] 4.2703ms 4.1713ms 239.7330 Ops/s 239.4214 Ops/s $\color{#35bf28}+0.13\%$
test_ddpg_speed[True-None] 1.5276ms 1.4118ms 708.3395 Ops/s 702.5808 Ops/s $\color{#35bf28}+0.82\%$
test_ddpg_speed[True-backward] 2.5195ms 2.4295ms 411.6140 Ops/s 411.4667 Ops/s $\color{#35bf28}+0.04\%$
test_ddpg_speed[reduce-overhead-None] 1.8011ms 1.4038ms 712.3287 Ops/s 706.9950 Ops/s $\color{#35bf28}+0.75\%$
test_ddpg_speed[reduce-overhead-backward] 2.5414ms 2.4318ms 411.2106 Ops/s 379.8253 Ops/s $\textbf{\color{#35bf28}+8.26\%}$
test_sac_speed[False-None] 8.4577ms 8.0033ms 124.9482 Ops/s 123.8688 Ops/s $\color{#35bf28}+0.87\%$
test_sac_speed[False-backward] 11.8700ms 11.3432ms 88.1583 Ops/s 87.5777 Ops/s $\color{#35bf28}+0.66\%$
test_sac_speed[True-None] 2.5466ms 2.1671ms 461.4475 Ops/s 451.3132 Ops/s $\color{#35bf28}+2.25\%$
test_sac_speed[True-backward] 4.3094ms 4.1627ms 240.2295 Ops/s 209.8360 Ops/s $\textbf{\color{#35bf28}+14.48\%}$
test_sac_speed[reduce-overhead-None] 2.5632ms 2.1575ms 463.4967 Ops/s 449.3661 Ops/s $\color{#35bf28}+3.14\%$
test_sac_speed[reduce-overhead-backward] 4.2837ms 4.1532ms 240.7779 Ops/s 211.8435 Ops/s $\textbf{\color{#35bf28}+13.66\%}$
test_redq_speed[False-None] 11.6586ms 10.7479ms 93.0418 Ops/s 94.0379 Ops/s $\color{#d91a1a}-1.06\%$
test_redq_speed[False-backward] 23.7621ms 19.1394ms 52.2484 Ops/s 55.6260 Ops/s $\textbf{\color{#d91a1a}-6.07\%}$
test_redq_speed[True-None] 4.7252ms 4.4415ms 225.1482 Ops/s 223.6843 Ops/s $\color{#35bf28}+0.65\%$
test_redq_speed[True-backward] 10.4185ms 10.0170ms 99.8307 Ops/s 102.6623 Ops/s $\color{#d91a1a}-2.76\%$
test_redq_speed[reduce-overhead-None] 4.6955ms 4.4896ms 222.7380 Ops/s 210.4697 Ops/s $\textbf{\color{#35bf28}+5.83\%}$
test_redq_speed[reduce-overhead-backward] 10.9416ms 10.4231ms 95.9408 Ops/s 97.7458 Ops/s $\color{#d91a1a}-1.85\%$
test_redq_deprec_speed[False-None] 11.6958ms 11.2681ms 88.7462 Ops/s 86.9812 Ops/s $\color{#35bf28}+2.03\%$
test_redq_deprec_speed[False-backward] 16.9732ms 16.3577ms 61.1334 Ops/s 60.2671 Ops/s $\color{#35bf28}+1.44\%$
test_redq_deprec_speed[True-None] 4.3063ms 3.7381ms 267.5128 Ops/s 262.3663 Ops/s $\color{#35bf28}+1.96\%$
test_redq_deprec_speed[True-backward] 8.2076ms 7.8251ms 127.7945 Ops/s 117.9288 Ops/s $\textbf{\color{#35bf28}+8.37\%}$
test_redq_deprec_speed[reduce-overhead-None] 3.9770ms 3.6778ms 271.8999 Ops/s 269.5773 Ops/s $\color{#35bf28}+0.86\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.0012ms 7.8015ms 128.1797 Ops/s 113.2492 Ops/s $\textbf{\color{#35bf28}+13.18\%}$
test_td3_speed[False-None] 8.0572ms 8.0108ms 124.8308 Ops/s 123.4316 Ops/s $\color{#35bf28}+1.13\%$
test_td3_speed[False-backward] 11.5832ms 10.9485ms 91.3366 Ops/s 90.1604 Ops/s $\color{#35bf28}+1.30\%$
test_td3_speed[True-None] 1.8736ms 1.8343ms 545.1762 Ops/s 532.2768 Ops/s $\color{#35bf28}+2.42\%$
test_td3_speed[True-backward] 3.8073ms 3.6989ms 270.3499 Ops/s 263.9311 Ops/s $\color{#35bf28}+2.43\%$
test_td3_speed[reduce-overhead-None] 1.8543ms 1.8210ms 549.1609 Ops/s 540.5559 Ops/s $\color{#35bf28}+1.59\%$
test_td3_speed[reduce-overhead-backward] 3.8481ms 3.7219ms 268.6828 Ops/s 264.6520 Ops/s $\color{#35bf28}+1.52\%$
test_cql_speed[False-None] 28.2216ms 26.3799ms 37.9076 Ops/s 37.5920 Ops/s $\color{#35bf28}+0.84\%$
test_cql_speed[False-backward] 39.3562ms 36.0846ms 27.7127 Ops/s 27.4154 Ops/s $\color{#35bf28}+1.08\%$
test_cql_speed[True-None] 13.5225ms 12.7801ms 78.2468 Ops/s 77.9470 Ops/s $\color{#35bf28}+0.38\%$
test_cql_speed[True-backward] 21.3596ms 18.9140ms 52.8709 Ops/s 53.7759 Ops/s $\color{#d91a1a}-1.68\%$
test_cql_speed[reduce-overhead-None] 13.1562ms 12.7020ms 78.7280 Ops/s 78.7460 Ops/s $\color{#d91a1a}-0.02\%$
test_cql_speed[reduce-overhead-backward] 20.2574ms 18.9880ms 52.6648 Ops/s 53.0031 Ops/s $\color{#d91a1a}-0.64\%$
test_a2c_speed[False-None] 5.8445ms 5.5854ms 179.0387 Ops/s 178.8543 Ops/s $\color{#35bf28}+0.10\%$
test_a2c_speed[False-backward] 12.6488ms 12.2863ms 81.3912 Ops/s 81.1403 Ops/s $\color{#35bf28}+0.31\%$
test_a2c_speed[True-None] 4.0561ms 3.7264ms 268.3557 Ops/s 257.9722 Ops/s $\color{#35bf28}+4.03\%$
test_a2c_speed[True-backward] 9.1913ms 8.9053ms 112.2922 Ops/s 113.8416 Ops/s $\color{#d91a1a}-1.36\%$
test_a2c_speed[reduce-overhead-None] 4.1296ms 3.7604ms 265.9267 Ops/s 265.3501 Ops/s $\color{#35bf28}+0.22\%$
test_a2c_speed[reduce-overhead-backward] 9.2543ms 8.9761ms 111.4068 Ops/s 110.7364 Ops/s $\color{#35bf28}+0.61\%$
test_ppo_speed[False-None] 6.7919ms 6.0521ms 165.2321 Ops/s 165.9107 Ops/s $\color{#d91a1a}-0.41\%$
test_ppo_speed[False-backward] 13.3879ms 12.9222ms 77.3863 Ops/s 78.5631 Ops/s $\color{#d91a1a}-1.50\%$
test_ppo_speed[True-None] 4.0256ms 3.7100ms 269.5396 Ops/s 268.5681 Ops/s $\color{#35bf28}+0.36\%$
test_ppo_speed[True-backward] 8.9428ms 8.6397ms 115.7450 Ops/s 108.8698 Ops/s $\textbf{\color{#35bf28}+6.32\%}$
test_ppo_speed[reduce-overhead-None] 4.2831ms 3.7006ms 270.2247 Ops/s 261.7434 Ops/s $\color{#35bf28}+3.24\%$
test_ppo_speed[reduce-overhead-backward] 9.4989ms 9.0654ms 110.3091 Ops/s 105.5088 Ops/s $\color{#35bf28}+4.55\%$
test_reinforce_speed[False-None] 5.1492ms 4.7098ms 212.3227 Ops/s 211.3439 Ops/s $\color{#35bf28}+0.46\%$
test_reinforce_speed[False-backward] 7.8524ms 7.5825ms 131.8832 Ops/s 131.8024 Ops/s $\color{#35bf28}+0.06\%$
test_reinforce_speed[True-None] 3.3037ms 2.9258ms 341.7860 Ops/s 338.7229 Ops/s $\color{#35bf28}+0.90\%$
test_reinforce_speed[True-backward] 8.2567ms 7.9381ms 125.9752 Ops/s 110.4712 Ops/s $\textbf{\color{#35bf28}+14.03\%}$
test_reinforce_speed[reduce-overhead-None] 3.2699ms 2.9091ms 343.7460 Ops/s 334.1215 Ops/s $\color{#35bf28}+2.88\%$
test_reinforce_speed[reduce-overhead-backward] 8.4182ms 8.1264ms 123.0563 Ops/s 116.7527 Ops/s $\textbf{\color{#35bf28}+5.40\%}$
test_iql_speed[False-None] 26.0554ms 20.8063ms 48.0623 Ops/s 47.6182 Ops/s $\color{#35bf28}+0.93\%$
test_iql_speed[False-backward] 37.0843ms 31.3894ms 31.8579 Ops/s 31.0892 Ops/s $\color{#35bf28}+2.47\%$
test_iql_speed[True-None] 9.2377ms 8.7449ms 114.3527 Ops/s 111.4114 Ops/s $\color{#35bf28}+2.64\%$
test_iql_speed[True-backward] 17.5662ms 17.1279ms 58.3841 Ops/s 57.8743 Ops/s $\color{#35bf28}+0.88\%$
test_iql_speed[reduce-overhead-None] 8.9511ms 8.7459ms 114.3389 Ops/s 114.0214 Ops/s $\color{#35bf28}+0.28\%$
test_iql_speed[reduce-overhead-backward] 18.1351ms 17.6583ms 56.6305 Ops/s 55.8001 Ops/s $\color{#35bf28}+1.49\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.6862ms 6.0345ms 165.7129 Ops/s 166.3456 Ops/s $\color{#d91a1a}-0.38\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6443ms 0.3400ms 2.9409 KOps/s 3.2585 KOps/s $\textbf{\color{#d91a1a}-9.75\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5265ms 0.2658ms 3.7617 KOps/s 3.2738 KOps/s $\textbf{\color{#35bf28}+14.90\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0861ms 5.8093ms 172.1386 Ops/s 176.4482 Ops/s $\color{#d91a1a}-2.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6518ms 0.3248ms 3.0784 KOps/s 3.5862 KOps/s $\textbf{\color{#d91a1a}-14.16\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5481ms 0.3073ms 3.2543 KOps/s 3.7737 KOps/s $\textbf{\color{#d91a1a}-13.76\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6234ms 1.3766ms 726.4263 Ops/s 728.9793 Ops/s $\color{#d91a1a}-0.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5088ms 1.2831ms 779.3483 Ops/s 777.8082 Ops/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0966ms 5.9613ms 167.7489 Ops/s 169.2608 Ops/s $\color{#d91a1a}-0.89\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9887ms 0.4469ms 2.2377 KOps/s 2.0204 KOps/s $\textbf{\color{#35bf28}+10.76\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7706ms 0.4751ms 2.1050 KOps/s 2.2552 KOps/s $\textbf{\color{#d91a1a}-6.66\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.9398ms 5.7840ms 172.8900 Ops/s 175.1347 Ops/s $\color{#d91a1a}-1.28\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7485ms 0.3478ms 2.8748 KOps/s 798.9608 Ops/s $\textbf{\color{#35bf28}+259.82\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4812ms 0.2675ms 3.7379 KOps/s 3.5656 KOps/s $\color{#35bf28}+4.83\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9945ms 5.7708ms 173.2862 Ops/s 172.2855 Ops/s $\color{#35bf28}+0.58\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.2543ms 0.2923ms 3.4217 KOps/s 3.4359 KOps/s $\color{#d91a1a}-0.41\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8186ms 0.3051ms 3.2775 KOps/s 3.8547 KOps/s $\textbf{\color{#d91a1a}-14.97\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.1173ms 5.9795ms 167.2391 Ops/s 167.0298 Ops/s $\color{#35bf28}+0.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.9677ms 0.5031ms 1.9876 KOps/s 2.3056 KOps/s $\textbf{\color{#d91a1a}-13.79\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7037ms 0.5074ms 1.9708 KOps/s 2.3985 KOps/s $\textbf{\color{#d91a1a}-17.83\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.5603s 16.2280ms 61.6221 Ops/s 192.4011 Ops/s $\textbf{\color{#d91a1a}-67.97\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 4.1768ms 1.7280ms 578.6977 Ops/s 413.3285 Ops/s $\textbf{\color{#35bf28}+40.01\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.9049ms 1.2414ms 805.5107 Ops/s 841.8692 Ops/s $\color{#d91a1a}-4.32\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.9767ms 5.1183ms 195.3767 Ops/s 54.0269 Ops/s $\textbf{\color{#35bf28}+261.63\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.6400ms 2.0846ms 479.7043 Ops/s 480.2131 Ops/s $\color{#d91a1a}-0.11\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.1479ms 1.2243ms 816.7822 Ops/s 897.5610 Ops/s $\textbf{\color{#d91a1a}-9.00\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 6.9595ms 5.3367ms 187.3813 Ops/s 183.0187 Ops/s $\color{#35bf28}+2.38\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 12.0009ms 2.1431ms 466.6115 Ops/s 452.6489 Ops/s $\color{#35bf28}+3.08\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.5110ms 1.3513ms 740.0483 Ops/s 968.1388 Ops/s $\textbf{\color{#d91a1a}-23.56\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 35.7621ms 33.7502ms 29.6295 Ops/s 29.8975 Ops/s $\color{#d91a1a}-0.90\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.7430ms 17.7116ms 56.4601 Ops/s 56.3232 Ops/s $\color{#35bf28}+0.24\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 36.4915ms 34.3063ms 29.1491 Ops/s 28.2976 Ops/s $\color{#35bf28}+3.01\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.4150ms 17.9785ms 55.6221 Ops/s 53.8673 Ops/s $\color{#35bf28}+3.26\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 37.7123ms 36.1384ms 27.6714 Ops/s 27.7290 Ops/s $\color{#d91a1a}-0.21\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.8290ms 19.4712ms 51.3578 Ops/s 50.4087 Ops/s $\color{#35bf28}+1.88\%$

vmoens added 7 commits October 19, 2025 20:56
@vmoens
Update
1591d0e 
[ghstack-poisoned]
@vmoens
Update
2c479eb 
[ghstack-poisoned]
@vmoens
Update
1d4c308 
[ghstack-poisoned]
@vmoens
Update
3092841 
[ghstack-poisoned]
@vmoens
Update
323ed9d 
[ghstack-poisoned]
@vmoens
Update
1cb3227 
[ghstack-poisoned]
@vmoens
Update
826ff12 
[ghstack-poisoned]
This was referenced Oct 22, 2025
Open
Open
vmoens added 3 commits October 22, 2025 12:31
@vmoens
Update
f310600 
[ghstack-poisoned]
@vmoens
Update
f7046b6 
[ghstack-poisoned]
@vmoens
Update
1ff164a 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens