Skip to content

[Feature] SAC Trainer #3191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 25 commits into
base: gh/vmoens/154/base
Choose a base branch
from
Open

[Feature] SAC Trainer #3191

wants to merge 25 commits into from
+4 −0

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 14, 2025 •
edited
Loading

@vmoens
Update
4d6443e 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3191

Note: Links to docs will display an error until the docs builds have been completed.

❌ 7 New Failures, 8 Unrelated Failures

As of commit b7b0605 with merge base 92c20cd (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Oct 14, 2025
Merged
Merged
Merged
Merged
Merged
Open
Open
Open
Open
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 14, 2025
vmoens added 7 commits October 16, 2025 15:00
@vmoens
Update
ec4e903 
[ghstack-poisoned]
@vmoens
Update
58b15b5 
[ghstack-poisoned]
@vmoens
Update
3e42d1f 
[ghstack-poisoned]
@vmoens
Update
3ffb582 
[ghstack-poisoned]
@vmoens
Update
b0b8b39 
[ghstack-poisoned]
@vmoens
Update
3b61b27 
[ghstack-poisoned]
@vmoens
Update
fd9dec0 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 18, 2025
Open
vmoens added 4 commits October 18, 2025 19:00
@vmoens
Update
26026d4 
[ghstack-poisoned]
@vmoens
Update
a411944 
[ghstack-poisoned]
@vmoens
Update
7739792 
[ghstack-poisoned]
@vmoens
Update
f9a29c8 
[ghstack-poisoned]
@vmoens vmoens added the enhancement New feature or request label Oct 20, 2025
@vmoens
Update
579b61b 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
a3a0abc 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
072605a 
[ghstack-poisoned]
@github-actions
Copy link

github-actions bot commented Oct 20, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}30$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 84.2972μs 82.5072μs 12.1202 KOps/s 12.2197 KOps/s $\color{#d91a1a}-0.81\%$
test_tensor_to_bytestream_speed[torch.save] 0.1420ms 0.1406ms 7.1120 KOps/s 7.0690 KOps/s $\color{#35bf28}+0.61\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1268s 0.1261s 7.9323 Ops/s 7.9723 Ops/s $\color{#d91a1a}-0.50\%$
test_tensor_to_bytestream_speed[numpy] 2.8556μs 2.8472μs 351.2218 KOps/s 351.7945 KOps/s $\color{#d91a1a}-0.16\%$
test_tensor_to_bytestream_speed[safetensors] 44.4992μs 42.8003μs 23.3643 KOps/s 22.4260 KOps/s $\color{#35bf28}+4.18\%$
test_simple 0.5667s 0.5620s 1.7792 Ops/s 1.7128 Ops/s $\color{#35bf28}+3.87\%$
test_transformed 1.2318s 1.1369s 0.8796 Ops/s 0.8760 Ops/s $\color{#35bf28}+0.41\%$
test_serial 1.6624s 1.6570s 0.6035 Ops/s 0.5858 Ops/s $\color{#35bf28}+3.03\%$
test_parallel 1.1804s 1.1103s 0.9007 Ops/s 0.8544 Ops/s $\textbf{\color{#35bf28}+5.41\%}$
test_step_mdp_speed[True-True-True-True-True] 0.1627ms 44.3544μs 22.5457 KOps/s 22.0299 KOps/s $\color{#35bf28}+2.34\%$
test_step_mdp_speed[True-True-True-True-False] 56.0510μs 25.6007μs 39.0615 KOps/s 38.9075 KOps/s $\color{#35bf28}+0.40\%$
test_step_mdp_speed[True-True-True-False-True] 48.5810μs 24.9764μs 40.0378 KOps/s 37.9210 KOps/s $\textbf{\color{#35bf28}+5.58\%}$
test_step_mdp_speed[True-True-True-False-False] 84.6110μs 14.1577μs 70.6330 KOps/s 69.6642 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-True-False-True-True] 83.0520μs 48.4663μs 20.6329 KOps/s 20.3714 KOps/s $\color{#35bf28}+1.28\%$
test_step_mdp_speed[True-True-False-True-False] 69.9420μs 27.9678μs 35.7554 KOps/s 35.7112 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[True-True-False-False-True] 54.4210μs 27.8002μs 35.9710 KOps/s 33.8934 KOps/s $\textbf{\color{#35bf28}+6.13\%}$
test_step_mdp_speed[True-True-False-False-False] 47.4310μs 16.8230μs 59.4425 KOps/s 57.4470 KOps/s $\color{#35bf28}+3.47\%$
test_step_mdp_speed[True-False-True-True-True] 0.1279ms 50.7328μs 19.7111 KOps/s 19.2322 KOps/s $\color{#35bf28}+2.49\%$
test_step_mdp_speed[True-False-True-True-False] 59.8820μs 30.3542μs 32.9443 KOps/s 32.3074 KOps/s $\color{#35bf28}+1.97\%$
test_step_mdp_speed[True-False-True-False-True] 52.7510μs 27.3782μs 36.5254 KOps/s 34.8793 KOps/s $\color{#35bf28}+4.72\%$
test_step_mdp_speed[True-False-True-False-False] 40.2300μs 16.4274μs 60.8738 KOps/s 57.9569 KOps/s $\textbf{\color{#35bf28}+5.03\%}$
test_step_mdp_speed[True-False-False-True-True] 0.1051ms 52.5560μs 19.0273 KOps/s 18.2709 KOps/s $\color{#35bf28}+4.14\%$
test_step_mdp_speed[True-False-False-True-False] 0.1066ms 32.8566μs 30.4353 KOps/s 29.7726 KOps/s $\color{#35bf28}+2.23\%$
test_step_mdp_speed[True-False-False-False-True] 70.8120μs 30.8140μs 32.4528 KOps/s 32.2033 KOps/s $\color{#35bf28}+0.77\%$
test_step_mdp_speed[True-False-False-False-False] 60.0420μs 19.2198μs 52.0297 KOps/s 50.7713 KOps/s $\color{#35bf28}+2.48\%$
test_step_mdp_speed[False-True-True-True-True] 88.6320μs 50.4332μs 19.8282 KOps/s 19.0636 KOps/s $\color{#35bf28}+4.01\%$
test_step_mdp_speed[False-True-True-True-False] 60.9410μs 31.0076μs 32.2501 KOps/s 31.8085 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[False-True-True-False-True] 2.5667ms 32.0842μs 31.1680 KOps/s 29.8410 KOps/s $\color{#35bf28}+4.45\%$
test_step_mdp_speed[False-True-True-False-False] 42.9100μs 18.4011μs 54.3444 KOps/s 52.3288 KOps/s $\color{#35bf28}+3.85\%$
test_step_mdp_speed[False-True-False-True-True] 81.5120μs 52.5725μs 19.0214 KOps/s 18.4633 KOps/s $\color{#35bf28}+3.02\%$
test_step_mdp_speed[False-True-False-True-False] 61.1210μs 33.1070μs 30.2051 KOps/s 29.4262 KOps/s $\color{#35bf28}+2.65\%$
test_step_mdp_speed[False-True-False-False-True] 64.5420μs 34.1450μs 29.2869 KOps/s 27.8204 KOps/s $\textbf{\color{#35bf28}+5.27\%}$
test_step_mdp_speed[False-True-False-False-False] 50.0610μs 21.0873μs 47.4219 KOps/s 46.3091 KOps/s $\color{#35bf28}+2.40\%$
test_step_mdp_speed[False-False-True-True-True] 84.5410μs 55.8519μs 17.9045 KOps/s 17.5804 KOps/s $\color{#35bf28}+1.84\%$
test_step_mdp_speed[False-False-True-True-False] 69.6210μs 35.9821μs 27.7916 KOps/s 27.1101 KOps/s $\color{#35bf28}+2.51\%$
test_step_mdp_speed[False-False-True-False-True] 59.8010μs 33.9982μs 29.4133 KOps/s 27.9270 KOps/s $\textbf{\color{#35bf28}+5.32\%}$
test_step_mdp_speed[False-False-True-False-False] 93.7010μs 21.1963μs 47.1780 KOps/s 45.7809 KOps/s $\color{#35bf28}+3.05\%$
test_step_mdp_speed[False-False-False-True-True] 95.9210μs 58.5278μs 17.0859 KOps/s 16.7926 KOps/s $\color{#35bf28}+1.75\%$
test_step_mdp_speed[False-False-False-True-False] 63.8010μs 38.2310μs 26.1568 KOps/s 25.4638 KOps/s $\color{#35bf28}+2.72\%$
test_step_mdp_speed[False-False-False-False-True] 78.2720μs 36.1334μs 27.6752 KOps/s 26.6060 KOps/s $\color{#35bf28}+4.02\%$
test_step_mdp_speed[False-False-False-False-False] 51.5010μs 23.5761μs 42.4158 KOps/s 41.8051 KOps/s $\color{#35bf28}+1.46\%$
test_values[generalized_advantage_estimate-True-True] 10.3372ms 10.1466ms 98.5556 Ops/s 98.4667 Ops/s $\color{#35bf28}+0.09\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.4769ms 17.7394ms 56.3716 Ops/s 56.5532 Ops/s $\color{#d91a1a}-0.32\%$
test_values[td0_return_estimate-False-False] 0.2477ms 0.1310ms 7.6364 KOps/s 7.8984 KOps/s $\color{#d91a1a}-3.32\%$
test_values[td1_return_estimate-False-False] 28.4939ms 27.9669ms 35.7566 Ops/s 35.3388 Ops/s $\color{#35bf28}+1.18\%$
test_values[vec_td1_return_estimate-False-False] 18.1158ms 17.7812ms 56.2392 Ops/s 56.7512 Ops/s $\color{#d91a1a}-0.90\%$
test_values[td_lambda_return_estimate-True-False] 44.4941ms 42.3025ms 23.6393 Ops/s 23.6928 Ops/s $\color{#d91a1a}-0.23\%$
test_values[vec_td_lambda_return_estimate-True-False] 18.0691ms 17.7759ms 56.2561 Ops/s 56.2749 Ops/s $\color{#d91a1a}-0.03\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.9622ms 8.8177ms 113.4084 Ops/s 113.1031 Ops/s $\color{#35bf28}+0.27\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7036ms 1.4837ms 674.0059 Ops/s 641.4637 Ops/s $\textbf{\color{#35bf28}+5.07\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4685ms 0.4210ms 2.3752 KOps/s 2.3981 KOps/s $\color{#d91a1a}-0.95\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 35.3263ms 29.5912ms 33.7938 Ops/s 28.6154 Ops/s $\textbf{\color{#35bf28}+18.10\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.9062ms 1.7369ms 575.7473 Ops/s 579.1973 Ops/s $\color{#d91a1a}-0.60\%$
test_dqn_speed[False-None] 1.6313ms 1.4515ms 688.9506 Ops/s 696.3188 Ops/s $\color{#d91a1a}-1.06\%$
test_dqn_speed[False-backward] 2.2715ms 1.9789ms 505.3431 Ops/s 500.0567 Ops/s $\color{#35bf28}+1.06\%$
test_dqn_speed[True-None] 0.6343ms 0.5360ms 1.8657 KOps/s 1.7912 KOps/s $\color{#35bf28}+4.16\%$
test_dqn_speed[True-backward] 1.0166ms 0.9837ms 1.0165 KOps/s 844.9698 Ops/s $\textbf{\color{#35bf28}+20.30\%}$
test_dqn_speed[reduce-overhead-None] 0.6791ms 0.5245ms 1.9066 KOps/s 1.7752 KOps/s $\textbf{\color{#35bf28}+7.40\%}$
test_dqn_speed[reduce-overhead-backward] 1.0202ms 0.9713ms 1.0295 KOps/s 1.0124 KOps/s $\color{#35bf28}+1.69\%$
test_ddpg_speed[False-None] 3.2958ms 2.9477ms 339.2493 Ops/s 341.4338 Ops/s $\color{#d91a1a}-0.64\%$
test_ddpg_speed[False-backward] 4.3287ms 4.2421ms 235.7342 Ops/s 239.6826 Ops/s $\color{#d91a1a}-1.65\%$
test_ddpg_speed[True-None] 1.4865ms 1.4150ms 706.7229 Ops/s 624.3783 Ops/s $\textbf{\color{#35bf28}+13.19\%}$
test_ddpg_speed[True-backward] 2.4808ms 2.4283ms 411.8149 Ops/s 411.0097 Ops/s $\color{#35bf28}+0.20\%$
test_ddpg_speed[reduce-overhead-None] 1.5452ms 1.4060ms 711.2151 Ops/s 696.2404 Ops/s $\color{#35bf28}+2.15\%$
test_ddpg_speed[reduce-overhead-backward] 2.5291ms 2.4085ms 415.1922 Ops/s 403.7455 Ops/s $\color{#35bf28}+2.84\%$
test_sac_speed[False-None] 8.4403ms 7.9730ms 125.4234 Ops/s 123.6024 Ops/s $\color{#35bf28}+1.47\%$
test_sac_speed[False-backward] 11.7829ms 11.3312ms 88.2522 Ops/s 88.0256 Ops/s $\color{#35bf28}+0.26\%$
test_sac_speed[True-None] 2.3043ms 2.1752ms 459.7194 Ops/s 460.5692 Ops/s $\color{#d91a1a}-0.18\%$
test_sac_speed[True-backward] 4.3103ms 4.1636ms 240.1755 Ops/s 214.4442 Ops/s $\textbf{\color{#35bf28}+12.00\%}$
test_sac_speed[reduce-overhead-None] 2.3130ms 2.1760ms 459.5631 Ops/s 452.5387 Ops/s $\color{#35bf28}+1.55\%$
test_sac_speed[reduce-overhead-backward] 4.4102ms 4.1774ms 239.3842 Ops/s 220.6298 Ops/s $\textbf{\color{#35bf28}+8.50\%}$
test_redq_speed[False-None] 11.2569ms 10.5604ms 94.6938 Ops/s 92.1491 Ops/s $\color{#35bf28}+2.76\%$
test_redq_speed[False-backward] 23.0033ms 18.5866ms 53.8022 Ops/s 51.1835 Ops/s $\textbf{\color{#35bf28}+5.12\%}$
test_redq_speed[True-None] 4.9929ms 4.6038ms 217.2142 Ops/s 215.5294 Ops/s $\color{#35bf28}+0.78\%$
test_redq_speed[True-backward] 10.8213ms 10.1919ms 98.1172 Ops/s 91.0744 Ops/s $\textbf{\color{#35bf28}+7.73\%}$
test_redq_speed[reduce-overhead-None] 4.8880ms 4.5083ms 221.8144 Ops/s 210.9437 Ops/s $\textbf{\color{#35bf28}+5.15\%}$
test_redq_speed[reduce-overhead-backward] 11.0046ms 10.4058ms 96.1000 Ops/s 91.4727 Ops/s $\textbf{\color{#35bf28}+5.06\%}$
test_redq_deprec_speed[False-None] 11.9648ms 11.4712ms 87.1750 Ops/s 88.2293 Ops/s $\color{#d91a1a}-1.19\%$
test_redq_deprec_speed[False-backward] 17.1858ms 16.4669ms 60.7280 Ops/s 60.8028 Ops/s $\color{#d91a1a}-0.12\%$
test_redq_deprec_speed[True-None] 4.1101ms 3.7384ms 267.4942 Ops/s 256.1702 Ops/s $\color{#35bf28}+4.42\%$
test_redq_deprec_speed[True-backward] 8.1320ms 7.8662ms 127.1263 Ops/s 121.2741 Ops/s $\color{#35bf28}+4.83\%$
test_redq_deprec_speed[reduce-overhead-None] 3.8666ms 3.6544ms 273.6417 Ops/s 265.9446 Ops/s $\color{#35bf28}+2.89\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.2876ms 7.8694ms 127.0744 Ops/s 122.2885 Ops/s $\color{#35bf28}+3.91\%$
test_td3_speed[False-None] 8.3506ms 8.1037ms 123.4002 Ops/s 123.1866 Ops/s $\color{#35bf28}+0.17\%$
test_td3_speed[False-backward] 11.5961ms 11.0668ms 90.3601 Ops/s 90.5353 Ops/s $\color{#d91a1a}-0.19\%$
test_td3_speed[True-None] 1.9253ms 1.8579ms 538.2310 Ops/s 539.8981 Ops/s $\color{#d91a1a}-0.31\%$
test_td3_speed[True-backward] 4.0145ms 3.7174ms 269.0046 Ops/s 269.8774 Ops/s $\color{#d91a1a}-0.32\%$
test_td3_speed[reduce-overhead-None] 1.8704ms 1.8275ms 547.2018 Ops/s 547.7690 Ops/s $\color{#d91a1a}-0.10\%$
test_td3_speed[reduce-overhead-backward] 3.9605ms 3.7673ms 265.4413 Ops/s 266.0423 Ops/s $\color{#d91a1a}-0.23\%$
test_cql_speed[False-None] 29.3779ms 26.7364ms 37.4022 Ops/s 37.2629 Ops/s $\color{#35bf28}+0.37\%$
test_cql_speed[False-backward] 36.9112ms 36.2828ms 27.5612 Ops/s 27.4817 Ops/s $\color{#35bf28}+0.29\%$
test_cql_speed[True-None] 13.3645ms 12.9037ms 77.4972 Ops/s 78.5870 Ops/s $\color{#d91a1a}-1.39\%$
test_cql_speed[True-backward] 19.5042ms 19.0188ms 52.5796 Ops/s 52.4784 Ops/s $\color{#35bf28}+0.19\%$
test_cql_speed[reduce-overhead-None] 13.2268ms 12.7882ms 78.1970 Ops/s 73.9647 Ops/s $\textbf{\color{#35bf28}+5.72\%}$
test_cql_speed[reduce-overhead-backward] 19.7118ms 19.1253ms 52.2867 Ops/s 51.9218 Ops/s $\color{#35bf28}+0.70\%$
test_a2c_speed[False-None] 5.8453ms 5.5976ms 178.6491 Ops/s 181.3313 Ops/s $\color{#d91a1a}-1.48\%$
test_a2c_speed[False-backward] 12.7410ms 12.3503ms 80.9696 Ops/s 81.8137 Ops/s $\color{#d91a1a}-1.03\%$
test_a2c_speed[True-None] 3.9615ms 3.7864ms 264.1004 Ops/s 265.5826 Ops/s $\color{#d91a1a}-0.56\%$
test_a2c_speed[True-backward] 9.2963ms 8.9006ms 112.3516 Ops/s 111.8839 Ops/s $\color{#35bf28}+0.42\%$
test_a2c_speed[reduce-overhead-None] 4.2121ms 3.7954ms 263.4766 Ops/s 267.9445 Ops/s $\color{#d91a1a}-1.67\%$
test_a2c_speed[reduce-overhead-backward] 9.3045ms 9.0735ms 110.2116 Ops/s 109.8719 Ops/s $\color{#35bf28}+0.31\%$
test_ppo_speed[False-None] 6.3140ms 6.0891ms 164.2281 Ops/s 166.6159 Ops/s $\color{#d91a1a}-1.43\%$
test_ppo_speed[False-backward] 13.2503ms 12.9198ms 77.4006 Ops/s 76.6961 Ops/s $\color{#35bf28}+0.92\%$
test_ppo_speed[True-None] 3.8427ms 3.7287ms 268.1883 Ops/s 270.8751 Ops/s $\color{#d91a1a}-0.99\%$
test_ppo_speed[True-backward] 9.0515ms 8.7151ms 114.7439 Ops/s 115.5698 Ops/s $\color{#d91a1a}-0.71\%$
test_ppo_speed[reduce-overhead-None] 3.8184ms 3.6639ms 272.9324 Ops/s 270.4855 Ops/s $\color{#35bf28}+0.90\%$
test_ppo_speed[reduce-overhead-backward] 9.4416ms 8.9609ms 111.5961 Ops/s 96.9761 Ops/s $\textbf{\color{#35bf28}+15.08\%}$
test_reinforce_speed[False-None] 5.1870ms 4.7017ms 212.6882 Ops/s 210.7248 Ops/s $\color{#35bf28}+0.93\%$
test_reinforce_speed[False-backward] 7.8499ms 7.6347ms 130.9801 Ops/s 130.6083 Ops/s $\color{#35bf28}+0.28\%$
test_reinforce_speed[True-None] 3.3623ms 2.9513ms 338.8283 Ops/s 333.7721 Ops/s $\color{#35bf28}+1.51\%$
test_reinforce_speed[True-backward] 8.4543ms 8.0730ms 123.8691 Ops/s 121.5869 Ops/s $\color{#35bf28}+1.88\%$
test_reinforce_speed[reduce-overhead-None] 3.7274ms 2.9749ms 336.1412 Ops/s 328.0067 Ops/s $\color{#35bf28}+2.48\%$
test_reinforce_speed[reduce-overhead-backward] 8.4098ms 8.1305ms 122.9944 Ops/s 122.2019 Ops/s $\color{#35bf28}+0.65\%$
test_iql_speed[False-None] 24.9613ms 20.7164ms 48.2708 Ops/s 48.8903 Ops/s $\color{#d91a1a}-1.27\%$
test_iql_speed[False-backward] 33.4824ms 31.6081ms 31.6374 Ops/s 32.0621 Ops/s $\color{#d91a1a}-1.32\%$
test_iql_speed[True-None] 11.9096ms 9.2767ms 107.7971 Ops/s 114.1593 Ops/s $\textbf{\color{#d91a1a}-5.57\%}$
test_iql_speed[True-backward] 18.1264ms 17.1735ms 58.2293 Ops/s 56.5855 Ops/s $\color{#35bf28}+2.90\%$
test_iql_speed[reduce-overhead-None] 9.1956ms 8.7589ms 114.1691 Ops/s 109.5407 Ops/s $\color{#35bf28}+4.23\%$
test_iql_speed[reduce-overhead-backward] 18.0594ms 17.5820ms 56.8764 Ops/s 56.8607 Ops/s $\color{#35bf28}+0.03\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5558ms 6.0905ms 164.1914 Ops/s 166.4488 Ops/s $\color{#d91a1a}-1.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5264ms 0.3101ms 3.2250 KOps/s 2.8691 KOps/s $\textbf{\color{#35bf28}+12.41\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6906ms 0.3368ms 2.9688 KOps/s 3.0270 KOps/s $\color{#d91a1a}-1.92\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.3202ms 5.8079ms 172.1796 Ops/s 173.6459 Ops/s $\color{#d91a1a}-0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7220ms 0.3143ms 3.1813 KOps/s 2.7647 KOps/s $\textbf{\color{#35bf28}+15.07\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5142ms 0.3048ms 3.2804 KOps/s 3.5276 KOps/s $\textbf{\color{#d91a1a}-7.01\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6627ms 1.3623ms 734.0552 Ops/s 698.6351 Ops/s $\textbf{\color{#35bf28}+5.07\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6376ms 1.1997ms 833.5550 Ops/s 792.9994 Ops/s $\textbf{\color{#35bf28}+5.11\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.5208ms 6.1411ms 162.8365 Ops/s 170.2154 Ops/s $\color{#d91a1a}-4.34\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.9537ms 0.4744ms 2.1080 KOps/s 2.1405 KOps/s $\color{#d91a1a}-1.52\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.5748ms 0.4262ms 2.3463 KOps/s 2.4003 KOps/s $\color{#d91a1a}-2.25\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.9365ms 5.8327ms 171.4482 Ops/s 173.7950 Ops/s $\color{#d91a1a}-1.35\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6793ms 0.2943ms 3.3974 KOps/s 3.0757 KOps/s $\textbf{\color{#35bf28}+10.46\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4985ms 0.2708ms 3.6932 KOps/s 3.2362 KOps/s $\textbf{\color{#35bf28}+14.12\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0307ms 5.7641ms 173.4882 Ops/s 174.0714 Ops/s $\color{#d91a1a}-0.34\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8325ms 0.3218ms 3.1078 KOps/s 2.7405 KOps/s $\textbf{\color{#35bf28}+13.40\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6570ms 0.3351ms 2.9841 KOps/s 2.8971 KOps/s $\color{#35bf28}+3.00\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.3954ms 5.9666ms 167.6001 Ops/s 169.0161 Ops/s $\color{#d91a1a}-0.84\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.5338s 1.2325ms 811.3746 Ops/s 1.9583 KOps/s $\textbf{\color{#d91a1a}-58.57\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6563ms 0.4776ms 2.0940 KOps/s 2.0206 KOps/s $\color{#35bf28}+3.63\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.7447ms 5.0891ms 196.4988 Ops/s 189.0974 Ops/s $\color{#35bf28}+3.91\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 8.4275ms 2.1106ms 473.8001 Ops/s 420.5194 Ops/s $\textbf{\color{#35bf28}+12.67\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.9971ms 1.2127ms 824.6356 Ops/s 916.5616 Ops/s $\textbf{\color{#d91a1a}-10.03\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.9654ms 5.1488ms 194.2216 Ops/s 54.8025 Ops/s $\textbf{\color{#35bf28}+254.40\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.6044ms 2.0461ms 488.7418 Ops/s 700.7001 Ops/s $\textbf{\color{#d91a1a}-30.25\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.2806ms 1.2374ms 808.1639 Ops/s 749.1858 Ops/s $\textbf{\color{#35bf28}+7.87\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.5120s 15.5027ms 64.5047 Ops/s 181.2906 Ops/s $\textbf{\color{#d91a1a}-64.42\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 12.0014ms 2.1468ms 465.8179 Ops/s 455.7472 Ops/s $\color{#35bf28}+2.21\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.3574ms 1.0268ms 973.9029 Ops/s 720.0375 Ops/s $\textbf{\color{#35bf28}+35.26\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 35.1157ms 33.2289ms 30.0943 Ops/s 29.9792 Ops/s $\color{#35bf28}+0.38\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.2521ms 17.3922ms 57.4970 Ops/s 56.4663 Ops/s $\color{#35bf28}+1.83\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 37.1557ms 34.3174ms 29.1398 Ops/s 29.2685 Ops/s $\color{#d91a1a}-0.44\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.7757ms 17.8486ms 56.0267 Ops/s 56.5329 Ops/s $\color{#d91a1a}-0.90\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 37.6114ms 35.5081ms 28.1626 Ops/s 26.9536 Ops/s $\color{#35bf28}+4.49\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.5896ms 19.3071ms 51.7945 Ops/s 51.1815 Ops/s $\color{#35bf28}+1.20\%$

vmoens added 7 commits October 19, 2025 20:56
@vmoens
Update
39f3850 
[ghstack-poisoned]
@vmoens
Update
70d35e9 
[ghstack-poisoned]
@vmoens
Update
72d9116 
[ghstack-poisoned]
@vmoens
Update
dfaa0fa 
[ghstack-poisoned]
@vmoens
Update
264938d 
[ghstack-poisoned]
@vmoens
Update
f2901e8 
[ghstack-poisoned]
@vmoens
Update
4051ef3 
[ghstack-poisoned]
This was referenced Oct 22, 2025
Open
Open
vmoens added 3 commits October 22, 2025 12:31
@vmoens
Update
cb20e00 
[ghstack-poisoned]
@vmoens
Update
70768b5 
[ghstack-poisoned]
@vmoens
Update
b7b0605 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens