fbgemm op ci test fail #638
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Pull Request resolved: facebookresearch#474 Reviewed By: czxttkl Differential Revision: D28312845 fbshipit-source-id: abb039d445a1228bb11ffb6103744854b209b3dc
Summary: Pull Request resolved: facebookresearch#476 Add a n-gram MLP for synthetic reward attribution. This model uses an MLP to predict each step's reward. Compared with single-step reward model, it uses n-gram with a context window centered around each step and zero padding. Reviewed By: czxttkl Differential Revision: D28362111 fbshipit-source-id: 624de95f14b7fedb79ccb0cd47cb811b651fab04
Summary: Pull Request resolved: facebookresearch#472 Distributed readers are not supported yet, as shown in the test plan below czxttkl. Reviewed By: czxttkl Differential Revision: D28292330 fbshipit-source-id: 0f03d27fdba75740ab9590747ae025c6da6ce9fa
Summary: Pull Request resolved: facebookresearch#478 Reviewed By: bankawas Differential Revision: D28427686 fbshipit-source-id: b53a9f974f9c2ee615fb453b5efe48b9de487dbf
…ch#479) Summary: Pull Request resolved: facebookresearch#479 Making these changes can finally get us distributed training for reward networks (hopefully. Still need to wait for the workflow to finish). Fix the error asked in https://fb.workplace.com/groups/pytorchLightning/permalink/455491295468768/. Reviewed By: gji1 Differential Revision: D28318470 fbshipit-source-id: fe3836ef49864a20af07511a10e25c0d1a20ba0d
Summary: Pull Request resolved: facebookresearch#480 Lower the number of training samples & threshold, use Adam instead of SGD. Reviewed By: j-jiafei Differential Revision: D28464831 fbshipit-source-id: 918329290be62bd846507e2bd3697af4c3e710db
…bookresearch#470) Summary: Pull Request resolved: facebookresearch#470 Reviewed By: czxttkl Differential Revision: D28093192 fbshipit-source-id: 6b260c3e8d49c8b302e40066e2be49a0bfe96688
Summary: Pull Request resolved: facebookresearch#477 Add ConvNet support to n-gram synthetic reward network. Reviewed By: czxttkl Differential Revision: D28402551 fbshipit-source-id: c2201be3d71c32977c2f19b69e5a0abcaf0a855d
Summary: Pull Request resolved: facebookresearch#481 Add LSTM synthetic reward net. Reviewed By: czxttkl Differential Revision: D28448615 fbshipit-source-id: e8c77ef8c7b4ad69fcda2fd432cc018cfb7495cd
Summary: Pull Request resolved: facebookresearch#482 as titled. Also support discrete action. Reviewed By: j-jiafei Differential Revision: D28248528 fbshipit-source-id: bf87afa18914e9331177b22f0c9a823ac2ba2337
…h#483) Summary: Pull Request resolved: facebookresearch#483 As title. Reviewed By: czxttkl Differential Revision: D28551285 fbshipit-source-id: 3cc14daa930399daa0880c8569f8f36b46c1ff94
Summary: Pull Request resolved: facebookresearch#484 Refactoring so that we can use spark transform to bulk eval synthetic reward models. Things changed: 1. Improve API for defining models. In `reagent/models/synthetic_reward.py`, we create `SyntheticRewardNet`, which takes in different architecture implementations with standardized input/output shapes. 2. Net builders will build different architectures to construct `SyntheticRewardNet`. So we follow a composite pattern in net builders. 3. All net builders now share the same `build_serving_module` method. 4. Improve test methods so they share as much code as possible between different architectures. Reviewed By: j-jiafei Differential Revision: D28549704 fbshipit-source-id: 535a6191b6cfc4c55ed8b4f8c366af77ceac5c79
Summary: Added binary_difference_scorer to discrete_dqn.py Reviewed By: czxttkl Differential Revision: D28691568 fbshipit-source-id: dd9fe5518b13aea2acb94dae10823cdfd9253926
…cebookresearch#485) Summary: Pull Request resolved: facebookresearch#485 As title. Reviewed By: czxttkl Differential Revision: D28790947 fbshipit-source-id: 26405326402a0b913731c2a9ccb4badde4b47a9b
…s set up, required in lightning 1.3.3 Summary: with move to lightning 1.3 (D28792413), MDNRNNTrainer cannot call self.log() without setting up a LoggerConnector Reviewed By: kandluis Differential Revision: D28825504 fbshipit-source-id: 145028b62647f7466d44833bde0c0d4fb4c6d729
Summary: Data module for CFEval Reviewed By: gji1 Differential Revision: D28661138 fbshipit-source-id: c248600105bad5e66c717deb1fc0dee44d415005
…esearch#486) Summary: Pull Request resolved: facebookresearch#486 1. Add batch norm to single-step synthetic reward network; 2. Add layer norm to single-step, ngram fc and ngram conv net synthetic reward network; The normalization helps mitigate the problem of zero predictions from the use of MSE and sigmoid output layer. Reviewed By: czxttkl Differential Revision: D28888793 fbshipit-source-id: c041e0602880b270f10acba91d77b1cb4d8d17a2
Summary: Pull Request resolved: facebookresearch#415 Currently, we have some test failures (https://app.circleci.com/pipelines/github/facebookresearch/ReAgent/1460/workflows/ecc21254-779b-4a89-a40d-ea317e839d96/jobs/8655) because we miss some latest features. Reviewed By: MisterTea Differential Revision: D26977836 fbshipit-source-id: 9243d194ddf5c62895c9f1369830309c379fd7dd
Summary: A standalone workflow to train reward models for discrete-action contextual bandit problems. Reviewed By: kittipatv Differential Revision: D28937902 fbshipit-source-id: 9d3a28a195654eb9892f9aba56c499ccc59079c2
Summary: As titled. Otherwise for very large datasets we see the Presto memory limit error. Reviewed By: j-jiafei Differential Revision: D29020301 fbshipit-source-id: a35198cf0da83f2fc454e92844d6a7ea17e2b8f7
…DQNBase (facebookresearch#475) Summary: Pull Request resolved: facebookresearch#475 As titled. Mimicking changes done in D25377364 (facebookresearch@7584cd1). 1) Create a data module class `ParametricDqnDataModule` inheriting from `ManualDataModule`, and move implementation of following methods from `ParametricDQNBase` to it: - `should_generate_eval_dataset` - `run_feature_identification` - `query_data` - `build_batch_preprocessor` Methods that were not implemented are left unimplemented in `ParametricDqnDataModule`. 2) Create `get_data_module()` method in `ParametricDQNBase` which returns a `ParametricDqnDataModule` object. Reviewed By: czxttkl Differential Revision: D26888159 fbshipit-source-id: 2e4ce8eaa0e2a5871b0746f36a83506ce0bd7707
…ter) to github/third-party/PyTorchLightning/pytorch-lightning Summary: ### Manual - (ephemeral*) make `ResultCollection._extract_batch_size` a class method - (ephtermal) commented out the MisconfigurationException in https://fburl.com/diffusion/agbk3mxc - reagent/gym/tests/test_gym.py: wrap EpisodicDataset with dataloader before passing it to .fit() to fix the type checker error \* ephemeral means that the change are made in-place in Lightning and will disappear after another sync. ### Automatic ### New commit log messages cdcc483e CHANGELOG update after v1.3.6 release (#7988) 7978a537 Ipynb update (#8004) c6e02e48 [feat] Allow overriding optimizer_zero_grad and/or optimizer_step when using accumulate_grad_batches (#7980) eebdc910 progressive restoring of trainer state (#7652) 3fece17f [feat] Add `{,load_}state_dict` to `ResultCollection` 1/n (#7948) 906de2a7 [feat] Named Parameter Groups in `LearningRateMonitor` (#7987) 5647087f New speed documentation (#7665) 55494e87 Fix Special Tests (#7841) bc2c2db2 Do not override the logged epoch in `logged_metrics` (#7982) 21342165 Change `WarningCache` to subclass `set` (#7995) 4ffba600 Add predict hook test (#7973) 917cf836 [doc] Add more reference around predict_step (#7997) d2983c7c [fix] Enable manual optimization DeepSpeed (#7970) b093a9e6 Support `save_hyperparameters()` in LightningModule dataclass (#7992) 341adad8 Loop Refactor 2/N - Remove Old Training Loop (#7985) b71aa55b Make optimizers skippable when using amp (#7975) 0004216f Easier configurability of callbacks that should always be present in LightningCLI (#7964) 78a14a3f Add `tpu_spawn_debug` to plugin registry (#7933) 92024df2 Pt 1.9 breaking fix: __iter__ type hint (#7993) b2e9fa81 Improvements related to save of config file by LightningCLI (#7963) 971908a1 Loop Refactor 1/N - Training Loop (#7871) 560b1970 Standardize positional datamodule and argument names (#7431) 0974d66c Add docs for IPUs (#7923) 024cf23c Remove convert_to_half, suggest using `model.half` (#7974) Reviewed By: colin2328 Differential Revision: D29203448 fbshipit-source-id: 0e866b869bda06349828ec4fc61af19e4ea21f0e
…research#490) Summary: Pull Request resolved: facebookresearch#490 Fix world model simulation. The previous failure is due to that the world model is not loaded properly from warmstart path. Also, this diff updates `prepare_data()` API. `prepare_data()` is now assumed to not return setup data, following pytorch lightning's API. Reviewed By: kittipatv Differential Revision: D29157160 fbshipit-source-id: 7d52e12793b8bbc827bb2a14567993a7f63dd54c
…ookresearch#496) Summary: Pull Request resolved: facebookresearch#496 Offline Batch RL runs were failing on import error, which arose from missing init.py file Reviewed By: czxttkl Differential Revision: D29284160 fbshipit-source-id: 4e69941028f5d00bc0ef7dc30049929a9d44c306
Summary: Fixes : pytorch/pytorch#24892 In the paper : https://arxiv.org/pdf/1908.03265.pdf Liyuan Liu et al. suggested a new optimization algorithm with an essence of similar to Adam Algorithm. It has been discussed in the paper that, without warmup heuristic, in the early stage of adaptive optimization / learning algorithms sometimes we can get undesirable large variance which can slow overall convergence process. Authors proposed the idea of rectification of variance of adaptive learning rate when it is expected to be high. Differing from the paper, we selected variance tractability cut-off as 5 instead of 4. This adjustment is common practice, and could be found in the code-repository and also tensorflow swift optim library as well : https://github.com/LiyuanLucasLiu/RAdam/blob/2f03dd197022da442c6a15c47321f4335d113a3f/radam/radam.py#L156 https://github.com/tensorflow/swift-apis/blob/f51ee4618d652a2419e998bf9418ad80bda67454/Sources/TensorFlow/Optimizers/MomentumBased.swift#L638 Pull Request resolved: pytorch/pytorch#58968 Reviewed By: gchanan Differential Revision: D29241736 Pulled By: iramazanli fbshipit-source-id: 288b9b1f3125fdc6c7a7bb23fde1ea5c201c0448
Differential Revision: D29241736 (facebookresearch@0ff3634) Original commit changeset: 288b9b1f3125 fbshipit-source-id: 56c4ec98647c6f1822b130726741a1c9ca193670
Summary: Pull Request resolved: facebookresearch#487 We shouldn't need to yield the placeholder loss. Differential Revision: D29111772 fbshipit-source-id: 0971221583bd9a5de770860ff15cc80eb8d749c3
Summary: According to the original [SlateQ paper](https://arxiv.org/abs/1905.12767) (p28, 2nd paragraph, last sentence), the discount factor `gamma` will be scaled by the time difference in this way: `gamma^((t2-t1)/time_scale)`. Here, `t1` and `t2` are the timestamps between the current and the next state-action pairs within a training sample, and the `time_scale` is a hyperparameter that can scale up/down the time difference. This diff implements this mechanism by adding a `discount_time_scale` parameter to `SlateQTrainer`. Its value is the `time_scale` in the formula above. If this parameter is not set, i.e., `None`, we will keep the discount factor as it is. Reviewed By: kittipatv Differential Revision: D29297804 fbshipit-source-id: 5bd9101a2fe3b1b3d9817a3233357cab197e8ce8
Summary: Fixes : pytorch/pytorch#5804 In the paper : https://openreview.net/forum?id=OM0jvwB8jIp57ZJjtNEZ Timothy Dozat suggested a new optimization algorithm with an essence of combination of NAG and Adam algorithms. It is known that the idea of momentum can be improved with the Nesterov acceleration in optimization algorithms, and Dozat is investigating to apply this idea to momentum component of Adam algorithm. Author provided experiment evidence in their work to show excellence of the idea. In this PR we are implementing the proposed algorithm NAdam in the mentioned paper. Author has a preliminary work http://cs229.stanford.edu/proj2015/054_report.pdf where he shows the decay base constant should be taken as 0.96 which we also followed the same phenomenon here in this implementation similar to Keras. Moreover, implementation / coding practice have been followed similar to Keras in some other places as well: https://github.com/tensorflow/tensorflow/blob/f9d386849581d15d72f6f1f96f12aac230a8edbe/tensorflow/python/keras/optimizer_v2/nadam.py Pull Request resolved: pytorch/pytorch#59009 Reviewed By: gchanan, vincentqb Differential Revision: D29220375 Pulled By: iramazanli fbshipit-source-id: 4b4bb4b15f7e16f7527f368bbf4207ed345751aa
Summary: Fixes : pytorch/pytorch#24892 In the paper : https://arxiv.org/pdf/1908.03265.pdf Liyuan Liu et al. suggested a new optimization algorithm with an essence of similar to Adam Algorithm. It has been discussed in the paper that, without warmup heuristic, in the early stage of adaptive optimization / learning algorithms sometimes we can get undesirable large variance which can slow overall convergence process. Authors proposed the idea of rectification of variance of adaptive learning rate when it is expected to be high. Differing from the paper, we selected variance tractability cut-off as 5 instead of 4. This adjustment is common practice, and could be found in the code-repository and also tensorflow swift optim library as well : https://github.com/LiyuanLucasLiu/RAdam/blob/2f03dd197022da442c6a15c47321f4335d113a3f/radam/radam.py#L156 https://github.com/tensorflow/swift-apis/blob/f51ee4618d652a2419e998bf9418ad80bda67454/Sources/TensorFlow/Optimizers/MomentumBased.swift#L638 Pull Request resolved: pytorch/pytorch#58968 Reviewed By: vincentqb Differential Revision: D29310601 Pulled By: iramazanli fbshipit-source-id: b7bd487f72f1074f266687fd9c0c6be264a748a9
…#189) Summary: X-link: facebookresearch/d2go#189 X-link: facebookresearch/recipes#14 Pull Request resolved: facebookresearch#616 ### New commit log messages - [9b011606f Add callout items to the Docs landing page (#12196)](Lightning-AI/pytorch-lightning#12196) Reviewed By: edward-io Differential Revision: D34687261 fbshipit-source-id: 3ef6be5169a855582384f9097a962d2261625882
Summary: Pull Request resolved: facebookresearch#617 Improve the reinforce trainer by 1. Allowing reward mean subtraction without normalization, 2. Providing the option to log training loss and ips ratio mean per epoch. Reviewed By: alexnikulkov Differential Revision: D34688279 fbshipit-source-id: 50e94140fbf2182523e03c350f7bbe6812cb6e74
Summary: Pull Request resolved: facebookresearch#618 as titled Reviewed By: sinannasir Differential Revision: D34587407 fbshipit-source-id: 738aa3fb580716628330efa65a8c5ca7596aff14
Summary: Pull Request resolved: facebookresearch#615 as titled Reviewed By: PavlosApo Differential Revision: D34677139 fbshipit-source-id: 9fa8a0884d8f4abf0c7ca47fa669932d739a2d4c
Summary: Pull Request resolved: facebookresearch#619 as titled Reviewed By: alexnikulkov Differential Revision: D34940029 fbshipit-source-id: 9f6add38bd7f03f6811b6f4c51db431a1412660c
Summary: Pull Request resolved: facebookresearch#620 Officially import torchrec Reviewed By: alexnikulkov Differential Revision: D34942469 fbshipit-source-id: d4d47f4e90ff99f738f27c0720fd5462f40abe86
Summary: Pull Request resolved: facebookresearch#621 As the creative ranking project runs only 1 epoch, enable per-batch logging to TensorBoard, as did in the SAC trainer in ReAgent. Reviewed By: czxttkl Differential Revision: D35100625 fbshipit-source-id: 37bf361a4f668665de7691731467755c37b31067
Differential Revision: D35275827 fbshipit-source-id: e1e402f8a07f97e3243318bb0101e2943a40c48c
Differential Revision: D35313194 fbshipit-source-id: 30b3f317f90b2e736453ae5162caad765fbfa414
…->context, action->arm (facebookresearch#624) Summary: Pull Request resolved: facebookresearch#624 1. Rename state -> context 2. Rename action -> arm 3. Add capability to read context-arm features from the input 4. Remove action probability from contextual bandit input (will add back in when we add algorithms which require it) 5. Improve offset validation in `FixedLengthSequences` transform Differential Revision: D35372899 fbshipit-source-id: b00fa256aec344a2d7fcf2034e1f00132fef62f3
…kresearch#625) Summary: Pull Request resolved: facebookresearch#625 Tittle Differential Revision: D35417882 fbshipit-source-id: 74bf4799cebce3f8f35f0b83fd7fd9825c34c7c2
Summary: Pull Request resolved: facebookresearch#626 use pure pytorch operators to perform array length check Reviewed By: alexnikulkov Differential Revision: D35423434 fbshipit-source-id: 397879eb2d0cbbcaaf9624e9b4cbead2f445263e
Summary: Pull Request resolved: facebookresearch#627 Removing an unused parameter Disable interaction features by default Reviewed By: czxttkl Differential Revision: D35442407 fbshipit-source-id: fdc0fd3137226565656b8feddbdffdb054026fe2
Summary: Pull Request resolved: facebookresearch#628 Fixing some device issues in ReAgent code Reviewed By: alexnikulkov Differential Revision: D34995851 fbshipit-source-id: 2f0376c2d53b7797e6193deffa95ca162bd1153a
Differential Revision: D35589581 fbshipit-source-id: b08bb906c6703876a3be2be5345f69342d123a1c
…search#629) Summary: Pull Request resolved: facebookresearch#629 The attributes weren't registered properly, so they weren't pushed to the device when `model.to(device)` was called Reviewed By: soudia Differential Revision: D35560710 fbshipit-source-id: 67492e7f64829750e395bdec85e04b7fb6fff04c
…arch#630) Summary: Pull Request resolved: facebookresearch#630 Removing device assignment following changes in D35560710 Reviewed By: alexnikulkov Differential Revision: D35656985 fbshipit-source-id: 423124fdc9615c74476152f39e259bcf1f9f94d0
Differential Revision: D35791227 fbshipit-source-id: a9bea27928d8da3f413c341d9cccfa6d14fdcc6f
… greedy action selection is working (facebookresearch#632) Summary: Pull Request resolved: facebookresearch#632 This diff fixes a runtime error with the EpsilonGreedyActionSampler, and adds a unit test to ensure that greedy action selection via this class computes correct log probs and returns the expected actions. Reviewed By: czxttkl Differential Revision: D35783785 fbshipit-source-id: e6d64ea0dbd643e3887ed47497f37c005c518276
Differential Revision: D35968031 fbshipit-source-id: 80d19aab074a8f4aaea544a56b7309b46901f1cc
Summary: Pull Request resolved: facebookresearch#633 add `test_shift_kjt_by_one` and `test_reorder_data_kjt` `test_reorder_data_kjt` will reorder data within each key `test_shift_kjt_by_one` will left shift data by one within each key The two functions will be used in the Ads LTV project. Reviewed By: alexnikulkov Differential Revision: D35970439 fbshipit-source-id: dfc67f00216bcb575e4c9fb439ec570dc96f0951
Summary: Pull Request resolved: facebookresearch#634 When KeyedJaggedTensor doesn't have weights, `.weights()` will throw an assertion error. We should use `.weights_or_none()` to check if a KJT has weights. Reviewed By: BerenLuthien Differential Revision: D36005910 fbshipit-source-id: b075ef9949b44fc1186bc124fd42a00e3c9d77f3
|
This pull request was exported from Phabricator. Differential Revision: D36133994 |
Summary: Pull Request resolved: facebookresearch#638 Differential Revision: D36133994 fbshipit-source-id: d76a42b84f3eab4196a5d4f8210f3f37c5edf55e
|
This pull request was exported from Phabricator. Differential Revision: D36133994 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Differential Revision: D36133994