Skip to content

Commit

Permalink
update changelog
Browse files Browse the repository at this point in the history
  • Loading branch information
thammegowda committed Mar 15, 2022
1 parent 36e1f44 commit 0f5a4e4
Showing 1 changed file with 14 additions and 3 deletions.
17 changes: 14 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,17 @@
# v0.7 - WIP
-

# v0.7 - 20220315
- Big improvements:
- Autocast / mixed precision: `bfloat16` instead of `float16`. Now we can train larger models on larger batches using 16bit float ops without loss becoming infinity!
- WARNING: we need pytorch 1.10 or newer. Please upgrade!
- validation BLEU scores are computed without teacher forcing i.e., similar to inference. BLEU is more realistic estimate of test time bleu
- WARNING: validations can be slower. Dont use too big validation set
- schedule:
- `inverse_sqrt` support scaler multiplier term, similar to `noam`
- `inverse_root` schedule added, generalization of `inverse_sqrt`
- fixes
- `rtg.prep` CLI arguments works now
- optimizer state loading now works while resuming training
- parent model will be recreated if missing even after _PREPARED flag exists


# v0.6.1 : 20220128
- `rtg.fork` accepts multiple to_dir; thus supports cloning multiple times at once
Expand Down

0 comments on commit 0f5a4e4

Please sign in to comment.