Replace js-tiktoken BPE merge algorithm with faster heap based algorithm#101
Open
mikolalysenko wants to merge 3 commits intodqbd:mainfrom
Open
Replace js-tiktoken BPE merge algorithm with faster heap based algorithm#101mikolalysenko wants to merge 3 commits intodqbd:mainfrom
mikolalysenko wants to merge 3 commits intodqbd:mainfrom
Commits
Commits on Apr 19, 2024
Commits on Oct 2, 2024
- committed
