Codeprep Versions Save

A toolkit for pre-processing large source code corpora

v1.0.5

3 years ago
  • Add workaround for calculation vocabulary on OSx (#11)

v1.0.1

4 years ago

Changes:

  • Fix training custom bpe codes (Thanks to @mir-am : pull request #6 )
  • Fix corpus pre-processing on Windows

v1.0.0

4 years ago