Ugly Alpha 1
rust_tokenizers:3.1.2

total uncompressed bytes

554.3 KB

total files

36

Fix

NewInclude { include: [ "src/**/*", "README.md", "protos/sentencepiece_model.proto", "build.rs", ], has_build_script: true, }

7 wasted files

total waste: 67.4 KB

  1. tests/test_gpt2_cased.rs : 14.7 KB
  2. tests/test_roberta_cased.rs : 13.7 KB
  3. tests/test_sentence_piece.rs : 10.0 KB
  4. tests/test_ctrl_cased.rs : 10.0 KB
  5. tests/test_bert_uncased.rs : 9.2 KB
  6. tests/test_openai_gpt_uncased.rs : 9.1 KB
  7. tests/test_utils.rs : 578 B