mjbommar
/

binary-tokenizer-001-4k

Feature Extraction

binary-analysis

binary-tokenization

byte-pair-encoding

reverse-engineering

malware-analysis

executable-analysis

Model card Files Files and versions

mjbommar commited on Nov 13, 2025

Commit

0674cd8

·

verified ·

1 Parent(s): bbc3b0b

Upload binary-tokenizer-001-4k tokenizer

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -37,7 +37,7 @@ A cross-platform BPE tokenizer for binary executables and machine code. Trained
 ## Training Configuration
 **Training Corpus**:
-- Source: `/nas4/data/glaurung-data/binaries-small/`
 - Size: ~13 GB
 - Files: 30,738 binary files
 - Platforms: Linux (ELF), Windows (PE), macOS (Mach-O), Android (APK)

 ## Training Configuration
 **Training Corpus**:
+- Source: [`mjbommar/binary-30k-tokenized`](https://huggingface.co/datasets/mjbommar/binary-30k-tokenized)
 - Size: ~13 GB
 - Files: 30,738 binary files
 - Platforms: Linux (ELF), Windows (PE), macOS (Mach-O), Android (APK)