Pretrained models from the paper "Predicting the Order of Upcoming Tokens Improves Language Modeling"
Zayd Muhammad Kawakibi Zuhri PRO
zaydzuhri
AI & ML interests
I really like watching loss go down
Recent Activity
updated
a model
about 1 hour ago
zaydzuhri/top-1B-ratio090-4096-batch8x2-steps200000-20251209-082436
published
a model
1 day ago
zaydzuhri/top-1B-ratio090-4096-batch8x2-steps200000-20251209-082436
updated
a model
3 days ago
zaydzuhri/top-1B-ratio090-4096-batch8x2-steps200000-20251208-112022
Organizations
None yet
models
99
zaydzuhri/top-1B-ratio090-4096-batch8x2-steps200000-20251209-082436
Updated
zaydzuhri/top-1B-ratio090-4096-batch8x2-steps200000-20251208-112022
Updated
zaydzuhri/top-340M-ratio090-4096-batch16-steps100000-20251204-171738
Updated
zaydzuhri/top-340M-ratio010-4096-batch16-steps100000-20251203-130428
Updated
zaydzuhri/top-340M-ratio075-4096-batch16-steps100000-20251202-073245
Updated
zaydzuhri/top-340M-ratio025-4096-batch16-steps100000-20251128-073203
Updated
zaydzuhri/mtp-gsm8k-1.8B-4096-model
2B
•
Updated
•
48
zaydzuhri/top-gsm8k-1.8B-4096-model
2B
•
Updated
•
41
zaydzuhri/vanilla-gsm8k-1.8B-4096-model
2B
•
Updated
•
35
zaydzuhri/top-340M-window1024-4096-batch128-steps100000-20251118-062653
Updated
datasets
10
zaydzuhri/hendrycks_math_text
Viewer
•
Updated
•
12.5k
•
25
zaydzuhri/OpenMathInstruct-2-Text
Viewer
•
Updated
•
22M
•
88
zaydzuhri/stack-edu-python
Viewer
•
Updated
•
25.3M
•
49
zaydzuhri/stack-edu
Viewer
•
Updated
•
99.5M
•
32
zaydzuhri/kreyol-mt-cleaned
Viewer
•
Updated
•
903k
•
57
zaydzuhri/the_pile_tokenized_5percent_truncated_packed_v2
Viewer
•
Updated
•
2.46M
•
83
zaydzuhri/the_pile_tokenized_5percent_truncated_packed
Viewer
•
Updated
•
2.11M
•
81
zaydzuhri/the_pile_tokenized_5percent_truncated
Viewer
•
Updated
•
6M
•
220
zaydzuhri/the_pile_tokenized_5percent
Viewer
•
Updated
•
6M
•
119
zaydzuhri/the_pile_tokenized_6k
Viewer
•
Updated
•
6k
•
12