We deleted the Embedding Layer -- INTRO Our Collins-Embedding-3M NoesisLab/Collins-Embedding-3M Most "small" models are just giant vocab tables in a trench coat. Collins-3M changes that. By using 2-Universal Hashing and Chernoff-bound noise suppression, weβve collapsed the embedding space into a fixed O(1) hash-map. * STSB: 0.7114 (Beating many 100M+ models) * Size: 3M (Edge-ready, IoT-ready) * Tech: Randomized Sign-Hashing + RoPE positional injection. Built by NoesisLab
if you like it give the demo a little star and send a shoutout to : @MaxLSB@jddqd and @GAD-cell for absolutely obliterating the pareto frontier of the french language understanding .
In short, the students won. They did so by fine-tuning LFM2. LFM2 is a foundation built by Liquid AI. Liquid AI is a $2 billion startup from MIT.
the total budget of the students was zero . the total budget required (currently) for someone with no access to a computer is zero : https://huggingface.co/unsloth-jobs
it's actually not a mystery why a well-ressourced language like french would actually get history made by highschoolers' side project : a pareto frontier they've currently held for almost six months now ...
their supreme reign will continue until ressource allocation improves.
if you like it give the demo a little star and send a shoutout to : @MaxLSB@jddqd and @GAD-cell for absolutely obliterating the pareto frontier of the french language understanding .
4 replies
Β·
reacted toprithivMLmods'spost with πππ€β€οΈ23 days ago