catalystsec/MiniMax-M2.5-4bit-DWQ

This model was quantized to 4-bit using DWQ with mlx-lm version 0.30.7.

Parameter	Value
DWQ learning rate	3e-7
Batch size	1
Dataset	allenai/tulu-3-sft-mixture
Initial validation loss	0.077
Final validation loss	0.053
Relative KL reduction	≈31 %
Tokens processed	≈1.11 M

Perplexity

Evaluated on 210 samples of 512 tokens from the default mlx-lm calibration data.

Model	Perplexity
3-bit	7.802
3-bit DWQ	7.434
4-bit	6.581
4-bit DWQ	6.431

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("catalystsec/MiniMax-M2.5-4bit-DWQ")
prompt = "hello"

if tokenizer.chat_template is not None:
    prompt = tokenizer.apply_chat_template(
        [{"role": "user", "content": prompt}],
        add_generation_prompt=True,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)
print(response)