Teen-Different/qwen2.5-1.5b-arc1-decoded-dft
Text Generation • 2B • Updated
• 8
Tarun Reddi PRO
Teen-Different
AI & ML interests
Generative AI, Modular AI Systems, Reinforcement Learning
Recent Activity
updated
a model 1 day ago
Teen-Different/qwen2.5-1.5b-arc1-decoded-dft published
a model 1 day ago
Teen-Different/qwen2.5-1.5b-arc1-decoded-dft published
a model 1 day ago
Teen-Different/qwen2.5-1.5b-arc1-encoded-dft Organizations
updated a model 1 day ago
published 2 models 1 day ago
updated a dataset 2 days ago
published a dataset 2 days ago
updated a dataset 2 days ago
published a dataset 2 days ago
updated a dataset 2 days ago
published a dataset 2 days ago
updated a dataset 2 days ago
published a dataset 2 days ago
updated a model 12 days ago
published a model 12 days ago
updated 2 models 14 days ago
published 2 models 14 days ago
posted an
update about 1 month ago
Post
1024
Safety Alignment Collapses Without apply_chat_template(): An Empirical Study
This weekend, I ran an experiment on the safety alignment of several small-scale open models (Qwen2.5, Qwen3, Gemma-3, SmolLM). My objective was to measure the robustness of refusal mechanisms when deviating from canonical chat templates.
The finding: Safety guarantees effectively collapse when apply_chat_template() is omitted.
METHODOLOGY
I evaluated models in two states:
• In-Distribution: Input wrapped in standard <|im_start|> instruction tokens
• Out-of-Distribution: Input provided as a raw string
For scalable evaluation, I used Qwen3Guard-Gen-4B as an automated judge, classifying responses as Safe, Unsafe, or Controversial.
KEY FINDINGS: REFUSAL COLLAPSE
When "Assistant" formatting tokens are removed, models undergo a distributional shift—reverting from a helpful assistant to a raw completion engine.
Gemma-3: 100% refusal (aligned) → 60% (raw)
Qwen3: 80% refusal (aligned) → 40% (raw)
SmolLM2-1.7B: 0% → 0% (no safety tuning to begin with)
QUALITATIVE FAILURES
The failure modes were not minor. Without the template, models that previously refused harmful queries began outputting high-fidelity harmful content:
• Explosives: Qwen3 generated technical detonation mechanisms
• Explicit content: Requests flatly refused by aligned models were fulfilled with graphic narratives by unaligned versions
This suggests instruction tuning acts as a "soft mask" over the pre-training distribution rather than removing harmful latent knowledge.
👉 Read the full analysis: https://teendifferent.substack.com/p/apply_chat_template-is-the-safety
💻 Reproduction Code: https://github.com/REDDITARUN/experments/tree/main/llm_alignment
This weekend, I ran an experiment on the safety alignment of several small-scale open models (Qwen2.5, Qwen3, Gemma-3, SmolLM). My objective was to measure the robustness of refusal mechanisms when deviating from canonical chat templates.
The finding: Safety guarantees effectively collapse when apply_chat_template() is omitted.
METHODOLOGY
I evaluated models in two states:
• In-Distribution: Input wrapped in standard <|im_start|> instruction tokens
• Out-of-Distribution: Input provided as a raw string
For scalable evaluation, I used Qwen3Guard-Gen-4B as an automated judge, classifying responses as Safe, Unsafe, or Controversial.
KEY FINDINGS: REFUSAL COLLAPSE
When "Assistant" formatting tokens are removed, models undergo a distributional shift—reverting from a helpful assistant to a raw completion engine.
Gemma-3: 100% refusal (aligned) → 60% (raw)
Qwen3: 80% refusal (aligned) → 40% (raw)
SmolLM2-1.7B: 0% → 0% (no safety tuning to begin with)
QUALITATIVE FAILURES
The failure modes were not minor. Without the template, models that previously refused harmful queries began outputting high-fidelity harmful content:
• Explosives: Qwen3 generated technical detonation mechanisms
• Explicit content: Requests flatly refused by aligned models were fulfilled with graphic narratives by unaligned versions
This suggests instruction tuning acts as a "soft mask" over the pre-training distribution rather than removing harmful latent knowledge.
👉 Read the full analysis: https://teendifferent.substack.com/p/apply_chat_template-is-the-safety
💻 Reproduction Code: https://github.com/REDDITARUN/experments/tree/main/llm_alignment
- 2 replies
updated a model about 2 months ago
published a model 2 months ago