psssd-cat/llm-base

Fine-tuned Llama 3.2 1B for Docker Commands

This model is a fine-tuned version of Cagatayd/llama3.2-1B-Instruct-Egitim on the MattCoddity/dockerNLcommands dataset.

Training Details

• Base Model: Cagatayd/llama3.2-1B-Instruct-Egitim
• Dataset: MattCoddity/dockerNLcommands
• Training Method: LoRA (Low-Rank Adaptation)
• Quantization: 8-bit
• LoRA Rank: 32
• Training Steps: 60
• Final Accuracy: 80.69%

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("psssd-cat/llm-base")
tokenizer = AutoTokenizer.from_pretrained("psssd-cat/llm-base")

# Example usage
text = "Find all running docker containers"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Configuration

• Learning Rate: 3e-5
• Batch Size: 2 per device
• Gradient Accumulation Steps: 4
• FP16: Enabled
• Warmup Steps: 5

Model Performance

• Evaluation Loss: 1.147
• Mean Token Accuracy: 80.69%
• Training Loss: 2.094

---

Generated with Claude Code