train_multirc_123_1764974431

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 0.5540
• Num Input Tokens Seen: 264547520

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1392	1.0	6130	0.1364	13255424
0.0776	2.0	12260	0.1306	26471216
0.0857	3.0	18390	0.1191	39694112
0.024	4.0	24520	0.1130	52929744
0.0926	5.0	30650	0.1113	66152480
0.0768	6.0	36780	0.1107	79389648
0.0824	7.0	42910	0.1156	92621824
0.0343	8.0	49040	0.1119	105830544
0.0227	9.0	55170	0.1227	119047920
0.0474	10.0	61300	0.1268	132272272
0.0968	11.0	67430	0.1396	145487264
0.1417	12.0	73560	0.1472	158737232
0.0084	13.0	79690	0.1756	171979232
0.0019	14.0	85820	0.1946	185199728
0.0014	15.0	91950	0.2322	198426688
0.0046	16.0	98080	0.2587	211640976
0.0021	17.0	104210	0.2946	224870720
0.0003	18.0	110340	0.3278	238102672
0.0007	19.0	116470	0.3455	251320768
0.0004	20.0	122600	0.3436	264547520

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1