t5-base-controllable-qa-generator

This model is a fine-tuned version of t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8687

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• training_steps: 2000

Training results

Training Loss	Epoch	Step	Validation Loss
7.2453	2.3855	50	3.5197
2.0995	4.7711	100	1.9805
1.8109	7.1446	150	1.7060
1.6062	9.5301	200	1.5023
1.3582	11.9157	250	1.3483
1.3196	14.2892	300	1.2469
1.2131	16.6747	350	1.1814
1.1552	19.0482	400	1.1317
1.122	21.4337	450	1.0944
1.046	23.8193	500	1.0690
1.051	26.1928	550	1.0470
1.0073	28.5783	600	1.0285
0.9617	30.9639	650	1.0111
0.9889	33.3373	700	0.9974
0.9275	35.7229	750	0.9843
0.923	38.0964	800	0.9723
0.9385	40.4819	850	0.9607
0.931	42.8675	900	0.9519
0.8915	45.2410	950	0.9439
0.939	47.6265	1000	0.9360
0.9003	50.0	1050	0.9281
0.875	52.3855	1100	0.9206
0.8751	54.7711	1150	0.9152
0.8616	57.1446	1200	0.9095
0.8805	59.5301	1250	0.9040
0.8509	61.9157	1300	0.8989
0.8196	64.2892	1350	0.8942
0.8495	66.6747	1400	0.8908
0.8655	69.0482	1450	0.8874
0.9224	71.4337	1500	0.8829
0.8371	73.8193	1550	0.8807
0.8107	76.1928	1600	0.8784
0.8333	78.5783	1650	0.8756
0.8475	80.9639	1700	0.8747
0.8075	83.3373	1750	0.8729
0.8352	85.7229	1800	0.8712
0.8516	88.0964	1850	0.8700
0.8115	90.4819	1900	0.8694
0.8477	92.8675	1950	0.8687
0.8289	95.2410	2000	0.8687

Framework versions

• PEFT 0.16.0
• Transformers 4.53.3
• Pytorch 2.6.0+cu124
• Datasets 4.1.1
• Tokenizers 0.21.2