目标:

构建一个轻量但专业的人物领域问答/推理模型

🚀 阶段一:SFT(监督微调):
步骤一:构造人物领域问题集
这些问题可以来自公开资料、专业文献、论坛、教材等。

人物数据数量:168条

实验一:只加垂类数据集看看效果
GRADIO_SERVER_PORT=8103 CUDA_VISIBLE_DEVICES=1,2,5,7 llamafactory-cli train
–stage sft
–do_train
–model_name_or_path /workspace/codes/deepseek/Qwen3-0.6B
–dataset person
–eval_dataset train_eval
–dataset_dir ./data
–template qwen
–finetuning_type lora
–output_dir ./saves/Qwen3-0.6B/lora/sft_person_1
–overwrite_cache
–overwrite_output_dir
–cutoff_len 1024
–preprocessing_num_workers 16
–per_device_train_batch_size 1
–per_device_eval_batch_size 1
–gradient_accumulation_steps 16
–lr_scheduler_type cosine
–logging_steps 10
–warmup_steps 10
–save_steps 100
–eval_steps 100
–save_strategy steps
–eval_strategy steps
–load_best_model_at_end
–learning_rate 5e-6
–num_train_epochs 30
–lora_rank 64
–lora_alpha 128
–val_size 0
–plot_loss
–fp16

更多推荐