Has anyone tried training DSpark / DeepSpec with the Open-PerfectBlend setup for qwen3.5 family like 4b ,9b ..etc ?
Hi u/everyone , I’m trying to understand the practical training time and compute requirements for the DSpark / DeepSpec setup using the mlabonne/open-perfectblend dataset. The config I’m looking at is close to the paper setup: Dataset: mlabonne/open-perfectblend Samples: ~1.3M Data mix: math, code, chat, and instruction following Epochs: 10 Global batch size: 512 Max sequence length: 4096 Precision: bf16 Optimizer: AdamW LR: 6e-4 with cosine decay and warmup Total steps: ~25k From my rough calculation, this
评论
?
参与讨论