What would you require before calling an inference change “quality-neutral”? [D]

Two GSM8K scores: 96.89% → 96.82%, across all 1,319 test questions. These are from NVIDIA’s Ling-3.0-flash comparison on one DGX Spark. The optimized configuration changes the MoE backend, adds an FP8 output head and enables NEXTN speculative decoding together. Both runs use the FP4 checkpoint. Aggregate scores summarize the total. A paired view of the 1,319 answers would show where the two configurations agree and where they trade wins and losses. For a claim of no meaningful quality regression, what evidence would you consider sufficient: paired per-question outcomes, repeated seeds, a declared equivalence margin, another task set? Especially when a precision change and a decoding optimization ship together, which checks give the most information for the evaluation budget?