How does MTP actually improve performance? Haven't understood the verification process
Hi there, does anyone know how MTP works? I was trying to wrap my head around it, but things aren't adding up. For example, the input string I give it is "look up, " and I want it to output "the sky is blue", except the MTP heads miss the mark and tell me "the sky is green". How does the model verify that "green" is wrong in a decent amount of time? From what I understand, the first prefill step is standard and populates the caches for the various layers using "look up, " as context. Then the last layer doe
评论
?
参与讨论