Neuralwatt says cache misses can use up to 140% more inference energy

Neuralwatt CEO Chad Gibson says the software that sends prompts to an AI model can nearly double the energy required to serve a request. In an analysis published October 16th, Neuralwatt compared prompt-cache reuse across agent clients and found that small differences in how they assemble prompts can translate into substantial extra prefill work.