curious if Jev-like models might accelerate us towards on-device AI native functionality on mobile devices

strong LLMs are a long way from running on phones:

• slow memory bandwidth
• only highly quantized MoE models will fit
• lots of issues with power/heat/etc

Folks are building NPUs for the next gen of mobile but targeting pretty modestly sized LLMs

There’s lots of mobile UX that would benefit from fast/cheap AI decision models - from notifications, typing/texting, to in-app experiences like inboxes/calendars/etc

I always figured the other solution would be a model-on-a-chip that would encode an older-yet-useful model into the actual phone hardware itself, but maybe this will move faster?