Build intelligent Android apps: On-device inference

Posted by Caren Chang, Developer Relations Engineer, Android Developer Relations

Welcome back to the blog post series " Build intelligent Android apps " where we take a basic Android app and transform it into a personalized, intelligent, and agentic experience. In our previous post we introduced Jetpacker, the demo app we'll use throughout this series.

In this blog post, we will share how you can use Gemini Nano through ML Kit’s Prompt API to build intelligent on-device features.

Building intelligent on-device features refers to the ability to process prompts and data directly on a device without sending data to a server. This offers a few advantages:

User data can be processed locally on the device, preserving user privacy

Functionality of the model is reliable even with spotty or no internet connection

No additional cloud inference cost, since everything runs on the user’s hardware

With the benefits of on-device in mind, we identified three features to add in Jetpacker that can improve the user experience: summarizing trip itineraries, managing expenses, and capturing voice notes.

On-device features in Jetpacker: Summarizing trip itineraries, managing expenses, and voice notes

High quality tailored summarization of short texts

The itinerary screen gives users a quick overview of all activities for a given trip. Since this screen contains a lot of information, it can quickly become overwhelming. To help users prepare without feeling overwhelmed, we can add a ‘ Get ready for your trip’ section at the top.

The romantic Paris trip is summarized as a classic Parisian adventure blending art, sights, and delicious food. A tip and some useful phrases are also added.

By inputting a trip itinerary and asking an LLM to summarize it, we can generate a quick summary of the trip along with packing tips and useful local phrases. This is a great use case for an on-device model for several reasons:

Performance and quality: Both the input and output text are relativel…