LLM features in apps: chat, streaming, sources, errors and cost

A large language model makes it easy to add a text box that answers anything, and that is the problem. An empty box with a blinking cursor asks the user to invent the feature themselves. Most people type one vague question, get a vague answer and never open it again. The model is rarely the weak part of an LLM feature. The interface around it usually is.

This guide covers the design decisions that make these features work in a mobile app: choosing the right shape, turning loose language into something the user can check, handling latency, showing where answers come from, failing honestly, and keeping personal data and costs under control. Products in this area change quickly, so the specifics here stick to what the platforms themselves document.

Contains affiliate links. As an Amazon Associate I earn from qualifying purchases.

Chat is one shape among several

Chat suits open questions, back-and-forth refinement and support. For many tasks a narrower shape works better, because the user doesn't have to explain what they want.

Pattern	Works well for	Watch out for
Full chat screen	Support, research, planning with follow-up questions	Blank-page paralysis; long histories that cost more with each turn
One-tap action on existing content	Summarizing a thread, rewriting a message, tidying a note	People need to see and keep the original
Natural language that fills a form	Dates, reminders, filters, expense entries	Every parsed field must be visible and editable
Inline suggestions	Replies, tags, titles, next steps	Suggestions that interrupt typing
Invisible processing	Sorting, tagging, search ranking	Mistakes nobody can see or correct

If you do build chat, don't open on an empty screen. Offer three or four starting points that use the person's own data, such as a summary of this week's notes, the same way a good empty state in onboarding offers one useful action instead of a blank list.

Natural language in, structured data out

For many apps the safest pattern is to let people type or say something loose, then show the result as ordinary interface. "Dentist every six months starting in March" becomes a filled-in reminder form with a title, a date and a repeat rule, which the person checks and saves. Nothing happens until they confirm.

Ask the model for structured output instead of parsing prose. Apple's Foundation Models framework, which gives apps on iOS 26 access to the on-device model behind Apple Intelligence on supported devices, can generate directly into Swift types, and the main server-side model APIs offer similar schema-based output. Validate the result anyway, and treat every action with consequences (sending, paying, deleting, booking) as something the user approves explicitly.

Latency and streaming

Generation takes time, and on a phone it also competes with tunnels and elevators. For long text, stream the response so the first words arrive while the rest is generated, and give people a stop control. For structured results you must validate before showing, don't stream half a form; show a short progress state, then the finished result.

A few details separate careful streaming from the rest. Don't pull the scroll position down while someone is reading earlier text. Keep partial output if the connection drops, and offer a retry. Phones suspend apps when people switch away, so for anything long, generate on your server and store the result, so it is waiting when the person returns. For screen reader users, announce the finished answer once rather than every fragment; our accessibility checklist covers the screen reader basics.

On-device models remove the network wait and keep text on the phone. Apple's framework needs a device with Apple Intelligence turned on, and on Android the ML Kit GenAI APIs use Gemini Nano through the AICore system service on supported devices, for tasks such as summarizing, proofreading and rewriting. Both depend on the hardware, so design the fallback for phones that can't run them.

Showing where answers come from

When an answer draws on your content, such as help articles, product data or the user's own documents, show which items it used and let people open them with one tap. Put the reference next to the statement it supports rather than in a list at the bottom. If retrieval finds nothing relevant, say so plainly instead of letting the model answer from general knowledge. "Nothing in your notes mentions this" is a useful answer. A confident guess is not.

Errors, refusals and wrong answers

Users can't tell one kind of model failure from another, so the interface has to do it for them. Separate the cases and give each its own message:

Network failure or timeout: keep what the person typed and offer a retry.
Usage limit reached: say when it resets, in plain words.
The model declined: explain briefly what it won't do and offer another route, such as human support.
No good answer: say the app doesn't know, rather than filling the gap.
Wrong answer: make it easy to flag, edit or undo.

Never show a raw provider error. On Google Play, apps that generate content with AI must let users report or flag offensive content without leaving the app, according to Google's AI-generated content policy. A small flag control on each generated answer covers that and gives you a steady stream of real failures to learn from.

Privacy

Decide what leaves the device, and send no more than the task needs: the note being summarized, not the whole account. Apple's App Review Guidelines (5.1.2(i)) require apps to clearly disclose where personal data will be shared with third parties, including third-party AI, and to get explicit permission before doing so. Under GDPR, a model provider that processes users' personal data on your behalf needs a data processing agreement, and its retention and training settings are worth checking before launch. Reflect all of this in the App Store privacy details and Google Play's Data safety form, and let people see and delete their history.

Cost control

Model providers usually charge for the amount of text going in and coming out, and in a typical chat the whole conversation so far is sent again with every new message. Long conversations get more expensive with each turn, so cost needs design as well as engineering:

Keep provider API keys on your server and call the model from there. A key inside the app can be extracted.
Trim or summarize older turns instead of resending everything.
Send simple tasks to smaller, cheaper models, or to on-device models where they exist.
Use the provider's prompt caching, if it offers one, for long instructions that repeat.
Set per-user limits and show them honestly, with a clear message when someone reaches one.
Set spending alerts, so a bug or an abusive account shows up within hours rather than on the monthly invoice.

Questions

Should every LLM feature be a chat?

No. Chat fits open-ended tasks. For a defined job, a button on the content or a natural-language field that fills a form is quicker to use and easier to trust.

Do users need to know a feature uses AI?

On iOS, if personal data goes to a third-party AI service, Apple requires disclosure and explicit permission. Beyond the rules, labeling generated text helps people decide how far to trust it, and our guide on AI tools, no-code and outsourcing looks at the build side of the same question.

Is on-device generation good enough?

For short, focused tasks such as summaries, rewrites and pulling fields out of text, it is often a good fit, and it keeps data on the phone. It runs only on supported devices, so plan the fallback before the feature.

How do we keep costs predictable?

Run model calls from your server, cap conversation length and per-user usage, and set spending alerts. Then watch the cost per active user every week, the same way you watch the crash rate.