HYBRID ON-DEVICE AI ROUTING CHECKLIST (2026)

Use this as a working document in a 60–90 minute product + eng review.

1) PICK 1–2 FLOWS TO MIGRATE FIRST
- List your top 3 AI-driven user journeys (the ones that drive retention or revenue).
- For each, write the “network unavailable” version of the flow.
- Circle the flow where speed + privacy are clearly part of the value (not just nice-to-have).

2) ROUTE BY CONSTRAINTS (NOT BY MODEL FASHION)
For the chosen flow, answer:
- Latency: does the user perceive delays as broken UI (typing, editing, scanning, meeting capture)? If yes, local-first.
- Sensitivity: does the request touch personal notes, messages, contacts, calendar, health, finance, or regulated data? If yes, default local.
- Offline: do users operate in weak connectivity (field work, travel, secure facilities)? If yes, local baseline.
- Complexity: is it classification/extraction/short generation? Good local candidate. Is it long-context reasoning or heavy generation? Cloud candidate.

3) DEFINE A TWO-TIER ARCHITECTURE
- Tier A (local): intent detection + safety pre-check + small generation/extraction.
- Tier B (cloud): deep reasoning or expensive generation with minimal necessary context.
- Add a cloud fallback path for Tier A failures (first release) so you can ship without betting the company.

4) MINIMIZE CONTEXT MOVEMENT
- Write down exactly what context is allowed to leave the device.
- Default to redaction: send structured facts, not raw documents.
- Cache locally where possible (indexes, embeddings, summaries), and define retention rules.

5) SECURITY / PRIVACY REALITY CHECK
- Assume the on-device model file can be copied.
- Encrypt local caches; treat logs as sensitive data.
- Store secrets in OS-provided secure storage (Keychain/Keystore/DPAPI where applicable).
- Decide what happens on a compromised/jailbroken device (deny features, reduce context, or require server auth).

6) SHIPPING PLAN (LOW DRAMA)
- Phase 1: ship local-first for one narrow capability + cloud fallback.
- Phase 2: move routing on-device (reduce unnecessary cloud calls).
- Phase 3: make “process on-device” a visible product setting and document it.
- Phase 4: tighten fallback rules (cloud becomes escalation, not default).

7) ACCEPTANCE TESTS (BEFORE YOU CELEBRATE)
- Airplane mode: can the core flow complete end-to-end?
- Cold start: does the first interaction feel instant enough?
- Worst device: does it still work on older supported hardware?
- Data boundary: can you explain, in one paragraph, what leaves the device and why?

If you can’t pass the acceptance tests, don’t argue about model sizes. Fix the flow and routing first.