↑ ↓ navigate↵ selectesc close

Introducing hybrid on-device inference using Firebase AI Logic: Web Architecture & Implementation Guide

CodingCat.dev tutorial cover image
Alex Patterson
Alex Patterson

Calling frontier multimodal models directly from a web or mobile client is a security minefield if you expose raw API keys or hardcode brittle system prompts in your frontend bundle. Following the announcement of Introducing hybrid on-device inference using Firebase AI Logic by Siddharth Gupta, Erik Eldridge on the Official Firebase Blog (2025-06-09), here is our hands-on CodingCat.dev breakdown of what actually shipped, how it works under the hood, and how we wire it into production TypeScript, Next.js, Astro, SvelteKit, React, and Angular applications. The Firebase team announced a powerful new experimental feature in the Firebase AI Logic client SDK for Web: hybrid on-device inference!

The Mental Model: Why Introducing hybrid on-device inference using Firebase AI Logic Matters for Web Devs

As a Google Developer Expert (GDE) for Firebase and Senior Solutions Engineer at CodingCat.dev, I evaluate every Firebase update through one lens: does it reduce boilerplate while keeping our runtime security and cloud bills predictable? Firebase AI Logic lets you invoke Gemini and Imagen models from client and edge apps using the Firebase Web SDK while keeping your API credentials off the wire, enforcing Firebase App Check, and managing server-side prompt templates centrally.

AJ Pro-Tip from Alex: Combine Firebase AI Logic with server prompt templates, Firebase Remote Config feature flags, and project spend caps so you can iterate on prompts or swap Gemini models instantly without redeploying your Next.js or Astro frontend.

How It Works: Technical Breakdown & Implementation

Harness the power of on-device AI models with a seamless fallback to cloud-hosted models

This ensures your generative AI-powered features are always available to your users, online or offline.

Chrome on Desktop supports Gemini Nano, a powerful on-device model that can be downloaded and hosted in the browser on-demand. Running AI models directly on a user’s device offers some great advantages:

  • Enhanced privacy: Sensitive data can stay right on the user’s device.
  • Offline availability: Your app’s AI features can work even without an internet connection.
  • Cost savings: On-device inference is available at no cost, reducing your application’s operational expenses.
  • On-device performance: You can often get high performance thanks to hardware acceleration.

You can build AI features that utilize all these advantages of an on-device model, but then fallback to a cloud-hosted model when its capabilities are needed.

On-Device and In-Cloud Inference

The Firebase AI Logic client SDK for Web has a hybrid on-device extension that uses the proposed W3C Prompt API for on-device model capabilities. It uses built-in, on-device models (like Gemini Nano in Chrome) when it’s available. If that’s not an option on a particular device or browser, the SDK falls back to using Gemini models on the server. This maximizes your reach, ensuring that your users can access your AI features, regardless of their device capabilities.

How it works

Using hybrid on-device inference is straightforward. The Firebase AI Logic client SDK for Web automatically handles the logic of checking for on-device model availability and falling back to a cloud-hosted model if necessary.

Here’s a look at how you can implement hybrid inference in your web app:

// Initialize the Gemini Developer API backend service
const

// Create a `GenerativeModel` instance
// Set the mode, for example to use on-device model when possible
const

// run inference and log result
const
console.

This implementation is almost identical to a regular getGenerativeModel call, but with the addition of the mode parameter that sets how you want to access a model. The SDK takes care of the rest!

Furthermore, you can also configure this code to always use either an on-device or cloud-hosted model for inference by switching the `mode` parameter:

// Run inference only on device. Does not fallback to cloud-hosted models.
const

// Run inference only on the cloud-hosted models.
const

If you’re curious, you can try out our demo and view the codebase.

Production Web Architecture & Security Checklist

Whether we are deploying on Firebase App Hosting, Cloudflare Workers, or Vercel, shipping AI features to production requires defense-in-depth across three layers:

  • Enforce Firebase App Check & Attestation: Protect your web endpoints, Cloud Functions, Data Connect GraphQL resolvers, and Firebase AI Logic calls against unauthorized replay or billing abuse using Firebase App Check with reCAPTCHA Enterprise.
  • Validate Locally with the Firebase Emulator Suite: Run deterministic integration tests against local Firestore, Auth, Functions, and Data Connect PostgreSQL emulators before promoting schema or rule changes to production.
  • Keep Server & Client Boundaries Type-Safe: Use the modular Firebase Web SDK and generated TypeScript bindings so SSR frameworks like Next.js, Astro, SvelteKit, and Angular serialize state cleanly without hydration flickers.

Conclusion

Accelerate your full-stack web and AI development by pairing Introducing hybrid on-device inference using Firebase AI Logic with strict Security Rules, Firebase App Check, and local emulator testing. Come tell us on X (@CodingCatDev) and LinkedIn (CodingCat.dev) how you are using Firebase in your stack, and subscribe to the CodingCat.dev YouTube Channel so you don't miss what we build next!

Watch Our Related Firebase Video Tutorial on YouTube

Want to see how we architect and debug Firebase web applications live? Watch our related CodingCat.dev session Firebase SQL with Data Connect below, and make sure to Subscribe to CodingCat.dev on YouTube for weekly full-stack web and AI deep dives:

Resources, Official Docs & Related CodingCat.dev Guides

Frequently Asked Questions (FAQ)

What are the key capabilities introduced in Introducing hybrid on-device inference using Firebase AI Logic?

Introducing hybrid on-device inference using Firebase AI Logic (announced 2025-06-09 by the Firebase team) enhances the platform for Firebase, Google Cloud, Web Dev, AI, TypeScript workloads. The Firebase team announced a powerful new experimental feature in the Firebase AI Logic client SDK for Web: hybrid on-device inference! Developers can review the primary announcement on the Firebase Blog alongside our implementation patterns and production checklist above.

How do web developers use Introducing hybrid on-device inference using Firebase AI Logic with Next.js, SvelteKit, React, or Angular?

Web developers can integrate these features using the modular Firebase JavaScript/TypeScript SDK, Firebase Data Connect web bindings, and Firebase App Hosting. Always pair client-side SDK initialization with Firebase App Check and strict Security Rules, and test locally with the Firebase Local Emulator Suite before deploying.

Where can I watch video tutorials on Firebase Web and AI development?

You can watch our hands-on Firebase video walkthroughs on the CodingCat.dev YouTube Channel, including Firebase SQL with Data Connect, and follow us on X (@CodingCatDev) for weekly full-stack tutorials.