↑ ↓ navigate↵ selectesc close

Building a cooking assistant with Firebase AI Logic: Web Architecture & Implementation Guide

CodingCat.dev tutorial cover image
Alex Patterson
Alex Patterson

Calling frontier multimodal models directly from a web or mobile client is a security minefield if you expose raw API keys or hardcode brittle system prompts in your frontend bundle. Following the announcement of Building a cooking assistant with Firebase AI Logic by Marina Coelho on the Official Firebase Blog (2026-05-29), here is our hands-on CodingCat.dev breakdown of what actually shipped, how it works under the hood, and how we wire it into production TypeScript, Next.js, Astro, SvelteKit, React, and Angular applications. Last week, I decided to put Firebase AI Logic to the test by adding a new feature to the Friendly Meals Android app. Picture this: you’re in the kitchen, your hands are covered in flour and you need to check what’s the next step in the recipe. The next step requires handling raw chicken, and while you’re at it, you remember that you need to get more garlic because you just used the last clove!

The Mental Model: Why Building a cooking assistant with Firebase AI Logic Matters for Web Devs

As a Google Developer Expert (GDE) for Firebase and Senior Solutions Engineer at CodingCat.dev, I evaluate every Firebase update through one lens: does it reduce boilerplate while keeping our runtime security and cloud bills predictable? Firebase AI Logic lets you invoke Gemini and Imagen models from client and edge apps using the Firebase Web SDK while keeping your API credentials off the wire, enforcing Firebase App Check, and managing server-side prompt templates centrally.

AJ Pro-Tip from Alex: Combine Firebase AI Logic with server prompt templates, Firebase Remote Config feature flags, and project spend caps so you can iterate on prompts or swap Gemini models instantly without redeploying your Next.js or Astro frontend.

How It Works: Technical Breakdown & Implementation

Wouldn’t it be easier if you could leave your phone pointed at your cooking area and ask questions as you cook? Wouldn’t it be so handy if a cooking assistant could add the missing ingredients to your grocery list without lifting a finger?

That’s what I wanted to build: a real-time cooking assistant combining the conversational speed of the Gemini Live API with the power of function calling. In this post, I’ll show you how you can leverage bidirectional streaming video and client-side function calling to build more complex and helpful features.

Firebase AI Logic

Firebase AI Logic provides secure client SDKs that lets mobile and web developers interact directly with Google’s generative models. Rather than routing streaming video and audio through a custom backend (and figuring out secure authentication on your own, or even how to deal with WebSockets), Firebase AI Logic manages a secure connection for you and protects your resources when you add Firebase Authentication and Firebase App Check to your projects.

The Friendly Meals cooking assistant uses a Gemini Live model to establish a bidirectional stream. The user shows the cooking area and speaks to the model, the model processes the video stream, understands intent having the current recipe as context, and streams audio responses back quickly.

This model can also handle function calling, a capability that is very handy when you need to extend the model’s knowledge and enable it to act like an agent, triggering actions and performing tasks on your behalf.

How Friendly Meals implements Gemini Live

Let’s inspect the key changes on Friendly Meals to make this cooking companion a reality.

Because I’m building my video assistant on Android, the AndroidManifest.xml requires permission to capture microphone and camera data:

The first two lines are requesting the necessary permissions. The last line is telling the Android system and the Google Play Store that the app has some camera features, but these features are not required for the app to work. This is important because it allows users without a camera to still download your app and use the features that don’t rely on it.

Friendly Meals has a dedicated class to handle the lifespan of the streaming session: LiveAIRemoteDataSource. It begins by initializing a LiveGenerativeModel instance, passing the appropriate model name, a voice character profile, and the response modality:

suspend
  val
    speechConfig 
    responseModality 
  }

  val
  val

  val
    modelName 
    generationConfig 
    systemInstruction 
  )

  return
    liveModel.
  } 
    null
  }
}

companion
  //Live Model Config
  private

  //Remote Config Keys
  private
  private
}

It also needs some system instructions to define the culinary expert persona, and specify what recipe the user is currently cooking. Notice that both prompt and model name are stored in Firebase Remote Config. This allows me to store them on the server and update them at any time, without requiring my users to install another version of your app if I need to modify either model or prompt.

These are the instructions I stored in Remote Config:

You are a helpful live cooking assistant. The user is currently preparing the following recipe:
Title: {{title}}
Prep time: {{prepTime}}, Cook time: {{cookTime}}, Servings: {{servings}}

Ingredients:
{{ingredients}}

Instructions:
{{instructions}}

The user will stream real-time video of their cooking and ask questions like "Is this the expected texture of the recipe?".
Confirm or deny accurately based on the recipe context and the video content. Be concise and helpful.
If the user asks you to add an ingredient or item to their grocery list or shopping list, call the addIngredientToGroceryList function.

I then use Kotlin’s replace function to format your prompt:

private
  return
    .
    .
    .
    .
    .
    .

Quick note: We highly recommend protecting your model configuration, system instructions and prompt with Server Prompt Templates in Firebase AI Logic. Bidirectional streaming is not yet supported by templates, but you should prioritize using them for all the models and capabilities we support.

To handle the bidirectional streaming, the LiveAssistantViewModel calls the setupLiveSession function shown above, and stores the current session in the liveSession object. The view model is responsible for starting and ending the live session according to the taps performed by the user:

  • Starts session when the user taps on the “Live Cooking Assistant” button, available in the Recipe screen
  • Ends session when the user taps on the button to close the Assistant, available in the navigation bar at the top of the screen

The view model uses the liveSession object to send video frames to the Gemini Live model, as seen on the function below:

fun
  if

  val

  // Limit sending frames to once per second to conserve bandwidth and processing
  if
  lastFrameTime 

  launchCatchingIO
    val
    bitmap.
    val
    liveSession?.
  }
}

Enabling hands-free actions with function calling

A conversational UI is great, but a true assistant needs to affect the application state. If a user says, “Hey chef, I just used the last of the olive oil, add it to my shopping list,” or “We’re out of garlic,” the model shouldn’t just reply, “Okay, I’ve noted that.” It needs to execute that action inside the app. Let’s see how I achieved that with function calling.

First, I declared my tool to manage grocery items in LiveAIRemoteDataSource:

private
  FunctionDeclaration
    name 
    description 
    parameters 
  )
))

Next, I added the new tool directly to the LiveModel configuration, when I’m creating an instance of the live model:

val
  modelName 
  generationConfig 
  systemInstruction 
  tools 
)

Lastly, I updated the view model to send a session handler as a parameter when starting a new session. This handler is of type FunctionResponsePart and defines what happens if the addIngredientToGroceryList tool is called:

private
  if
    val
    val
      is
      else
    }?.

    if
      val

      if
        launchCatching
          val
            userId 
            name 
            checked 
          )
          databaseRepository.
        }
      }
    }

    return
      functionCall.name,
      JsonObject
      functionCall.id
    )
  }

  return
}

The code above checks if the function being called is the one registered previously. If so, it performs some checks on the data: First, checks if it’s a JSON object to get the content, or if it’s a normal string to trim and clear the surroundings. Second, it checks if the resulting ingredient name is null or blank. Last, it gets the current logged-in user and stores the ingredient in their grocery list.

Now when the user asks the cooking assistant to add garlic (or any other ingredient!) to their list, the model will be able to detect the user’s intent, map it to the addIngredientToGroceryList tool, extract the string “garlic”, and trigger the function.

Going to production

Before shipping your apps to production, make sure you are protecting them with App Check. By adding App Check, you ensure only your legitimate app running on untampered devices can call any Firebase backend services. It helps protect your backend from abuse, such as billing fraud, phishing, app impersonation, and data poisoning.

Check out the App Check documentation to learn more about it.

Build your own real-time features

With client-side streaming and secure client-to-server management handled entirely via Firebase AI Logic, you don’t need a single line of backend infrastructure to bring agentic voice and video experiences into your mobile and web apps.

Head over to the Friendly Meals GitHub repository to review the codebase in full, see how the app manages UI states during active sessions, and start building hands-free, real-time features into your own apps!

Production Web Architecture & Security Checklist

Whether we are deploying on Firebase App Hosting, Cloudflare Workers, or Vercel, shipping AI features to production requires defense-in-depth across three layers:

  • Enforce Firebase App Check & Attestation: Protect your web endpoints, Cloud Functions, Data Connect GraphQL resolvers, and Firebase AI Logic calls against unauthorized replay or billing abuse using Firebase App Check with reCAPTCHA Enterprise.
  • Validate Locally with the Firebase Emulator Suite: Run deterministic integration tests against local Firestore, Auth, Functions, and Data Connect PostgreSQL emulators before promoting schema or rule changes to production.
  • Keep Server & Client Boundaries Type-Safe: Use the modular Firebase Web SDK and generated TypeScript bindings so SSR frameworks like Next.js, Astro, SvelteKit, and Angular serialize state cleanly without hydration flickers.

Conclusion

Accelerate your full-stack web and AI development by pairing Building a cooking assistant with Firebase AI Logic with strict Security Rules, Firebase App Check, and local emulator testing. Come tell us on X (@CodingCatDev) and LinkedIn (CodingCat.dev) how you are using Firebase in your stack, and subscribe to the CodingCat.dev YouTube Channel so you don't miss what we build next!

Watch Our Related Firebase Video Tutorial on YouTube

Want to see how we architect and debug Firebase web applications live? Watch our related CodingCat.dev session Firebase Security Rules: Effortless control over your app's data. below, and make sure to Subscribe to CodingCat.dev on YouTube for weekly full-stack web and AI deep dives:

Resources, Official Docs & Related CodingCat.dev Guides

Frequently Asked Questions (FAQ)

What are the key capabilities introduced in Building a cooking assistant with Firebase AI Logic?

Building a cooking assistant with Firebase AI Logic (announced 2026-05-29 by the Firebase team) enhances the platform for Firebase, Google Cloud, Web Dev, AI, TypeScript, Security workloads. Last week, I decided to put Firebase AI Logic to the test by adding a new feature to the Friendly Meals Android app. Picture this: you’re in the kitchen, your hands are covered in flour and you need to check what’s the next step in the recipe. The next step requires handling raw chicken, and while you’re at it, you remember that you need to get more garlic because you just used the last clove! Developers can review the primary announcement on the Firebase Blog alongside our implementation patterns and production checklist above.

How do web developers use Building a cooking assistant with Firebase AI Logic with Next.js, SvelteKit, React, or Angular?

Web developers can integrate these features using the modular Firebase JavaScript/TypeScript SDK, Firebase Data Connect web bindings, and Firebase App Hosting. Always pair client-side SDK initialization with Firebase App Check and strict Security Rules, and test locally with the Firebase Local Emulator Suite before deploying.

Where can I watch video tutorials on Firebase Web and AI development?

You can watch our hands-on Firebase video walkthroughs on the CodingCat.dev YouTube Channel, including Firebase Security Rules: Effortless control over your app's data., and follow us on X (@CodingCatDev) for weekly full-stack tutorials.