↑ ↓ navigate↵ selectesc close

Eval-driven development: How we build better agent skills for Firebase: Web Architecture & Implementation Guide

CodingCat.dev tutorial cover image
Alex Patterson
Alex Patterson

If you have ever watched an AI coding agent hallucinate deprecated Firebase v8 syntax or deploy wide-open Firestore rules, you know that raw LLM prompts are not enough for production backends. Following the announcement of Eval-driven development: How we build better agent skills for Firebase by Charlotte Liang on the Official Firebase Blog (2026-08-11), here is our hands-on CodingCat.dev breakdown of what actually shipped, how it works under the hood, and how we wire it into production TypeScript, Next.js, Astro, SvelteKit, React, and Angular applications. Earlier, the Firebase team introduced Firebase Agent Skills, our way of giving AI agents the structured cheat sheets they need to write better Firebase code. the agent skills help coding agents navigate Firebase’s latest APIs, avoid deprecated features, and follow our best practices.

The Mental Model: Why Eval-driven development: How we build better agent skills for Firebase Matters for Web Devs

As a Google Developer Expert (GDE) for Firebase and Senior Solutions Engineer at CodingCat.dev, I evaluate every Firebase update through one lens: does it reduce boilerplate while keeping our runtime security and cloud bills predictable? Instead of relying on stale training weights, wiring up the Model Context Protocol (MCP) and structured Firebase agent skills gives your IDE or CLI direct, schema-aware tools to provision resources, inspect security rules, and run deterministic CLI deployments.

AJ Pro-Tip from Alex: Never let an autonomous coding agent push Firestore or Data Connect rules straight to production without a gate. Always require the agent to run your test suite against the Firebase Local Emulator Suite first, and audit every generated rule for strict request.auth.uid ownership checks.

How It Works: Technical Breakdown & Implementation

The agentic toolkit: CLI, MCP servers, and skills

Skills don’t operate in a vacuum - they need great tools as well. We provide two types of tools for agents to use: the Firebase CLI and the MCP servers.

  • CLIs are loved by AI agents for their straightforward execution and terminal-based workflows.
  • MCP servers are “agent friendly” because of the explicit tool definitions that let agents understand exactly what a tool does and what arguments it requires.

So, where do agent skills fit into this picture? Skills are the conductor that orchestrates everything together. To get more accurate results, I recommend developers enable and install both the Firebase Agent Skills and the Firebase MCP server.

The Firebase local MCP server includes tools to access the Google Developer Knowledge MCP server. This connection seamlessly guides agents to search and read official Google documentation, including Firebase.

You might wonder: What is the difference between an agent skill and official documentation?

While documentation provides comprehensive reference material, skills are highly focused, outcome-driven cheat sheets. They are specifically designed to onboard users to Firebase products, enforce core principles, highlight best practices, and provide preemptive guidance for tasks where agents are known to struggle. For example, a skill can help a developer add a feature in their app that securely uses Firestore database, everything from provisioning in the cloud to writing the app code for interacting with the database. We even have agent skills that review and help to write robust Firebase Security Rules to protect user data.

Furthermore, skills act as a vital bridge for agent knowledge. Because LLMs are limited by training data cut-off dates, they won’t know about recent releases and changes. A skill can explicitly instruct the agent to use the CLI or an MCP server tool to implement the feature (or to read the latest documentation), ensuring the model can discover and utilize brand-new features on the fly.

The need for measurement

As Firebase expanded this toolkit and wrote more agent skills, we ran into a critical question: how do we actually know if these skills are helping agents succeed?

We couldn’t just create skills and hope for the best. An agent might misinterpret a prompt, fail to activate the right skill, or get tripped up by a fragile workflow. We needed a way to programmatically measure an agent’s success rate.

To solve this, we built a set of evals, automated prompt suites that score how well an agent completes specific Critical User Journeys (CUJs).

We use a few different types of evals to understand exactly how agents use our products:

Single skill evaluations: This is the simplest and most important type of eval. It provides the agent with a single skill, for instance the firebase-firestore skill, and evaluates its core functionality against 5-10 specific cases per CUJ.

Here’s an example of what skill evaluation looks like:

prompt
expectations
  - 
  -

Skill activation evaluations: Having a great skill doesn’t matter if the agent doesn’t know when to use it! Activation evals test the skill’s frontmatter (name and description). We run a mix of positive and negative prompts to ensure the agent activates the skill when needed, without over-activating on irrelevant prompts.

prompt
expected_skills
  -

E2E multi-skill evaluations: These are the ultimate test. We provide the agent with all the skills and ask it to create and deploy an app that uses multiple Firebase products. These evals interact with a real Firebase project to ensure the skills work together in a realistic environment.

prompt
expectations
  - 
  - 
  - 
  - 
  - 
  -

When we first started developing the agent skills, it was easier and faster to just write what we thought were important instructions for the agent. However, with our evolution to evaluation-driven development of skills, we now know what instructions are important because we can now start with the test and then write and iterate on skills until the agent succeeds.

Here is what the eval-driven development iteration loop looks like today, which we start adopting not only to refine the skills, but to uncover friction in the CLI and MCP servers as well:

Write the eval first Before we write any agent skills, we define the expected outcome for a task. We set strict LLM-as-a-judge expectations for what a successful Firebase implementation looks like.

Establish a baseline We prompt the agent to complete the task without providing the agent any of the Firebase agent skills, which lets us capture a baseline failure rate.

Add and refine skills Next, we write the skill. We look for what failed in the baseline evaluation, and include specific procedural steps, mandatory flags, and “gotchas” into the agent skills to address the agent’s exact pain points.

Hill climb We re-run the evaluation. If the score goes up, we keep the changes. We iterate on this loop until the agent consistently passes the eval.

By adopting an eval-first mindset and building a cohesive toolkit of skills and servers, we stopped guessing what agents might misunderstand and started programmatically proving it. To show just how much of a difference this makes, we compared the baseline performance of an agent without skills against an agent equipped with our tuned Firebase agent skills.

To interpret the data below, the following definitions describe how resources are measured:

  • Input Tokens: The number of tokens the model reads. This includes the original prompt, agent skills, tool definitions (such as CLI or MCP tool descriptions), error messages, and web searches. A reduction in input tokens suggests the agent spent less wasted time searching for context.
  • Output Tokens: The actual response provided to the user, including code written and internal “thinking” tokens. A reduction in output tokens suggests a more concise answer, less effort spent rewriting code, and a more streamlined reasoning process.

Eval results demonstrate that integrating Skills raises pass rate from 31.7% to 78.0% alongside substantial reductions in token overhead and execution time.

An important impact of eval-driven development is that it doesn’t just help us write better skills, it helps us build better tools.

When we watch an agent repeatedly fail a task even after we’ve given it a well-written skill, it acts as a harsh usability test. It’s a strong signal that the underlying workflow or command is too complex, fragile, or confusing. We can take these insights and use them to improve the Firebase CLI and our MCP tools, streamlining the developer experience for both AI agents and human developers.

We’ve written an evaluation suite for Firebase CLI usage, with ~50 cases covering common developer tasks, and we’ve started to use this suite to optimize the CLI for agents. In the past few weeks, we’ve used this approach to improve help text, provide smoother login for agents, and fix cases where agents were blocked by interactive prompts. We think this approach is the best way to build effective, agent friendly tooling, and we expect to use it much more in the coming months.

Production Web Architecture & Security Checklist

Whether we are deploying on Firebase App Hosting, Cloudflare Workers, or Vercel, shipping AI features to production requires defense-in-depth across three layers:

  • Enforce Firebase App Check & Attestation: Protect your web endpoints, Cloud Functions, Data Connect GraphQL resolvers, and Firebase AI Logic calls against unauthorized replay or billing abuse using Firebase App Check with reCAPTCHA Enterprise.
  • Validate Locally with the Firebase Emulator Suite: Run deterministic integration tests against local Firestore, Auth, Functions, and Data Connect PostgreSQL emulators before promoting schema or rule changes to production.
  • Keep Server & Client Boundaries Type-Safe: Use the modular Firebase Web SDK and generated TypeScript bindings so SSR frameworks like Next.js, Astro, SvelteKit, and Angular serialize state cleanly without hydration flickers.

Conclusion

Accelerate your full-stack web and AI development by pairing Eval-driven development: How we build better agent skills for Firebase with strict Security Rules, Firebase App Check, and local emulator testing. Come tell us on X (@CodingCatDev) and LinkedIn (CodingCat.dev) how you are using Firebase in your stack, and subscribe to the CodingCat.dev YouTube Channel so you don't miss what we build next!

Watch Our Related Firebase Video Tutorial on YouTube

Want to see how we architect and debug Firebase web applications live? Watch our related CodingCat.dev session Firebase Security Rules: Effortless control over your app's data. below, and make sure to Subscribe to CodingCat.dev on YouTube for weekly full-stack web and AI deep dives:

Resources, Official Docs & Related CodingCat.dev Guides

Frequently Asked Questions (FAQ)

What are the key capabilities introduced in Eval-driven development: How we build better agent skills for Firebase?

Eval-driven development: How we build better agent skills for Firebase (announced 2026-08-11 by the Firebase team) enhances the platform for Firebase, Google Cloud, Web Dev, AI, MCP, Security workloads. Earlier, the Firebase team introduced Firebase Agent Skills, our way of giving AI agents the structured cheat sheets they need to write better Firebase code. the agent skills help coding agents navigate Firebase’s latest APIs, avoid deprecated features, and follow our best practices. Developers can review the primary announcement on the Firebase Blog alongside our implementation patterns and production checklist above.

How do web developers use Eval-driven development: How we build better agent skills for Firebase with Next.js, SvelteKit, React, or Angular?

Web developers can integrate these features using the modular Firebase JavaScript/TypeScript SDK, Firebase Data Connect web bindings, and Firebase App Hosting. Always pair client-side SDK initialization with Firebase App Check and strict Security Rules, and test locally with the Firebase Local Emulator Suite before deploying.

Where can I watch video tutorials on Firebase Web and AI development?

You can watch our hands-on Firebase video walkthroughs on the CodingCat.dev YouTube Channel, including Firebase Security Rules: Effortless control over your app's data., and follow us on X (@CodingCatDev) for weekly full-stack tutorials.