> Canonical URL: https://codingcat.dev/post/client-side-webgpu-rag-transformers-js-mediapipe-opfs

# Zero-Backend Client-Side RAG in the Browser with WebGPU, Transformers.js, MediaPipe, and OPFS

✏️ Blog PostJune 9, 2026[![](/logos/tags/webgpu-symbol-light.svg)![](/logos/tags/webgpu-symbol-dark.svg)WebGPU](/topic/webgpu)[![](/logos/tags/transformers-js-symbol-light.svg)![](/logos/tags/transformers-js-symbol-dark.svg)Transformers.js](/topic/transformers-js)[#Web AI](/topic/web-ai)[#RAG](/topic/rag)[#Web Standards](/topic/web-technologies)[#AI & Agents](/topic/ai)[![](/logos/tags/rust-symbol-light.svg)![](/logos/tags/rust-symbol-dark.svg)Rust](/topic/rust)[![](/logos/tags/chrome-built-in-ai-symbol-light.svg)![](/logos/tags/chrome-built-in-ai-symbol-dark.svg)Chrome Built-in AI](/topic/chrome-built-in-ai)

# Zero-Backend Client-Side RAG in the Browser with WebGPU, Transformers.js, MediaPipe, and OPFS

![Alex Patterson and AJ the Cat demonstrating zero-backend client-side WebGPU RAG with OPFS vector storage](https://cdn.sanity.io/media-libraries/mlttp0BZbCpV/images/107fa0e8c1aed86a7cc97ee7885d5df2b1dd9bfe-1376x768.jpg?w=1376&q=82&fit=max&auto=format)[![Alex Patterson](https://cdn.sanity.io/media-libraries/mlttp0BZbCpV/images/a606dfaa381aab08ddbe16a33fe1d3e1fb37c927-400x400.jpg?w=96&h=96&q=82&fit=max&auto=format)Alex Patterson](/author/alex-patterson)

Most teams building semantic search or Retrieval-Augmented Generation (RAG) for documentation portals reach straight for a cloud vector database and an external embedding API. That works fine until you need instant sub-10ms keystroke search, offline availability, or strict zero-egress privacy for user documents. With [**WebGPU compute shaders**](https://developer.chrome.com/docs/web-platform/webgpu), **Transformers.js v3**, **Google MediaPipe**, and the `Origin Private File System (OPFS)`, you can run the entire embedding, vector indexing, and synthesis pipeline directly inside the user's browser tab.

## The Mental Model: Local WebGPU Shaders + OPFS Binary Vector Cache

Why did client-side embeddings feel sluggish three years ago? Because running matrix multiplication on a single JavaScript main thread blocks UI rendering, and WebGL was designed for rasterizing triangles rather than general-purpose tensor math. **WebGPU**  gives browser Web Workers direct access to GPU compute pipelines (`GPUBuffer` and WGSL compute shaders), while `OPFS` provides high-throughput binary file storage so quantized ONNX models and pre-computed `Float32Array` vector indexes load from disk in milliseconds on repeat visits.

![Dark-mode Architecture Diagram comparing server-bound cloud vector pipelines against zero-backend client-side WebGPU RAG with Transformers.js and OPFS](https://cdn.sanity.io/media-libraries/mlttp0BZbCpV/images/7b8fc02e94a8ead7890df267e1130241c7fa324a-1376x768.jpg?w=1376&q=82&fit=max&auto=format)Architecture Diagram: Cloud-Bound Vector Pipeline (left, Coral anti-pattern) vs. Zero-Backend Client-Side WebGPU RAG with OPFS and On-Device Gemini Nano (right, Mint recommended pattern).

## Browser Compute Runtimes Comparison: WebAssembly vs. WebGL vs. WebGPU vs. WebNN

Browser Compute BackendHardware Target384-d Embedding Latency (Batch=1)Quantization Support & Best Use CaseWASM SIMD (Threads)Multi-core CPU18ms to 45msq8 / int8 universal fallback for devices without discrete or integrated WebGPU.WebGL 2.0Legacy GPU Fragment Shaders12ms to 30msfp16 / fp32 texture hacks; lacks native storage buffers and compute workgroups.WebGPU (navigator.gpu)Modern GPU Compute Units1.8ms to 5.5msq4f16 / fp16 / int8 via WGSL compute shaders; ideal for Transformers.js v3 and MediaPipe.WebNN (navigator.ml)Dedicated NPU / OS ML Accelerator1.2ms to 4.0msDirect OS-level graph compilation (DirectML, CoreML, NNAPI) for ultra-low battery drain.

## Step 1: Generating 384-Dimensional Embeddings on WebGPU in a Web Worker

To keep Main Thread Interaction to Next Paint (INP) under 16ms, always initialize your embedding pipeline inside a dedicated Web Worker and probe `navigator.gpu.requestAdapter()` to select `device: 'webgpu'` with graceful fallback to `'wasm'`:

Copied! 🐾

```
// src/workers/embedding-worker.ts
import { pipeline, env, type FeatureExtractionPipeline } from "@huggingface/transformers";

// Cache quantized ONNX weights in the browser Cache API / OPFS
env.allowLocalModels = false;
env.useBrowserCache = true;

let extractorPromise: Promise | null = null;

async function detectOptimalDevice(): Promise {
  if (typeof navigator !== "undefined" && "gpu" in navigator) {
    try {
      const adapter = await (navigator as unknown as { gpu: GPU }).gpu.requestAdapter();
      if (adapter) return "webgpu";
    } catch (_) {
      // Fallback to WASM SIMD
    }
  }
  return "wasm";
}

export async function getEmbeddingExtractor(): Promise {
  if (!extractorPromise) {
    const device = await detectOptimalDevice();
    extractorPromise = pipeline(
      "feature-extraction",
      "Snowflake/snowflake-arctic-embed-xs",
      {
        device,
        dtype: device === "webgpu" ? "fp16" : "q8",
      }
    );
  }
  return extractorPromise;
}

export async function embedTextToFloat32(text: string): Promise {
  const extractor = await getEmbeddingExtractor();
  const output = await extractor(text, {
    pooling: "cls",
    normalize: true,
  });
  return new Float32Array(output.data as Float32Array);
}
```

## Step 2: Persisting Binary Vector Indexes in Origin Private File System (OPFS)

Storing thousands of JSON-serialized float arrays in `localStorage` or naive IndexedDB object stores creates massive serialization overhead. Instead, pack your normalized `Float32Array` embeddings contiguously into a binary file inside `navigator.storage.getDirectory()` (OPFS) and run a single-pass dot-product scan:

Copied! 🐾

```
// src/lib/vector/opfs-vector-index.ts
export interface VectorMetadata {
  id: string;
  title: string;
  slug: string;
  excerpt: string;
}

export class OpfsFlatVectorIndex {
  private readonly dim: number;
  private vectors: Float32Array = new Float32Array(0);
  private docs: VectorMetadata[] = [];

  constructor(dimensions = 384) {
    this.dim = dimensions;
  }

  async saveToOpfs(filename = "codingcat-vectors-384d.bin"): Promise {
    const root = await navigator.storage.getDirectory();
    const fileHandle = await root.getFileHandle(filename, { create: true });
    const writable = await fileHandle.createWritable();
    await writable.write(this.vectors.buffer);
    await writable.close();

    const metaHandle = await root.getFileHandle(`${filename}.meta.json`, { create: true });
    const metaWritable = await metaHandle.createWritable();
    await metaWritable.write(JSON.stringify(this.docs));
    await metaWritable.close();
  }

  async loadFromOpfs(filename = "codingcat-vectors-384d.bin"): Promise {
    try {
      const root = await navigator.storage.getDirectory();
      const fileHandle = await root.getFileHandle(filename);
      const file = await fileHandle.getFile();
      const arrayBuffer = await file.arrayBuffer();
      this.vectors = new Float32Array(arrayBuffer);

      const metaHandle = await root.getFileHandle(`${filename}.meta.json`);
      const metaFile = await metaHandle.getFile();
      this.docs = JSON.parse(await metaFile.text()) as VectorMetadata[];
      return true;
    } catch {
      return false;
    }
  }

  /**
   * Since embeddings are L2-normalized at extraction time,
   * Cosine Similarity equals the raw dot product.
   */
  searchTopK(queryVec: Float32Array, k = 5): Array {
    const count = this.docs.length;
    const dim = this.dim;
    const scores: Array = [];

    for (let i = 0; i  b.score - a.score);
    return scores.slice(0, k).map(({ idx, score }) => ({
      ...this.docs[idx],
      score,
    }));
  }
}
```

## AJ Pro-Tip: Ship Pre-Computed Binary Embeddings from Your Astro Build

> AJ Pro-Tip: Pre-Compute Corpus Embeddings at Build Time, Embed Only the Query in the Browser. Do not force every visitor's GPU to embed your entire 500-article blog archive on first page load. Generate your `codingcat-vectors-384d.bin` binary artifact once during your Astro static/SSR build (less than 750KB for 500 articles at `Float32` or 375KB at `Float16`), cache it in OPFS, and only run the WebGPU embedding pipeline on the user's 10-word search query.

## **Try the Live Browser AI & WebMCP Playground**

You can test every pattern from this article live right now in our interactive [**Web AI & WebMCP Playground**](/demos/web-ai) on CodingCat.dev, or inspect our production discovery manifests on the [**Developers & Agent APIs**](/developers) page. If you are organizing a developer conference or engineering summit, check out my Web AI talk abstracts on [**alexpatterson.dev**](https://alexpatterson.dev) and explore more hands-on [**Courses**](/courses), [**Podcasts**](/podcasts), and [**Blog Posts**](/blog) right here on CodingCat.dev.

✏️ Visual Architecture Diagram

## how this post connects together.

Follow AJ from first principles to production-ready architecture.

Let’s build it!✏️ Visual Diagram: Zero-Backend Client-Side RAG in the Browser with…✓ Good PatternVISUAL DIAGRAM[Diagram Walkthrough]Step-by-step diagram breakdown for Zero-Backend Client-Side RAG in the Browser with WebGPU, Transformers.js, MediaPipe, and OPFS: start with Zero, implement Hands-On Implementation, and ship to production.AAA Contrast Verified▶ Watch on @CodingCatDev YouTubeRelated Course & Video

## Watch the Related Video Deep Dive: Local-First Web Apps: PGlite (WASM Postgres) & Real-Time Sync

Run full Postgres inside the browser via WebAssembly with PGlite, persist to OPFS, sync state in real time with ElectricSQL Shapes, and build zero-latency offline-capable web apps.

[View Full Page →](/course/local-first-web-apps-pglite-electric-sql)![](https://cdn.sanity.io/images/hfh83o0w/production/c0aaa4007ae983fdc5ee4bcb4b2afadfaf6a44a2-1376x768.png?rect=6,0,1365,768&w=1920&h=1080&q=82&fit=max&auto=format)[@CodingCatDev62.5K / 100K Subscribers Goal](https://www.youtube.com/@codingcatdev?sub_confirmation=1)[Subscribe on YouTube→](https://www.youtube.com/@codingcatdev?sub_confirmation=1)[Watch on YouTube App ↗](https://www.youtube.com/watch?v=ZD8j6K3-ArU&list=UUnKZ8gEb78zXKMi1ns-IQ2g)

## related deep dives.

[All Blog Posts&rarr;](/blog)[![Chrome DevTools MCP: Self-Healing Agent Visual & Video Inspection](https://cdn.sanity.io/images/hfh83o0w/production/a8275a17a27ec4774212b57a9655a174df309bb6-1376x768.png?w=1376&q=82&fit=max&auto=format)](/course/chrome-devtools-mcp-visual-agent-loops)November 4, 2026[![](/logos/tags/typescript-symbol-light.svg)![](/logos/tags/typescript-symbol-dark.svg)TypeScript](/topic/typescript)[#AI & Agents](/topic/ai)[![](/logos/tags/react-symbol-light.svg)![](/logos/tags/react-symbol-dark.svg)React](/topic/react)

### [Chrome DevTools MCP: Self-Healing Agent Visual & Video Inspection](/course/chrome-devtools-mcp-visual-agent-loops)

Give your AI agent eyes inside the browser. Use Chrome DevTools MCP so your agent inspects its own rendered video frames, WebGL canvases, and DOM layouts before exporting.

[![Alex Patterson](https://cdn.sanity.io/media-libraries/mlttp0BZbCpV/images/a606dfaa381aab08ddbe16a33fe1d3e1fb37c927-400x400.jpg?w=96&h=96&q=82&fit=max&auto=format)Alex Patterson](/author/alex-patterson)[![Three.js & WebGPU in Production: Interactive 3D Mascots & Viewers](https://cdn.sanity.io/images/hfh83o0w/production/c6726c757067aa8e8de8e01cb7c1ac17c948ef97-1376x768.png?w=1376&q=82&fit=max&auto=format)](/course/threejs-webgpu-3d-mascots)November 4, 2026[![](/logos/tags/typescript-symbol-light.svg)![](/logos/tags/typescript-symbol-dark.svg)TypeScript](/topic/typescript)[#AI & Agents](/topic/ai)[![](/logos/tags/threejs-symbol-light.svg)![](/logos/tags/threejs-symbol-dark.svg)Three.js](/topic/threejs)[+1](/course/threejs-webgpu-3d-mascots)

### [Three.js & WebGPU in Production: Interactive 3D Mascots & Viewers](/course/threejs-webgpu-3d-mascots)

Build our production `<Aj3dMascotViewer />` component in Astro and Three.js. Master GLTFLoader, studio rim lighting, clamped cursor pupil tracking, and deterministic `window.seek(t)` scrubbing.

[![Alex Patterson](https://cdn.sanity.io/media-libraries/mlttp0BZbCpV/images/a606dfaa381aab08ddbe16a33fe1d3e1fb37c927-400x400.jpg?w=96&h=96&q=82&fit=max&auto=format)Alex Patterson](/author/alex-patterson)