> Canonical URL: https://codingcat.dev/post/client-side-webgpu-rag-transformers-js-mediapipe-opfs

# Zero-Backend Client-Side RAG in the Browser with WebGPU, Transformers.js, MediaPipe, and OPFS

✏️ Blog PostJune 9, 2026[AI & MCP Agents](/topic/ai)

# Zero-Backend Client-Side RAG in the Browser with WebGPU, Transformers.js, MediaPipe, and OPFS

![Alex Patterson and AJ the Cat demonstrating zero-backend client-side WebGPU RAG with OPFS vector storage](https://cdn.sanity.io/images/hfh83o0w/production/107fa0e8c1aed86a7cc97ee7885d5df2b1dd9bfe-1376x768.jpg?w=1376&q=82&fit=max&auto=format)[![Alex Patterson](https://cdn.sanity.io/images/hfh83o0w/production/a606dfaa381aab08ddbe16a33fe1d3e1fb37c927-400x400.jpg?w=96&h=96&q=82&fit=max&auto=format)Alex Patterson](/author/alex-patterson)✏️ Visual Architecture Diagram

## how this post connects together.

Follow AJ from first principles to production-ready architecture.

Let’s build it!✏️ Visual Diagram: Zero-Backend Client-Side RAG in the Browser with…✓ Good PatternVISUAL DIAGRAM[Diagram Walkthrough]Step-by-step diagram breakdown for Zero-Backend Client-Side RAG in the Browser with WebGPU, Transformers.js, MediaPipe, and OPFS: start with Zero, implement Hands-On Implementation, and ship to production.AAA Contrast Verified

Most teams building semantic search or Retrieval-Augmented Generation (RAG) for documentation portals reach straight for a cloud vector database and an external embedding API. That works fine until you need instant sub-10ms keystroke search, offline availability, or strict zero-egress privacy for user documents. With [**WebGPU compute shaders**](https://developer.chrome.com/docs/web-platform/webgpu), **Transformers.js v3**, **Google MediaPipe**, and the `Origin Private File System (OPFS)`, you can run the entire embedding, vector indexing, and synthesis pipeline directly inside the user's browser tab.

## The Mental Model: Local WebGPU Shaders + OPFS Binary Vector Cache

Why did client-side embeddings feel sluggish three years ago? Because running matrix multiplication on a single JavaScript main thread blocks UI rendering, and WebGL was designed for rasterizing triangles rather than general-purpose tensor math. **WebGPU**  gives browser Web Workers direct access to GPU compute pipelines (`GPUBuffer` and WGSL compute shaders), while `OPFS` provides high-throughput binary file storage so quantized ONNX models and pre-computed `Float32Array` vector indexes load from disk in milliseconds on repeat visits.

![Dark-mode Architecture Diagram comparing server-bound cloud vector pipelines against zero-backend client-side WebGPU RAG with Transformers.js and OPFS](https://cdn.sanity.io/images/hfh83o0w/production/7b8fc02e94a8ead7890df267e1130241c7fa324a-1376x768.jpg?w=1376&q=82&fit=max&auto=format)Architecture Diagram: Cloud-Bound Vector Pipeline (left, Coral anti-pattern) vs. Zero-Backend Client-Side WebGPU RAG with OPFS and On-Device Gemini Nano (right, Mint recommended pattern).

## Browser Compute Runtimes Comparison: WebAssembly vs. WebGL vs. WebGPU vs. WebNN

Browser Compute BackendHardware Target384-d Embedding Latency (Batch=1)Quantization Support & Best Use CaseWASM SIMD (Threads)Multi-core CPU18ms to 45msq8 / int8 universal fallback for devices without discrete or integrated WebGPU.WebGL 2.0Legacy GPU Fragment Shaders12ms to 30msfp16 / fp32 texture hacks; lacks native storage buffers and compute workgroups.WebGPU (navigator.gpu)Modern GPU Compute Units1.8ms to 5.5msq4f16 / fp16 / int8 via WGSL compute shaders; ideal for Transformers.js v3 and MediaPipe.WebNN (navigator.ml)Dedicated NPU / OS ML Accelerator1.2ms to 4.0msDirect OS-level graph compilation (DirectML, CoreML, NNAPI) for ultra-low battery drain.

## Step 1: Generating 384-Dimensional Embeddings on WebGPU in a Web Worker

To keep Main Thread Interaction to Next Paint (INP) under 16ms, always initialize your embedding pipeline inside a dedicated Web Worker and probe `navigator.gpu.requestAdapter()` to select `device: 'webgpu'` with graceful fallback to `'wasm'`:

```
// src/workers/embedding-worker.ts
import { pipeline, env, type FeatureExtractionPipeline } from "@huggingface/transformers";

// Cache quantized ONNX weights in the browser Cache API / OPFS
env.allowLocalModels = false;
env.useBrowserCache = true;

let extractorPromise: Promise | null = null;

async function detectOptimalDevice(): Promise {
  if (typeof navigator !== "undefined" && "gpu" in navigator) {
    try {
      const adapter = await (navigator as unknown as { gpu: GPU }).gpu.requestAdapter();
      if (adapter) return "webgpu";
    } catch (_) {
      // Fallback to WASM SIMD
    }
  }
  return "wasm";
}

export async function getEmbeddingExtractor(): Promise {
  if (!extractorPromise) {
    const device = await detectOptimalDevice();
    extractorPromise = pipeline(
      "feature-extraction",
      "Snowflake/snowflake-arctic-embed-xs",
      {
        device,
        dtype: device === "webgpu" ? "fp16" : "q8",
      }
    );
  }
  return extractorPromise;
}

export async function embedTextToFloat32(text: string): Promise {
  const extractor = await getEmbeddingExtractor();
  const output = await extractor(text, {
    pooling: "cls",
    normalize: true,
  });
  return new Float32Array(output.data as Float32Array);
}
```

## Step 2: Persisting Binary Vector Indexes in Origin Private File System (OPFS)

Storing thousands of JSON-serialized float arrays in `localStorage` or naive IndexedDB object stores creates massive serialization overhead. Instead, pack your normalized `Float32Array` embeddings contiguously into a binary file inside `navigator.storage.getDirectory()` (OPFS) and run a single-pass dot-product scan:

```
// src/lib/vector/opfs-vector-index.ts
export interface VectorMetadata {
  id: string;
  title: string;
  slug: string;
  excerpt: string;
}

export class OpfsFlatVectorIndex {
  private readonly dim: number;
  private vectors: Float32Array = new Float32Array(0);
  private docs: VectorMetadata[] = [];

  constructor(dimensions = 384) {
    this.dim = dimensions;
  }

  async saveToOpfs(filename = "codingcat-vectors-384d.bin"): Promise {
    const root = await navigator.storage.getDirectory();
    const fileHandle = await root.getFileHandle(filename, { create: true });
    const writable = await fileHandle.createWritable();
    await writable.write(this.vectors.buffer);
    await writable.close();

    const metaHandle = await root.getFileHandle(`${filename}.meta.json`, { create: true });
    const metaWritable = await metaHandle.createWritable();
    await metaWritable.write(JSON.stringify(this.docs));
    await metaWritable.close();
  }

  async loadFromOpfs(filename = "codingcat-vectors-384d.bin"): Promise {
    try {
      const root = await navigator.storage.getDirectory();
      const fileHandle = await root.getFileHandle(filename);
      const file = await fileHandle.getFile();
      const arrayBuffer = await file.arrayBuffer();
      this.vectors = new Float32Array(arrayBuffer);

      const metaHandle = await root.getFileHandle(`${filename}.meta.json`);
      const metaFile = await metaHandle.getFile();
      this.docs = JSON.parse(await metaFile.text()) as VectorMetadata[];
      return true;
    } catch {
      return false;
    }
  }

  /**
   * Since embeddings are L2-normalized at extraction time,
   * Cosine Similarity equals the raw dot product.
   */
  searchTopK(queryVec: Float32Array, k = 5): Array {
    const count = this.docs.length;
    const dim = this.dim;
    const scores: Array = [];

    for (let i = 0; i  b.score - a.score);
    return scores.slice(0, k).map(({ idx, score }) => ({
      ...this.docs[idx],
      score,
    }));
  }
}
```

## AJ Pro-Tip: Ship Pre-Computed Binary Embeddings from Your Astro Build

> AJ Pro-Tip: Pre-Compute Corpus Embeddings at Build Time, Embed Only the Query in the Browser. Do not force every visitor's GPU to embed your entire 500-article blog archive on first page load. Generate your `codingcat-vectors-384d.bin` binary artifact once during your Astro static/SSR build (less than 750KB for 500 articles at `Float32` or 375KB at `Float16`), cache it in OPFS, and only run the WebGPU embedding pipeline on the user's 10-word search query.

## **Try the Live Browser AI & WebMCP Playground**

You can test every pattern from this article live right now in our interactive [**Web AI & WebMCP Playground**](/demos/web-ai) on CodingCat.dev, or inspect our production discovery manifests on the [**Developers & Agent APIs**](/developers) page. If you are organizing a developer conference or engineering summit, check out my Web AI talk abstracts on [**alexpatterson.dev**](https://alexpatterson.dev) and explore more hands-on [**Courses**](/courses), [**Podcasts**](/podcasts), and [**Blog Posts**](/blog) right here on CodingCat.dev.

## recent blog posts.

[All Blog Posts→](/blog)[![The Heads of AI wearing black company logo t-shirts and shaking hands in a circular web of cloud and GPU investments](https://cdn.sanity.io/images/hfh83o0w/production/d1d83d596ce5899d797cd3aaf1bd0142038f282a-1376x768.jpg?w=1376&q=82&fit=max&auto=format)](/post/anthropic-s1-ipo-circular-ai-economy-etf-risk)October 26, 2026[![](/logos/amplify/amplify.svg)![](/logos/amplify/amplify.svg)AWS Amplify](/topic/amplify)[AI & MCP Agents](/topic/ai)

### [Anthropic's S-1, $518B in cloud commitments, and circular AI capex](/post/anthropic-s1-ipo-circular-ai-economy-etf-risk)

Anthropic just filed its Form S-1 for a Nasdaq IPO under ticker ANT, revealing $518B in take-or-pay cloud commitments to Google, AWS, and Microsoft. Here is how Mark Zuckerberg, Elon Musk, Jensen Huang, Sundar Pichai, Satya Nadella, Tim Cook, and Dario Amodei built a circular revenue machine, and what happens to your S&P 500 ETF if it stalls.

[![Alex Patterson](https://cdn.sanity.io/images/hfh83o0w/production/a606dfaa381aab08ddbe16a33fe1d3e1fb37c927-400x400.jpg?w=96&h=96&q=82&fit=max&auto=format)Alex Patterson](/author/alex-patterson)[![PHP2Shell (CVE-2026-109618): How AI Found a Decade-Old Bug in 70% of the Web](https://cdn.sanity.io/images/hfh83o0w/production/e83065d0a4e2f841ce64d3f4aea8ddbbf07e5cb9-1376x768.jpg?w=1376&q=82&fit=max&auto=format)](/post/php2shell-cve-2026-109618-ai-php-security-scramble)October 22, 2026[![](/logos/amplify/amplify.svg)![](/logos/amplify/amplify.svg)AWS Amplify](/topic/amplify)[AI & MCP Agents](/topic/ai)

### [PHP2Shell (CVE-2026-109618): How AI Found a Decade-Old Bug in 70% of the Web](/post/php2shell-cve-2026-109618-ai-php-security-scramble)

Two critical pre-authentication RCE flaws in PHP (CVE-2026-109618 and CVE-2026-109619), uncovered by AI models from OpenAI and Google Gemini, sparked a global patching scramble across 70% of the web. Here is how PHP2Shell works and how to patch your servers.

[![Alex Patterson](https://cdn.sanity.io/images/hfh83o0w/production/a606dfaa381aab08ddbe16a33fe1d3e1fb37c927-400x400.jpg?w=96&h=96&q=82&fit=max&auto=format)Alex Patterson](/author/alex-patterson)