Lesson 4: Mapping Spoken Syllables to AJ's 4 Mouth Visemes (`smile-open`, `talking`, `surprised-o`, `grin`)

1. The 15-Second Hook

Welcome to Lesson 4 of faster-whisper on Local GPU: Word-Level Captions & Mascot Lip-Sync on CodingCat.dev. Today we break down Mapping Spoken Syllables to AJ's 4 Mouth Visemes (`smile-open`, `talking`, `surprised-o`, `grin`) and see how it powers our deterministic 3D AJ and code-rendered video pipeline.

2. Architecture Mental Model (Visual Diagram)

Look at the architecture diagram: on the left, vibe-prompted animations suffer from timer drift and unreviewed layout bugs. On the right, 4 Visemes and Lip-Sync enforce a deterministic frame contract.

3. Show Me the Code (tools/video-studio/synthesize-voiceover.py)

from faster_whisper import WhisperModel

whisper = WhisperModel("base.en", device="cuda", compute_type="float16")
segments, info = whisper.transcribe(wav_path, word_timestamps=True)
for seg in segments:
    for w in (seg.words or []):
        print(f"{w.start:.3f}s -> {w.end:.3f}s: {w.word}")

In tools/video-studio/synthesize-voiceover.py, we wire up the production implementation so both our browser runtime and headless renderer stay 100 percent synchronized.

4. Production Trade-Off Matrix

When we compare this deterministic 2026 pattern against legacy workflows in our trade-off matrix, we get exact frame reproducibility, automated critique gates, and zero licensing cost.

5. Verdict & AJ Pro-Tip

AJ Pro-Tip: Always verify frame 300 twice during your contact-sheet critique pass to confirm your 4 Visemes setup has zero non-deterministic state leaks.

Adopt 4 Visemes inside your studio folder today so every future launch film and 3D mascot animation builds on a reusable harness. Hit Subscribe and let's continue!