shifathosn
/
filmcraft
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
filmcraft/crates/speech
History
Manuel Camacho beebdbcd39 M7.12: natural Text to Speech voices (Kokoro-82M in pure Rust) 
- filmcraft-tts (feature kokoro): Kokoro-82M on candle, implemented from
  the StyleTTS 2 / iSTFTNet / ALBERT papers, config.json and the
  checkpoint's tensor shapes; official .pth read with candle's restricted
  pickle reader; voice packs from their zip. Neural Voice: tts-text
  phonemize -> Kokoro per chunk -> pauses -> pitch via audio-dsp.
- catalog: 9 US English voices (model card grade C+ or better) + CMUdict,
  every file pinned to a revision and SHA-256 (336 MB).
- speech: generic download_files (same verified downloader).
- engine: tts.render (background synthesis into Session::tts_cache),
  tts.downloadVoices (background job, cancellable), voices list with
  package state; feature neural-voices (default on in the desktop app).
- ui: natural/basic voice groups, consent box (size, sources, licences),
  download progress + Cancel, synthesis never blocks the window;
  ui.set {tts:{...}} for agents.

Measured: Whisper-base WER on 50 original sentences 2.1-3.6 % for all 9
voices (built-in robotic voice 96.6 %); 2.2-2.3x real time on CPU.
Gates: fmt, clippy, layers, assets, wasm pass; tests pass except
shortcuts_tests::assign_reassigns_conflicts_and_undoes (fails on main)
and the frame pool test (flaky under load, passes alone).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 		2026-10-08 06:52:30 -07:00
..
examples Clippy 1.99: fix new lints so `cargo xtask ci` is green again 2026-10-04 15:55:23 -04:00
src M7.12: natural Text to Speech voices (Kokoro-82M in pure Rust) 2026-10-08 06:52:30 -07:00
tests Clippy 1.99: fix new lints so `cargo xtask ci` is green again 2026-10-04 15:55:23 -04:00
Cargo.toml M10.5: Text-based editing / transcripts 2026-10-01 23:25:49 -04:00
README.md M10.5: Text-based editing / transcripts 2026-10-01 23:25:49 -04:00

README.md

filmcraft-speech

Speech-to-text for FilmCraft's Text panel ▸ Transcript (L2). User-facing behaviour, commands and model handling: docs/transcripts.md.

  • Transcriber trait (mono 16 kHz in, word-timed filmcraft_project::Transcript out) and FixedTranscriber for tests and agents.
  • models: Whisper model catalogue (pinned revisions, SHA-256, sizes, licences) and, with feature download, the verified downloader (ureq + rustls + RustCrypto). Weights are never bundled.
  • whisper (feature whisper): Whisper inference on candle (CPU, pure Rust).
  • mel, vad, diarize: log-mel front end, word-bound tightening, MFCC-clustering speaker labels.

Both features are off by default and are never enabled for wasm (cargo xtask wasm checks the crate without them).

References

Implemented from the published description of the model: A. Radford et al., "Robust Speech Recognition via Large-Scale Weak Supervision" (OpenAI, 2022), the model cards and configuration files published with the weights (config.json, generation_config.json, tokenizer.json), and the candle tensor library's public API. Weights: OpenAI Whisper, MIT licence.