shifathosn
/
filmcraft
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
filmcraft/crates/speech
History
Brandon Thomas fc18a6962e M10.5: Text-based editing / transcripts 
- crates/speech (L2): Transcriber trait, FixedTranscriber, Whisper model catalogue (pinned,
  SHA-256 verified) with optional downloader (feature `download`: ureq + rustls + RustCrypto),
  pure-Rust Whisper inference on candle with word timestamps (feature `whisper`), log-mel, VAD
  word tightening, MFCC speaker clustering. Both features off by default and for wasm.
- project: per-media-item transcripts (Project::transcripts); schema v9 with a no-op v8->v9
  migration and a v8 fixture.
- edit::transcript: sequence transcript derived from clip transcripts, search, paragraphs,
  word ranges, pause/filler detection, ripple delete of many ranges, caption layout.
- engine: transcript.* commands (generate, set, delete, inspect, search, select, extract, lift,
  renameSpeaker, removeFillers, removePauses, createCaptions, models, downloadModel);
  Session::transcriber for hosts/tests; features `whisper` / `speech-download` (also on the
  desktop and CLI apps). Without whisper, transcript.generate fails with a clear error.
- ui: Text panel Transcript tab (Transcribe button, speaker paragraphs of clickable words,
  Shift+click selection -> In/Out, extract/lift/remove fillers/create captions, search) with
  text.transcript.* automation ids; headless UI test.
- docs/transcripts.md, crates/speech/README.md, architecture and project-files updates.
  whisper-tiny measures 12.2% WER over 797 words on local samples.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 		2026-10-01 23:25:49 -04:00
..
examples M10.5: Text-based editing / transcripts 2026-10-01 23:25:49 -04:00
src M10.5: Text-based editing / transcripts 2026-10-01 23:25:49 -04:00
tests M10.5: Text-based editing / transcripts 2026-10-01 23:25:49 -04:00
Cargo.toml M10.5: Text-based editing / transcripts 2026-10-01 23:25:49 -04:00
README.md M10.5: Text-based editing / transcripts 2026-10-01 23:25:49 -04:00

README.md

filmcraft-speech

Speech-to-text for FilmCraft's Text panel ▸ Transcript (L2). User-facing behaviour, commands and model handling: docs/transcripts.md.

  • Transcriber trait (mono 16 kHz in, word-timed filmcraft_project::Transcript out) and FixedTranscriber for tests and agents.
  • models: Whisper model catalogue (pinned revisions, SHA-256, sizes, licences) and, with feature download, the verified downloader (ureq + rustls + RustCrypto). Weights are never bundled.
  • whisper (feature whisper): Whisper inference on candle (CPU, pure Rust).
  • mel, vad, diarize: log-mel front end, word-bound tightening, MFCC-clustering speaker labels.

Both features are off by default and are never enabled for wasm (cargo xtask wasm checks the crate without them).

References

Implemented from the published description of the model: A. Radford et al., "Robust Speech Recognition via Large-Scale Weak Supervision" (OpenAI, 2022), the model cards and configuration files published with the weights (config.json, generation_config.json, tokenizer.json), and the candle tensor library's public API. Weights: OpenAI Whisper, MIT licence.