shifathosn
/
filmcraft
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Windows: NVENC H.265 (HEVC) Main hardware encoding for Export 

One NVENC session, ring and NV12 path now serve H.264 and HEVC (Codec enum,
Profile::HevcMain). HEVC Main 8-bit 4:2:0 SDR BT.709, MP4/MOV hvc1, parameter
sets only in an hvcC built from the SPS the encoder wrote. The H.265 format is
available through a cached NVENC probe; choosing it is the opt-in, declines are
counted errors. ABI asserts regenerated from nvEncodeAPI.h 12.1 (no drift) and
extended for HEVC; tests, docs and roadmap updated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Davi Andrei 2026-10-08 16:28:50 -03:00
parent 5231852443
commit deebe2e5b2
19 changed files with 1498 additions and 97 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
ATTRIBUTION.md
View File

@ -146,7 +146,7 @@ the original copyright and permission notice in its header.
| File | Author | Source | Licence |
|---|---|---|---|
| `crates/platform/src/nvenc/ffi.rs` | NVIDIA Corporation (`nvEncodeAPI.h`); FilmCraft contributors (Rust transcription) | Structures, function table and constants transcribed from `nvEncodeAPI.h`, NVIDIA Video Codec SDK, API 12.1 (https://developer.nvidia.com/video-codec-sdk) | MIT (Copyright (c) 2010-2023 NVIDIA Corporation; notice kept in the file header) |
| `crates/platform/src/nvenc/ffi.rs` | NVIDIA Corporation (`nvEncodeAPI.h`); FilmCraft contributors (Rust transcription) | Structures (including `NV_ENC_CONFIG_HEVC` and its bit-field masks), function table, codec / profile GUIDs (H.264, HEVC Main) and constants (`NV_ENC_LEVEL_HEVC_*`, tier, CU size) transcribed from `nvEncodeAPI.h`, NVIDIA Video Codec SDK, API 12.1 (https://developer.nvidia.com/video-codec-sdk) | MIT (Copyright (c) 2010-2023 NVIDIA Corporation; notice kept in the file header) |
## Downloaded at runtime

2
Cargo.lock generated
View File

@ -2097,10 +2097,12 @@ dependencies = [
name = "filmcraft-platform"
version = "0.4.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-codecs",
"filmcraft-color",
"filmcraft-export",
"filmcraft-frame",
"filmcraft-hevc",
"filmcraft-isobmff",
"filmcraft-media",
"filmcraft-project",

6
README.md
View File

@ -185,7 +185,7 @@ No FFmpeg inside. The video codecs, AAC, Opus and the containers are our own Rus
| **Matroska / WebM** | ✓ | | Lacing, Cues, header stripping, HDR colour metadata |
| **Stills** | ✓ | ✓ | Import PNG, JPEG, GIF, WebP, TIFF and BMP; export PNG sequences and animated GIF |
Also imported: MPEG-2 / MPEG-1 video, AC-3, MP2, MXF (OP1a / OP-Atom), MPEG transport and program streams (AVCHD, broadcast, DVD), Ogg and image sequences. Not yet: HEVC and AV1 export, camera RAW, E-AC-3. Hardware decoding works on macOS (VideoToolbox) and Windows (Media Foundation), with Linux to follow; H.264 export can use NVIDIA's encoder on Windows (opt-in) ([#30](https://github.com/storytold/filmcraft/issues/30)).
Also imported: MPEG-2 / MPEG-1 video, AC-3, MP2, MXF (OP1a / OP-Atom), MPEG transport and program streams (AVCHD, broadcast, DVD), Ogg and image sequences. Not yet: AV1 export, camera RAW, E-AC-3. Hardware decoding works on macOS (VideoToolbox) and Windows (Media Foundation), with Linux to follow; H.264 export can use NVIDIA's encoder on Windows (opt-in), and H.265 (HEVC Main, 8-bit) export works through it (NVENC on Windows, VideoToolbox on macOS) ([#30](https://github.com/storytold/filmcraft/issues/30)).
<br>
@ -287,9 +287,9 @@ We track two numbers ([ROADMAP.md](ROADMAP.md#honest-assessment-2026-10-05)):
The biggest gaps today:
- **Speed on big footage.** Hardware decoding works on macOS and Windows; Linux has no hardware path yet. Blend modes and the most common effects run on the GPU, but Lumetri, keys and export rendering still run on the CPU; H.264 encoding can use the hardware encoder on macOS and on Windows with an NVIDIA GPU (opt-in, Export ▸ Hardware encoding), and H.265 export is hardware-only there ([#30](https://github.com/storytold/filmcraft/issues/30)).
- **Speed on big footage.** Hardware decoding works on macOS and Windows; Linux has no hardware path yet. Blend modes and the most common effects run on the GPU, but Lumetri, keys and export rendering still run on the CPU; H.264 encoding can use the hardware encoder on macOS and on Windows with an NVIDIA GPU (opt-in, Export ▸ Hardware encoding), and H.265 export is hardware-only on both (NVENC; Main 10 and HDR not yet) ([#30](https://github.com/storytold/filmcraft/issues/30)).
- **No plugins.** No VST3 / Audio Units or OpenFX hosting.
- **Delivery codecs.** H.264 is our only delivery-codec export; no HEVC or AV1 export yet.
- **Delivery codecs.** H.264 is our only software delivery-codec export; H.265 (8-bit Main, SDR) exports only through a hardware encoder (NVIDIA on Windows, VideoToolbox on macOS); no AV1 export yet.
- **Real-world media and platforms.** Our decoders are bit-exact on conformance streams, but camera and phone files in the wild are less tested. Windows and Linux get far less testing than macOS.
- **AI features.** Few so far; speech to text is optional and off by default.

10
ROADMAP.md
View File

@ -35,8 +35,8 @@ before choosing work.
| **Feature breadth** (menus, commands, panels) | ~86–95% | **~85%** | Menu diff against Premiere 26.5; effect, transition and audio-effect catalogues complete by test. The long tail is mostly there. |
| **Correctness in depth** | not tracked | **~60–70%** | A first-time contributor found four real bugs in core paths within hours (#5–#8: AIFF import failed, Slide left linked audio behind, transitions didn't follow ripple trims, export ignored start/end times). User reports: Color Matte always grey with no picker (#29); ALSA audio error on Linux (#23). Settings are "most wired", not all. |
| **Codecs and media** | ~90% | **~75%** | Our decoders are bit-exact on conformance streams. Real camera and phone media (variable frame rate, damaged files, unusual containers) is far less tested. No camera RAW (RED, BRAW, ARRIRAW), no E-AC-3. |
| **Export** | ~92% | **~70%** | Delivery: H.264 only (ProRes, DNxHR and image sequences cover mastering). No HEVC or AV1 export. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. |
| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software) and Windows (Media Foundation / Direct3D 11 DXVA, H.264, HEVC Main / Main 10, VP9 and AV1 4:2:0 8/10-bit, #30). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects and export rendering are CPU; H.264 encoding can use VideoToolbox on macOS (opt-in per export, no B-frames) and NVENC on Windows with an NVIDIA GPU (opt-in per export), and H.265 (HEVC) export exists there only, through the same hardware encoder. Linux has no hardware path. 8K isn't real-time (AV1 is on Windows with a hardware decoder). The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
| **Export** | ~92% | **~70%** | Delivery: H.264, plus H.265 (8-bit Main, SDR) through a hardware encoder only (NVENC on Windows, VideoToolbox on macOS); ProRes, DNxHR and image sequences cover mastering. No AV1 export, no HDR or 10-bit HEVC. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. |
| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software) and Windows (Media Foundation / Direct3D 11 DXVA, H.264, HEVC Main / Main 10, VP9 and AV1 4:2:0 8/10-bit, #30). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects and export rendering are CPU; H.264 encoding can use VideoToolbox on macOS (opt-in per export, no B-frames) and NVENC on Windows with an NVIDIA GPU (opt-in per export), and H.265 (HEVC) export exists there only, through the same hardware encoders (macOS and Windows). Linux has no hardware path. 8K isn't real-time (AV1 is on Windows with a hardware decoder). The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
| **Stability** | not tracked | **improving, unproven** | The never-crash pass (no panics in product code, last-resort guards) landed on 2026-10-04. There's no field record from real users' projects yet. |
| **Plugins and pro ecosystem** | not tracked | **~0–10%** | No audio plugin hosting (VST3 / Audio Units) and no third-party video effects (OpenFX). For many professional editors this alone rules FilmCraft out. Team Projects and Productions are out of scope by design. |
| **AI features** | partial | **~25–35%** | Speech to text exists but is off in default builds (the `whisper` feature). Enhance Speech is a DSP chain, not a model. Auto Reframe is approximate. No Generative Extend, and no media-intelligence search or auto colour. |
@ -51,7 +51,7 @@ In priority order. Agents should prefer this work over adding more checklist ite
- hardware decode on Linux (VA-API);
- zero-copy decoded frames into wgpu;
- the remaining effects on the GPU: Lumetri, keys, Vignette, Video Limiter, masks (31 common effects are done);
- GPU export, then hardware encode: H.264 and H.265 on macOS are in (H.264 opt-in; H.265 has no other encoder), H.264 on NVENC (Windows, opt-in); HEVC on Windows and Linux remain.
- GPU export, then hardware encode: H.264 and H.265 on macOS are in (H.264 opt-in; H.265 has no other encoder), H.264 and H.265 (Main, 8-bit) on NVENC (Windows); Main 10 / HDR HEVC, Linux and other Windows vendors remain.
2. **A measured parity number.** Add `cargo xtask parity`, run in CI. It should cover menus,
commands, effects, transitions, panels, preferences and formats against the Premiere snapshot in
`plan/premiere/`. It should report presence and fidelity separately (exact, approximate, stub).
@ -154,11 +154,13 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
## Running now
- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 31 common effects on the GPU (GPU2). Windows: Media Foundation / Direct3D 11 H.264 + HEVC decode (HW2) and VP9 + AV1 (HW3), NVENC H.264 export (HW4, opt-in). Next: VA-API (Linux) decode, zero-copy upload, Lumetri on the GPU.
- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 31 common effects on the GPU (GPU2). Windows: Media Foundation / Direct3D 11 H.264 + HEVC decode (HW2) and VP9 + AV1 (HW3), NVENC H.264 export (HW4, opt-in) and H.265 export. Next: VA-API (Linux) decode, zero-copy upload, Lumetri on the GPU.
- **Next:** [Where we are lacking](#where-we-are-lacking) items 1–5. GPU export waits for the export-crate work (#19 and the never-crash follow-up) to land.
## Log
- **2026-10-08:** H.265 (HEVC Main, 8-bit 4:2:0, SDR) export on Windows through NVENC (#30): the NVENC session, ring and NV12 path of the H.264 encoder now serve both codecs (the H.264 export is byte-for-byte unchanged); the format is available where the GPU has an HEVC encoder, choosing it is the opt-in, and `hvcC` is built from the SPS the encoder wrote. ffprobe reads `hevc` / `Main` / `hvc1` / `yuv420p` / BT.709; our own HEVC decoder decodes it at 48.9 dB luma PSNR (RTX 5060). Main 10 and HDR remain.
- **2026-10-06:** 21 community PRs landed (#19, #28, #38, #39, #41–#60). Highlights: hardware decoding is now actually on in the desktop app (#41: `register()` sat inside a `log::info!` that never ran, so #33's VideoToolbox path only worked in the CLI and benchmarks); decode memory on long timelines cut sharply (frame pool, shared GOP-cache budget, positional file reads); exports are byte-identical across machines (#19); crash fix for odd track names over MCP (#44); ripple trim, ripple delete and speed changes move split-edit clips once (#56–#58); Japanese interface using an installed system font (#38; no fonts bundled); MCP annotations, resources, export progress and cancellation (#28).
- **2026-10-05:** honest assessment added (checklist ~87% vs ready for real work ~50–60%). Hardware acceleration started (#30): blend modes on the GPU (#32); VideoToolbox decode + `platform` crate, the one crate allowed `unsafe` (#33). Community fixes landed: AIFF import, export start/end ranges, transitions following ripple trims, Slide moving linked audio (#5–#8).

2
crates/platform/Cargo.toml
View File

@ -21,10 +21,12 @@ type_complexity = "allow"
undocumented_unsafe_blocks = "deny"
[dependencies]
filmcraft-bitstream = { workspace = true }
filmcraft-codecs = { workspace = true }
filmcraft-color = { workspace = true }
filmcraft-export = { workspace = true }
filmcraft-frame = { workspace = true }
filmcraft-hevc = { workspace = true }
filmcraft-isobmff = { workspace = true }
filmcraft-time = { workspace = true }
log = { workspace = true }

64
crates/platform/README.md
View File

@ -61,7 +61,7 @@ let availability = filmcraft_platform::register(); // Available("VideoToolbox")
- After a `flush` (which drains the MFT) the MFT only restarts at an IDR picture; the GOP cache
always seeks after a flush, and a caller that continues from the middle of a GOP gets an error
that `HybridDecoder` answers by replaying the run in software.
- **Windows: NVIDIA NVENC H.264 encoding** (`nvenc/`), 8-bit SDR 4:2:0 for Export. The driver's
- **Windows: NVIDIA NVENC H.264 and H.265 (HEVC) encoding** (`nvenc/`), 8-bit SDR 4:2:0 for Export. The driver's
`nvEncodeAPI64.dll` (API 12.1, no CUDA or SDK) is loaded at run time, so machines without NVIDIA
still start. RGBA is converted with the software encoder's own BT.709 limited conversion into NV12
input buffers (a ring of eight); the encoder runs preset P5 with high-quality tuning, CABAC (CAVLC
@ -69,7 +69,9 @@ let availability = filmcraft_platform::register(); // Available("VideoToolbox")
distance; the parameter sets go into `avcC`. Export ▸ Hardware encoding (off by default) selects it.
It declines two-pass VBR, HDR, MXF, interlaced output, sizes outside NVENC's limits and systems
without an NVIDIA GPU or driver, and the software encoder runs instead. A failure during an export
ends it with an error, since a hardware stream cannot be finished in software.
ends it with an error, since a hardware stream cannot be finished in software. The same session,
ring and NV12 path encode H.265, see [Hardware H.265 (HEVC) encoding (Windows,
NVENC)](#hardware-h265-hevc-encoding-windows-nvenc).
- **Other systems:** `register()` does nothing and returns `Availability::Unavailable`.
- **`HybridDecoder`** (`hybrid.rs`, safe code): the hardware decoder plus the means to build our
software decoder for the same `SampleEntry` (`filmcraft_codecs::software_video_decoder`). On a
@ -135,6 +137,51 @@ The sample entry's `hvcC` record is the one VideoToolbox wrote for the stream (r
format description's sample description extension atoms, with the VPS / SPS / PPS), so profile,
level and flags are the encoder's own. If it is missing the export stops with that reason.
## Hardware H.265 (HEVC) encoding (Windows, NVENC)
One codec enum (`nvenc::Codec`, chosen by `Profile::HevcMain` the way `VtProfile::HevcMain` does it
for VideoToolbox) parameterises the NVENC session: the codec and profile GUIDs, the capability
query, the `hevcConfig` member of the codec-configuration union, how NAL units are told apart
(`(b0 >> 1) & 0x3f`) and the level numbering (`level × 30`, so 4.1 is 123; the export leaves the
level to the encoder). The H.264 path is unchanged: the same export is the same file, byte for byte.
- **Choosing the format is the opt-in,** as on macOS: Export ▸ Format ▸ H.265 (HEVC) or
`"format": "hevc"`. `register()` hands `nvenc::hevc_available` to
`filmcraft_export::register_format_probe`: one small HEVC session (a 640×360 encoder, created on
the first question and the answer kept; about half a second in a fresh process, nearly all of it
opening the Direct3D 11 device and the NVENC session, which every NVENC export pays too). The
Hardware encoding toggle governs H.264 only: with H.265 selected NVENC is used whether it is Auto
or Off, because nothing else can encode it.
- **What it writes:** HEVC Main, 8-bit 4:2:0, SDR BT.709 limited range (the VUI says so, with the
frame rate as `time_scale / num_units_in_tick`), progressive, preset P5 with high-quality tuning,
constant or one-pass variable bitrate, an IDR at every keyframe distance, one B-frame when the GPU
reports support and the GOP is longer than two pictures (shorter ones are written without), in
MP4 (`hvc1`) or QuickTime. Pictures are converted exactly like the H.264 path's. The coded size is
a whole number of 32-pixel coding tree blocks and the SPS carries a conformance window (1080 is
coded as 1088 and cropped back); the `hvcC` is only accepted when the cropped size is the export's.
- **Samples are length-prefixed (4 bytes) with the parameter sets only in the `hvcC`:** VPS, SPS,
PPS, access unit delimiters and end-of-sequence / bitstream markers are dropped; slices and SEI
stay. `nvEncGetSequenceParams` returns all three parameter sets.
- **The `hvcC` is built from the SPS the encoder wrote** (`nvenc/hevc.rs`, parsed with
`filmcraft_hevc`): profile space, tier, profile, compatibility flags and level from the SPS's
profile / tier / level; the 48 general constraint flags and `sps_temporal_id_nesting_flag` read from
the SPS bits that carry them; chroma format, bit depths and temporal layers from the SPS; the
arrays complete (`array_completeness` 1). Nothing is a constant of ours except what the format
leaves unspecified (`avg_frame_rate`, `constant_frame_rate`, `min_spatial_segmentation_idc` and
`parallelism_type` are 0). It is built, and validated as Main 8-bit 4:2:0 of the right size, when
the encoder is created, so a stream we cannot describe is a declined export and never a default
sample entry.
- **Timestamps are the H.264 path's:** dts is the decode index minus the B-frame delay, composition
offsets are `pts - dts` in frame durations, and the media starts `delay` frames early (edit list).
- **There is no fallback encoder.** A request NVENC cannot take is counted in
`export.hardware.declined` and ends the export with "H.265 export with NVENC: …" and the reason:
HDR, interlaced output, two-pass (`ExportSettings::validate` refuses it first), non-square pixels,
MXF, sizes above 65535, odd sizes, sizes outside the GPU's limits (129×33 to 8192×8192 on the RTX
5060). On a machine without an HEVC encoder the format is not available and an export fails with
"encoder not available yet". Frames, sessions and declines are the same `export.hardware` counters
as for H.264.
- Main 10 / HDR is not done (see Not yet).
## Guarantees
- **Never undecodable:** the factory declines (returns `None`, so the software decoder is used)
@ -161,7 +208,11 @@ level and flags are the encoder's own. If it is missing the export stops with th
| `tests/hardware_encode.rs` (macOS) | what the hardware path takes and declines; round trip through our software decoder (every picture, in order, luma PSNR above 30 dB, keyframes no further apart than asked, no composition offsets); an export through `filmcraft_export` that decodes in our decoder and in ffmpeg / ffprobe (profile, size, frame count, BT.709); the built-in encoder still exporting everything hardware declines; exact output size at sizes that are not multiples of 16; hostile configurations (zero, huge, odd sizes, frame rates, bitrates, keyframe intervals, wrong planes) give errors and never panic; encoders dropped at any point do not crash or hang. The same for **HEVC**: Main profile, 8-bit 4:2:0, `hvc1` entry with VPS / SPS / PPS and 4-byte lengths, MP4 and QuickTime, AAC audio, two-pass refused, the format list agreeing with the probe, ffprobe reading `codec_name=hevc`, `profile=Main`, `codec_tag_string=hvc1`, `pix_fmt=yuv420p`, BT.709 |
| `tests/nvenc.rs` (Windows, NVIDIA) | H.264 from NVENC (1280×720, 6 Mbps, 72 frames) decodes with our decoder at worst 46.9 dB luma PSNR; IDR at 0, 24 and 48; dts / pts right |
| `tests/nvenc_export.rs` (Windows, NVIDIA) | Export with hardware encoding against the software encoder through the export pipeline: the two decoded files at worst 54.8 dB luma PSNR; ffmpeg decodes the file without errors; declined cases go to the software encoder; the counters |
| `src/nvenc/abi_tests.rs` (Windows) | FFI structs' sizes, alignments, field offsets, constants and GUIDs against a C compiler's view of NVIDIA's `nvEncodeAPI.h` (12.1) |
| `tests/nvenc_hevc.rs` (Windows, NVIDIA with HEVC) | HEVC from NVENC (1280×720, 6 Mbps, 72 frames, keyframe every 24) decodes with our HEVC decoder at worst 48.9 dB luma / 51.0 dB chroma PSNR; IDR at 0, 24, 48; dts strictly increasing, pts a permutation, `sps_max_num_reorder_pics` within the dts shift; no parameter sets in the samples; the `hvcC`'s profile / tier / level bytes are the SPS's own; VUI BT.709 limited and timing `(1, 24)`; hostile configurations, sizes, planes and encoders dropped mid-stream give errors, never panics; keyframes every 1–2 pictures without B-frames |
| `tests/nvenc_hevc_export.rs` (Windows, NVIDIA with HEVC) | H.265 export through the real pipeline, MP4 and QuickTime, toggle Auto and Off: counters (72 frames, 1 session, 0 declined), our decoder against the software H.264 export at worst 54.8 dB luma PSNR, ffprobe `codec_name=hevc`, `profile=Main`, `codec_tag_string=hvc1`, `pix_fmt=yuv420p`, BT.709 limited, 72 frames, 24/1, keyframes at 0 and 48, start 0, `ffmpeg -xerror` clean; with AAC audio; 1920×1080 and 642×362 cropped back from the coded size; every decline (HDR, analysis pass, two-pass, interlaced, non-square pixels, MXF, sizes over 65535, odd sizes, sizes outside the GPU's limits) an error naming NVENC, counted once; H.264 with hardware encoding Off never touches NVENC |
| `tests/nvenc_hevc_probe.rs` (Windows) | the HEVC probe in a fresh process: its cost, the cached answer, `available(Hevc)` following it after `register()` (twice) |
| `src/nvenc/abi_tests.rs` (Windows) | FFI structs' sizes, alignments, field offsets, constants, GUIDs and the bit-field masks of the `flags` words (H.264 and HEVC) against a C compiler's view of NVIDIA's `nvEncodeAPI.h` (12.1) |
| `src/nvenc/{mod,hevc}.rs` unit tests (Windows) | HEVC NAL types, stripping and parameter-set splitting (including `IDR_N_LP`, whose header byte reads as a PPS in H.264, and one-byte or truncated NAL units); the `hvcC` built from real VPS / SPS / PPS, every truncation and bit flip of the SPS, wrong profile, size and NAL types; HEVC levels |
Fixtures are made with ffmpeg into `target/fixtures/platform/` (generator only, never linked);
tests skip without ffmpeg or without a hardware decoder.
@ -188,6 +239,7 @@ playback with no dropped frames at Full, 1/2 and 1/4. Details in
## Not yet
Zero-copy upload of decoded pictures into wgpu textures (`CVPixelBuffer`s on macOS, Direct3D 11
textures on Windows); B-frames and 10-bit / HDR HEVC (Main 10) in hardware encoding; VA-API (Linux)
decoders; field-coded H.264; HEVC, 10-bit and HDR encoding with NVENC, and encoders from other
vendors on Windows (through Media Foundation); VP9 / AV1 4:4:4 and 12-bit on Windows.
textures on Windows); B-frames in the VideoToolbox encoders and 10-bit / HDR HEVC (Main 10) in
hardware encoding; VA-API (Linux) decoders; field-coded H.264; 10-bit and HDR encoding with NVENC
(Main 10 HEVC, 4:4:4), AV1 with NVENC, H.265 with the other Windows vendors and on Linux, and encoders
from other vendors on Windows (through Media Foundation); VP9 / AV1 4:4:4 and 12-bit on Windows.

9
crates/platform/src/lib.rs
View File

@ -6,7 +6,8 @@
//! (`avcC`) and HEVC (`hvcC`) streams, 8- and 10-bit, 4:2:0 and 4:2:2; on other systems
//! registration does nothing and reports [`Availability::Unavailable`]. It also registers a
//! hardware H.264 encoder factory (`filmcraft_export::register_encoder`) that only acts when an
//! export asks for it (`ExportSettings::hardware_encoding` = `Auto`), see [`hardware_encode`].
//! export asks for it (`ExportSettings::hardware_encoding` = `Auto`), see [`hardware_encode`]; on
//! macOS and on Windows (NVENC, [`nvenc`]) it also makes the H.265 export format available.
//!
//! Hardware decoding never makes a file undecodable:
//!
@ -67,6 +68,12 @@ pub fn register() -> Availability {
filmcraft_codecs::hw::set_hw_backend("VideoToolbox");
Availability::Available("VideoToolbox")
}
// Export ▸ H.265: NVENC is the only HEVC encoder on Windows, so the format is available when a
// small HEVC session opens (asked once, when the format list first asks)
#[cfg(target_os = "windows")]
{
filmcraft_export::register_format_probe(filmcraft_export::Format::Hevc, nvenc::hevc_available);
}
#[cfg(target_os = "windows")]
{
static ENCODERS: std::sync::Once = std::sync::Once::new();

66
crates/platform/src/nvenc/abi_tests.rs
View File

@ -2,7 +2,9 @@
//! alignments, field offsets, constants and GUIDs. Generated by running a C program (built with
//! MSVC from the NVIDIA header) that prints them; the header itself is not in this repository.
//! To regenerate, put NVIDIA's MIT-licensed `nvEncodeAPI.h` 12.1 in `ffnvcodec/` next to a program
//! printing `sizeof` / `offsetof` of the structures of `src/nvenc/ffi.rs` and the constants it uses.
//! printing `sizeof` / `alignof` / `offsetof` of the structures of `src/nvenc/ffi.rs`, the constants
//! and GUIDs it uses and, for the bit-fields of the `flags` words (`offsetof` cannot take those), the
//! word with that one field set (see docs/testing.md). H.264 and HEVC are both covered.
#![allow(clippy::unreadable_literal)]
use super::ffi::*;
@ -111,9 +113,38 @@ fn layouts_match_the_header() {
assert_eq!(offset_of!(NV_ENC_CONFIG_H264, numRefL1), 208, "NV_ENC_CONFIG_H264.numRefL1");
assert_eq!(offset_of!(NV_ENC_CONFIG_H264, reserved1), 212, "NV_ENC_CONFIG_H264.reserved1");
assert_eq!(offset_of!(NV_ENC_CONFIG_H264, reserved2), 1280, "NV_ENC_CONFIG_H264.reserved2");
assert_eq!(
(size_of::<NV_ENC_CONFIG_HEVC_VUI_PARAMETERS>(), align_of::<NV_ENC_CONFIG_HEVC_VUI_PARAMETERS>()),
(112, 4),
"NV_ENC_CONFIG_HEVC_VUI_PARAMETERS"
);
assert_eq!((size_of::<NV_ENC_CONFIG_HEVC>(), align_of::<NV_ENC_CONFIG_HEVC>()), (1560, 8), "NV_ENC_CONFIG_HEVC");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, level), 0, "NV_ENC_CONFIG_HEVC.level");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, tier), 4, "NV_ENC_CONFIG_HEVC.tier");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, minCUSize), 8, "NV_ENC_CONFIG_HEVC.minCUSize");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, maxCUSize), 12, "NV_ENC_CONFIG_HEVC.maxCUSize");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, idrPeriod), 20, "NV_ENC_CONFIG_HEVC.idrPeriod");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, intraRefreshPeriod), 24, "NV_ENC_CONFIG_HEVC.intraRefreshPeriod");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, intraRefreshCnt), 28, "NV_ENC_CONFIG_HEVC.intraRefreshCnt");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, maxNumRefFramesInDPB), 32, "NV_ENC_CONFIG_HEVC.maxNumRefFramesInDPB");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, ltrNumFrames), 36, "NV_ENC_CONFIG_HEVC.ltrNumFrames");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, vpsId), 40, "NV_ENC_CONFIG_HEVC.vpsId");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, spsId), 44, "NV_ENC_CONFIG_HEVC.spsId");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, ppsId), 48, "NV_ENC_CONFIG_HEVC.ppsId");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, sliceMode), 52, "NV_ENC_CONFIG_HEVC.sliceMode");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, sliceModeData), 56, "NV_ENC_CONFIG_HEVC.sliceModeData");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, maxTemporalLayersMinus1), 60, "NV_ENC_CONFIG_HEVC.maxTemporalLayersMinus1");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, hevcVUIParameters), 64, "NV_ENC_CONFIG_HEVC.hevcVUIParameters");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, ltrTrustMode), 176, "NV_ENC_CONFIG_HEVC.ltrTrustMode");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, useBFramesAsRef), 180, "NV_ENC_CONFIG_HEVC.useBFramesAsRef");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, numRefL0), 184, "NV_ENC_CONFIG_HEVC.numRefL0");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, numRefL1), 188, "NV_ENC_CONFIG_HEVC.numRefL1");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, reserved1), 192, "NV_ENC_CONFIG_HEVC.reserved1");
assert_eq!(offset_of!(NV_ENC_CONFIG_HEVC, reserved2), 1048, "NV_ENC_CONFIG_HEVC.reserved2");
assert_eq!((size_of::<NV_ENC_CODEC_CONFIG>(), align_of::<NV_ENC_CODEC_CONFIG>()), (1792, 8), "NV_ENC_CODEC_CONFIG");
assert_eq!(offset_of!(NV_ENC_CODEC_CONFIG, h264Config), 0, "NV_ENC_CODEC_CONFIG.h264Config");
assert_eq!(offset_of!(NV_ENC_CODEC_CONFIG, reserved), 0, "NV_ENC_CODEC_CONFIG.reserved");
assert_eq!(offset_of!(NV_ENC_CODEC_CONFIG, hevcConfig), 0, "NV_ENC_CODEC_CONFIG.hevcConfig");
assert_eq!((size_of::<NV_ENC_CONFIG>(), align_of::<NV_ENC_CONFIG>()), (3584, 8), "NV_ENC_CONFIG");
assert_eq!(offset_of!(NV_ENC_CONFIG, version), 0, "NV_ENC_CONFIG.version");
assert_eq!(offset_of!(NV_ENC_CONFIG, profileGUID), 4, "NV_ENC_CONFIG.profileGUID");
@ -353,6 +384,21 @@ fn constants_match_the_header() {
assert_eq!(NV_ENC_LOCK_BITSTREAM_VER as i64, 4043374604, "NV_ENC_LOCK_BITSTREAM_VER");
assert_eq!(NV_ENC_LOCK_INPUT_BUFFER_VER as i64, 1895890956, "NV_ENC_LOCK_INPUT_BUFFER_VER");
assert_eq!(NV_ENC_SEQUENCE_PARAM_PAYLOAD_VER as i64, 1895890956, "NV_ENC_SEQUENCE_PARAM_PAYLOAD_VER");
assert_eq!(NV_ENC_TIER_HEVC_MAIN as i64, 0, "NV_ENC_TIER_HEVC_MAIN");
assert_eq!(NV_ENC_HEVC_CUSIZE_AUTOSELECT as i64, 0, "NV_ENC_HEVC_CUSIZE_AUTOSELECT");
assert_eq!(NV_ENC_LEVEL_HEVC_1 as i64, 30, "NV_ENC_LEVEL_HEVC_1");
assert_eq!(NV_ENC_LEVEL_HEVC_2 as i64, 60, "NV_ENC_LEVEL_HEVC_2");
assert_eq!(NV_ENC_LEVEL_HEVC_21 as i64, 63, "NV_ENC_LEVEL_HEVC_21");
assert_eq!(NV_ENC_LEVEL_HEVC_3 as i64, 90, "NV_ENC_LEVEL_HEVC_3");
assert_eq!(NV_ENC_LEVEL_HEVC_31 as i64, 93, "NV_ENC_LEVEL_HEVC_31");
assert_eq!(NV_ENC_LEVEL_HEVC_4 as i64, 120, "NV_ENC_LEVEL_HEVC_4");
assert_eq!(NV_ENC_LEVEL_HEVC_41 as i64, 123, "NV_ENC_LEVEL_HEVC_41");
assert_eq!(NV_ENC_LEVEL_HEVC_5 as i64, 150, "NV_ENC_LEVEL_HEVC_5");
assert_eq!(NV_ENC_LEVEL_HEVC_51 as i64, 153, "NV_ENC_LEVEL_HEVC_51");
assert_eq!(NV_ENC_LEVEL_HEVC_52 as i64, 156, "NV_ENC_LEVEL_HEVC_52");
assert_eq!(NV_ENC_LEVEL_HEVC_6 as i64, 180, "NV_ENC_LEVEL_HEVC_6");
assert_eq!(NV_ENC_LEVEL_HEVC_61 as i64, 183, "NV_ENC_LEVEL_HEVC_61");
assert_eq!(NV_ENC_LEVEL_HEVC_62 as i64, 186, "NV_ENC_LEVEL_HEVC_62");
assert_eq!(NVENCAPI_VERSION as i64, 16777228, "NVENCAPI_VERSION");
}
@ -364,4 +410,22 @@ fn guids_match_the_header() {
assert_eq!(NV_ENC_H264_PROFILE_HIGH_GUID.to_u128(), 0xe7cbc3094f7a4b89af2ad537c92be310u128, "NV_ENC_H264_PROFILE_HIGH_GUID");
assert_eq!(NV_ENC_PRESET_P4_GUID.to_u128(), 0x90a7b826df064862b9d2cd6d73a08681u128, "NV_ENC_PRESET_P4_GUID");
assert_eq!(NV_ENC_PRESET_P5_GUID.to_u128(), 0x21c6e6b4297a4cba998fb6cbde72ade3u128, "NV_ENC_PRESET_P5_GUID");
assert_eq!(NV_ENC_CODEC_HEVC_GUID.to_u128(), 0x790cdc8845224d7b9425bda9975f7603u128, "NV_ENC_CODEC_HEVC_GUID");
assert_eq!(NV_ENC_HEVC_PROFILE_MAIN_GUID.to_u128(), 0xb514c39ab55b40fa878ff1253b4dfdecu128, "NV_ENC_HEVC_PROFILE_MAIN_GUID");
}
/// The bit-field masks of the `flags` words (the C compiler's layout of each bit-field set alone).
#[test]
fn bit_field_masks_match_the_header() {
assert_eq!(H264_OUTPUT_AUD, 0x40, "NV_ENC_CONFIG_H264.outputAUD");
assert_eq!(H264_REPEAT_SPSPPS, 0x1000, "NV_ENC_CONFIG_H264.repeatSPSPPS");
assert_eq!(HEVC_OUTPUT_AUD, 0x10, "NV_ENC_CONFIG_HEVC.outputAUD");
assert_eq!(HEVC_DISABLE_SPSPPS, 0x40, "NV_ENC_CONFIG_HEVC.disableSPSPPS");
assert_eq!(HEVC_REPEAT_SPSPPS, 0x80, "NV_ENC_CONFIG_HEVC.repeatSPSPPS");
assert_eq!(HEVC_ENABLE_INTRA_REFRESH, 0x100, "NV_ENC_CONFIG_HEVC.enableIntraRefresh");
assert_eq!(HEVC_CHROMA_FORMAT_MASK, 0x600, "NV_ENC_CONFIG_HEVC.chromaFormatIDC");
assert_eq!(HEVC_PIXEL_BIT_DEPTH_MASK, 0x3800, "NV_ENC_CONFIG_HEVC.pixelBitDepthMinus8");
assert_eq!(HEVC_OUTPUT_RECOVERY_POINT_SEI, 0x40000, "NV_ENC_CONFIG_HEVC.outputRecoveryPointSEI");
assert_eq!(HEVC_CHROMA_FORMAT_MASK >> HEVC_CHROMA_FORMAT_SHIFT, 3, "chromaFormatIDC is 2 bits");
assert_eq!(HEVC_PIXEL_BIT_DEPTH_MASK >> HEVC_PIXEL_BIT_DEPTH_SHIFT, 7, "pixelBitDepthMinus8 is 3 bits");
}

120
crates/platform/src/nvenc/export.rs
View File

@ -1,7 +1,14 @@
//! NVENC as an Export encoder: [`factory`] is registered in front of the software H.264 encoder
//! (`filmcraft_export::register_encoder`) and takes an export only when Export ▸ Hardware encoding
//! is Auto and NVENC can do what the settings ask; otherwise it returns `None` and the software
//! encoder runs, exactly as before.
//! (`filmcraft_export::register_encoder`).
//!
//! - H.264: it takes an export only when Export ▸ Hardware encoding is Auto and NVENC can do what
//! the settings ask; otherwise it returns `None` and the software encoder runs, exactly as before.
//! - H.265 (HEVC Main, 8-bit 4:2:0, SDR BT.709, progressive): there is no software encoder, so the
//! Hardware encoding toggle does not apply (as with VideoToolbox) and choosing the format is the
//! opt-in. What NVENC cannot do (HDR, interlaced, two-pass, non-square pixels, sizes outside its
//! limits...) is counted as declined and ends the export with an error that says why; when this
//! machine has no HEVC encoder at all, [`filmcraft_export::available`] says so and the factory
//! returns `None` (the export's own "encoder not available" error).
//!
//! A hardware encoder that fails in the middle of an export (a lost device) ends the export with an
//! error naming it; unlike a decoder it cannot be replayed into the software encoder, because the
@ -13,11 +20,11 @@ use filmcraft_export::{
use filmcraft_isobmff::{AvcConfig, SampleEntry};
use filmcraft_time::FrameRate;
use super::{Config, NvencH264, Profile};
use super::{Codec, Config, Nvenc, Profile};
/// The NVENC export encoder.
struct NvencEncoder {
enc: NvencH264,
enc: Nvenc,
w: u32,
h: u32,
rate: FrameRate,
@ -46,7 +53,11 @@ impl VideoEncoder for NvencEncoder {
let (sps, pps) = self.enc.parameter_sets();
// `config` declines sizes above u16::MAX
let (w, h) = (u16::try_from(self.w).unwrap_or(u16::MAX), u16::try_from(self.h).unwrap_or(u16::MAX));
SampleEntry::avc(AvcConfig::new(vec![sps.to_vec()], vec![pps.to_vec()], 4), w, h)
match self.enc.codec() {
Codec::H264 => SampleEntry::avc(AvcConfig::new(vec![sps.to_vec()], vec![pps.to_vec()], 4), w, h),
// `Nvenc::new` built and validated the record, so there is always one for HEVC
Codec::Hevc => SampleEntry::hevc(self.enc.hevc_config().cloned().unwrap_or_default(), w, h),
}
}
fn timescale(&self) -> u32 {
@ -54,6 +65,9 @@ impl VideoEncoder for NvencEncoder {
}
fn encode(&mut self, f: &EncoderFrame) -> Result<Vec<EncodedPacket>> {
if f.hdr.is_some() {
return Err(ExportError::Unsupported("NVENC does not take HDR pictures here".into()));
}
if f.width != self.w || f.height != self.h {
return Err(ExportError::Encode(format!("NVENC: a {}x{} picture for a {}x{} encoder", f.width, f.height, self.w, self.h)));
}
@ -77,17 +91,27 @@ impl VideoEncoder for NvencEncoder {
}
}
/// Why NVENC does not take this export (the software encoder does), or the encoder's configuration.
/// Why NVENC does not take this export (the software encoder does, for H.264), or the encoder's
/// configuration.
fn config(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -> std::result::Result<Config, String> {
if format != Format::H264 || s.format.is_mxf() {
return Err("only MP4 / MOV H.264 exports".into());
let hevc = format == Format::Hevc;
if !(format == Format::H264 || hevc) || s.format.is_mxf() {
return Err("only MP4 / MOV H.264 and H.265 exports".into());
}
if s.signal.is_hdr() {
return Err("HDR (8-bit H.264 here carries it with software signalling)".into());
return Err(if hevc {
"HDR (hardware H.265 here is 8-bit SDR Main)".into()
} else {
"HDR (8-bit H.264 here carries it with software signalling)".into()
});
}
if !matches!(s.h264_pass, H264Pass::Single) {
if !matches!(s.h264_pass, H264Pass::Single) || (hevc && s.bitrate_mode == BitrateMode::Vbr2Pass) {
return Err("two-pass VBR".into());
}
// the hardware path writes the aspect ratio of H.264 only
if hevc && s.pixel_aspect.is_some_and(|(n, d)| n != d) {
return Err("non-square pixels".into());
}
if s.field_order != FieldOrder::Progressive {
return Err("interlaced output".into());
}
@ -110,12 +134,14 @@ fn config(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -
max_bitrate_kbps: s.max_bitrate_kbps.filter(|m| *m >= kbps).unwrap_or(kbps / 2 * 3),
cbr: s.bitrate_mode == BitrateMode::Cbr,
keyint: s.keyframe_distance.filter(|k| *k > 0).unwrap_or_else(|| (f64::from(fps.0) / f64::from(fps.1) * 2.0).round().max(1.0) as u32),
profile: match s.h264_profile {
H264Profile::Baseline => Profile::Baseline,
H264Profile::Main => Profile::Main,
H264Profile::High => Profile::High,
profile: match (hevc, s.h264_profile) {
(true, _) => Profile::HevcMain,
(false, H264Profile::Baseline) => Profile::Baseline,
(false, H264Profile::Main) => Profile::Main,
(false, H264Profile::High) => Profile::High,
},
level: s.h264_level,
// the H.264 level setting means nothing for H.265: the encoder picks the level
level: if hevc { None } else { s.h264_level },
sar: s.pixel_aspect,
bframes: true,
})
@ -123,6 +149,9 @@ fn config(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -
/// The Export encoder factory (see the module documentation).
pub fn factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -> Option<Result<Box<dyn VideoEncoder>>> {
if format == Format::Hevc {
return hevc_factory(w, h, rate, s);
}
if s.hardware_encoding != HardwareEncoding::Auto || format != Format::H264 {
return None;
}
@ -135,7 +164,7 @@ pub fn factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettin
Ok(c) => c,
Err(why) => return declined(&why),
};
match NvencH264::new(&cfg) {
match Nvenc::new(&cfg) {
Ok(enc) => {
filmcraft_export::note_hw_encode_session();
Some(Ok(Box::new(NvencEncoder { enc, w, h, rate, y: Vec::new(), u: Vec::new(), v: Vec::new() })))
@ -144,6 +173,32 @@ pub fn factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettin
}
}
/// The H.265 side of [`factory`]: the only HEVC encoder on Windows, so the Hardware encoding toggle
/// does not apply and a request NVENC cannot take is an error, not a fall-through.
fn hevc_factory(w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -> Option<Result<Box<dyn VideoEncoder>>> {
let declined = |why: &str| {
log::info!("hardware H.265 encoding declined: {why}");
filmcraft_export::note_hw_encode_declined();
Some(Err(ExportError::Unsupported(format!("H.265 export with NVENC: {why}"))))
};
let cfg = match config(Format::Hevc, w, h, rate, s) {
Ok(c) => c,
Err(why) => return declined(&why),
};
match Nvenc::new(&cfg) {
Ok(enc) => {
filmcraft_export::note_hw_encode_session();
Some(Ok(Box::new(NvencEncoder { enc, w, h, rate, y: Vec::new(), u: Vec::new(), v: Vec::new() })))
}
Err(why) => {
log::info!("hardware H.265 encoding declined: {why}");
filmcraft_export::note_hw_encode_declined();
// no HEVC encoder here at all: the export's own "encoder not available" error
if super::hevc_available() { Some(Err(ExportError::Unsupported(format!("H.265 export with NVENC: {why}")))) } else { None }
}
}
}
#[cfg(test)]
mod tests {
use super::*;
@ -157,6 +212,37 @@ mod tests {
assert_eq!(composition_offset(1 << 40, 0, 1), i32::MAX);
}
#[test]
fn hevc_takes_what_nvenc_can_write_and_declines_the_rest() {
let ok = ExportSettings { format: Format::Hevc, bitrate_kbps: 4000, keyframe_distance: Some(48), ..Default::default() };
let c = config(Format::Hevc, 1280, 720, FrameRate::FPS_24, &ok).unwrap();
assert_eq!((c.profile, c.level, c.keyint, c.fps), (Profile::HevcMain, None, 48, (24, 1)));
// the H.264 level and profile settings mean nothing to HEVC
let h = ExportSettings { h264_level: Some(41), h264_profile: H264Profile::Baseline, ..ok.clone() };
assert_eq!(config(Format::Hevc, 1280, 720, FrameRate::FPS_24, &h).unwrap().level, None);
// the Hardware encoding toggle does not matter for HEVC (it has no software encoder)
let off = ExportSettings { hardware_encoding: HardwareEncoding::Off, ..ok.clone() };
assert!(config(Format::Hevc, 1280, 720, FrameRate::FPS_24, &off).is_ok());
for (what, s) in [
("HDR", ExportSettings { signal: filmcraft_export::ColorSignal::PQ, ..ok.clone() }),
("two-pass", ExportSettings { bitrate_mode: BitrateMode::Vbr2Pass, ..ok.clone() }),
("analysis pass", ExportSettings { h264_pass: H264Pass::First, ..ok.clone() }),
("interlaced", ExportSettings { field_order: FieldOrder::UpperFirst, ..ok.clone() }),
("non-square pixels", ExportSettings { pixel_aspect: Some((4, 3)), ..ok.clone() }),
("MXF", ExportSettings { format: Format::MxfOp1a, ..ok.clone() }),
] {
assert!(config(Format::Hevc, 1280, 720, FrameRate::FPS_24, &s).is_err(), "{what}");
}
// hostile sizes and frame rates
assert!(config(Format::Hevc, 70_000, 64, FrameRate::FPS_24, &ok).is_err());
assert!(config(Format::Hevc, 64, 70_000, FrameRate::FPS_24, &ok).is_err());
for rate in [FrameRate { num: 0, den: 1 }, FrameRate { num: 24, den: 0 }, FrameRate { num: -24, den: 1 }, FrameRate { num: i64::MAX, den: 1 }] {
assert!(config(Format::Hevc, 1280, 720, rate, &ok).is_err(), "{rate:?}");
}
// other formats are not NVENC's
assert!(config(Format::ProRes, 1280, 720, FrameRate::FPS_24, &ok).is_err());
}
#[test]
fn oversized_pictures_are_declined() {
let s = ExportSettings { format: Format::H264, hardware_encoding: HardwareEncoding::Auto, ..Default::default() };

68
crates/platform/src/nvenc/ffi.rs
View File

@ -30,7 +30,7 @@
//! offsets are checked against a C compiler's by the generated `abi_tests.rs`.
//!
//! API 12.1 is targeted on purpose: it is understood by every driver since 531.x, and the encoder
//! features used here (H.264, presets, rate control) are the same in later versions.
//! features used here (H.264 and HEVC, presets, rate control) are the same in later versions.
//!
//! FFI module (docs/adr/0001-platform-ffi.md): plain `repr(C)` data, no logic.
@ -82,6 +82,22 @@ pub const NV_ENC_PIC_TYPE_I: u32 = 2;
pub const NV_ENC_PIC_TYPE_IDR: u32 = 3;
pub const NV_ENC_PIC_FLAG_EOS: u32 = 0x8;
pub const NV_ENC_LEVEL_AUTOSELECT: u32 = 0;
/// HEVC levels are `level × 30` (4.1 is 123), unlike H.264's `level × 10`.
pub const NV_ENC_LEVEL_HEVC_1: u32 = 30;
pub const NV_ENC_LEVEL_HEVC_2: u32 = 60;
pub const NV_ENC_LEVEL_HEVC_21: u32 = 63;
pub const NV_ENC_LEVEL_HEVC_3: u32 = 90;
pub const NV_ENC_LEVEL_HEVC_31: u32 = 93;
pub const NV_ENC_LEVEL_HEVC_4: u32 = 120;
pub const NV_ENC_LEVEL_HEVC_41: u32 = 123;
pub const NV_ENC_LEVEL_HEVC_5: u32 = 150;
pub const NV_ENC_LEVEL_HEVC_51: u32 = 153;
pub const NV_ENC_LEVEL_HEVC_52: u32 = 156;
pub const NV_ENC_LEVEL_HEVC_6: u32 = 180;
pub const NV_ENC_LEVEL_HEVC_61: u32 = 183;
pub const NV_ENC_LEVEL_HEVC_62: u32 = 186;
pub const NV_ENC_TIER_HEVC_MAIN: u32 = 0;
pub const NV_ENC_HEVC_CUSIZE_AUTOSELECT: u32 = 0;
pub const NV_ENC_H264_ENTROPY_CODING_MODE_CABAC: u32 = 1;
pub const NV_ENC_VUI_VIDEO_FORMAT_UNSPECIFIED: u32 = 5;
pub const NV_ENC_VUI_COLOR_PRIMARIES_BT709: u32 = 1;
@ -97,9 +113,11 @@ pub const NV_ENC_CAPS_WIDTH_MIN: u32 = 45;
pub const NV_ENC_CAPS_HEIGHT_MIN: u32 = 46;
pub const NV_ENC_CODEC_H264_GUID: GUID = GUID::from_values(0x6bc82762, 0x4e63, 0x4ca4, [0xaa, 0x85, 0x1e, 0x50, 0xf3, 0x21, 0xf6, 0xbf]);
pub const NV_ENC_CODEC_HEVC_GUID: GUID = GUID::from_values(0x790cdc88, 0x4522, 0x4d7b, [0x94, 0x25, 0xbd, 0xa9, 0x97, 0x5f, 0x76, 0x03]);
pub const NV_ENC_H264_PROFILE_BASELINE_GUID: GUID = GUID::from_values(0x0727bcaa, 0x78c4, 0x4c83, [0x8c, 0x2f, 0xef, 0x3d, 0xff, 0x26, 0x7c, 0x6a]);
pub const NV_ENC_H264_PROFILE_MAIN_GUID: GUID = GUID::from_values(0x60b5c1d4, 0x67fe, 0x4790, [0x94, 0xd5, 0xc4, 0x72, 0x6d, 0x7b, 0x6e, 0x6d]);
pub const NV_ENC_H264_PROFILE_HIGH_GUID: GUID = GUID::from_values(0xe7cbc309, 0x4f7a, 0x4b89, [0xaf, 0x2a, 0xd5, 0x37, 0xc9, 0x2b, 0xe3, 0x10]);
pub const NV_ENC_HEVC_PROFILE_MAIN_GUID: GUID = GUID::from_values(0xb514c39a, 0xb55b, 0x40fa, [0x87, 0x8f, 0xf1, 0x25, 0x3b, 0x4d, 0xfd, 0xec]);
pub const NV_ENC_PRESET_P4_GUID: GUID = GUID::from_values(0x90a7b826, 0xdf06, 0x4862, [0xb9, 0xd2, 0xcd, 0x6d, 0x73, 0xa0, 0x86, 0x81]);
pub const NV_ENC_PRESET_P5_GUID: GUID = GUID::from_values(0x21c6e6b4, 0x297a, 0x4cba, [0x99, 0x8f, 0xb6, 0xcb, 0xde, 0x72, 0xad, 0xe3]);
@ -235,10 +253,58 @@ pub const H264_OUTPUT_AUD: u32 = 1 << 6;
pub const H264_REPEAT_SPSPPS: u32 = 1 << 12;
pub const H264_OUTPUT_RECOVERY_POINT_SEI: u32 = 1 << 9;
/// `NV_ENC_CONFIG_HEVC_VUI_PARAMETERS` is a typedef of the H.264 VUI structure.
pub type NV_ENC_CONFIG_HEVC_VUI_PARAMETERS = NV_ENC_CONFIG_H264_VUI_PARAMETERS;
#[repr(C)]
#[derive(Clone, Copy)]
pub struct NV_ENC_CONFIG_HEVC {
pub level: u32,
pub tier: u32,
pub minCUSize: u32,
pub maxCUSize: u32,
/// useConstrainedIntraPred:1, disableDeblockAcrossSliceBoundary:1, outputBufferingPeriodSEI:1,
/// outputPictureTimingSEI:1, outputAUD:1, enableLTR:1, disableSPSPPS:1, repeatSPSPPS:1,
/// enableIntraRefresh:1, chromaFormatIDC:2, pixelBitDepthMinus8:3, enableFillerDataInsertion:1,
/// enableConstrainedEncoding:1, enableAlphaLayerEncoding:1, singleSliceIntraRefresh:1,
/// outputRecoveryPointSEI:1, outputTimeCodeSEI:1, reserved:12.
pub flags: u32,
pub idrPeriod: u32,
pub intraRefreshPeriod: u32,
pub intraRefreshCnt: u32,
pub maxNumRefFramesInDPB: u32,
pub ltrNumFrames: u32,
pub vpsId: u32,
pub spsId: u32,
pub ppsId: u32,
pub sliceMode: u32,
pub sliceModeData: u32,
pub maxTemporalLayersMinus1: u32,
pub hevcVUIParameters: NV_ENC_CONFIG_HEVC_VUI_PARAMETERS,
pub ltrTrustMode: u32,
pub useBFramesAsRef: u32,
pub numRefL0: u32,
pub numRefL1: u32,
pub reserved1: [u32; 214],
pub reserved2: [*mut c_void; 64],
}
pub const HEVC_OUTPUT_AUD: u32 = 1 << 4;
pub const HEVC_DISABLE_SPSPPS: u32 = 1 << 6;
pub const HEVC_REPEAT_SPSPPS: u32 = 1 << 7;
pub const HEVC_ENABLE_INTRA_REFRESH: u32 = 1 << 8;
/// `chromaFormatIDC` (2 bits) and `pixelBitDepthMinus8` (3 bits) share the flags word.
pub const HEVC_CHROMA_FORMAT_SHIFT: u32 = 9;
pub const HEVC_CHROMA_FORMAT_MASK: u32 = 3 << HEVC_CHROMA_FORMAT_SHIFT;
pub const HEVC_PIXEL_BIT_DEPTH_SHIFT: u32 = 11;
pub const HEVC_PIXEL_BIT_DEPTH_MASK: u32 = 7 << HEVC_PIXEL_BIT_DEPTH_SHIFT;
pub const HEVC_OUTPUT_RECOVERY_POINT_SEI: u32 = 1 << 18;
#[repr(C)]
#[derive(Clone, Copy)]
pub union NV_ENC_CODEC_CONFIG {
pub h264Config: NV_ENC_CONFIG_H264,
pub hevcConfig: NV_ENC_CONFIG_HEVC,
pub reserved: [u32; 320],
align: [*mut c_void; 0],
}

194
crates/platform/src/nvenc/hevc.rs Normal file
View File

@ -0,0 +1,194 @@
//! What is specific to HEVC in the NVENC backend (safe code): the level codes, and the `hvcC`
//! record built from the parameter sets the encoder wrote.
//!
//! Nothing in the record is invented: the profile, tier, compatibility flags, level, chroma format,
//! bit depths and temporal layers come from the SPS (parsed with `filmcraft_hevc`), and the 48
//! general constraint flags and `sps_temporal_id_nesting_flag` are read from the SPS bits that carry
//! them (`parse_ptl` skips the flags and `Sps` does not keep the nesting flag). The array
//! completeness flags are 1 and the parameter sets are only in the record, never in the samples
//! (the `hvc1` sample entry).
use filmcraft_bitstream::unescape_rbsp;
use filmcraft_hevc::params::Sps;
use filmcraft_isobmff::{HevcConfig, HevcNalArray};
use super::{Codec, nal_type};
/// `NV_ENC_LEVEL_HEVC_*` for a level × 10 (4.1 is 41): HEVC levels are numbered `level × 30`.
/// `None` for a number that is not an HEVC level.
pub fn level_code(level_x10: u8) -> Option<u8> {
matches!(level_x10, 10 | 20 | 21 | 30 | 31 | 40 | 41 | 50 | 51 | 52 | 60 | 61 | 62).then(|| level_x10.saturating_mul(3))
}
/// Bytes of the SPS RBSP before the general profile / tier / level record: `sps_video_parameter_set_id`,
/// `sps_max_sub_layers_minus1` and `sps_temporal_id_nesting_flag` share the first one.
const PTL_START: usize = 1;
/// General profile space / tier / profile (1 byte), compatibility flags (4), constraint flags (6), level (1).
const PTL_LEN: usize = 12;
/// The `hvcC` record of a stream of `size` pictures (width, height) whose parameter sets are `vps`,
/// `sps` and `pps` (NAL units with their headers, without start codes or length prefixes). An error
/// says why the stream is not one this backend writes: HEVC Main, 8-bit 4:2:0.
pub fn hevc_config(vps: &[u8], sps: &[u8], pps: &[u8], size: (u32, u32)) -> Result<HevcConfig, String> {
for (nal, kind, name) in [(vps, 32u8, "VPS"), (sps, 33, "SPS"), (pps, 34, "PPS")] {
if nal_type(Codec::Hevc, nal) != Some(kind) || nal.len() <= 2 {
return Err(format!("the encoder's {name} is not a {name} NAL unit"));
}
// the record stores each length in 16 bits
if u16::try_from(nal.len()).is_err() {
return Err(format!("the {name} is too long for an hvcC record"));
}
}
let rbsp = unescape_rbsp(sps.get(2..).unwrap_or_default());
let parsed = Sps::parse(&rbsp).map_err(|e| format!("unreadable HEVC SPS: {e}"))?;
let ptl_bytes = rbsp.get(PTL_START..PTL_START + PTL_LEN).ok_or("the HEVC SPS is truncated before its profile / tier / level")?;
// bytes 5..11 of the record are the 48 general constraint indicator flags
let constraints = ptl_bytes.get(5..11).ok_or("the HEVC SPS is truncated before its constraint flags")?;
let general_constraint_indicator_flags = constraints.iter().fold(0u64, |acc, b| (acc << 8) | u64::from(*b));
let temporal_id_nested = rbsp.first().is_some_and(|b| b & 1 != 0);
let ptl = &parsed.ptl;
if ptl.profile_space != 0 || ptl.profile_idc != 1 {
return Err(format!("the encoder wrote HEVC profile {} (space {}), not Main", ptl.profile_idc, ptl.profile_space));
}
if parsed.chroma_format_idc != 1 || parsed.bit_depth_luma != 8 || parsed.bit_depth_chroma != 8 {
return Err(format!(
"the encoder wrote chroma format {} at {}/{} bits, not 8-bit 4:2:0",
parsed.chroma_format_idc, parsed.bit_depth_luma, parsed.bit_depth_chroma
));
}
let (_, _, w, h) = parsed.crop_rect();
if (w, h) != size {
return Err(format!("the encoder wrote {w}x{h} pictures for a {}x{} export", size.0, size.1));
}
Ok(HevcConfig {
general_profile_space: ptl.profile_space,
general_tier_flag: ptl.tier,
general_profile_idc: ptl.profile_idc,
general_profile_compatibility_flags: ptl.compatibility,
general_constraint_indicator_flags,
general_level_idc: ptl.level_idc,
// 0: not specified, which is a valid value for both
min_spatial_segmentation_idc: 0,
parallelism_type: 0,
chroma_format_idc: u8::try_from(parsed.chroma_format_idc).map_err(|_| "chroma format".to_string())?,
bit_depth_luma: u8::try_from(parsed.bit_depth_luma).map_err(|_| "bit depth".to_string())?,
bit_depth_chroma: u8::try_from(parsed.bit_depth_chroma).map_err(|_| "bit depth".to_string())?,
// 0: unspecified average rate, and the frame rate may not be constant
avg_frame_rate: 0,
constant_frame_rate: 0,
num_temporal_layers: u8::try_from(parsed.max_sub_layers_minus1.saturating_add(1)).map_err(|_| "temporal layers".to_string())?,
temporal_id_nested,
length_size: 4,
arrays: vec![
HevcNalArray { completeness: true, nal_type: 32, nalus: vec![vps.to_vec()] },
HevcNalArray { completeness: true, nal_type: 33, nalus: vec![sps.to_vec()] },
HevcNalArray { completeness: true, nal_type: 34, nalus: vec![pps.to_vec()] },
],
})
}
#[cfg(test)]
mod tests {
use super::*;
// The parameter sets NVENC wrote for a 1280x720, 24 fps, B-frame HEVC Main stream (RTX 5060, driver
// 617.14): the coded size is 1280x736 and a conformance window crops it to 1280x720.
const VPS: &str = "40010c01ffff01600000030090000003000003005d9940c0000003004000000614";
const SPS: &str = "42010101600000030090000003000003005da00280802e1f1396654a421191bff0c05a8080808a0000030002000003003010";
const PPS: &str = "4401c1937c0cc9";
fn unhex(h: &str) -> Vec<u8> {
(0..h.len()).step_by(2).filter_map(|i| u8::from_str_radix(h.get(i..i + 2)?, 16).ok()).collect()
}
fn record() -> HevcConfig {
hevc_config(&unhex(VPS), &unhex(SPS), &unhex(PPS), (1280, 720)).unwrap()
}
#[test]
fn hevc_levels_are_thirty_times_the_level() {
assert_eq!(level_code(41), Some(123));
assert_eq!(level_code(10), Some(30));
assert_eq!(level_code(21), Some(63));
assert_eq!(level_code(62), Some(186));
for bad in [0, 11, 42, 53, 70, 255] {
assert_eq!(level_code(bad), None, "{bad}");
}
}
#[test]
fn the_record_is_the_sps_it_came_from() {
let c = record();
let sps = unhex(SPS);
let rbsp = unescape_rbsp(&sps[2..]);
// profile / tier / compatibility / constraint flags / level are the SPS's own bytes
assert_eq!(&c.to_bytes()[1..13], &rbsp[1..13]);
assert_eq!((c.general_profile_space, c.general_tier_flag, c.general_profile_idc), (0, false, 1));
assert_eq!(c.general_profile_compatibility_flags, 0x6000_0000, "Main and Main 10");
// progressive_source_flag and frame_only_constraint_flag
assert_eq!(c.general_constraint_indicator_flags, 0x9000_0000_0000);
assert_eq!(c.general_level_idc, 93, "level 3.1");
assert_eq!((c.chroma_format_idc, c.bit_depth_luma, c.bit_depth_chroma), (1, 8, 8));
assert_eq!((c.num_temporal_layers, c.temporal_id_nested, c.length_size), (1, true, 4));
assert_eq!(c.arrays.iter().map(|a| (a.nal_type, a.completeness, a.nalus.len())).collect::<Vec<_>>(), vec![(32, true, 1), (33, true, 1), (34, true, 1)]);
assert_eq!((c.vps(), c.sps(), c.pps()), (vec![&unhex(VPS)[..]], vec![&sps[..]], vec![&unhex(PPS)[..]]));
assert_eq!(HevcConfig::parse(&c.to_bytes()).unwrap(), c);
}
#[test]
fn the_picture_size_has_to_match_after_cropping() {
let (v, s, p) = (unhex(VPS), unhex(SPS), unhex(PPS));
for wrong in [(1280, 736), (1920, 1080), (1280, 719), (0, 0)] {
let e = hevc_config(&v, &s, &p, wrong).unwrap_err();
assert!(e.contains("1280x720"), "{e}");
}
}
#[test]
fn a_stream_that_is_not_main_8_bit_420_is_refused() {
// The same SPS with the general profile changed (bits of profile_idc in the first PTL byte),
// the chroma format or the bit depth changed would be other streams: edit the PTL byte only
// (it sits before any emulation-prevention byte).
let (v, s, p) = (unhex(VPS), unhex(SPS), unhex(PPS));
let mut other_profile = s.clone();
other_profile[3] = (other_profile[3] & !0x1f) | 2; // Main 10
assert!(hevc_config(&v, &other_profile, &p, (1280, 720)).unwrap_err().contains("not Main"));
let mut other_space = s.clone();
other_space[3] |= 0x40;
assert!(hevc_config(&v, &other_space, &p, (1280, 720)).is_err());
}
#[test]
fn truncated_and_corrupt_parameter_sets_are_errors_never_panics() {
let (v, s, p) = (unhex(VPS), unhex(SPS), unhex(PPS));
// every truncation of the SPS
for n in 0..s.len() {
assert!(hevc_config(&v, &s[..n], &p, (1280, 720)).is_err(), "SPS cut at {n}");
}
// a bit flip anywhere in the SPS: an error or a record, but never a panic
for byte in 0..s.len() {
for bit in 0..8 {
let mut m = s.clone();
m[byte] ^= 1 << bit;
let r = std::panic::catch_unwind(|| hevc_config(&v, &m, &p, (1280, 720)));
assert!(r.is_ok(), "panic with bit {bit} of byte {byte} flipped");
}
}
// wrong NAL types, empty and one-byte NAL units
let cases: [(&[u8], &[u8], &[u8]); 5] = [
(&[], &[], &[]),
(&[0x40], &[0x42], &[0x44]),
(&[0x40, 0x01, 0xAA], &[0x42, 0x01], &[0x44, 0x01, 0xBB]),
(&[0x42, 0x01, 0xAA], &s, &p),
(&v, &p, &s),
];
for (v, s, p) in cases {
assert!(hevc_config(v, s, p, (64, 64)).is_err(), "{v:?} {s:?} {p:?}");
}
// a parameter set too long for the record's 16-bit lengths
let mut huge = s.clone();
huge.resize(70_000, 0x55);
assert!(hevc_config(&v, &huge, &p, (1280, 720)).is_err());
}
}

265
crates/platform/src/nvenc/mod.rs
View File

@ -1,23 +1,27 @@
//! NVIDIA NVENC H.264 hardware encoding (Windows).
//! NVIDIA NVENC hardware encoding, H.264 and H.265 (HEVC) Main (Windows).
//!
//! The encoder runs on the GPU's NVENC engine through the driver's `nvEncodeAPI64.dll` (API 12.1,
//! [`ffi`]; no CUDA, no SDK to install): pictures go in as NV12 input buffers, the Annex B output
//! comes back as length-prefixed H.264 samples with the parameter sets split out for the `avcC`.
//! [`export`] plugs it into Export as an alternative to the software encoder.
//! comes back as length-prefixed samples with the parameter sets split out for the `avcC` (H.264)
//! or the `hvcC` (HEVC, see [`hevc`]). One session, ring and NV12 path serves both codecs; the
//! codec only chooses the GUIDs, the codec configuration and how NAL units are told apart.
//! [`export`] plugs it into Export: as an alternative to the software encoder for H.264, and as the
//! only encoder of the H.265 format (which has no software encoder).
//!
//! ```text
//! RGBA (the export pipeline) ──► BT.709 limited 4:2:0 (the software encoder's conversion)
//! ──► NV12 input buffer ──NVENC──► Annex B ──► length-prefixed samples + avcC
//! ──► NV12 input buffer ──NVENC──► Annex B ──► length-prefixed samples + avcC / hvcC
//! ```
//!
//! `unsafe` is confined to `ffi` (data) and `session` (every driver call); this module is safe
//! code. Hardware encoding never replaces an export that works in software: the factory declines
//! what NVENC cannot do (no NVIDIA GPU or driver, sizes, HDR, two-pass, MXF...) and the software
//! encoder takes over.
//! code. Hardware H.264 encoding never replaces an export that works in software: the factory
//! declines what NVENC cannot do (no NVIDIA GPU or driver, sizes, HDR, two-pass, MXF...) and the
//! software encoder takes over. A declined HEVC export is an error that says why.
pub mod export;
#[allow(unsafe_code)]
mod ffi;
pub mod hevc;
#[allow(unsafe_code)]
mod session;
@ -26,15 +30,29 @@ mod abi_tests;
use std::collections::VecDeque;
pub use self::session::{Caps, Params};
use filmcraft_isobmff::HevcConfig;
pub use self::session::{Caps, Codec, Params};
use self::session::{Locked, Session, Submitted};
/// H.264 profile (`profile_idc` 66 / 77 / 100).
/// H.264 profile (`profile_idc` 66 / 77 / 100), or HEVC Main (8-bit 4:2:0).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Profile {
Baseline,
Main,
High,
/// H.265 Main: chooses the HEVC codec (like `VtProfile::HevcMain` for VideoToolbox).
HevcMain,
}
impl Profile {
/// The codec this profile belongs to.
pub fn codec(self) -> Codec {
match self {
Profile::HevcMain => Codec::Hevc,
_ => Codec::H264,
}
}
}
/// What to encode.
@ -50,7 +68,7 @@ pub struct Config {
/// Frames between IDR pictures.
pub keyint: u32,
pub profile: Profile,
/// Level × 10 (41 = 4.1); `None` lets the encoder pick.
/// Level × 10 (41 = 4.1), for either codec; `None` lets the encoder pick.
pub level: Option<u8>,
/// Sample aspect ratio.
pub sar: Option<(u32, u32)>,
@ -71,11 +89,16 @@ pub struct Packet {
/// Pictures in flight: input / output buffer pairs.
const RING: usize = 8;
/// An NVENC H.264 encoder.
pub struct NvencH264 {
/// An NVENC encoder (H.264, or HEVC when the profile is [`Profile::HevcMain`]).
pub struct Nvenc {
session: Session,
codec: Codec,
/// Empty for H.264.
vps: Vec<u8>,
sps: Vec<u8>,
pps: Vec<u8>,
/// The `hvcC` record (HEVC only), built from the parameter sets the encoder wrote.
hvcc: Option<HevcConfig>,
/// Frames the output is delayed by reordering (0 or 1).
delay: u32,
free: Vec<usize>,
@ -86,12 +109,41 @@ pub struct NvencH264 {
size: (u32, u32),
}
/// The H.264 encoder (the name it had before HEVC joined it).
pub type NvencH264 = Nvenc;
/// Whether this system has an NVIDIA GPU with a driver that has NVENC.
pub fn available() -> bool {
Session::open().is_ok()
}
impl NvencH264 {
/// Whether this system's NVENC can encode HEVC: a session opened and an HEVC encoder created for a
/// small picture, once (the answer is kept). What makes the H.265 export format available.
pub fn hevc_available() -> bool {
static AVAILABLE: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
*AVAILABLE.get_or_init(|| {
let probe = Config {
width: 640,
height: 360,
fps: (30, 1),
bitrate_kbps: 2000,
max_bitrate_kbps: 3000,
cbr: false,
keyint: 30,
profile: Profile::HevcMain,
level: None,
sar: None,
bframes: false,
};
let result = Nvenc::new(&probe);
if let Err(why) = &result {
log::info!("no hardware HEVC encoder: {why}");
}
result.is_ok()
})
}
impl Nvenc {
/// Open an encoder, or say why NVENC does not take this configuration.
pub fn new(cfg: &Config) -> Result<Self, String> {
let (w, h) = (cfg.width, cfg.height);
@ -101,13 +153,22 @@ impl NvencH264 {
if cfg.fps.0 == 0 || cfg.fps.1 == 0 {
return Err("frame rate".into());
}
let codec = cfg.profile.codec();
let level = match (codec, cfg.level) {
(Codec::Hevc, Some(l)) => Some(hevc::level_code(l).ok_or_else(|| format!("{}.{} is not an HEVC level", l / 10, l % 10))?),
(_, l) => l,
};
let mut session = Session::open()?;
let caps = session.caps()?;
let caps = session.caps(codec)?;
if w < caps.min_size.0 || h < caps.min_size.1 || w > caps.max_size.0 || h > caps.max_size.1 {
return Err(format!("{w}x{h} is outside NVENC's {}x{} - {}x{}", caps.min_size.0, caps.min_size.1, caps.max_size.0, caps.max_size.1));
}
let bframes = u32::from(cfg.bframes && cfg.profile != Profile::Baseline && caps.max_bframes >= 1);
// NVENC wants a GOP longer than the B-frame pattern: HEVC with a keyframe every one or two
// pictures is simply written without B-frames (H.264 keeps declining those, to the software encoder)
let room_for_bframes = codec == Codec::H264 || cfg.keyint > 2;
let bframes = u32::from(cfg.bframes && cfg.profile != Profile::Baseline && caps.max_bframes >= 1 && room_for_bframes);
let params = Params {
codec,
width: w,
height: h,
fps: cfg.fps,
@ -117,17 +178,47 @@ impl NvencH264 {
gop: cfg.keyint.max(1),
profile: match cfg.profile {
Profile::Baseline => 0,
Profile::Main => 1,
Profile::Main | Profile::HevcMain => 1,
Profile::High => 2,
},
level: cfg.level,
level,
sar: cfg.sar,
bframes,
};
session.initialize(&params, RING)?;
let (sps, pps) = split_parameter_sets(&session.sequence_params()?)?;
let params = session.sequence_params()?;
let (vps, sps, pps, hvcc) = match codec {
Codec::H264 => {
let (sps, pps) = split_parameter_sets(&params)?;
(Vec::new(), sps, pps, None)
}
Codec::Hevc => {
let (vps, sps, pps) = split_hevc_parameter_sets(&params)?;
// validated now: a stream whose parameter sets we cannot describe is a declined export
let record = hevc::hevc_config(&vps, &sps, &pps, (w, h))?;
(vps, sps, pps, Some(record))
}
};
let slots = session.slots();
Ok(Self { session, sps, pps, delay: bframes, free: (0..slots).rev().collect(), pending: VecDeque::new(), ready: 0, emitted: 0, size: (w, h) })
Ok(Self {
session,
codec,
vps,
sps,
pps,
hvcc,
delay: bframes,
free: (0..slots).rev().collect(),
pending: VecDeque::new(),
ready: 0,
emitted: 0,
size: (w, h),
})
}
/// The codec this encoder writes.
pub fn codec(&self) -> Codec {
self.codec
}
/// The sequence and picture parameter sets (NAL units without start codes).
@ -135,6 +226,16 @@ impl NvencH264 {
(&self.sps, &self.pps)
}
/// The video parameter set (HEVC; empty for H.264).
pub fn vps(&self) -> &[u8] {
&self.vps
}
/// The `hvcC` record of an HEVC stream (`None` for H.264).
pub fn hevc_config(&self) -> Option<&HevcConfig> {
self.hvcc.as_ref()
}
/// Frames the decoding time runs behind the presentation time (B-frame reordering).
pub fn delay(&self) -> u32 {
self.delay
@ -193,7 +294,7 @@ impl NvencH264 {
let r = self.session.read(slot);
self.free.push(slot);
let o = r?;
let data = annex_b_to_length_prefixed(&o.data)?;
let data = annex_b_to_length_prefixed_for(self.codec, &o.data)?;
let k = self.emitted;
self.emitted = self.emitted.saturating_add(1);
let pts = i64::try_from(o.pts).unwrap_or(i64::MAX);
@ -256,16 +357,41 @@ pub fn annex_b_nals(data: &[u8]) -> Vec<&[u8]> {
/// Annex B to 4-byte length-prefixed NAL units, dropping parameter sets (they live in the `avcC`)
/// and access unit delimiters. Errors on a stream without NAL units.
pub fn annex_b_to_length_prefixed(data: &[u8]) -> Result<Vec<u8>, String> {
annex_b_to_length_prefixed_for(Codec::H264, data)
}
/// The NAL unit type of a NAL unit (`None` when it is too short to have a header).
pub fn nal_type(codec: Codec, nal: &[u8]) -> Option<u8> {
match codec {
Codec::H264 => nal.first().map(|b| b & 0x1f),
// two header bytes: forbidden_zero_bit, nal_unit_type (6), layer id (6), temporal id plus 1 (3)
Codec::Hevc => match nal {
[b, _, ..] => Some((b >> 1) & 0x3f),
_ => None,
},
}
}
/// [`annex_b_to_length_prefixed`] for a codec: H.264 drops SPS / PPS / AUD (types 7 to 9); HEVC drops
/// VPS / SPS / PPS / AUD (32 to 35) and end-of-sequence / end-of-bitstream markers (36, 37), since
/// the file is one coded video sequence. A NAL unit too short for its header is dropped.
pub fn annex_b_to_length_prefixed_for(codec: Codec, data: &[u8]) -> Result<Vec<u8>, String> {
let nals = annex_b_nals(data);
if nals.is_empty() {
return Err("the encoder produced no NAL units".into());
}
let mut out = Vec::with_capacity(data.len());
for n in nals {
if n.first().is_none_or(|b| matches!(b & 0x1f, 7..=9)) {
let dropped = match (codec, nal_type(codec, n)) {
(_, None) => true,
(Codec::H264, Some(t)) => matches!(t, 7..=9),
(Codec::Hevc, Some(t)) => matches!(t, 32..=37),
};
if dropped {
continue;
}
out.extend_from_slice(&(n.len() as u32).to_be_bytes());
let len = u32::try_from(n.len()).map_err(|_| "a NAL unit longer than 4 GiB".to_string())?;
out.extend_from_slice(&len.to_be_bytes());
out.extend_from_slice(n);
}
if out.is_empty() {
@ -284,6 +410,16 @@ pub fn split_parameter_sets(data: &[u8]) -> Result<(Vec<u8>, Vec<u8>), String> {
}
}
/// The VPS, SPS and PPS NAL units of an Annex B byte string from an HEVC encoder.
pub fn split_hevc_parameter_sets(data: &[u8]) -> Result<(Vec<u8>, Vec<u8>, Vec<u8>), String> {
let nals = annex_b_nals(data);
let find = |t: u8| nals.iter().find(|n| nal_type(Codec::Hevc, n) == Some(t)).map(|n| n.to_vec());
match (find(32), find(33), find(34)) {
(Some(v), Some(s), Some(p)) => Ok((v, s, p)),
_ => Err("the encoder returned no VPS / SPS / PPS".into()),
}
}
#[cfg(test)]
mod tests {
use super::*;
@ -336,4 +472,89 @@ mod tests {
assert!(annex_b_to_length_prefixed(&[0, 0, 1, 0, 0, 0, 1]).is_err());
assert!(split_parameter_sets(&[0, 0, 1]).is_err());
}
// HEVC NAL unit headers: type in bits 1..=6 of the first byte (VPS 0x40, SPS 0x42, PPS 0x44, AUD 0x46,
// EOS 0x48, EOB 0x4a, IDR_W_RADL 0x26, IDR_N_LP 0x28, TRAIL_R 0x02, prefix SEI 0x4e, suffix SEI 0x50)
#[test]
fn hevc_nal_types_come_from_the_second_header_bit_range() {
assert_eq!(nal_type(Codec::Hevc, &[0x40, 0x01]), Some(32));
assert_eq!(nal_type(Codec::Hevc, &[0x26, 0x01, 9]), Some(19));
assert_eq!(nal_type(Codec::Hevc, &[0x28, 0x01]), Some(20));
assert_eq!(nal_type(Codec::Hevc, &[0x4e, 0x01]), Some(39));
assert_eq!(nal_type(Codec::Hevc, &[0x50, 0x01]), Some(40));
// the H.264 reading of the same bytes is different (and wrong for HEVC)
assert_eq!(nal_type(Codec::H264, &[0x28, 0x01]), Some(8));
// no header, no type
assert_eq!(nal_type(Codec::Hevc, &[]), None);
assert_eq!(nal_type(Codec::Hevc, &[0x40]), None);
assert_eq!(nal_type(Codec::H264, &[]), None);
}
#[test]
fn hevc_samples_keep_slices_and_sei_and_lose_parameter_sets_and_delimiters() {
let s = [
&[0, 0, 0, 1, 0x46, 0x01, 0x50][..], // AUD
&[0, 0, 0, 1, 0x40, 0x01, 1, 2], // VPS
&[0, 0, 0, 1, 0x42, 0x01, 3], // SPS
&[0, 0, 1, 0x44, 0x01, 4], // PPS
&[0, 0, 1, 0x4e, 0x01, 5, 5], // prefix SEI
&[0, 0, 0, 1, 0x28, 0x01, 6, 6, 6], // IDR_N_LP: its header byte is 0x28, a PPS to an H.264 reader
&[0, 0, 1, 0x26, 0x01, 7], // IDR_W_RADL
&[0, 0, 1, 0x02, 0x01, 8], // TRAIL_R
&[0, 0, 1, 0x50, 0x01, 9], // suffix SEI
&[0, 0, 1, 0x48, 0x01], // end of sequence
&[0, 0, 1, 0x4a, 0x01], // end of bitstream
]
.concat();
let out = annex_b_to_length_prefixed_for(Codec::Hevc, &s).unwrap();
let expected: Vec<u8> = [
&[0, 0, 0, 4, 0x4e, 0x01, 5, 5][..],
&[0, 0, 0, 5, 0x28, 0x01, 6, 6, 6],
&[0, 0, 0, 3, 0x26, 0x01, 7],
&[0, 0, 0, 3, 0x02, 0x01, 8],
&[0, 0, 0, 3, 0x50, 0x01, 9],
]
.concat();
assert_eq!(out, expected);
// the same stream read as H.264 loses and keeps the wrong NAL units
assert_ne!(annex_b_to_length_prefixed(&s).unwrap(), expected);
}
#[test]
fn hevc_parameter_sets_are_found_by_type() {
let s = [0, 0, 0, 1, 0x40, 0x01, 1, 0, 0, 0, 1, 0x42, 0x01, 2, 0, 0, 1, 0x44, 0x01, 3, 0, 0, 1, 0x26, 0x01, 4];
assert_eq!(split_hevc_parameter_sets(&s).unwrap(), (vec![0x40, 1, 1], vec![0x42, 1, 2], vec![0x44, 1, 3]));
// an H.264 SPS / PPS pair has none of them; one missing is an error
assert!(split_hevc_parameter_sets(&[0, 0, 0, 1, 0x67, 1, 2, 0, 0, 1, 0x68, 3]).is_err());
assert!(split_hevc_parameter_sets(&[0, 0, 0, 1, 0x40, 0x01, 1, 0, 0, 1, 0x42, 0x01, 2]).is_err());
}
#[test]
fn hostile_hevc_streams_are_errors() {
// empty, no start code, only parameter sets / delimiters / markers, one-byte NAL units
for s in [
&[][..],
&[0, 0, 1],
&[1, 2, 3],
&[0, 0, 1, 0x46, 0x01, 0x50],
&[0, 0, 0, 1, 0x40, 0x01],
&[0, 0, 1, 0x26],
&[0, 0, 1, 0x48, 0x01],
&[0, 0, 1, 0xff],
] {
assert!(annex_b_to_length_prefixed_for(Codec::Hevc, s).is_err(), "{s:?}");
}
// truncated slices are kept as they are (the decoder reports them), never panic
assert!(annex_b_to_length_prefixed_for(Codec::Hevc, &[0, 0, 1, 0x26, 0x01]).is_ok());
assert!(split_hevc_parameter_sets(&[]).is_err());
assert!(split_hevc_parameter_sets(&[0, 0, 1, 0x40]).is_err());
}
#[test]
fn the_hevc_profile_picks_the_hevc_codec() {
assert_eq!(Profile::HevcMain.codec(), Codec::Hevc);
for p in [Profile::Baseline, Profile::Main, Profile::High] {
assert_eq!(p.codec(), Codec::H264);
}
}
}

120
crates/platform/src/nvenc/session.rs
View File

@ -1,4 +1,4 @@
//! An NVENC H.264 encode session: the driver library, the Direct3D 11 device it is opened on, the
//! An NVENC (H.264 or HEVC) encode session: the driver library, the Direct3D 11 device it is opened on, the
//! encoder configuration, and the input / output buffers (every call into the driver).
//!
//! FFI module (docs/adr/0001-platform-ffi.md): every `unsafe` block has a `// SAFETY:` comment, the
@ -99,7 +99,23 @@ fn nvidia_device() -> Result<ID3D11Device, String> {
Err("no NVIDIA adapter".into())
}
/// What the encoder says it can do for H.264.
/// The codec a session encodes.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Codec {
H264,
Hevc,
}
impl Codec {
fn guid(self) -> windows::core::GUID {
match self {
Codec::H264 => NV_ENC_CODEC_H264_GUID,
Codec::Hevc => NV_ENC_CODEC_HEVC_GUID,
}
}
}
/// What the encoder says it can do for a codec.
#[derive(Clone, Copy, Debug)]
pub struct Caps {
pub max_bframes: u32,
@ -110,6 +126,7 @@ pub struct Caps {
/// Encoder settings (already validated by the caller).
#[derive(Clone, Debug)]
pub struct Params {
pub codec: Codec,
pub width: u32,
pub height: u32,
pub fps: (u32, u32),
@ -117,8 +134,9 @@ pub struct Params {
pub max_bitrate: u32,
pub cbr: bool,
pub gop: u32,
/// 0 baseline, 1 main, 2 high.
/// H.264 only: 0 baseline, 1 main, 2 high (HEVC is always Main).
pub profile: u8,
/// The codec's own level code: `level × 10` for H.264, `level × 30` for HEVC.
pub level: Option<u8>,
pub sar: Option<(u32, u32)>,
/// B-frames between references (0 or 1).
@ -181,7 +199,8 @@ fn status(what: &str, s: NvStatus, enc: Handle, api: &Api) -> String {
})
})
.flatten()
.filter(|d| !d.is_empty());
// the driver's text is sometimes not text at all (HEVC parameter errors): keep it only when readable
.filter(|d| !d.is_empty() && d.chars().all(|c| c.is_ascii_graphic() || c == ' '));
match detail {
Some(d) => format!("{what} failed ({s}): {d}"),
None => format!("{what} failed ({s})"),
@ -221,8 +240,8 @@ impl Session {
Ok(Session { api, enc, _device: device, slots: Vec::new(), size: (0, 0), out_size: 0 })
}
/// The encoder's limits for H.264.
pub fn caps(&self) -> Result<Caps, String> {
/// The encoder's limits for `codec` (an error when the GPU cannot encode it).
pub fn caps(&self, codec: Codec) -> Result<Caps, String> {
let get = |cap: u32| -> Result<u32, String> {
let f = self.api.list.nvEncGetEncodeCaps.ok_or("no nvEncGetEncodeCaps")?;
// SAFETY: a valid session, a zeroed parameter block with its version, one int out.
@ -231,9 +250,9 @@ impl Session {
p.version = NV_ENC_CAPS_PARAM_VER;
p.capsToQuery = cap;
let mut v = 0i32;
let s = f(self.enc, NV_ENC_CODEC_H264_GUID, &mut p, &mut v);
let s = f(self.enc, codec.guid(), &mut p, &mut v);
if s != NV_ENC_SUCCESS {
return Err(status("querying the encoder", s, self.enc, self.api));
return Err(status(if codec == Codec::Hevc { "querying the HEVC encoder" } else { "querying the encoder" }, s, self.enc, self.api));
}
Ok(v.max(0) as u32)
}
@ -256,16 +275,17 @@ impl Session {
let mut preset: Box<NV_ENC_PRESET_CONFIG> = Box::new(std::mem::zeroed());
preset.version = NV_ENC_PRESET_CONFIG_VER;
preset.presetCfg.version = NV_ENC_CONFIG_VER;
let s = preset_fn(self.enc, NV_ENC_CODEC_H264_GUID, NV_ENC_PRESET_P5_GUID, NV_ENC_TUNING_INFO_HIGH_QUALITY, &mut *preset);
let s = preset_fn(self.enc, p.codec.guid(), NV_ENC_PRESET_P5_GUID, NV_ENC_TUNING_INFO_HIGH_QUALITY, &mut *preset);
if s != NV_ENC_SUCCESS {
return Err(status("reading the encoder preset", s, self.enc, self.api));
}
let mut cfg: Box<NV_ENC_CONFIG> = Box::new(preset.presetCfg);
cfg.version = NV_ENC_CONFIG_VER;
cfg.profileGUID = match p.profile {
0 => NV_ENC_H264_PROFILE_BASELINE_GUID,
1 => NV_ENC_H264_PROFILE_MAIN_GUID,
_ => NV_ENC_H264_PROFILE_HIGH_GUID,
cfg.profileGUID = match (p.codec, p.profile) {
(Codec::Hevc, _) => NV_ENC_HEVC_PROFILE_MAIN_GUID,
(Codec::H264, 0) => NV_ENC_H264_PROFILE_BASELINE_GUID,
(Codec::H264, 1) => NV_ENC_H264_PROFILE_MAIN_GUID,
(Codec::H264, _) => NV_ENC_H264_PROFILE_HIGH_GUID,
};
cfg.gopLength = p.gop.max(1);
cfg.frameIntervalP = 1 + p.bframes as i32;
@ -277,28 +297,62 @@ impl Session {
// a one-second buffer, the usual for streaming-style rate control
rc.vbvBufferSize = rc.maxBitRate;
rc.vbvInitialDelay = rc.vbvBufferSize;
let h = &mut cfg.encodeCodecConfig.h264Config;
h.idrPeriod = p.gop.max(1);
h.level = p.level.map_or(NV_ENC_LEVEL_AUTOSELECT, u32::from);
h.flags &= !(H264_OUTPUT_AUD | H264_REPEAT_SPSPPS);
h.entropyCodingMode = if p.profile == 0 { 2 } else { NV_ENC_H264_ENTROPY_CODING_MODE_CABAC };
h.maxNumRefFrames = 0;
// BT.709 limited range, like our own encoder's default signalling
let vui = &mut h.h264VUIParameters;
vui.videoSignalTypePresentFlag = 1;
vui.videoFormat = NV_ENC_VUI_VIDEO_FORMAT_UNSPECIFIED;
vui.videoFullRangeFlag = 0;
vui.colourDescriptionPresentFlag = 1;
vui.colourPrimaries = NV_ENC_VUI_COLOR_PRIMARIES_BT709;
vui.transferCharacteristics = NV_ENC_VUI_TRANSFER_CHARACTERISTIC_BT709;
vui.colourMatrix = NV_ENC_VUI_MATRIX_COEFFS_BT709;
vui.timingInfoPresentFlag = 1;
vui.numUnitInTicks = p.fps.1;
vui.timeScale = p.fps.0.saturating_mul(2);
match p.codec {
Codec::H264 => {
let h = &mut cfg.encodeCodecConfig.h264Config;
h.idrPeriod = p.gop.max(1);
h.level = p.level.map_or(NV_ENC_LEVEL_AUTOSELECT, u32::from);
h.flags &= !(H264_OUTPUT_AUD | H264_REPEAT_SPSPPS);
h.entropyCodingMode = if p.profile == 0 { 2 } else { NV_ENC_H264_ENTROPY_CODING_MODE_CABAC };
h.maxNumRefFrames = 0;
// BT.709 limited range, like our own encoder's default signalling
let vui = &mut h.h264VUIParameters;
vui.videoSignalTypePresentFlag = 1;
vui.videoFormat = NV_ENC_VUI_VIDEO_FORMAT_UNSPECIFIED;
vui.videoFullRangeFlag = 0;
vui.colourDescriptionPresentFlag = 1;
vui.colourPrimaries = NV_ENC_VUI_COLOR_PRIMARIES_BT709;
vui.transferCharacteristics = NV_ENC_VUI_TRANSFER_CHARACTERISTIC_BT709;
vui.colourMatrix = NV_ENC_VUI_MATRIX_COEFFS_BT709;
vui.timingInfoPresentFlag = 1;
vui.numUnitInTicks = p.fps.1;
vui.timeScale = p.fps.0.saturating_mul(2);
}
Codec::Hevc => {
let h = &mut cfg.encodeCodecConfig.hevcConfig;
h.level = p.level.map_or(NV_ENC_LEVEL_AUTOSELECT, u32::from);
h.tier = NV_ENC_TIER_HEVC_MAIN;
h.minCUSize = NV_ENC_HEVC_CUSIZE_AUTOSELECT;
h.maxCUSize = NV_ENC_HEVC_CUSIZE_AUTOSELECT;
h.idrPeriod = p.gop.max(1);
// parameter sets only in the first IDR (and the hvcC), no access unit delimiters, 8-bit 4:2:0
h.flags &= !(HEVC_OUTPUT_AUD
| HEVC_DISABLE_SPSPPS
| HEVC_REPEAT_SPSPPS
| HEVC_ENABLE_INTRA_REFRESH
| HEVC_CHROMA_FORMAT_MASK
| HEVC_PIXEL_BIT_DEPTH_MASK);
h.flags |= 1 << HEVC_CHROMA_FORMAT_SHIFT;
h.maxNumRefFramesInDPB = 0;
// BT.709 limited range, like the H.264 path; an HEVC VUI counts frames, not fields:
// frame rate = time_scale / num_units_in_tick
let vui = &mut h.hevcVUIParameters;
vui.videoSignalTypePresentFlag = 1;
vui.videoFormat = NV_ENC_VUI_VIDEO_FORMAT_UNSPECIFIED;
vui.videoFullRangeFlag = 0;
vui.colourDescriptionPresentFlag = 1;
vui.colourPrimaries = NV_ENC_VUI_COLOR_PRIMARIES_BT709;
vui.transferCharacteristics = NV_ENC_VUI_TRANSFER_CHARACTERISTIC_BT709;
vui.colourMatrix = NV_ENC_VUI_MATRIX_COEFFS_BT709;
vui.timingInfoPresentFlag = 1;
vui.numUnitInTicks = p.fps.1;
vui.timeScale = p.fps.0;
}
}
let mut init: Box<NV_ENC_INITIALIZE_PARAMS> = Box::new(std::mem::zeroed());
init.version = NV_ENC_INITIALIZE_PARAMS_VER;
init.encodeGUID = NV_ENC_CODEC_H264_GUID;
init.encodeGUID = p.codec.guid();
init.presetGUID = NV_ENC_PRESET_P5_GUID;
init.encodeWidth = p.width;
init.encodeHeight = p.height;
@ -363,7 +417,7 @@ impl Session {
self.slots.len()
}
/// The SPS and PPS as one Annex B byte string.
/// The parameter sets as one Annex B byte string (SPS and PPS; VPS, SPS and PPS for HEVC).
pub fn sequence_params(&self) -> Result<Vec<u8>, String> {
let f = self.api.list.nvEncGetSequenceParams.ok_or("no nvEncGetSequenceParams")?;
let mut buf = vec![0u8; 1024];

306
crates/platform/tests/nvenc_hevc.rs Normal file
View File

@ -0,0 +1,306 @@
//! NVENC H.265 (HEVC Main) encoding (Windows, NVIDIA GPU with an HEVC encoder): the stream decodes
//! with our own HEVC decoder to pictures close to the source, keyframes and timestamps are right, the
//! samples carry no parameter sets and the `hvcC` describes the SPS it was built from. Skips
//! without an NVIDIA GPU / driver with HEVC encoding.
#![cfg(target_os = "windows")]
use filmcraft_bitstream::unescape_rbsp;
use filmcraft_hevc::params::Sps;
use filmcraft_isobmff::{HevcConfig, SampleEntry};
use filmcraft_platform::nvenc::{Codec, Config, Nvenc, Packet, Profile, annex_b_to_length_prefixed_for, hevc_available, nal_type};
/// A moving test picture: gradient background, a moving box, a little texture.
fn rgba(w: usize, h: usize, i: usize) -> Vec<u8> {
let mut v = vec![0u8; w * h * 4];
for y in 0..h {
for x in 0..w {
let o = (y * w + x) * 4;
let (bx, by) = ((i * 7) % (w - 64), (i * 3) % (h - 64));
let in_box = x >= bx && x < bx + 64 && y >= by && y < by + 64;
let tex = ((x * 31 + y * 17 + i * 5) % 23) as u8;
v[o] = if in_box { 230 } else { (x * 200 / w) as u8 + tex };
v[o + 1] = if in_box { 40 } else { (y * 200 / h) as u8 + tex };
v[o + 2] = if in_box { 60 } else { 90 + tex };
v[o + 3] = 255;
}
}
v
}
fn config(w: u32, h: u32) -> Config {
Config {
width: w,
height: h,
fps: (24, 1),
bitrate_kbps: 6000,
max_bitrate_kbps: 9000,
cbr: false,
keyint: 24,
profile: Profile::HevcMain,
level: None,
sar: None,
bframes: true,
}
}
/// NVENC sessions are a limited resource on consumer GPUs: one test at a time.
static SERIAL: std::sync::Mutex<()> = std::sync::Mutex::new(());
type Sources = Vec<(Vec<u8>, Vec<u8>, Vec<u8>)>;
fn encode_all(cfg: &Config, frames: usize) -> Option<(Nvenc, Vec<Packet>, Sources)> {
let mut enc = match Nvenc::new(cfg) {
Ok(e) => e,
Err(why) => {
eprintln!("SKIPPED: {why}");
return None;
}
};
let (w, h) = (cfg.width as usize, cfg.height as usize);
let (mut y, mut u, mut v) = (Vec::new(), Vec::new(), Vec::new());
let (mut packets, mut sources) = (Vec::new(), Vec::new());
for i in 0..frames {
filmcraft_export::rgba_to_yuv420_8(&rgba(w, h, i), w, h, &mut y, &mut u, &mut v);
sources.push((y.clone(), u.clone(), v.clone()));
packets.extend(enc.encode(&y, &u, &v, i as u64).unwrap());
}
packets.extend(enc.flush().unwrap());
Some((enc, packets, sources))
}
fn psnr(a: &[u8], b: &[u8]) -> f64 {
let mse = a.iter().zip(b).map(|(x, y)| (f64::from(*x) - f64::from(*y)).powi(2)).sum::<f64>() / a.len() as f64;
if mse == 0.0 { 99.0 } else { 10.0 * (255.0f64 * 255.0 / mse).log10() }
}
/// The NAL unit types of a length-prefixed sample.
fn sample_nal_types(sample: &[u8]) -> Vec<u8> {
let mut out = Vec::new();
let mut pos = 0;
while let Some(len) = sample.get(pos..pos + 4) {
let n = u32::from_be_bytes([len[0], len[1], len[2], len[3]]) as usize;
pos += 4;
let nal = sample.get(pos..pos + n).expect("a NAL unit inside the sample");
out.push(nal_type(Codec::Hevc, nal).expect("a NAL header"));
pos += n;
}
assert_eq!(pos, sample.len(), "samples are made of whole NAL units");
out
}
fn hex(b: &[u8]) -> String {
b.iter().map(|x| format!("{x:02x}")).collect()
}
#[test]
fn the_stream_decodes_to_the_source() {
let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
let (w, h) = (1280u32, 720u32);
let Some((enc, packets, sources)) = encode_all(&config(w, h), 72) else { return };
assert_eq!(enc.codec(), Codec::Hevc);
assert_eq!(packets.len(), 72, "one packet per picture");
// timestamps: decoding order, presentation = frame number, dts = k - delay
let delay = i64::from(enc.delay());
for (k, p) in packets.iter().enumerate() {
assert_eq!(p.dts, k as i64 - delay, "dts of packet {k}");
assert!(p.pts >= p.dts, "pts {} before dts {}", p.pts, p.dts);
}
assert!(packets.windows(2).all(|p| p[0].dts < p[1].dts), "dts strictly increases");
let mut pts: Vec<i64> = packets.iter().map(|p| p.pts).collect();
pts.sort_unstable();
assert_eq!(pts, (0..72).collect::<Vec<_>>(), "every picture once");
// keyframes every 24 pictures (IDR), nothing else flagged
let keys: Vec<i64> = packets.iter().filter(|p| p.key).map(|p| p.pts).collect();
assert_eq!(keys, vec![0, 24, 48], "IDR pictures");
eprintln!("B-frame delay {delay}");
// no parameter sets, delimiters or end markers in the samples; the keyframes are IDR slices
for (k, p) in packets.iter().enumerate() {
let types = sample_nal_types(&p.data);
assert!(!types.is_empty(), "packet {k} has no NAL units");
assert!(types.iter().all(|t| !(32..=37).contains(t)), "packet {k} carries NAL types {types:?}");
if p.key {
assert!(types.iter().any(|t| matches!(t, 19 | 20)), "keyframe {k}: {types:?}");
}
}
// the record describes the SPS it came from
let hvcc = enc.hevc_config().expect("an hvcC record").clone();
let (sps_nal, vps_nal) = (enc.parameter_sets().0.to_vec(), enc.vps().to_vec());
eprintln!("VPS {}\nSPS {}\nPPS {}", hex(&vps_nal), hex(&sps_nal), hex(enc.parameter_sets().1));
let rbsp = unescape_rbsp(&sps_nal[2..]);
let sps = Sps::parse(&rbsp).unwrap();
// the coded size is a whole number of coding tree blocks; the conformance window crops it back
assert!(sps.width >= w && sps.height >= h);
let (_, _, shown_w, shown_h) = sps.crop_rect();
assert_eq!((shown_w, shown_h), (w, h));
assert!(sps.max_num_reorder <= enc.delay(), "SPS reorder {} but the dts shift is {}", sps.max_num_reorder, enc.delay());
assert_eq!(&hvcc.to_bytes()[1..13], &rbsp[1..13], "the record's profile / tier / level bytes are the SPS's");
assert_eq!(HevcConfig::parse(&hvcc.to_bytes()).unwrap(), hvcc);
assert_eq!((hvcc.general_profile_idc, hvcc.general_tier_flag, hvcc.chroma_format_idc, hvcc.bit_depth_luma, hvcc.bit_depth_chroma), (1, false, 1, 8, 8));
assert_eq!(usize::from(hvcc.num_temporal_layers), sps.max_sub_layers_minus1 as usize + 1);
assert!(hvcc.arrays.iter().all(|a| a.completeness && a.nalus.len() == 1));
assert_eq!(hvcc.arrays.iter().map(|a| a.nal_type).collect::<Vec<_>>(), vec![32, 33, 34]);
eprintln!(
"hvcC: level {} compat {:#010x} constraints {:#014x} nested {} layers {}",
hvcc.general_level_idc,
hvcc.general_profile_compatibility_flags,
hvcc.general_constraint_indicator_flags,
hvcc.temporal_id_nested,
hvcc.num_temporal_layers
);
// BT.709 limited range, timing of the frame rate (a frame count per tick, not fields)
let vui = sps.vui.clone().expect("a VUI");
assert_eq!((vui.colour_primaries, vui.transfer_characteristics, vui.matrix_coefficients, vui.full_range), (1, 1, 1, false));
assert_eq!(vui.timing, Some((1, 24)), "num_units_in_tick, time_scale");
// our own decoder decodes the samples
let entry = SampleEntry::hevc(hvcc, w as u16, h as u16);
let mut dec = filmcraft_codecs::software_video_decoder(&entry).unwrap();
let mut out = Vec::new();
for p in &packets {
out.extend(dec.decode(&p.data, p.pts).unwrap());
}
out.extend(dec.flush());
assert_eq!(out.len(), 72);
let mut worst = 99.0f64;
let mut worst_chroma = 99.0f64;
for f in &out {
let src = &sources[f.pts as usize];
let filmcraft_frame::PixelData::Yuv8 { planes, .. } = &f.frame.data else { panic!("8-bit planes") };
worst = worst.min(psnr(&planes[0], &src.0));
worst_chroma = worst_chroma.min(psnr(&planes[1], &src.1)).min(psnr(&planes[2], &src.2));
}
eprintln!("worst luma PSNR {worst:.1} dB, worst chroma {worst_chroma:.1} dB, {} bytes", packets.iter().map(|p| p.data.len()).sum::<usize>());
assert!(worst > 43.0, "luma PSNR {worst:.1} dB");
assert!(worst_chroma > 45.0, "chroma PSNR {worst_chroma:.1} dB");
}
#[test]
fn nothing_to_encode_with_is_an_error_not_a_crash() {
let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if !hevc_available() {
eprintln!("SKIPPED: no NVENC HEVC here");
return;
}
// sizes NVENC cannot take, a zero frame rate, an HEVC level that does not exist
for (w, h) in [(0, 0), (641, 360), (640, 361), (16, 16), (u32::MAX, u32::MAX), (100_000, 64)] {
assert!(Nvenc::new(&config(w, h)).is_err(), "{w}x{h}");
}
let mut c = config(640, 360);
c.fps = (0, 1);
assert!(Nvenc::new(&c).is_err());
c.fps = (24, 0);
assert!(Nvenc::new(&c).is_err());
c = config(640, 360);
c.level = Some(42);
assert!(Nvenc::new(&c).is_err());
// a level that exists is taken
c.level = Some(41);
let enc = Nvenc::new(&c).expect("level 4.1");
assert_eq!(enc.hevc_config().map(|h| h.general_level_idc), Some(123));
// pictures smaller than the encoder's size
let mut enc = Nvenc::new(&config(640, 360)).unwrap();
assert!(enc.encode(&[0; 10], &[0; 10], &[0; 10], 0).is_err());
}
#[test]
fn hostile_annex_b_never_panics() {
for s in [&[][..], &[0, 0, 1], &[1, 2, 3], &[0, 0, 1, 0x46, 0x01, 0x50], &[0, 0, 0, 1, 0x40], &[0, 0, 0, 1, 0x40, 0x01]] {
assert!(annex_b_to_length_prefixed_for(Codec::Hevc, s).is_err(), "{s:?}");
}
}
/// Run `f`, failing the test (not the process) if it panics.
fn calm<T>(what: &str, f: impl FnOnce() -> T) -> T {
match std::panic::catch_unwind(std::panic::AssertUnwindSafe(f)) {
Ok(v) => v,
Err(_) => panic!("{what}: panicked"),
}
}
#[test]
fn hostile_configurations_give_errors_not_crashes() {
let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if !hevc_available() {
eprintln!("SKIPPED: no NVENC HEVC here");
return;
}
let base = config(640, 360);
let variants: Vec<(&str, Config)> = vec![
("zero bitrate", Config { bitrate_kbps: 0, max_bitrate_kbps: 0, ..base.clone() }),
("huge bitrate", Config { bitrate_kbps: u32::MAX, max_bitrate_kbps: u32::MAX, ..base.clone() }),
("max below target", Config { bitrate_kbps: 6000, max_bitrate_kbps: 1, ..base.clone() }),
("zero cbr", Config { cbr: true, bitrate_kbps: 0, ..base.clone() }),
("cbr", Config { cbr: true, bitrate_kbps: 4000, ..base.clone() }),
("zero keyint", Config { keyint: 0, ..base.clone() }),
("keyint 2", Config { keyint: 2, ..base.clone() }),
("huge keyint", Config { keyint: u32::MAX, ..base.clone() }),
("keyint 1 (all intra)", Config { keyint: 1, ..base.clone() }),
("huge frame rate", Config { fps: (u32::MAX, 1), ..base.clone() }),
("huge frame duration", Config { fps: (1, u32::MAX), ..base.clone() }),
("huge aspect ratio", Config { sar: Some((u32::MAX, u32::MAX)), ..base.clone() }),
("zero aspect ratio", Config { sar: Some((0, 0)), ..base.clone() }),
("no B-frames", Config { bframes: false, ..base.clone() }),
("smallest size", Config { width: 130, height: 34, ..base.clone() }),
("odd size", Config { width: 321, height: 181, ..base.clone() }),
];
for (name, c) in variants {
match calm(name, || Nvenc::new(&c)) {
Ok(mut enc) => {
let (w, h) = (c.width as usize, c.height as usize);
let (cw, ch) = (w / 2, h / 2);
assert!(calm(name, || enc.encode(&[], &[], &[], 0)).is_err() || w == 0, "{name}: empty planes");
assert!(calm(name, || enc.encode(&vec![0; w * h - 1], &vec![128; cw * ch], &vec![128; cw * ch], 0)).is_err(), "{name}: short luma");
// a few real pictures, then the end; whatever comes out, nothing panics
for i in 0..3u8 {
let _ = calm(name, || enc.encode(&vec![16 + i; w * h], &vec![128; cw * ch], &vec![128; cw * ch], i as u64));
}
let _ = calm(name, || enc.flush());
}
Err(why) => eprintln!("{name}: declined ({why})"),
}
}
}
#[test]
fn an_encoder_dropped_mid_stream_releases_everything() {
let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if !hevc_available() {
eprintln!("SKIPPED: no NVENC HEVC here");
return;
}
// more sessions than a consumer GPU allows at once would fail the later ones: drop each early
for round in 0..12 {
let mut enc = Nvenc::new(&config(640, 360)).unwrap_or_else(|e| panic!("round {round}: {e}"));
let (mut y, mut u, mut v) = (Vec::new(), Vec::new(), Vec::new());
for i in 0..5 {
filmcraft_export::rgba_to_yuv420_8(&rgba(640, 360, i), 640, 360, &mut y, &mut u, &mut v);
enc.encode(&y, &u, &v, i as u64).unwrap();
}
// dropped with pictures in flight and no flush
}
}
#[test]
fn a_keyframe_every_one_or_two_pictures_is_written_without_b_frames() {
let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
// NVENC wants a GOP longer than the B-frame pattern: short GOPs drop the B-frames, they do not fail
for keyint in [1u32, 2] {
let cfg = Config { keyint, ..config(640, 360) };
let Some((enc, packets, _)) = encode_all(&cfg, 12) else { return };
assert_eq!(enc.delay(), 0, "keyint {keyint}");
assert_eq!(packets.len(), 12);
assert!(packets.iter().enumerate().all(|(k, p)| p.pts == k as i64 && p.dts == p.pts), "keyint {keyint}: no reordering");
let keys: Vec<i64> = packets.iter().filter(|p| p.key).map(|p| p.pts).collect();
assert_eq!(keys, (0..12).step_by(keyint as usize).collect::<Vec<i64>>(), "keyint {keyint}");
let entry = SampleEntry::hevc(enc.hevc_config().expect("an hvcC record").clone(), 640, 360);
let mut dec = filmcraft_codecs::software_video_decoder(&entry).unwrap();
let mut n = 0;
for p in &packets {
n += dec.decode(&p.data, p.pts).unwrap().len();
}
assert_eq!(n + dec.flush().len(), 12, "keyint {keyint}");
}
}

303
crates/platform/tests/nvenc_hevc_export.rs Normal file
View File

@ -0,0 +1,303 @@
//! Export as H.265 (HEVC Main, NVENC) through the real export pipeline: MP4 and QuickTime files that
//! ffprobe (an external oracle only) describes as `hvc1` Main 8-bit 4:2:0 BT.709, that ffmpeg decodes
//! without errors and that our own decoder decodes to the pictures of the software H.264 export; the
//! counters; and what NVENC declines (an error that says why, counted, never a crash). Its own test
//! binary: the encoder registry and the counters are process-wide. Skips without an NVIDIA GPU with
//! an HEVC encoder.
#![cfg(target_os = "windows")]
use std::path::Path;
use std::process::Command;
use std::sync::Arc;
use filmcraft_export::{
BitrateMode, ColorSignal, ExportSettings, FieldOrder, Format, H264Pass, HardwareEncoding, Multiplexer, Progress, export, hw_encode_stats,
};
use filmcraft_media::generators::GeneratorSource;
use filmcraft_media::{DemoScene, FrameRequest, Generator, MediaSource};
use filmcraft_platform::nvenc::export::factory;
use filmcraft_platform::nvenc::hevc_available;
use filmcraft_project::{ItemKind, Label, MediaClip, MediaRef, Project, SequenceSettings, TrackKind};
use filmcraft_render::SourceMap;
use filmcraft_time::{FrameRate, Tick, TimeRange};
/// The encoder counters are process-wide: one test at a time.
static SERIAL: std::sync::Mutex<()> = std::sync::Mutex::new(());
fn project(w: u32, h: u32, frames: i64) -> (Arc<Project>, filmcraft_project::ItemId, SourceMap) {
let rate = FrameRate::FPS_24;
let mut p = Project::new("x");
let g = GeneratorSource::new(Generator::Demo(DemoScene::OceanSunset), w, h, rate, rate.tick_of(frames));
let info = g.info().clone();
let clip = p.add_item(
&info.name.clone(),
Label::Iris,
ItemKind::Media(MediaClip {
media: MediaRef::Generator(g.generator.clone()),
info,
interpret: Default::default(),
mark_in: None,
mark_out: None,
markers: vec![],
offline: false,
proxy: None,
identity: None,
}),
None,
);
let seq = p.new_sequence("s", SequenceSettings { width: w, height: h, frame_rate: rate, ..Default::default() }, 1, 1, None);
let v = p.make_track_item(clip, TrackKind::Video, Tick::ZERO, TimeRange::new(Tick::ZERO, rate.tick_of(frames)), rate).unwrap();
p.sequence_mut(seq).unwrap().video_tracks[0].items.push(v);
let mut m = SourceMap::default();
m.0.insert(clip, Arc::new(g));
(Arc::new(p), seq, m)
}
fn tmp(name: &str) -> String {
let d = std::env::temp_dir().join(format!("fc-nvenc-hevc-{}", std::process::id()));
std::fs::create_dir_all(&d).unwrap();
d.join(name).to_string_lossy().to_string()
}
fn decode(path: &str) -> Vec<Arc<filmcraft_frame::VideoFrame>> {
let bytes: Arc<[u8]> = std::fs::read(path).unwrap().into();
let src = filmcraft_codecs::open_bytes(path, bytes).unwrap();
let rate = src.info().frame_rate();
let n = rate.frame_at(src.info().duration);
(0..n).map(|i| src.video_frame(FrameRequest::full(rate.tick_of(i))).unwrap()).collect()
}
fn luma_psnr(a: &filmcraft_frame::VideoFrame, b: &filmcraft_frame::VideoFrame) -> f64 {
let (x, y) = (a.luma8(), b.luma8());
let mse = x.iter().zip(&y).map(|(p, q)| (f64::from(*p) - f64::from(*q)).powi(2)).sum::<f64>() / x.len() as f64;
if mse == 0.0 { 99.0 } else { 10.0 * (255.0f64 * 255.0 / mse).log10() }
}
fn hevc(path: String) -> ExportSettings {
ExportSettings { format: Format::Hevc, path, include_audio: false, bitrate_kbps: 4000, bitrate_mode: BitrateMode::Vbr1Pass, ..Default::default() }
}
/// `key=value` lines of `ffprobe -show_entries` for the first video stream.
fn probe(ffprobe: &Path, path: &str) -> std::collections::HashMap<String, String> {
let entries = "stream=codec_name,profile,codec_tag_string,pix_fmt,color_space,color_transfer,color_primaries,color_range,width,height,nb_frames,r_frame_rate,start_time,level";
let out = Command::new(ffprobe).args(["-v", "error", "-select_streams", "v:0", "-show_entries", entries, "-of", "default=nw=1", path]).output().unwrap();
assert!(out.status.success(), "ffprobe: {}", String::from_utf8_lossy(&out.stderr));
String::from_utf8_lossy(&out.stdout).lines().filter_map(|l| l.split_once('=')).map(|(k, v)| (k.to_string(), v.to_string())).collect()
}
/// Presentation times (in frames at 24 fps) of the packets ffprobe flags as keyframes.
fn keyframes(ffprobe: &Path, path: &str) -> Vec<i64> {
let out = Command::new(ffprobe)
.args(["-v", "error", "-select_streams", "v:0", "-show_entries", "packet=pts_time,flags", "-of", "csv=p=0", path])
.output()
.unwrap();
let mut keys: Vec<i64> = String::from_utf8_lossy(&out.stdout)
.lines()
.filter_map(|l| l.split_once(','))
.filter(|(_, flags)| flags.starts_with('K'))
.filter_map(|(t, _)| t.parse::<f64>().ok())
.map(|t| (t * 24.0).round() as i64)
.collect();
keys.sort_unstable();
keys
}
/// ffmpeg decodes the whole file with `-xerror` and says nothing.
fn assert_ffmpeg_decodes(ffmpeg: &Path, path: &str) {
let out = Command::new(ffmpeg).args(["-v", "error", "-xerror", "-i", path, "-f", "null", "-"]).output().unwrap();
assert!(out.status.success() && out.stderr.is_empty(), "ffmpeg: {}", String::from_utf8_lossy(&out.stderr));
}
fn delta(before: filmcraft_export::HwEncodeStats) -> (u64, u64, u64) {
let after = hw_encode_stats();
(after.frames - before.frames, after.sessions - before.sessions, after.declined - before.declined)
}
#[test]
fn hevc_exports_play_everywhere() {
let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
filmcraft_platform::register();
assert_eq!(filmcraft_export::available(Format::Hevc), hevc_available(), "the format is available iff NVENC can encode HEVC");
if !hevc_available() {
eprintln!("SKIPPED: no NVENC HEVC here");
return;
}
let (p, seq, m) = project(1280, 720, 72);
// the software H.264 export of the same frames is the reference for our decoder's pictures
let soft_path = tmp("soft.mp4");
let soft = ExportSettings { format: Format::H264, hardware_encoding: HardwareEncoding::Off, ..hevc(soft_path.clone()) };
let before = hw_encode_stats();
export(&p, seq, &soft, &m, &Progress::default()).unwrap();
assert_eq!(delta(before), (0, 0, 0), "H.264 with hardware encoding Off never touches NVENC");
let reference = decode(&soft_path);
let ffprobe = filmcraft_testkit::ffprobe_or_skip("nvenc hevc export");
let ffmpeg = filmcraft_testkit::ffmpeg_or_skip("nvenc hevc export");
for (name, multiplexer, toggle) in [
("hevc.mp4", Multiplexer::Mp4, HardwareEncoding::Auto),
("hevc.mov", Multiplexer::Mov, HardwareEncoding::Auto),
// HEVC has no software encoder: the toggle does not apply
("hevc-off.mp4", Multiplexer::Mp4, HardwareEncoding::Off),
] {
let path = tmp(name);
let s = ExportSettings { multiplexer, hardware_encoding: toggle, ..hevc(path.clone()) };
let before = hw_encode_stats();
let report = export(&p, seq, &s, &m, &Progress::default()).unwrap_or_else(|e| panic!("{name}: {e}"));
assert_eq!(report.frames, 72, "{name}");
assert_eq!(delta(before), (72, 1, 0), "{name}: every picture through one NVENC session, nothing declined");
// our own decoder: 72 pictures close to the software export's
let pictures = decode(&path);
assert_eq!(pictures.len(), 72, "{name}");
let worst = pictures.iter().zip(&reference).map(|(a, b)| luma_psnr(a, b)).fold(99.0, f64::min);
eprintln!(
"{name}: {} bytes (software H.264 {}), worst luma PSNR against the software export {worst:.1} dB",
std::fs::metadata(&path).unwrap().len(),
std::fs::metadata(&soft_path).unwrap().len()
);
assert!(worst > 48.0, "{name}: {worst:.1} dB");
let (Some(ffprobe), Some(ffmpeg)) = (&ffprobe, &ffmpeg) else { continue };
let v = probe(ffprobe, &path);
let get = |k: &str| v.get(k).map(String::as_str).unwrap_or("?");
assert_eq!((get("codec_name"), get("profile"), get("codec_tag_string"), get("pix_fmt")), ("hevc", "Main", "hvc1", "yuv420p"), "{name}: {v:?}");
assert_eq!((get("color_space"), get("color_transfer"), get("color_primaries"), get("color_range")), ("bt709", "bt709", "bt709", "tv"), "{name}: {v:?}");
assert_eq!((get("width"), get("height"), get("nb_frames"), get("r_frame_rate")), ("1280", "720", "72", "24/1"), "{name}: {v:?}");
assert!(get("start_time").parse::<f64>().is_ok_and(|t| t.abs() < 1e-3), "{name}: starts at {}", get("start_time"));
assert_eq!(keyframes(ffprobe, &path), vec![0, 48], "{name}: keyframes every 2 s (the default distance)");
assert_ffmpeg_decodes(ffmpeg, &path);
eprintln!("{name}: ffprobe {v:?}, ffmpeg -xerror decodes it cleanly");
}
}
#[test]
fn hevc_export_with_audio() {
let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
filmcraft_platform::register();
if !hevc_available() {
eprintln!("SKIPPED: no NVENC HEVC here");
return;
}
let (p, seq, m) = project(640, 360, 48);
let path = tmp("hevc-audio.mp4");
let s = ExportSettings { include_audio: true, keyframe_distance: Some(24), ..hevc(path.clone()) };
export(&p, seq, &s, &m, &Progress::default()).expect("export");
let bytes = std::fs::read(&path).unwrap();
let file = filmcraft_isobmff::open(bytes.as_slice()).unwrap();
assert!(file.track_of_kind(filmcraft_isobmff::TrackKind::Video).is_some());
let at = file.track_of_kind(filmcraft_isobmff::TrackKind::Audio).expect("an audio track");
assert!(matches!(&file.tracks[at].entries[0].codec, filmcraft_isobmff::CodecConfig::Aac(_)));
if let (Some(ffprobe), Some(ffmpeg)) = (filmcraft_testkit::ffprobe_or_skip("nvenc hevc export"), filmcraft_testkit::ffmpeg_or_skip("nvenc hevc export")) {
let out = Command::new(&ffprobe)
.args(["-v", "error", "-show_entries", "stream=codec_name,codec_type,codec_tag_string", "-of", "csv=p=0", &path])
.output()
.unwrap();
let streams = String::from_utf8_lossy(&out.stdout).to_string();
assert!(streams.contains("hevc,video,hvc1") && streams.contains("aac,audio,mp4a"), "{streams}");
assert_ffmpeg_decodes(&ffmpeg, &path);
assert_eq!(keyframes(&ffprobe, &path), vec![0, 24]);
}
}
#[test]
fn sizes_that_are_not_multiples_of_the_coding_block_are_cropped_back() {
let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
filmcraft_platform::register();
if !hevc_available() {
eprintln!("SKIPPED: no NVENC HEVC here");
return;
}
// 1080 is not a multiple of the 32-pixel coding tree block (the SPS carries a conformance window);
// 642x362 is even but a multiple of nothing
for (w, h) in [(1920u32, 1080u32), (642, 362)] {
let (p, seq, m) = project(w, h, 12);
let path = tmp(&format!("hevc-{w}x{h}.mp4"));
let before = hw_encode_stats();
export(&p, seq, &hevc(path.clone()), &m, &Progress::default()).unwrap_or_else(|e| panic!("{w}x{h}: {e}"));
assert_eq!(delta(before), (12, 1, 0), "{w}x{h}");
let pictures = decode(&path);
assert_eq!(pictures.len(), 12);
assert!(pictures.iter().all(|f| (f.width, f.height) == (w, h)), "{w}x{h}: decoded {}x{}", pictures[0].width, pictures[0].height);
if let (Some(ffprobe), Some(ffmpeg)) = (filmcraft_testkit::ffprobe_or_skip("nvenc hevc export"), filmcraft_testkit::ffmpeg_or_skip("nvenc hevc export"))
{
let v = probe(&ffprobe, &path);
assert_eq!((v["width"].as_str(), v["height"].as_str()), (w.to_string().as_str(), h.to_string().as_str()), "{v:?}");
assert_ffmpeg_decodes(&ffmpeg, &path);
}
}
}
#[test]
fn what_nvenc_cannot_do_is_an_error_that_says_why() {
let _one = SERIAL.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
filmcraft_platform::register();
let rate = FrameRate::FPS_24;
let base = hevc(tmp("declined.mp4"));
// (what, settings, size): each declined by the configuration alone, so even a machine without
// NVENC counts and explains it
let cases: Vec<(&str, ExportSettings, (u32, u32))> = vec![
("HDR (PQ)", ExportSettings { signal: ColorSignal::PQ, ..base.clone() }, (1280, 720)),
("HDR (HLG)", ExportSettings { signal: ColorSignal::HLG, ..base.clone() }, (1280, 720)),
("an analysis pass", ExportSettings { h264_pass: H264Pass::First, ..base.clone() }, (1280, 720)),
("two-pass VBR", ExportSettings { bitrate_mode: BitrateMode::Vbr2Pass, ..base.clone() }, (1280, 720)),
("non-square pixels", ExportSettings { pixel_aspect: Some((4, 3)), ..base.clone() }, (1280, 720)),
("MXF", ExportSettings { format: Format::MxfOp1a, ..base.clone() }, (1280, 720)),
("interlaced output", ExportSettings { field_order: FieldOrder::UpperFirst, ..base.clone() }, (1280, 720)),
("wider than an MP4 sample entry", base.clone(), (70_000, 64)),
("taller than an MP4 sample entry", base.clone(), (64, 70_000)),
];
for (what, s, (w, h)) in cases {
let before = hw_encode_stats();
let r = factory(Format::Hevc, w, h, rate, &s);
let Some(Err(e)) = r else { panic!("{what}: expected a declined request, got {}", if r.is_some() { "an encoder" } else { "None" }) };
assert!(e.to_string().contains("NVENC"), "{what}: {e}");
assert_eq!(delta(before), (0, 0, 1), "{what}: counted as declined, no session");
}
// interlaced output is refused by the export's own validation, before any encoder is asked
let (p, seq, m) = project(640, 360, 6);
let before = hw_encode_stats();
let s = ExportSettings { field_order: FieldOrder::UpperFirst, ..hevc(tmp("interlaced.mp4")) };
assert!(export(&p, seq, &s, &m, &Progress::default()).is_err());
assert_eq!(delta(before), (0, 0, 0));
if !hevc_available() {
// no HEVC encoder: the export's own "encoder not available" error, nothing written, no panic
let err = export(&p, seq, &hevc(tmp("none.mp4")), &m, &Progress::default()).expect_err("no encoder").to_string();
assert!(err.contains("encoder not available"), "{err}");
eprintln!("SKIPPED the NVENC-limit cases: no NVENC HEVC here");
return;
}
// sizes outside what the GPU's HEVC encoder takes, through the real export
let (p, seq, m) = project(640, 360, 6);
for (w, h) in [(64u32, 64u32), (16, 16), (10_000, 64)] {
let path = tmp(&format!("small-{w}x{h}.mp4"));
let s = ExportSettings { frame_size: Some((w, h)), ..hevc(path.clone()) };
let before = hw_encode_stats();
let r = export(&p, seq, &s, &m, &Progress::default());
match r {
Err(e) => {
let e = e.to_string();
eprintln!("{w}x{h}: {e}");
assert!(e.contains("NVENC"), "{w}x{h}: {e}");
assert_eq!(delta(before), (0, 0, 1), "{w}x{h}");
}
// a GPU that does take the size: a good file
Ok(_) => assert_eq!(delta(before).1, 1, "{w}x{h}"),
}
}
// the factory itself: an odd size and a size outside the limits are declined, a good one is taken
let before = hw_encode_stats();
assert!(matches!(factory(Format::Hevc, 641, 360, rate, &base), Some(Err(_))));
assert!(matches!(factory(Format::Hevc, 20_000, 20_000, rate, &base), Some(Err(_))));
assert_eq!(delta(before), (0, 0, 2));
let before = hw_encode_stats();
assert!(matches!(factory(Format::Hevc, 640, 360, rate, &base), Some(Ok(_))));
assert_eq!(delta(before), (0, 1, 0));
// H.264 with hardware encoding Off is still not NVENC's business, HEVC's toggle does not matter
let before = hw_encode_stats();
let off = ExportSettings { format: Format::H264, hardware_encoding: HardwareEncoding::Off, ..base.clone() };
assert!(factory(Format::H264, 640, 360, rate, &off).is_none());
assert!(matches!(factory(Format::Hevc, 640, 360, rate, &ExportSettings { hardware_encoding: HardwareEncoding::Off, ..base }), Some(Ok(_))));
assert_eq!(delta(before), (0, 1, 0));
}

29
crates/platform/tests/nvenc_hevc_probe.rs Normal file
View File

@ -0,0 +1,29 @@
//! The HEVC availability probe of the NVENC backend: it opens one small encoder session, once. Its own
//! test binary so that "first call" means a fresh process (driver load, Direct3D 11 device, NVENC
//! session and encoder initialisation all included).
#![cfg(target_os = "windows")]
use std::time::{Duration, Instant};
use filmcraft_export::Format;
use filmcraft_platform::nvenc::hevc_available;
#[test]
fn the_probe_runs_once_and_registers_the_format() {
let t0 = Instant::now();
let first = hevc_available();
let cold = t0.elapsed();
let t1 = Instant::now();
let again = hevc_available();
let warm = t1.elapsed();
eprintln!("first HEVC probe in a fresh process: {cold:?} (available: {first}); the cached answer: {warm:?}");
assert_eq!(first, again);
assert!(warm < Duration::from_millis(50), "the answer is cached: {warm:?}");
// `register` makes `available(Hevc)` follow the probe; registering again changes nothing
assert!(!filmcraft_export::available(Format::Hevc) || first);
filmcraft_platform::register();
filmcraft_platform::register();
assert_eq!(filmcraft_export::available(Format::Hevc), first);
assert!(filmcraft_export::available(Format::H264));
}

4
docs/adr/0001-platform-ffi.md
View File

@ -28,7 +28,9 @@ Containment rules:
`clippy::undocumented_unsafe_blocks = "deny"`. Only the FFI modules (`videotoolbox`, `media_foundation::gpu` / `media_foundation::mft`, and
`nvenc::ffi` / `nvenc::session` on Windows) carry `#[allow(unsafe_code)]`; the rest of the crate
(the fallback logic in `hybrid`, the decoder logic in `media_foundation`, `annexb`, `biplanar`,
the encoder logic in `nvenc`) has no `unsafe`.
the encoder logic in `nvenc`, H.264 and H.265 alike) has no `unsafe`. NVENC's H.265 support
added no `unsafe` module: it reuses `nvenc::ffi` and `nvenc::session` (new data declarations and
a second codec-configuration member of an existing union).
2. Every `unsafe` block has a `// SAFETY:` comment saying why it is sound.
3. The public API is safe: no `pub unsafe fn`, no raw pointers or FFI types in public signatures;
failures are `Result`s. The crate keeps the never-crash `deny(clippy::unwrap_used, …)`

8
docs/architecture.md
View File

@ -72,7 +72,7 @@ and `filmcraft-cli`.
| `engine` | L4 | `Session`, command registry, undo history, media pool, jobs, interchange glue |
| `ui-egui` | L5 | the egui frontend: docking, panels, timeline, monitors, playback, control-channel handlers |
| `automation` | L5 | MCP server (`rmcp`, stdio), headless or bridged to the running app |
| `platform` | L5 | OS media FFI only: hardware video decoding (VideoToolbox H.264 / HEVC on macOS; Media Foundation / Direct3D 11 H.264 / HEVC on Windows; a no-op elsewhere) behind `codecs::VideoDecoder`, with transparent fallback to our decoders, and hardware H.264 encoding (VideoToolbox, opt-in) and H.265 encoding (VideoToolbox, the only H.265 encoder; the format exists only where a hardware encoder does) and NVIDIA NVENC H.264 encoding (Windows, opt-in) behind `export::VideoEncoder`; H.264 declines to the built-in encoder for what the hardware does not take. The one crate allowed `unsafe` ([ADR 0001](adr/0001-platform-ffi.md), [README](../crates/platform/README.md)) |
| `platform` | L5 | OS media FFI only: hardware video decoding (VideoToolbox H.264 / HEVC on macOS; Media Foundation / Direct3D 11 H.264 / HEVC on Windows; a no-op elsewhere) behind `codecs::VideoDecoder`, with transparent fallback to our decoders, and hardware H.264 encoding (VideoToolbox, opt-in) and H.265 encoding (VideoToolbox, the only H.265 encoder; the format exists only where a hardware encoder does) and NVIDIA NVENC H.264 (Windows, opt-in) and H.265 (Windows, the only H.265 encoder there) encoding behind `export::VideoEncoder`; H.264 declines to the built-in encoder for what the hardware does not take. The one crate allowed `unsafe` ([ADR 0001](adr/0001-platform-ffi.md), [README](../crates/platform/README.md)) |
| `filmcraft` | L6 | desktop binary: eframe/wgpu window, cpal audio output, file dialogs, native macOS menu, TCP control server |
| `filmcraft-cli` | L6 | headless CLI: `exec`, `run`, `inspect`, `describe`, `commands`, `import`, `export`, `render`, `probe`, `mcp`; `--bridge` targets the running app |
| `filmcraft-web` | L6 | the browser app (wasm32): eframe web runner on WebGPU/WebGL2, Blob-backed services, OPFS recovery, WebAudio, WebCodecs, `window.filmcraft` API ([web.md](web.md)) |
@ -273,7 +273,11 @@ file ──► codecs (MP4/MOV, MKV, audio) demux + decode, GOP-aware see
(the software encoder runs, counted in `export.hardware.declined`) two-pass VBR, HDR, MXF,
interlaced output, sizes outside NVENC's limits, and systems without an NVIDIA GPU or driver.
A failure during an export ends it with an error: the software encoder cannot take over a hardware
stream.
stream. The same NVENC session also encodes H.265 (HEVC Main, 8-bit 4:2:0, SDR; the `hvcC` is built
from the SPS the encoder wrote): `register()` hands `platform::nvenc::hevc_available` to
`filmcraft_export::register_format_probe`, so the H.265 format exists where an HEVC encoder does,
and choosing it is the opt-in (the Hardware encoding toggle governs H.264 only). With no software
H.265 encoder, what NVENC declines is counted and ends the export with an error that says why.
- **Compositor.** `render` is the reference for monitors, thumbnails and export. `render::plan`
turns a frame into GPU layers, each with its opacity and blend mode. All 27 blend modes run on the
GPU (`filmcraft-gpu`): Normal and Dissolve with fixed-function "over" blending, the others by

17
docs/testing.md
View File

@ -71,11 +71,18 @@ are never linked or shipped ([AGENTS.md](../AGENTS.md) §2).
without a Direct3D 11 video device or the HEVC / VP9 / AV1 codec extensions of the Microsoft Store. The VP9 and
AV1 fixtures (libvpx-vp9, libaom-av1; 360p, 1080p, 2160p, hidden alt-ref frames, two GOPs) need an ffmpeg with those encoders.
The NVENC tests (`crates/platform/tests/nvenc.rs` and `nvenc_export.rs`, Windows only) skip without an
NVIDIA GPU with NVENC. The FFI layout tests in `crates/platform/src/nvenc/abi_tests.rs` were generated
from a C program built with MSVC against NVIDIA's MIT-licensed `nvEncodeAPI.h` (12.1); to regenerate
them, print the sizes, alignments, offsets, constants and GUIDs of `src/nvenc/ffi.rs` from that
program and update the asserts.
The NVENC tests (`crates/platform/tests/nvenc.rs` and `nvenc_export.rs` for H.264, `nvenc_hevc.rs`,
`nvenc_hevc_export.rs` and `nvenc_hevc_probe.rs` for H.265; Windows only) skip, printing `SKIPPED`,
without an NVIDIA GPU with NVENC (H.265: with an HEVC encoder). The FFI layout tests in
`crates/platform/src/nvenc/abi_tests.rs` were generated from a C program built with MSVC
(`cl` after `vcvars64.bat`) against NVIDIA's MIT-licensed `nvEncodeAPI.h` (12.1); the header and the
program are not in the repository. To regenerate them, put the header in a scratch directory and
write a program that prints, for every type, field, constant and GUID of `src/nvenc/ffi.rs` that the
asserts name (H.264 and HEVC), `sizeof` / `alignof` / `offsetof`, the enum and macro values cast to
64-bit integers and the GUIDs as 128-bit hex; for the bit-fields of the `flags` words, which
`offsetof` cannot take, zero a structure, set that one field to its maximum and print the 32-bit word.
Compare the numbers with the asserts (a difference is drift, which must be understood before
anything is "fixed" to match) and add asserts for anything new.
### Pass criteria per codec