From ea79751a5c19c3fe312daad290c7c8fbc703fd24 Mon Sep 17 00:00:00 2001 From: storybored379 Date: Thu, 8 Oct 2026 20:40:01 -0700 Subject: [PATCH] platform: enable existing NVENC H.264 export on Linux Runtime-load the NVIDIA encode and CUDA driver libraries, retaining the primary context through session destruction. Preserve the Windows Direct3D path and default-off selection, and test driver rejection and deterministic software fallback. Co-Authored-By: OpenAI Codex --- Cargo.lock | 1 + crates/platform/Cargo.toml | 3 + crates/platform/README.md | 25 +++-- crates/platform/src/lib.rs | 4 +- crates/platform/src/nvenc/device.rs | 130 ++++++++++++++++++++++++++ crates/platform/src/nvenc/export.rs | 57 ++++++++++- crates/platform/src/nvenc/ffi.rs | 46 ++++++--- crates/platform/src/nvenc/mod.rs | 10 +- crates/platform/src/nvenc/session.rs | 81 +++++++--------- crates/platform/tests/nvenc.rs | 2 +- crates/platform/tests/nvenc_export.rs | 14 ++- docs/adr/0001-platform-ffi.md | 12 ++- docs/architecture.md | 4 +- 13 files changed, 308 insertions(+), 81 deletions(-) create mode 100644 crates/platform/src/nvenc/device.rs diff --git a/Cargo.lock b/Cargo.lock index 900806b9..8798f26a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2107,6 +2107,7 @@ dependencies = [ "filmcraft-render", "filmcraft-testkit", "filmcraft-time", + "libloading", "log", "objc2-core-foundation", "objc2-core-media", diff --git a/crates/platform/Cargo.toml b/crates/platform/Cargo.toml index f52c1290..f102eb14 100644 --- a/crates/platform/Cargo.toml +++ b/crates/platform/Cargo.toml @@ -49,6 +49,9 @@ windows = { version = "0.62", features = [ "Win32_System_LibraryLoader", ] } +[target.'cfg(all(target_os = "linux", target_pointer_width = "64"))'.dependencies] +libloading = "0.8" + [dev-dependencies] filmcraft-testkit = { workspace = true } filmcraft-media = { workspace = true } diff --git a/crates/platform/README.md b/crates/platform/README.md index eb361484..9f1be37c 100644 --- a/crates/platform/README.md +++ b/crates/platform/README.md @@ -61,9 +61,11 @@ let availability = filmcraft_platform::register(); // Available("VideoToolbox") - After a `flush` (which drains the MFT) the MFT only restarts at an IDR picture; the GOP cache always seeks after a flush, and a caller that continues from the middle of a GOP gets an error that `HybridDecoder` answers by replaying the run in software. -- **Windows: NVIDIA NVENC H.264 encoding** (`nvenc/`), 8-bit SDR 4:2:0 for Export. The driver's - `nvEncodeAPI64.dll` (API 12.1, no CUDA or SDK) is loaded at run time, so machines without NVIDIA - still start. RGBA is converted with the software encoder's own BT.709 limited conversion into NV12 +- **Windows and 64-bit Linux: NVIDIA NVENC H.264 encoding** (`nvenc/`), 8-bit SDR 4:2:0 for Export. The driver's + `nvEncodeAPI64.dll` (Windows) or `libnvidia-encode.so.1` (Linux), API 12.1, is loaded at run time, + so machines without NVIDIA still start. Windows uses a Direct3D 11 device; Linux retains a primary + CUDA context through the runtime-loaded `libcuda.so.1`, released after the encoder is destroyed. + No SDK, CUDA toolkit or NVIDIA binaries are bundled or required at build time. RGBA is converted with the software encoder's own BT.709 limited conversion into NV12 input buffers (a ring of eight); the encoder runs preset P5 with high-quality tuning, CABAC (CAVLC for Baseline), one B-frame when the profile and GPU allow it, and an IDR at every keyframe distance; the parameter sets go into `avcC`. Export ▸ Hardware encoding (off by default) selects it. @@ -159,9 +161,9 @@ level and flags are the encoder's own. If it is missing the export stops with th | `tests/fallback.rs` (every OS) | `HybridDecoder` with a stand-in hardware decoder failing after N samples (every sync sample ± a few, first / last sample, after a seek): output identical to the software decoder; in-band parameter sets identical to the sample entry's stay in hardware, different ones switch to software | | `tests/setting.rs` | Hardware decoding Off gives the software decoder through `make_video_decoder` and the media stack (no hardware frames); Auto gives VideoToolbox where available | | `tests/hardware_encode.rs` (macOS) | what the hardware path takes and declines; round trip through our software decoder (every picture, in order, luma PSNR above 30 dB, keyframes no further apart than asked, no composition offsets); an export through `filmcraft_export` that decodes in our decoder and in ffmpeg / ffprobe (profile, size, frame count, BT.709); the built-in encoder still exporting everything hardware declines; exact output size at sizes that are not multiples of 16; hostile configurations (zero, huge, odd sizes, frame rates, bitrates, keyframe intervals, wrong planes) give errors and never panic; encoders dropped at any point do not crash or hang. The same for **HEVC**: Main profile, 8-bit 4:2:0, `hvc1` entry with VPS / SPS / PPS and 4-byte lengths, MP4 and QuickTime, AAC audio, two-pass refused, the format list agreeing with the probe, ffprobe reading `codec_name=hevc`, `profile=Main`, `codec_tag_string=hvc1`, `pix_fmt=yuv420p`, BT.709 | -| `tests/nvenc.rs` (Windows, NVIDIA) | H.264 from NVENC (1280×720, 6 Mbps, 72 frames) decodes with our decoder at worst 46.9 dB luma PSNR; IDR at 0, 24 and 48; dts / pts right | -| `tests/nvenc_export.rs` (Windows, NVIDIA) | Export with hardware encoding against the software encoder through the export pipeline: the two decoded files at worst 54.8 dB luma PSNR; ffmpeg decodes the file without errors; declined cases go to the software encoder; the counters | -| `src/nvenc/abi_tests.rs` (Windows) | FFI structs' sizes, alignments, field offsets, constants and GUIDs against a C compiler's view of NVIDIA's `nvEncodeAPI.h` (12.1) | +| `tests/nvenc.rs` (Windows / Linux, NVIDIA) | H.264 from NVENC (1280×720, 6 Mbps, 72 frames) decodes with our decoder at worst 46.9 dB luma PSNR; IDR at 0, 24 and 48; dts / pts right | +| `tests/nvenc_export.rs` (Windows / Linux; fallback also tested without NVIDIA) | Export with hardware encoding against the software encoder through the export pipeline: the two decoded files at worst 54.8 dB luma PSNR; ffmpeg decodes the file without errors; declined cases go to the software encoder; the counters | +| `src/nvenc/abi_tests.rs` (Windows / Linux) | FFI structs' sizes, alignments, field offsets, constants and GUIDs against a C compiler's view of NVIDIA's `nvEncodeAPI.h` (12.1) | Fixtures are made with ffmpeg into `target/fixtures/platform/` (generator only, never linked); tests skip without ffmpeg or without a hardware decoder. @@ -191,3 +193,14 @@ Zero-copy upload of decoded pictures into wgpu textures (`CVPixelBuffer`s on mac textures on Windows); B-frames and 10-bit / HDR HEVC (Main 10) in hardware encoding; VA-API (Linux) decoders; field-coded H.264; HEVC, 10-bit and HDR encoding with NVENC, and encoders from other vendors on Windows (through Media Foundation); VP9 / AV1 4:4:4 and 12-bit on Windows. + +### Linux NVENC sources and limits + +The device binding uses NVIDIA's [CUDA Driver API 12.1 primary-context reference](https://docs.nvidia.com/cuda/archive/12.1.0/cuda-driver-api/group__CUDA__PRIMARY__CTX.html) +and the existing [Video Codec SDK 12.1 encode API](https://docs.nvidia.com/video-technologies/video-codec-sdk/12.1/nvenc-video-encoder-api-prog-guide/index.html). +It uses the first CUDA device. This backend handles H.264 only, on NVIDIA hardware, on Windows and +64-bit Linux; other formats and unsupported H.264 settings use the existing encoder selection. +Linux registers the export factory even without a driver; `register()`'s return value and +`registered()` still describe hardware **decoding**, which this backend does not provide. +Absent / old drivers decline once per export, with an informational log and +`perf.stats` → `export.hardware.declined`; hardware encoding Off never opens the driver. diff --git a/crates/platform/src/lib.rs b/crates/platform/src/lib.rs index 44875796..4219d9a1 100644 --- a/crates/platform/src/lib.rs +++ b/crates/platform/src/lib.rs @@ -35,7 +35,7 @@ pub mod hardware_encode; pub mod hybrid; #[cfg(target_os = "windows")] pub mod media_foundation; -#[cfg(target_os = "windows")] +#[cfg(any(target_os = "windows", all(target_os = "linux", target_pointer_width = "64")))] pub mod nvenc; #[cfg(target_os = "macos")] #[allow(unsafe_code)] @@ -59,6 +59,8 @@ pub enum Availability { /// harmless). Streams they do not take, and every stream while hardware decoding is Off, keep /// using FilmCraft's own decoders. pub fn register() -> Availability { + #[cfg(all(target_os = "linux", target_pointer_width = "64"))] + filmcraft_export::register_encoder(nvenc::export::factory); #[cfg(target_os = "macos")] { filmcraft_codecs::register_video_decoder(videotoolbox_factory); diff --git a/crates/platform/src/nvenc/device.rs b/crates/platform/src/nvenc/device.rs new file mode 100644 index 00000000..fe898795 --- /dev/null +++ b/crates/platform/src/nvenc/device.rs @@ -0,0 +1,130 @@ +//! Devices passed to nvEncOpenEncodeSessionEx. Public APIs are safe; handles stay in this module. +use std::ffi::c_void; + +#[cfg(target_os = "linux")] +pub use cuda::Device; +#[cfg(target_os = "windows")] +pub use directx::Device; + +#[cfg(target_os = "windows")] +mod directx { + use super::c_void; + use windows::Win32::Foundation::HMODULE; + use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN; + use windows::Win32::Graphics::Direct3D11::{D3D11_CREATE_DEVICE_FLAG, D3D11_SDK_VERSION, D3D11CreateDevice, ID3D11Device}; + use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter, IDXGIFactory1}; + use windows::core::Interface; + + /// NVIDIA's PCI vendor id. + const NVIDIA: u32 = 0x10DE; + + pub struct Device(ID3D11Device); + impl Device { + pub const KIND: u32 = super::super::ffi::NV_ENC_DEVICE_TYPE_DIRECTX; + pub fn new() -> Result { + nvidia_device().map(Self) + } + pub fn as_raw(&self) -> *mut c_void { + self.0.as_raw() + } + } + /// A Direct3D 11 device on the first NVIDIA adapter (what an NVENC session is opened on). + fn nvidia_device() -> Result { + // SAFETY: plain COM / D3D calls with valid out-pointers; the adapter outlives the call. + unsafe { + let factory: IDXGIFactory1 = CreateDXGIFactory1().map_err(|e| format!("no DXGI: {e}"))?; + let mut i = 0; + while let Ok(adapter) = factory.EnumAdapters1(i) { + i += 1; + let Ok(desc) = adapter.GetDesc1() else { continue }; + if desc.VendorId != NVIDIA { + continue; + } + let mut device = None; + let adapter: IDXGIAdapter = adapter.cast().map_err(|e| e.to_string())?; + if D3D11CreateDevice( + &adapter, + D3D_DRIVER_TYPE_UNKNOWN, + HMODULE::default(), + D3D11_CREATE_DEVICE_FLAG(0), + None, + D3D11_SDK_VERSION, + Some(&mut device), + None, + None, + ) + .is_ok() + && let Some(d) = device + { + return Ok(d); + } + } + } + Err("no NVIDIA adapter".into()) + } +} + +#[cfg(target_os = "linux")] +mod cuda { + use super::c_void; + type Init = unsafe extern "C" fn(u32) -> i32; + type GetDevice = unsafe extern "C" fn(*mut i32, i32) -> i32; + type Retain = unsafe extern "C" fn(*mut *mut c_void, i32) -> i32; + type Release = unsafe extern "C" fn(i32) -> i32; + + /// One retained primary CUDA context. Retaining does not change the calling thread's + /// current context; NVENC receives the explicit context handle. No toolkit is needed. + pub struct Device { + context: *mut c_void, + ordinal: i32, + release: Release, + _library: libloading::Library, + } + + impl Device { + pub const KIND: u32 = super::super::ffi::NV_ENC_DEVICE_TYPE_CUDA; + pub fn new() -> Result { + // SAFETY: CUDA driver API signatures from NVIDIA's public Driver API reference. + // Resolve all symbols before retaining; keep the library alive until after release. + unsafe { + let library = libloading::Library::new("libcuda.so.1").map_err(|e| format!("no NVIDIA CUDA driver: {e}"))?; + let init: libloading::Symbol = library.get(b"cuInit\0").map_err(|e| e.to_string())?; + let get: libloading::Symbol = library.get(b"cuDeviceGet\0").map_err(|e| e.to_string())?; + let retain: libloading::Symbol = library.get(b"cuDevicePrimaryCtxRetain\0").map_err(|e| e.to_string())?; + let release: Release = *library.get(b"cuDevicePrimaryCtxRelease_v2\0").map_err(|e| e.to_string())?; + let st = init(0); + if st != 0 { + return Err(format!("cuInit failed ({st})")); + } + let mut ordinal = 0; + let st = get(&mut ordinal, 0); + if st != 0 { + return Err(format!("no CUDA device ({st})")); + } + let mut context = std::ptr::null_mut(); + let st = retain(&mut context, ordinal); + if st != 0 { + return Err(format!("cuDevicePrimaryCtxRetain failed ({st})")); + } + if context.is_null() { + let _ = release(ordinal); + return Err("CUDA returned a null primary context".into()); + } + Ok(Self { context, ordinal, release, _library: library }) + } + } + pub fn as_raw(&self) -> *mut c_void { + self.context + } + } + + impl Drop for Device { + fn drop(&mut self) { + // SAFETY: balances this object's successful retain, after the NVENC session is + // destroyed. Release drops our reference, not other users' primary-context references. + unsafe { + (self.release)(self.ordinal); + } + } + } +} diff --git a/crates/platform/src/nvenc/export.rs b/crates/platform/src/nvenc/export.rs index 9d833147..fc77fe28 100644 --- a/crates/platform/src/nvenc/export.rs +++ b/crates/platform/src/nvenc/export.rs @@ -91,6 +91,9 @@ fn config(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) - if s.field_order != FieldOrder::Progressive { return Err("interlaced output".into()); } + if w == 0 || h == 0 || !w.is_multiple_of(2) || !h.is_multiple_of(2) { + return Err("NVENC needs nonzero even dimensions".into()); + } if w > u32::from(u16::MAX) || h > u32::from(u16::MAX) { return Err(format!("{w}x{h} does not fit an MP4 sample entry")); } @@ -107,7 +110,7 @@ fn config(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) - height: h, fps, bitrate_kbps: kbps, - max_bitrate_kbps: s.max_bitrate_kbps.filter(|m| *m >= kbps).unwrap_or(kbps / 2 * 3), + max_bitrate_kbps: s.max_bitrate_kbps.filter(|m| *m >= kbps).unwrap_or_else(|| kbps.saturating_add(kbps / 2)), cbr: s.bitrate_mode == BitrateMode::Cbr, keyint: s.keyframe_distance.filter(|k| *k > 0).unwrap_or_else(|| (f64::from(fps.0) / f64::from(fps.1) * 2.0).round().max(1.0) as u32), profile: match s.h264_profile { @@ -123,6 +126,17 @@ fn config(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) - /// The Export encoder factory (see the module documentation). pub fn factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -> Option>> { + factory_with(format, w, h, rate, s, NvencH264::new) +} + +fn factory_with( + format: Format, + w: u32, + h: u32, + rate: FrameRate, + s: &ExportSettings, + open: impl FnOnce(&Config) -> std::result::Result, +) -> Option>> { if s.hardware_encoding != HardwareEncoding::Auto || format != Format::H264 { return None; } @@ -135,7 +149,7 @@ pub fn factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettin Ok(c) => c, Err(why) => return declined(&why), }; - match NvencH264::new(&cfg) { + match open(&cfg) { Ok(enc) => { filmcraft_export::note_hw_encode_session(); Some(Ok(Box::new(NvencEncoder { enc, w, h, rate, y: Vec::new(), u: Vec::new(), v: Vec::new() }))) @@ -148,6 +162,45 @@ pub fn factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettin mod tests { use super::*; + #[test] + fn off_and_unsupported_settings_never_open_the_driver() { + let declined = |format, w, h, s: &ExportSettings| { + assert!(factory_with(format, w, h, FrameRate::FPS_24, s, |_| panic!("must not open the driver")).is_none()); + }; + let mut s = ExportSettings { hardware_encoding: HardwareEncoding::Off, ..Default::default() }; + declined(Format::H264, 640, 360, &s); + s.hardware_encoding = HardwareEncoding::Auto; + declined(Format::ProRes, 640, 360, &s); + declined(Format::H264, 641, 360, &s); + declined(Format::H264, 0, 360, &s); + s.h264_pass = H264Pass::First; + declined(Format::H264, 640, 360, &s); + s.h264_pass = H264Pass::Single; + s.field_order = FieldOrder::UpperFirst; + declined(Format::H264, 640, 360, &s); + } + + #[test] + fn missing_or_rejecting_driver_declines_for_software_fallback() { + let s = ExportSettings { hardware_encoding: HardwareEncoding::Auto, ..Default::default() }; + let mut attempted = false; + assert!( + factory_with(Format::H264, 640, 360, FrameRate::FPS_24, &s, |cfg| { + attempted = true; + assert_eq!((cfg.width, cfg.height), (640, 360)); + Err("no NVIDIA encoder driver".into()) + }) + .is_none() + ); + assert!(attempted); + } + + #[test] + fn bitrate_ceiling_does_not_wrap() { + let s = ExportSettings { bitrate_kbps: u32::MAX, ..Default::default() }; + assert_eq!(config(Format::H264, 640, 360, FrameRate::FPS_24, &s).unwrap().max_bitrate_kbps, u32::MAX); + } + #[test] fn composition_offsets_saturate() { assert_eq!(composition_offset(3, 1, 1001), 2002); diff --git a/crates/platform/src/nvenc/ffi.rs b/crates/platform/src/nvenc/ffi.rs index cae6fa0e..6fb6b72b 100644 --- a/crates/platform/src/nvenc/ffi.rs +++ b/crates/platform/src/nvenc/ffi.rs @@ -25,7 +25,7 @@ // OTHER DEALINGS IN THE SOFTWARE. //! The NVIDIA Video Codec SDK's encode API (`nvEncodeAPI.h`, API 12.1, MIT-licensed header by -//! NVIDIA) as the driver's `nvEncodeAPI64.dll` expects it: the structures this backend fills in, +//! NVIDIA) as the driver's `nvEncodeAPI64.dll` / `libnvidia-encode.so.1` expects it: the structures this backend fills in, //! the function table, and the constants it uses. Written from the public header; sizes and field //! offsets are checked against a C compiler's by the generated `abi_tests.rs`. //! @@ -38,7 +38,26 @@ use std::ffi::{c_char, c_int, c_void}; -use windows::core::GUID; +// NVIDIA's public header defines this same layout on non-Windows systems. +#[repr(C)] +#[derive(Clone, Copy)] +pub struct Guid { + data1: u32, + data2: u16, + data3: u16, + data4: [u8; 8], +} + +impl Guid { + pub const fn from_values(data1: u32, data2: u16, data3: u16, data4: [u8; 8]) -> Self { + Self { data1, data2, data3, data4 } + } + + #[cfg(test)] + pub const fn to_u128(self) -> u128 { + ((self.data1 as u128) << 96) | ((self.data2 as u128) << 80) | ((self.data3 as u128) << 64) | u64::from_be_bytes(self.data4) as u128 + } +} pub const NVENCAPI_VERSION: u32 = 12 | (1 << 24); @@ -70,6 +89,7 @@ pub const NV_ENC_ERR_ENCODER_BUSY: NvStatus = 18; pub const NV_ENC_ERR_LOCK_BUSY: NvStatus = 13; pub const NV_ENC_DEVICE_TYPE_DIRECTX: u32 = 0; +pub const NV_ENC_DEVICE_TYPE_CUDA: u32 = 1; pub const NV_ENC_MEMORY_HEAP_SYSMEM_CACHED: u32 = 2; pub const NV_ENC_BUFFER_FORMAT_NV12: u32 = 0x1; pub const NV_ENC_PARAMS_RC_VBR: u32 = 1; @@ -96,12 +116,12 @@ pub const NV_ENC_CAPS_HEIGHT_MAX: u32 = 17; pub const NV_ENC_CAPS_WIDTH_MIN: u32 = 45; pub const NV_ENC_CAPS_HEIGHT_MIN: u32 = 46; -pub const NV_ENC_CODEC_H264_GUID: GUID = GUID::from_values(0x6bc82762, 0x4e63, 0x4ca4, [0xaa, 0x85, 0x1e, 0x50, 0xf3, 0x21, 0xf6, 0xbf]); -pub const NV_ENC_H264_PROFILE_BASELINE_GUID: GUID = GUID::from_values(0x0727bcaa, 0x78c4, 0x4c83, [0x8c, 0x2f, 0xef, 0x3d, 0xff, 0x26, 0x7c, 0x6a]); -pub const NV_ENC_H264_PROFILE_MAIN_GUID: GUID = GUID::from_values(0x60b5c1d4, 0x67fe, 0x4790, [0x94, 0xd5, 0xc4, 0x72, 0x6d, 0x7b, 0x6e, 0x6d]); -pub const NV_ENC_H264_PROFILE_HIGH_GUID: GUID = GUID::from_values(0xe7cbc309, 0x4f7a, 0x4b89, [0xaf, 0x2a, 0xd5, 0x37, 0xc9, 0x2b, 0xe3, 0x10]); -pub const NV_ENC_PRESET_P4_GUID: GUID = GUID::from_values(0x90a7b826, 0xdf06, 0x4862, [0xb9, 0xd2, 0xcd, 0x6d, 0x73, 0xa0, 0x86, 0x81]); -pub const NV_ENC_PRESET_P5_GUID: GUID = GUID::from_values(0x21c6e6b4, 0x297a, 0x4cba, [0x99, 0x8f, 0xb6, 0xcb, 0xde, 0x72, 0xad, 0xe3]); +pub const NV_ENC_CODEC_H264_GUID: Guid = Guid::from_values(0x6bc82762, 0x4e63, 0x4ca4, [0xaa, 0x85, 0x1e, 0x50, 0xf3, 0x21, 0xf6, 0xbf]); +pub const NV_ENC_H264_PROFILE_BASELINE_GUID: Guid = Guid::from_values(0x0727bcaa, 0x78c4, 0x4c83, [0x8c, 0x2f, 0xef, 0x3d, 0xff, 0x26, 0x7c, 0x6a]); +pub const NV_ENC_H264_PROFILE_MAIN_GUID: Guid = Guid::from_values(0x60b5c1d4, 0x67fe, 0x4790, [0x94, 0xd5, 0xc4, 0x72, 0x6d, 0x7b, 0x6e, 0x6d]); +pub const NV_ENC_H264_PROFILE_HIGH_GUID: Guid = Guid::from_values(0xe7cbc309, 0x4f7a, 0x4b89, [0xaf, 0x2a, 0xd5, 0x37, 0xc9, 0x2b, 0xe3, 0x10]); +pub const NV_ENC_PRESET_P4_GUID: Guid = Guid::from_values(0x90a7b826, 0xdf06, 0x4862, [0xb9, 0xd2, 0xcd, 0x6d, 0x73, 0xa0, 0x86, 0x81]); +pub const NV_ENC_PRESET_P5_GUID: Guid = Guid::from_values(0x21c6e6b4, 0x297a, 0x4cba, [0x99, 0x8f, 0xb6, 0xcb, 0xde, 0x72, 0xad, 0xe3]); pub type Handle = *mut c_void; @@ -247,7 +267,7 @@ pub union NV_ENC_CODEC_CONFIG { #[derive(Clone, Copy)] pub struct NV_ENC_CONFIG { pub version: u32, - pub profileGUID: GUID, + pub profileGUID: Guid, pub gopLength: u32, pub frameIntervalP: i32, pub monoChromeEncoding: u32, @@ -279,8 +299,8 @@ pub struct NVENC_EXTERNAL_ME_HINT_COUNTS_PER_BLOCKTYPE { #[derive(Clone, Copy)] pub struct NV_ENC_INITIALIZE_PARAMS { pub version: u32, - pub encodeGUID: GUID, - pub presetGUID: GUID, + pub encodeGUID: Guid, + pub presetGUID: Guid, pub encodeWidth: u32, pub encodeHeight: u32, pub darWidth: u32, @@ -456,7 +476,7 @@ pub struct NV_ENCODE_API_FUNCTION_LIST { pub nvEncGetEncodeGUIDs: Unused, pub nvEncGetInputFormatCount: Unused, pub nvEncGetInputFormats: Unused, - pub nvEncGetEncodeCaps: Option NvStatus>, + pub nvEncGetEncodeCaps: Option NvStatus>, pub nvEncGetEncodePresetCount: Unused, pub nvEncGetEncodePresetGUIDs: Unused, pub nvEncGetEncodePresetConfig: Unused, @@ -488,7 +508,7 @@ pub struct NV_ENCODE_API_FUNCTION_LIST { pub nvEncRunMotionEstimationOnly: Unused, pub nvEncGetLastErrorString: Option *const c_char>, pub nvEncSetIOCudaStreams: Unused, - pub nvEncGetEncodePresetConfigEx: Option NvStatus>, + pub nvEncGetEncodePresetConfigEx: Option NvStatus>, pub nvEncGetSequenceParamEx: Unused, pub nvEncRestoreEncoderState: Unused, pub nvEncLookaheadPicture: Unused, diff --git a/crates/platform/src/nvenc/mod.rs b/crates/platform/src/nvenc/mod.rs index d0723adc..74e39bd9 100644 --- a/crates/platform/src/nvenc/mod.rs +++ b/crates/platform/src/nvenc/mod.rs @@ -1,7 +1,7 @@ -//! NVIDIA NVENC H.264 hardware encoding (Windows). +//! NVIDIA NVENC H.264 hardware encoding (Windows and 64-bit Linux). //! -//! The encoder runs on the GPU's NVENC engine through the driver's `nvEncodeAPI64.dll` (API 12.1, -//! [`ffi`]; no CUDA, no SDK to install): pictures go in as NV12 input buffers, the Annex B output +//! The encoder runs through the driver's `nvEncodeAPI64.dll` on Windows or +//! `libnvidia-encode.so.1` on Linux (API 12.1, [`ffi`]; no SDK to install): pictures go in as NV12 input buffers, the Annex B output //! comes back as length-prefixed H.264 samples with the parameter sets split out for the `avcC`. //! [`export`] plugs it into Export as an alternative to the software encoder. //! @@ -10,11 +10,13 @@ //! ──► NV12 input buffer ──NVENC──► Annex B ──► length-prefixed samples + avcC //! ``` //! -//! `unsafe` is confined to `ffi` (data) and `session` (every driver call); this module is safe +//! `unsafe` is confined to `ffi` (data), `device` (device lifetime) and `session` (driver calls); this module is safe //! code. Hardware encoding never replaces an export that works in software: the factory declines //! what NVENC cannot do (no NVIDIA GPU or driver, sizes, HDR, two-pass, MXF...) and the software //! encoder takes over. +#[allow(unsafe_code)] +mod device; pub mod export; #[allow(unsafe_code)] mod ffi; diff --git a/crates/platform/src/nvenc/session.rs b/crates/platform/src/nvenc/session.rs index 0aa362c8..5c0a69b4 100644 --- a/crates/platform/src/nvenc/session.rs +++ b/crates/platform/src/nvenc/session.rs @@ -1,4 +1,4 @@ -//! An NVENC H.264 encode session: the driver library, the Direct3D 11 device it is opened on, the +//! An NVENC H.264 encode session: the driver library, the device it is opened on, the //! encoder configuration, and the input / output buffers (every call into the driver). //! //! FFI module (docs/adr/0001-platform-ffi.md): every `unsafe` block has a `// SAFETY:` comment, the @@ -7,21 +7,14 @@ use std::ffi::{CStr, c_void}; use std::sync::OnceLock; -use windows::Win32::Foundation::HMODULE; -use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN; -use windows::Win32::Graphics::Direct3D11::{D3D11_CREATE_DEVICE_FLAG, D3D11_SDK_VERSION, D3D11CreateDevice, ID3D11Device}; -use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter, IDXGIFactory1}; -use windows::Win32::System::LibraryLoader::{GetProcAddress, LOAD_LIBRARY_SEARCH_SYSTEM32, LoadLibraryExW}; -use windows::core::{Interface, PCSTR, w}; - +use super::device::Device; use super::ffi::*; -/// NVIDIA's PCI vendor id. -const NVIDIA: u32 = 0x10DE; - /// The driver's entry points (loaded once). struct Api { list: NV_ENCODE_API_FUNCTION_LIST, + #[cfg(target_os = "linux")] + _library: libloading::Library, } // SAFETY: the function table is plain function pointers, immutable after loading; NVENC's API is @@ -35,7 +28,11 @@ fn api() -> Result<&'static Api, String> { API.get_or_init(load).as_ref().map_err(Clone::clone) } +#[cfg(target_os = "windows")] fn load() -> Result { + use windows::Win32::Foundation::HMODULE; + use windows::Win32::System::LibraryLoader::{GetProcAddress, LOAD_LIBRARY_SEARCH_SYSTEM32, LoadLibraryExW}; + use windows::core::{PCSTR, w}; // SAFETY: loads the driver's library from the system directory only. let module: HMODULE = unsafe { LoadLibraryExW(w!("nvEncodeAPI64.dll"), None, LOAD_LIBRARY_SEARCH_SYSTEM32) }.map_err(|_| "no NVIDIA encoder driver".to_string())?; @@ -64,39 +61,29 @@ fn load() -> Result { } } -/// A Direct3D 11 device on the first NVIDIA adapter (what an NVENC session is opened on). -fn nvidia_device() -> Result { - // SAFETY: plain COM / D3D calls with valid out-pointers; the adapter outlives the call. +#[cfg(target_os = "linux")] +fn load() -> Result { + // SAFETY: runtime-load NVIDIA's driver and use the signatures from nvEncodeAPI.h. + // The library lives alongside the immutable function table, for the process lifetime. unsafe { - let factory: IDXGIFactory1 = CreateDXGIFactory1().map_err(|e| format!("no DXGI: {e}"))?; - let mut i = 0; - while let Ok(adapter) = factory.EnumAdapters1(i) { - i += 1; - let Ok(desc) = adapter.GetDesc1() else { continue }; - if desc.VendorId != NVIDIA { - continue; - } - let mut device = None; - let adapter: IDXGIAdapter = adapter.cast().map_err(|e| e.to_string())?; - if D3D11CreateDevice( - &adapter, - D3D_DRIVER_TYPE_UNKNOWN, - HMODULE::default(), - D3D11_CREATE_DEVICE_FLAG(0), - None, - D3D11_SDK_VERSION, - Some(&mut device), - None, - None, - ) - .is_ok() - && let Some(d) = device - { - return Ok(d); - } + let library = libloading::Library::new("libnvidia-encode.so.1").map_err(|e| format!("no NVIDIA encoder driver: {e}"))?; + let version: libloading::Symbol = library.get(b"NvEncodeAPIGetMaxSupportedVersion\0").map_err(|e| e.to_string())?; + let create: libloading::Symbol = library.get(b"NvEncodeAPICreateInstance\0").map_err(|e| e.to_string())?; + let mut v = 0; + if version(&mut v) != NV_ENC_SUCCESS { + return Err("cannot query the NVENC version".into()); } + let ours = (NVENCAPI_VERSION & 0xf) << 4 | ((NVENCAPI_VERSION >> 24) & 0xf); + if v < ours { + return Err(format!("the NVIDIA driver's NVENC ({}.{}) is older than {}.{}", v >> 4, v & 0xf, ours >> 4, ours & 0xf)); + } + let mut list: NV_ENCODE_API_FUNCTION_LIST = std::mem::zeroed(); + list.version = NV_ENCODE_API_FUNCTION_LIST_VER; + if create(&mut list) != NV_ENC_SUCCESS { + return Err("NvEncodeAPICreateInstance failed".into()); + } + Ok(Api { list, _library: library }) } - Err("no NVIDIA adapter".into()) } /// What the encoder says it can do for H.264. @@ -159,15 +146,15 @@ struct Slot { pub struct Session { api: &'static Api, enc: Handle, - // keeps the Direct3D device (the session's device) alive - _device: ID3D11Device, + // Keeps the Direct3D device / retained CUDA context alive through session destruction. + _device: Device, slots: Vec, size: (u32, u32), out_size: u32, } // SAFETY: an encoder session and its buffers are driver objects used from one thread at a time (the -// session is `&mut`-driven); the Direct3D device is a free-threaded COM object. +// session is `&mut`-driven); Direct3D is free-threaded, and NVENC takes an explicit CUDA context. unsafe impl Send for Session {} /// A readable form of a driver status. @@ -192,14 +179,14 @@ impl Session { /// Open a session on the system's NVIDIA GPU, or say why not. pub fn open() -> Result { let api = api()?; - let device = nvidia_device()?; - // SAFETY: a zeroed parameter block with its version and the device's COM pointer; the + let device = Device::new()?; + // SAFETY: a zeroed parameter block with its version and the matching device pointer; the // device is kept in the session for as long as the encoder lives. On failure a non-null // handle is the driver's and is destroyed exactly once, here, before returning. let enc = unsafe { let mut p: NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS = std::mem::zeroed(); p.version = NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER; - p.deviceType = NV_ENC_DEVICE_TYPE_DIRECTX; + p.deviceType = Device::KIND; p.device = device.as_raw(); p.apiVersion = NVENCAPI_VERSION; let open = api.list.nvEncOpenEncodeSessionEx.ok_or("the driver has no nvEncOpenEncodeSessionEx")?; diff --git a/crates/platform/tests/nvenc.rs b/crates/platform/tests/nvenc.rs index e72638af..2d43680f 100644 --- a/crates/platform/tests/nvenc.rs +++ b/crates/platform/tests/nvenc.rs @@ -1,7 +1,7 @@ //! NVENC H.264 encoding (Windows, NVIDIA GPU): the stream decodes with our own decoder to pictures //! close to the source, keyframes and timestamps are right, and Export takes it only when asked. //! Skips without an NVIDIA GPU / driver with NVENC. -#![cfg(target_os = "windows")] +#![cfg(any(target_os = "windows", all(target_os = "linux", target_pointer_width = "64")))] use filmcraft_isobmff::{AvcConfig, SampleEntry}; use filmcraft_platform::nvenc::{Config, NvencH264, Packet, Profile}; diff --git a/crates/platform/tests/nvenc_export.rs b/crates/platform/tests/nvenc_export.rs index be82040c..e0d32a20 100644 --- a/crates/platform/tests/nvenc_export.rs +++ b/crates/platform/tests/nvenc_export.rs @@ -2,7 +2,7 @@ //! pipeline: the same frames, a file both decode, counters showing which encoder ran, and what NVENC //! declines (hardware encoding Off, two-pass, MXF...) going to the software encoder. Its own test //! binary: the encoder registry is process-wide. Skips without an NVIDIA GPU with NVENC. -#![cfg(target_os = "windows")] +#![cfg(any(target_os = "windows", all(target_os = "linux", target_pointer_width = "64")))] use std::sync::Arc; @@ -83,15 +83,19 @@ fn hardware_export_matches_the_software_export() { filmcraft_platform::register(); let (p, seq, m) = project(1280, 720, 72); let soft_path = tmp("soft.mp4"); + let before_soft = hw_encode_stats(); export(&p, seq, &settings(soft_path.clone(), HardwareEncoding::Off), &m, &Progress::default()).unwrap(); - assert_eq!(hw_encode_stats().sessions, 0, "Off: no hardware encoder"); + assert_eq!(hw_encode_stats(), before_soft, "Off: no hardware encoder attempt"); let hw_path = tmp("hw.mp4"); let before = hw_encode_stats(); let report = export(&p, seq, &settings(hw_path.clone(), HardwareEncoding::Auto), &m, &Progress::default()).unwrap(); let after = hw_encode_stats(); if after.sessions == before.sessions { - eprintln!("SKIPPED: no NVENC here (declined {})", after.declined - before.declined); + assert_eq!(after.declined - before.declined, 1, "Auto attempted hardware and reported the fallback"); + assert_eq!(after.frames, before.frames); + assert_eq!(std::fs::read(&soft_path).unwrap(), std::fs::read(&hw_path).unwrap(), "fallback preserves software determinism"); + eprintln!("no NVENC here: byte-identical software fallback verified"); return; } assert_eq!(report.frames, 72); @@ -135,7 +139,9 @@ fn what_nvenc_does_not_take_goes_to_the_software_encoder() { (0, 1) ); // an odd frame size is rounded to even by the pipeline, so NVENC still takes it - assert_eq!(run("odd.mp4", &|s| s.frame_size = Some((641, 359))), (24, 0)); + let hardware_available = filmcraft_platform::nvenc::available(); + let odd = run("odd.mp4", &|s| s.frame_size = Some((641, 359))); + assert_eq!(odd, if hardware_available { (24, 0) } else { (0, 1) }); // Off never touches the hardware encoder assert_eq!(run("off.mp4", &|s| s.hardware_encoding = HardwareEncoding::Off), (0, 0)); // the decoded file of a declined export is fine diff --git a/docs/adr/0001-platform-ffi.md b/docs/adr/0001-platform-ffi.md index d067c4b7..91e6c2c4 100644 --- a/docs/adr/0001-platform-ffi.md +++ b/docs/adr/0001-platform-ffi.md @@ -26,7 +26,7 @@ Containment rules: 1. The crate does not use `lints.workspace = true`. Its own `[lints]` table copies the workspace lints except `unsafe_code = "deny"` (not `forbid`), and adds `clippy::undocumented_unsafe_blocks = "deny"`. Only the FFI modules (`videotoolbox`, `media_foundation::gpu` / `media_foundation::mft`, and - `nvenc::ffi` / `nvenc::session` on Windows) carry `#[allow(unsafe_code)]`; the rest of the crate + `nvenc::ffi` / `nvenc::session` / `nvenc::device` on Windows and 64-bit Linux) carry `#[allow(unsafe_code)]`; the rest of the crate (the fallback logic in `hybrid`, the decoder logic in `media_foundation`, `annexb`, `biplanar`, the encoder logic in `nvenc`) has no `unsafe`. 2. Every `unsafe` block has a `// SAFETY:` comment saying why it is sound. @@ -101,3 +101,13 @@ there will not be one (pure Rust, clean-room: x265 is GPL), so `Format::Hevc` ex OS has a hardware encoder. Choosing the format is the opt-in, `filmcraft_export::available` asks a probe the platform crate registers, and what the hardware path does not take (two-pass, odd sizes, a machine without the encoder) is an error naming the reason instead of a different encoder. + +## Addendum: Linux NVENC H.264 encoding + +The existing NVENC session is shared with 64-bit Linux. Only device acquisition and driver loading +are platform-specific: Windows retains the Direct3D device; Linux retains the first CUDA device's +primary context through `libcuda.so.1`. `cuDevicePrimaryCtxRetain` does not push a context onto the +calling thread's stack. The retained reference is released after the NVENC session and its buffers; +other users' references are not reset. `libnvidia-encode.so.1` stays loaded with its function table. +Both libraries come from the installed NVIDIA driver, not this project. Export registration on +Linux does not claim that a hardware decoder is available. The same opt-in and fallback rules apply. diff --git a/docs/architecture.md b/docs/architecture.md index e0a7885c..fc51325a 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -263,12 +263,12 @@ file ──► codecs (MP4/MOV, MKV, audio) demux + decode, GOP-aware see Decoders run slices on rayon, so an export worker waiting inside a decode can pick up another frame of the same source. A request that finds the shared decoder busy decodes with a private decoder. -- **Hardware encoding.** On Windows, `platform::nvenc::export::factory` is registered with +- **Hardware encoding.** On Windows and 64-bit Linux, `platform::nvenc::export::factory` is registered with `filmcraft_export::register_encoder` (`register()` does this once), in front of the software H.264 encoder. It takes an export only when Export ▸ Hardware encoding (`ExportSettings.hardwareEncoding`, `HardwareEncoding::Auto`) is Auto, which is off by default: hardware streams differ from ours, and exports are otherwise byte-identical from run to run. NVENC (`platform::nvenc`, the driver's - `nvEncodeAPI64.dll` loaded at run time) then takes RGBA frames, converted to 4:2:0 by the software + `nvEncodeAPI64.dll` / `libnvidia-encode.so.1` loaded at run time) then takes RGBA frames, converted to 4:2:0 by the software encoder's own conversion, and produces the H.264 samples and `avcC` of an MP4 / MOV. It declines (the software encoder runs, counted in `export.hardware.declined`) two-pass VBR, HDR, MXF, interlaced output, sizes outside NVENC's limits, and systems without an NVIDIA GPU or driver.