ffmpeg keeps a trailing zero byte after the hvcC PPS that the in-band PPS lacks, so the byte-for-byte comparison read it as a parameter-set change and sent the stream to the software decoder from its first sample. A NAL unit never ends with 0x00 (H.264 7.4.1, H.265 7.4.2): those bytes are trailing_zero_8bits, so both sides are compared without them. The platform fallback test used a trailing zero byte as its changed SPS; it now changes nal_ref_idc.
Problem: the Matroska demuxer mapped A_MPEG/L2 and A_MPEG/L3 only, so a
Layer I track was reported as an unsupported codec even though the MP1/MP2
decoder (#809) reads Layer I.
Fix: add Codec::Mp1 for A_MPEG/L1 and route it through the same packet
duration and decoder path as Mp2. Test remuxes synthetic Layer I frames
into Matroska with ffmpeg and compares our decode (max error 3e-5).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Phone footage stamps its frames up to a millisecond after the frame grid
(a first frame 34.1 ms long at 30 fps). The frame shown at a time is the
last one starting at or before it, so a sequence frame asked at its exact
start got the picture before: the clip ran one frame behind and showed
frame 0 twice. The request is already rounded to the track timescale
(#73, as WebM in #83), but at 90 kHz a millisecond is 90 units.
When opening an MP4, the video samples of a file that is a constant rate
gone slightly off its grid are put back on it: the grid starts at the
first picture and steps by the nominal frame duration, and the file is
snapped only when every picture is less than a quarter of a frame from
its own grid point and no two pictures share a grid point. Anything
further off is variable frame rate and keeps all its times; a picture one
unit off is left where it is, since that can't be told from timescale
rounding. Only the video sample times move: the audio, the first picture
and the clip duration are unchanged.
File ▸ Import and the Media Browser offer `.aac` (it is in AUDIO_EXTENSIONS),
and Export ▸ AAC Audio writes ADTS `.aac` (#814), but no opener read them:
`audio::opener` only accepted mp3/mp2/flac/ogg/aif by extension, so every
`.aac` failed with "unrecognised format".
`AudioFileSource` now decodes a standalone ADTS stream with our AAC decoder
(ISO/IEC 13818-7 §6.2): frame after frame from the first header (after an
ID3v2 tag), resyncing past damage, skipping frames with another
configuration, keeping a frame that doesn't decode as silence, and
upsampling HE-AAC to its output rate. Decoding stops with an error past
4 GiB of samples instead of allocating without bound.
Tests encode a stereo tone with our encoder's ADTS output and import it
(rate, channels, duration, level), and run truncated, bit-flipped and
bad-length streams under catch_unwind.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Other formats' start and priming (AAC edit lists, …) are handled and
oracle-tested elsewhere; enable_gapless for them could move their audio.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
VP8 WebM showed black video: there was no VP8 decoder. New L0 crate
filmcraft-vp8, implemented from RFC 6386 sections 1-19 (tables extracted
from the RFC text, nothing from the attached reference decoder): every
coding tool, versions 0-3, hidden altref frames, size changes. Bit-exact
with libvpx on 62 libvpx VP8 test vectors, real-world FATE samples and
generated fixtures; never panics on corrupted input.
Wired into codecs for Matroska / WebM (V_VP8, including the alpha layer
of transparent WebM) and MP4 (vp08). VP8 colour is BT.601 (RFC 6386 9.2).
A hidden altref frame is a WebM block of its own with the time of the
frame that follows it: the VP8 time lookup now moves on to that shown
frame instead of repeating the previous one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vui_color took the VUI matrix, transfer and range but not the primaries,
so every decoded H.264 / HEVC frame reported BT.709 primaries. Stream info
read them, so a BT.2020 file without a colr box said BT.2020 in its info
and BT.709 on its frames (Rec709 instead of Rec2020 for the renderer).
Pass colour_primaries through, for the software and hardware decoders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without DefaultDuration the rate came from the median step between
neighbouring blocks in file order. Blocks are in decode order, so with
B-frames neighbours are two or more frames apart in presentation time and
a 25 fps clip reported 12.5. Step between the sorted presentation times
instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The MP4 reader took the AudioSpecificConfig's channel configuration as a
count, but it is an index into ISO/IEC 14496-3 Table 1.19: configuration 7
(7.1) reported seven channels and dropped the last decoded one, and 12
(7.1 back, what ffmpeg 9 writes for 7.1) reported twelve. Count from the
table (with Amd 4: 11 → 7, 12 and 14 → 8, 13 → 24), and decode
configurations 11 and 12, reordered to the WAV/SMPTE order used for 5.1.
Checked against ffmpeg's decode channel by channel.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ffmpeg's CLI handles Matroska priming differently by version: 9.x trims
the 481-sample CodecDelay, 2023-24 static builds keep the priming
samples, so comparing from sample 2 put the priming ramp against our
tone (max error = peak 0.125). The .mkv case now searches lags within
two samples of 0 (trimmed) and of the stream's own CodecDelay (kept),
over the interior of both decodes; the plain .mp2 file stays at lag 0.
Review nits: the ID3v2 skip offset uses saturating_add (test with the
largest tag size); every Layer II table's sblimit is checked <= 32 at
compile time and clamped to SUBBANDS before indexing the per-subband
arrays (row() is a match, so it cannot index out of range).
Verified with ffmpeg 9.0.2 and the static N-71064 (2024-06) build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ffmpeg's CLI handles Matroska priming differently by version: 9.x trims
CodecDelay, 2023-24 static builds keep the priming samples (AAC 1024,
AC-3 / E-AC-3 256, MP3 1105), so a +-40-sample best-lag search against
its decode landed on a spurious minimum (4 or 24 samples).
matroska_audio_oracle now has ffmpeg encode a reference signal we
generate (low-passed xorshift noise) and asserts our decode lines up
with that signal to the sample, the stream's own CodecDelay trimmed:
version-independent, and it fails on the old ms-rounded trim (-16).
ffmpeg's decode is compared after a best-lag search bounded by the
CodecDelay; its lag is reported, the waveform must match (< 1e-3).
E-AC-3 joins the cases. eac3_oracle's .mkv case accepts lag 0 or the
256-sample priming; MP4 / MOV / TS stay strictly at lag 0.
Verified with ffmpeg 9.0.2 and the static N-71064 (2024-06) build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep FLAC beside main's new AAC Audio format (21 formats, both audio-only
presets), use is_mp4_with / is_mp4_video in audio_codec() and the Audio
Format picker so MP4 (incl. AV1) offers AAC or FLAC, and add flacLevel to
main's export.media params.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use ds64.dataSize for an RF64 data-size sentinel instead of the rest of
the file, so chunks after the audio are no longer decoded as samples.
Fixes#925
Transport and program streams listed a single audio stream, so file.import
and New Sequence From Clip kept one clip. Probe every audio PID (capped),
decode each independently, and place linked clips per stream. MP4/MKV
already did this; MXF same-rate PCM is still interleaved as one stream.
Pin GOP-cache eviction on the requested pts so a burst past the playhead
does not drop the target and walk the GOP again. Preroll only schedules
the frames that gate clock start. bench_playback no longer meters preroll
jobs as on-clock latency.
This is the seek stall from #848, not the ~3× per-core gap vs ffmpeg.
Problem: DPX import rejected the 10/12-bit packed layout (packing 0)
and the Y'CbCr descriptors (100 CbYCrY, 101 CbYACrYA, 102 CbYCr,
103 CbYCrA) with an "unsupported" error.
Fix: unpack packed components back to back, low bits first, each scan
line on a 32-bit boundary (as ffmpeg reads 12-bit packed files), and
convert Y'CbCr to RGB with the matrix the colorimetric field names
(BT.709; BT.601 for the 601 and composite codes; by picture size when
unspecified), in video range unless the reference codes span the full
code range; 4:2:2 chroma is shared by each pixel pair. Unit tests for
every new layout and hostile headers; an ffmpeg oracle test builds
packed 12-bit RGB and 8-bit 4:2:2 / 4:4:4 / 4:4:4:4 files from
ffmpeg's test picture and matches its decode within one 8-bit step.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A GopCache decodes with its lock held, and that decoding starts rayon work
(rotating iPhone frames, cropping, slice-threaded decoders). On the global pool
a worker that took part of it could, waiting inside it, pick up another render
job asking the same cache for a frame and block on the lock, while the lock
holder waited for the work on that worker's stack: exports of rotated iPhone
HEVC clips stopped making frames in about half the runs.
The locked decoding now runs on a separate decode pool, whose threads never wait
for a cache's lock; a caller that is a rayon worker waits on a plain thread so it
does no other rayon work while holding the lock. The cancel/catch-up/draft/
background hints travel with it (filmcraft_media::cancel::Context). Intra-only
streams crop and rotate before taking the lock again.
Second version of #662 (reverted): the regression test's stand-in decoder summed
with overflow, which panics in debug builds; it now wraps explicitly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: Premiere imports Targa and DPX as stills and numbered sequences.
FilmCraft exports both (#779) but could not read either: the extensions
were not importable and the image crate has no DPX reader, so even our
own Targa / DPX renders could not come back in.
Fix: clean-room decoders in filmcraft-media. Targa from the TGA 2.0 spec:
image types 1/2/3 and RLE 9/10/11, colour maps, 15/16/24/32-bit, 8/16-bit
grayscale, any origin. DPX from SMPTE ST 268: first element, luma / RGB /
RGBA / ABGR, 8/16-bit, filled 10/12-bit (methods A and B), either byte
order, padded lines, orientations 0-3. Both bound allocations by the
data present and reject hostile headers with errors. Stills and image
sequences share one decode_raster entry point; "tga" and "dpx" join the
importable still extensions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: symphonia rejects every Layer II frame whose bit rate / mode
combination ISO/IEC 11172-3 does not allow (mono above 192 kbit/s, stereo
below 64 kbit/s). ffmpeg writes mono MP2 at 384 kbit/s by default, so such
tracks played as silence in MPEG-TS / PS, Matroska and raw .mp2 files.
Fix: new L0 crate filmcraft-mpa, a clean-room Layer I / II decoder written
from ISO/IEC 11172-3 and 13818-3 (all modes incl. joint stereo, MPEG-1/2/2.5
rates, the disallowed combinations decoded with the allocation table their
bit rate per channel selects). PacketDecoder::mpeg_audio uses it for layers
I and II (packets of several frames too); raw .mp2 files decode with it,
resyncing after damage. Oracle tests vs ffmpeg: 73-84 dB SNR on ffmpeg /
twolame streams and synthetic Layer I / joint stereo frames; fuzzed damaged
frames never panic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: MP4 and QuickTime name MPEG audio without its layer (ffmpeg
stores MP2 as `mp4a` + esds object type 0x6B), and we always made the
layer III decoder for it. symphonia's decoder reads only the layer it
was made for, so every MP2 frame was rejected and the track played as
silence; MOV's `.mp2` sample entry was not recognised at all.
Fix: the MPEG audio decoder for MP4 / MOV is made on the first packet,
for the layer its frame header names (ISO/IEC 11172-3 2.4.2.3);
`.mp2` maps to the MPEG audio codec like `.mp3`; the track is labelled
"MPEG Audio" for layers I/II, as Matroska's is. Decoded tracks now match
ffmpeg sample-exactly (edit list trims the 481-sample encoder delay).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: ffmpeg stores the encoder priming as CodecDelay in ns (1024 AAC
samples at 48 kHz = 21.333 ms, 256 AC-3 samples at 44.1 kHz = 5.805 ms)
and block timestamps in ms. The demuxer subtracts the delay rounded to
1 ms, and fixed-length packets were anchored on that rounded stamp, so
the whole track played up to 16 samples late against ffmpeg's decode.
Only Opus undid the rounding.
Fix: one stamp-to-samples conversion for every Matroska audio codec
that undoes the demuxer's rounding and removes the exact CodecDelay
(rounded to the nearest sample; the element holds whole ns), with
checked math against hostile timebases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: files with E-AC-3 audio (MP4 ec-3, MOV, Matroska A_EAC3, MPEG-TS)
imported silent: filmcraft-ac3 refused bsid 11-16.
Fix: clean-room E-AC-3 decoding from ATSC A/52:2012 Annex E in filmcraft-ac3:
bsi and audfrm syntax, 1/2/3/6-block syncframes, reduced sample rates,
frame-based exponent strategies (exponents reused across short syncframes),
SNR offset strategies, default band structures, spectral extension
(translation, notch filter, noise blending). Independent substream 0 is
decoded; dependent substreams and other programs are skipped (7.1 plays as
its 5.1 core). AHT and enhanced coupling are refused as unsupported. The
audio blocks share the AC-3 path; AC-3 output is unchanged. filmcraft-codecs
routes E-AC-3 in MP4/MOV, Matroska and TS to the decoder; the TS splitter
joins dependent substream frames to their access unit.
Tests: ffmpeg oracle for raw E-AC-3 (64.8-91.0 dB SNR) and in MP4, MOV,
Matroska and TS (79.5-91.3 dB, random access identical), mutation fuzz,
synthetic frames for headers, substream skipping, refusals and exponent
reuse across 1-block syncframes, spectral extension unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In an open GOP (iPhone HEVC; x265 `open-gop`) each CRA after the first is
followed in the file by RASL pictures shown before it, predicted from the
GOP before. GopCache started a decoding run at `sync_before(i)`, the sync
sample at or before the wanted sample in decoding order: for a RASL
picture that is the CRA after it, and the decoders (software and
VideoToolbox) rightly leave a run's leading RASL pictures out. The frame
was never produced, so the request failed ("frame not produced") and the
clip's layer was left out: a black frame (or the track below) wherever a
seek landed on one of the 3-in-24 RASL frames of iPhone video, such as a
clip trimmed to start on one, the parked playhead, and parallel export.
- `decode_start`: a sample shown before its sync sample starts the run at
the sync sample before (the MPEG and MXF readers already did this in
their own `sync_before`; this covers MP4/MOV and Matroska too).
- A running decoder that started after that point is not continued for
it: it left the picture out.
Tests: a synthetic open-GOP stream in gop.rs, and an x265 open-GOP oracle
test (MP4 and Matroska, every frame from a fresh source and each CRA
before its leading pictures) against ffmpeg. Both fail without the fix.
Brandon Thomas
Marco Arnulfo
vangel76
lloyd
MarekSpodBiedry
MrCloudy2
JordyUSA
Mara Johnson
jahvari
Vikram Bhaskaran
Alex Brazh