shifathosn
/
filmcraft
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

M6.2: filmcraft-aac — clean-room AAC-LC encoder and decoder 

Encoder: transient-driven window switching (long/start/short/stop, grouping),
sine/KBD MDCT, MDCT-domain psychoacoustic model (Bark spreading, SFM tonality,
ATH, PE), TNS, per-band M/S, lambda rate-distortion loop with measured-error
scalefactor search and DP sectioning over all codebooks, bit reservoir with
fill elements (exact CBR), VBR quality scale, 1-8 channels (configs 1-7 + PCE
for 7ch), ASC + optional ADTS, 1024-sample priming reported.

Decoder: full AAC-LC syntax incl. PNS (correlated M/S noise), intensity stereo,
TNS, M/S, pulse data, both window shapes, PCE/DSE/FIL, ADTS helpers; never
panics on malformed input.

Tests use ffmpeg only as an oracle: our streams decode cleanly in ffmpeg
(CBR within 0.1%), our decoder matches ffmpeg to ~1e-7 on ffmpeg-encoded
streams (8-96 kHz, mono..7.1, IS/MS/TNS), plus fuzzing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Thomas 2026-09-30 07:30:13 -04:00
parent 8194dac6a8
commit c2872fbd8d
23 changed files with 6364 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
Cargo.lock generated
View File

@ -1163,6 +1163,15 @@ dependencies = [
"serde_json",
]
[[package]]
name = "filmcraft-aac"
version = "0.1.0"
dependencies = [
"filmcraft-bitstream",
"rayon",
"thiserror 2.0.21",
]
[[package]]
name = "filmcraft-bitstream"
version = "0.1.0"

1
Cargo.toml
View File

@ -17,6 +17,7 @@ filmcraft-geom = { path = "crates/geom" }
filmcraft-color = { path = "crates/color" }
filmcraft-bitstream = { path = "crates/bitstream" }
filmcraft-isobmff = { path = "crates/isobmff" }
filmcraft-aac = { path = "crates/aac" }
filmcraft-frame = { path = "crates/frame" }
filmcraft-media = { path = "crates/media" }
filmcraft-codecs = { path = "crates/codecs" }

19
crates/aac/Cargo.toml Normal file
View File

@ -0,0 +1,19 @@
[package]
name = "filmcraft-aac"
description = "Clean-room AAC-LC encoder and decoder (ISO/IEC 14496-3)"
version.workspace = true
edition.workspace = true
license.workspace = true
rust-version.workspace = true
[lints]
workspace = true
[features]
default = []
rayon = ["dep:rayon"]
[dependencies]
filmcraft-bitstream = { workspace = true }
thiserror = { workspace = true }
rayon = { workspace = true, optional = true }

188
crates/aac/README.md Normal file
View File

@ -0,0 +1,188 @@
# filmcraft-aac
A clean-room AAC-LC encoder and decoder (ISO/IEC 14496-3, MPEG-4 Audio, Low Complexity profile),
written from the standard and the public psychoacoustics literature. Layer L0: it depends only on
`filmcraft-bitstream` and `thiserror` (plus optional `rayon`), has no `unsafe`, and builds for
`wasm32-unknown-unknown`.
No FFmpeg/FAAC/FDK-AAC/libfaad or other GPL/LGPL code was read or ported. ffmpeg is used only as an
external **test oracle** (decoding our streams, producing reference streams).
## API
```rust
use filmcraft_aac::{Decoder, Encoder, EncoderConfig, BitrateMode, AdtsHeader, split_adts};
// Encoder: planar f32 (±1.0) → raw access units
let mut enc = Encoder::new(EncoderConfig::cbr(48_000, 2, 128_000))?; // or EncoderConfig::vbr(48_000, 2, 3.0)
let mut aus: Vec<Vec<u8>> = enc.encode(&[&left, &right]); // 0..n AUs per call, any chunk size
aus.extend(enc.flush()); // zero-pads and drains the look-ahead
let asc = enc.audio_specific_config(); // bytes for the MP4 `esds`
let delay = enc.priming_samples(); // 1024 → edit list media_time
let stats = enc.stats(); // frames, bytes, bitrate(), short blocks, M/S, TNS
// Decoder: AudioSpecificConfig + raw AUs → planar f32, 1024 samples per channel per AU
let mut dec = Decoder::new(&asc)?;
let pcm: Vec<Vec<f32>> = dec.decode(&aus[0])?;
// ADTS
let frames = split_adts(&adts_bytes)?; // [(AdtsHeader, raw payload)]
let mut dec = Decoder::from_adts(&frames[0].0)?; // channel config 0 → layout from the in-band PCE
```
`EncoderConfig` fields: `sample_rate` (the 13 AAC rates, 7.35–96 kHz), `channels` (1–8), `mode`
(`BitrateMode::Cbr(bps)` / `BitrateMode::Vbr(quality 1.0..=5.0)`), `bandwidth` (Hz, `None` = automatic
from bitrate/quality), `tns`, `ms_stereo`, `window_shape` (sine/KBD), `adts` (prefix each AU with a
7-byte ADTS header).
**Channel order** is AAC element order, for both encoder input and decoder output:
| channels | configuration | order |
|---|---|---|
| 1 | 1 | C |
| 2 | 2 | L R |
| 3 | 3 | C L R |
| 4 | 4 | C L R Cs |
| 5 | 5 | C L R Ls Rs |
| 6 | 6 | C L R Ls Rs LFE |
| 7 | 0 + PCE | C L R Ls Rs Cs LFE (PCE in the ASC; repeated in every AU when ADTS-framed) |
| 8 | 7 | C Lc Rc L R Ls Rs LFE |
`Decoder::layout()` reports the element layout (it follows a PCE for configuration 0).
## Encoder
Per 1024-sample frame and channel element:
1. **Transient detection**: energy of the first-difference (high-passed) signal in 128-sample
sub-blocks against the mean of the previous eight; one frame of look-ahead selects
`ONLY_LONG` / `LONG_START` / `EIGHT_SHORT` / `LONG_STOP`. Short windows are grouped by energy
similarity (an attack starts a new group). LFE is always long.
2. **MDCT** (2048/256 points) via a DCT-IV on an N/4-point radix-2 complex FFT; sine or KBD
(α = 4 long, 6 short) windows.
3. **Psychoacoustic model** in the MDCT domain: band energies, tonality from the spectral flatness of
a ≥16-line neighbourhood, Schroeder's spreading function in the Bark domain, tonality-dependent
masking offset (30 dB tonal … 6 dB noise), Terhardt threshold in quiet (capped at 40 dB SPL) as a
floor, pre-echo control (threshold ≤ 2× previous frame), perceptual entropy.
4. **TNS** on long windows: Levinson–Durbin (order 8) over the spectrum above 1.4 kHz; applied when the
prediction gain exceeds 1.4, with 4-bit reflection coefficients.
5. **M/S** per band (and group) for channel pairs (always common window): chosen when the estimated
perceptual cost of M and S (with the stricter L/R threshold, halved) is below that of L and R.
6. **Rate–distortion loop**: one multiplier λ scales every band's allowed noise. Each band gets the
coarsest scalefactor whose *measured* quantisation error meets its target (binary search,
`0.4054` rounding), scalefactor differences are kept within ±60, codebooks and sections are chosen
by dynamic programming over all 12 codebook states, and λ is bracketed and bisected until the frame
fits its budget (CBR) or fixed from the quality (VBR).
7. **Bit reservoir** (CBR): frame budgets follow the perceptual entropy (short-window frames get extra),
bounded by the 6144-bit-per-channel decoder buffer; overflow is padded with fill elements and the
final frame drains the reservoir, so the stream is exactly `frames × bitrate × 1024 / rate`.
Bandwidth: automatic from bitrate per full-bandwidth channel (e.g. 64 kbps/ch → 16 kHz, 96 → 19 kHz,
≥144 → 22 kHz) or from quality in VBR.
Delay: 1024 priming samples (`priming_samples()`); after `flush()` the decoded stream minus the priming
covers every input sample (`ceil(n / 1024) + 1` access units).
Performance (Apple Silicon, release, single thread): 10 s of 44.1 kHz stereo encodes at ~85× realtime
and decodes at ~1000× realtime. With the `rayon` feature channels are quantised in parallel inside the
rate loop.
## Decoder
Full AAC-LC syntax: SCE/CPE/LFE, DSE, FIL, PCE (reconfigures a configuration-0 stream), all window
sequences and both window shapes (shape of the previous frame for the left half), section data,
scalefactors, pulse data, TNS (all-pole synthesis, long and short), M/S (per band / all bands),
intensity stereo (in/out of phase, M/S-inverted), PNS (with correlated noise for `ms_used` noise bands
as specified), IMDCT and overlap-add. Output is planar f32 at ±1.0 full scale. All reads are bounds
checked: malformed access units return `Err`, never panic.
## Test results
All produced by `cargo test -p filmcraft-aac -- --nocapture` (ffmpeg 9.0 at `/opt/homebrew/bin` as
oracle; tests skip when it is absent). Fixtures go to `target/fixtures/aac/`.
### Encoder quality / bitrate (2 s synthetic signals, decoded by ffmpeg)
SNR is full-band; bwSNR is measured after low-passing both signals at the coded bandwidth;
segSNR averages 1024-sample segments above −50 dBFS (clamped to −10…60 dB). "us-vs-ff" is the
largest difference between our decoder and ffmpeg's on the same stream. Every stream decodes without
ffmpeg errors, no output clips (peak < 1.0), and CBR is within ±0.1% of target (test limit ±5%).
| signal | kbps | actual | bw Hz | SNR | bwSNR | segSNR | short frames | us-vs-ff |
|---|---|---|---|---|---|---|---|---|
| sweep 2ch 44.1k | 64 | 64.00 | 11000 | 12.1 | 36.1 | 43.6 | 1 | 5e-7 |
| sweep 2ch 44.1k | 128 | 127.92 | 16000 | 26.1 | 27.3 | 54.2 | 1 | 3e-7 |
| sweep 2ch 44.1k | 320 | 319.90 | 21609 | 46.6 | 46.6 | 59.6 | 1 | 3e-7 |
| white noise 2ch 44.1k | 64 | 64.00 | 11000 | 1.3 | 3.3 | 1.3 | 1 | 5e-7 |
| white noise 2ch 44.1k | 320 | 319.91 | 21609 | 16.4 | 16.5 | 16.6 | 1 | 1.5e-6 |
| speech-like 2ch 48k | 64 | 64.00 | 11000 | 13.6 | 14.0 | 15.2 | 13 | 2e-7 |
| speech-like 2ch 48k | 128 | 127.98 | 16000 | 21.3 | 23.0 | 20.7 | 13 | 2e-7 |
| speech-like 2ch 48k | 320 | 319.96 | 22000 | 31.2 | 31.5 | 31.3 | 13 | 2e-7 |
| chords 2ch 44.1k | 64 | 64.01 | 11000 | 12.4 | 12.4 | 20.6 | 4 | 2e-7 |
| chords 2ch 44.1k | 128 | 128.00 | 16000 | 20.1 | 20.1 | 38.5 | 4 | 2e-7 |
| chords 2ch 44.1k | 320 | 319.99 | 21609 | 35.9 | 35.9 | 55.5 | 4 | 2e-7 |
| clicks 2ch 48k | 64 | 64.00 | 11000 | 10.4 | 11.2 | 48.6 | 11 | 9e-8 |
| clicks 2ch 48k | 128 | 128.00 | 16000 | 20.2 | 25.1 | 53.9 | 11 | 2e-7 |
| clicks 2ch 48k | 320 | 319.95 | 22000 | 39.6 | 39.6 | 58.0 | 11 | 1e-7 |
| speech-like 1ch 44.1k | 64 | 64.00 | 16000 | 18.4 | 19.2 | 18.0 | 15 | 2e-7 |
| chords 5.1 48k | 320 | 320.00 | 16000 | 27.2 | 27.2 | 37.9 | 9 | 2e-7 |
| chords 3/4/5/7/8ch 48k | 192–448 | exact | 16000 | 26–28 | 26–28 | 38 | 7–13 | 2e-7 |
| chords 2ch 8 k … 96 k (10 rates) | 24–256 | exact | 3.9–21 k | 13–38 | | 17–47 | | ≤3e-7 |
Reference points from ffmpeg's own AAC encoder on the same signals (stereo 48 kHz, segSNR):
sweep 128k 39.7, speech 128k 21.2, chords 128k 32.8, clicks 128k 43.3 (ours: 52/20/39/54). Pure SNR
is not a perceptual measure: the model deliberately leaves noise under the masking threshold, and
full-band white noise is the worst case by design.
VBR (`Vbr(q)`, 44.1 kHz stereo): chords q1 62 kbps / q3 139 / q5 225 (segSNR 18 / 39 / 51);
speech-like q1 42 / q3 166 / q5 272 kbps (segSNR 13 / 29 / 41). Bitrate and quality grow
monotonically with q.
### Decoder accuracy vs ffmpeg (streams from `ffmpeg -c:a aac`, PNS off)
| stream | max abs error |
|---|---|
| stereo 44.1k 128k / 48k 256k / 96k 256k | 1.2e-7 / 1.2e-7 / 1.3e-7 |
| mono 48k 64k / 22.05k 32k / 16k 24k / 8k 16k | 1.5e-7 / 1.2e-7 / 1.2e-7 / 8.9e-8 |
| stereo 44.1k 96k with intensity stereo + M/S + TNS | 1.2e-7 |
| stereo 32k 48k with intensity stereo | 1.2e-7 |
| 3.0, 5.1 (320k), 7.1 wide with PCE (448k) | 1.2e-7, 1.5e-7, 1.3e-7 |
| stereo 44.1k 160k `-aac_coder fast` | 1.2e-7 |
Our own encoder's streams: ≤1.6e-6 against ffmpeg for every case above. PNS (ffmpeg's encoder does not
emit it, so streams are hand-built): total energy within 0.15 dB of ffmpeg; per-frame energy within
0.8 dB (random vectors differ). For noise bands flagged `ms_used` we produce identical L/R noise as
the standard specifies; ffmpeg produces independent noise there.
### Robustness
12 000 mutated access units (bit flips, byte overwrites, truncation, extension, splices, random
buffers) over mono/stereo/5.1/7-channel streams, 20 000 random `AudioSpecificConfig`s and 2 000 random
ADTS streams: no panics. The encoder accepts NaN/±inf/over-range input, any chunk size, and missing
channels.
## Clean-room notes
- Syntax, tables (scalefactor bands, TNS limits), filterbank, TNS, PNS, M/S and intensity stereo are
implemented from ISO/IEC 14496-3.
- The Huffman codebooks (`src/huffman_tables.rs`) are the normative values of the standard's Annex
4.A, recovered by black-box probing of the ffmpeg *binary* as an oracle: `tools/` contains the
Python (stdlib-only) scripts that craft single-frame streams repeating a candidate bit string, fit the
decoded output to the IMDCT basis, and walk the code tree. Every table is verified to be a complete
prefix code (Kraft sum exactly 1) covering each index once (unit test), and the decoder matches
ffmpeg to ~1e-7 on real streams.
- Psychoacoustics follow textbook models (Bark scale, Schroeder spreading, Terhardt ATH, Johnston-style
tonality/SFM offsets, perceptual entropy).
## Gaps / not implemented
- HE-AAC v1/v2 (SBR/PS): explicit signalling is parsed and only the AAC-LC core is decoded (half-rate,
no SBR); implicit SBR fill elements are skipped.
- AAC Main (prediction), SSR (gain control), LTP, ER/LD/ELD object types, coupling channel elements
(CCE → `Err(Unsupported)`), 960-sample frames.
- Encoder: no PNS or intensity stereo (decoder supports both), no TNS on short windows, no pulse data;
window shape is fixed per stream (sine or KBD). Short blocks at onsets cost segmental SNR on
speech-like material relative to long windows + TNS; perceptually they avoid pre-echo.
- No error concealment: a corrupt AU returns `Err`; the caller may substitute silence.
- ADTS with CRC is parsed (header skipped, CRC not verified); multi-block ADTS frames are split per
frame only.

436
crates/aac/src/config.rs Normal file
View File

@ -0,0 +1,436 @@
//! `AudioSpecificConfig` (ISO/IEC 14496-3 §1.6.2.1), `program_config_element` (§4.4.1.1) and ADTS
//! framing (§1.A.2.2).
use filmcraft_bitstream::{BitReader, BitWriter};
use crate::tables::{SAMPLE_RATES, table_index_for_rate};
use crate::{Error, Result};
/// Syntactic element types (`id_syn_ele`).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ElementType {
/// single_channel_element
Sce,
/// channel_pair_element
Cpe,
/// lfe_channel_element
Lfe,
}
impl ElementType {
pub fn channels(self) -> usize {
if self == ElementType::Cpe { 2 } else { 1 }
}
}
/// A `program_config_element` (only the parts that affect channel mapping are interpreted).
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub struct ProgramConfig {
pub element_instance_tag: u8,
pub object_type: u8,
pub sf_index: u8,
/// (is_cpe, tag) for front, side and back channel elements.
pub front: Vec<(bool, u8)>,
pub side: Vec<(bool, u8)>,
pub back: Vec<(bool, u8)>,
pub lfe: Vec<u8>,
pub assoc_data: Vec<u8>,
/// (is_ind_sw, tag)
pub cc: Vec<(bool, u8)>,
pub comment: Vec<u8>,
}
impl ProgramConfig {
pub fn parse(br: &mut BitReader) -> Result<ProgramConfig> {
let mut p = ProgramConfig {
element_instance_tag: br.read_bits(4)? as u8,
object_type: br.read_bits(2)? as u8,
sf_index: br.read_bits(4)? as u8,
..Default::default()
};
let nf = br.read_bits(4)?;
let ns = br.read_bits(4)?;
let nb = br.read_bits(4)?;
let nl = br.read_bits(2)?;
let na = br.read_bits(3)?;
let nc = br.read_bits(4)?;
if br.read_bits(1)? == 1 {
br.skip(4)?;
}
if br.read_bits(1)? == 1 {
br.skip(4)?;
}
if br.read_bits(1)? == 1 {
br.skip(3)?;
}
for (n, v) in [(nf, &mut p.front), (ns, &mut p.side), (nb, &mut p.back)] {
for _ in 0..n {
let cpe = br.read_bits(1)? == 1;
v.push((cpe, br.read_bits(4)? as u8));
}
}
for _ in 0..nl {
p.lfe.push(br.read_bits(4)? as u8);
}
for _ in 0..na {
p.assoc_data.push(br.read_bits(4)? as u8);
}
for _ in 0..nc {
let ind = br.read_bits(1)? == 1;
p.cc.push((ind, br.read_bits(4)? as u8));
}
br.byte_align();
let len = br.read_bits(8)?;
for _ in 0..len {
p.comment.push(br.read_bits(8)? as u8);
}
Ok(p)
}
pub fn write(&self, bw: &mut BitWriter) {
bw.write_bits(self.element_instance_tag as u32, 4);
bw.write_bits(self.object_type as u32, 2);
bw.write_bits(self.sf_index as u32, 4);
bw.write_bits(self.front.len() as u32, 4);
bw.write_bits(self.side.len() as u32, 4);
bw.write_bits(self.back.len() as u32, 4);
bw.write_bits(self.lfe.len() as u32, 2);
bw.write_bits(self.assoc_data.len() as u32, 3);
bw.write_bits(self.cc.len() as u32, 4);
bw.write_bits(0, 3); // no mixdowns
for v in [&self.front, &self.side, &self.back] {
for &(cpe, tag) in v {
bw.write_bits(cpe as u32, 1);
bw.write_bits(tag as u32, 4);
}
}
for &t in &self.lfe {
bw.write_bits(t as u32, 4);
}
for &t in &self.assoc_data {
bw.write_bits(t as u32, 4);
}
for &(ind, t) in &self.cc {
bw.write_bits(ind as u32, 1);
bw.write_bits(t as u32, 4);
}
bw.align_zero();
bw.write_bits(self.comment.len().min(255) as u32, 8);
for &b in self.comment.iter().take(255) {
bw.write_bits(b as u32, 8);
}
}
/// Elements in output channel order (front, side, back, LFE).
pub fn layout(&self) -> Vec<(ElementType, u8)> {
let mut out = Vec::new();
for v in [&self.front, &self.side, &self.back] {
for &(cpe, tag) in v {
out.push((if cpe { ElementType::Cpe } else { ElementType::Sce }, tag));
}
}
for &t in &self.lfe {
out.push((ElementType::Lfe, t));
}
out
}
}
/// Element layout of `channelConfiguration` 1..=7 (tags as conventionally assigned).
pub fn config_layout(channel_config: u8) -> Option<Vec<(ElementType, u8)>> {
use ElementType::*;
Some(match channel_config {
1 => vec![(Sce, 0)],
2 => vec![(Cpe, 0)],
3 => vec![(Sce, 0), (Cpe, 0)],
4 => vec![(Sce, 0), (Cpe, 0), (Sce, 1)],
5 => vec![(Sce, 0), (Cpe, 0), (Cpe, 1)],
6 => vec![(Sce, 0), (Cpe, 0), (Cpe, 1), (Lfe, 0)],
7 => vec![(Sce, 0), (Cpe, 0), (Cpe, 1), (Cpe, 2), (Lfe, 0)],
_ => return None,
})
}
/// Parsed `AudioSpecificConfig`.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct AudioSpecificConfig {
/// Audio object type of the core (2 = AAC LC).
pub object_type: u8,
/// Core sampling rate.
pub sample_rate: u32,
/// Index into the scalefactor-band tables (the signalled index, or derived from an explicit rate).
pub sf_index: u8,
pub channel_config: u8,
pub frame_length_flag: bool,
/// Explicitly signalled SBR/PS (HE-AAC v1/v2); only the AAC-LC core is decoded.
pub sbr: bool,
pub ps: bool,
/// Output (SBR) rate when `sbr` is set.
pub extension_sample_rate: Option<u32>,
pub pce: Option<ProgramConfig>,
}
fn read_aot(br: &mut BitReader) -> Result<u8> {
let a = br.read_bits(5)? as u8;
Ok(if a == 31 { 32 + br.read_bits(6)? as u8 } else { a })
}
fn read_rate(br: &mut BitReader) -> Result<(u8, u32)> {
let i = br.read_bits(4)? as u8;
if i == 15 {
let r = br.read_bits(24)?;
return Ok((table_index_for_rate(r), r));
}
let r = *SAMPLE_RATES.get(i as usize).ok_or(Error::InvalidConfig("reserved sampling frequency index"))?;
Ok((i, r))
}
impl AudioSpecificConfig {
/// A plain AAC-LC configuration.
pub fn lc(sf_index: u8, channel_config: u8) -> AudioSpecificConfig {
AudioSpecificConfig {
object_type: 2,
sample_rate: SAMPLE_RATES[sf_index as usize],
sf_index,
channel_config,
frame_length_flag: false,
sbr: false,
ps: false,
extension_sample_rate: None,
pce: None,
}
}
pub fn parse(data: &[u8]) -> Result<AudioSpecificConfig> {
let mut br = BitReader::new(data);
let mut aot = read_aot(&mut br)?;
let (sf_index, sample_rate) = read_rate(&mut br)?;
let channel_config = br.read_bits(4)? as u8;
let (mut sbr, mut ps, mut extension_sample_rate) = (false, false, None);
if aot == 5 || aot == 29 {
sbr = true;
ps = aot == 29;
let (_, ext) = read_rate(&mut br)?;
extension_sample_rate = Some(ext);
aot = read_aot(&mut br)?;
}
if aot != 2 {
return Err(Error::Unsupported(match aot {
1 => "AAC Main profile",
3 => "AAC SSR profile",
4 => "AAC LTP profile",
23 => "AAC-LD",
39 => "AAC-ELD",
_ => "audio object type other than AAC LC",
}));
}
let frame_length_flag = br.read_bits(1)? == 1;
if br.read_bits(1)? == 1 {
br.skip(14)?; // coreCoderDelay
}
let _extension_flag = br.read_bits(1)?;
let pce = if channel_config == 0 { Some(ProgramConfig::parse(&mut br)?) } else { None };
if frame_length_flag {
return Err(Error::Unsupported("960-sample frames"));
}
let cfg = AudioSpecificConfig { object_type: aot, sample_rate, sf_index, channel_config, frame_length_flag, sbr, ps, extension_sample_rate, pce };
if cfg.channels() == 0 {
return Err(Error::InvalidConfig("no channels"));
}
Ok(cfg)
}
/// Serialise (AAC-LC, no SBR signalling).
pub fn to_bytes(&self) -> Vec<u8> {
let mut bw = BitWriter::new();
bw.write_bits(self.object_type as u32, 5);
match crate::tables::sample_rate_index(self.sample_rate) {
Some(i) => bw.write_bits(i as u32, 4),
None => {
bw.write_bits(15, 4);
bw.write_bits(self.sample_rate, 24);
}
}
bw.write_bits(self.channel_config as u32, 4);
bw.write_bits(0, 3); // frameLengthFlag, dependsOnCoreCoder, extensionFlag
if self.channel_config == 0
&& let Some(p) = &self.pce
{
p.write(&mut bw);
}
bw.finish()
}
/// Elements in output channel order.
pub fn layout(&self) -> Vec<(ElementType, u8)> {
match &self.pce {
Some(p) if self.channel_config == 0 => p.layout(),
_ => config_layout(self.channel_config).unwrap_or_default(),
}
}
pub fn channels(&self) -> usize {
self.layout().iter().map(|e| e.0.channels()).sum()
}
}
/// A parsed ADTS header.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct AdtsHeader {
/// Audio object type (`profile + 1`).
pub object_type: u8,
pub sf_index: u8,
pub channel_config: u8,
/// Whole frame length including the header.
pub frame_length: usize,
pub buffer_fullness: u16,
/// Header length (7, or 9 with CRC).
pub header_length: usize,
pub raw_data_blocks: u8,
}
impl AdtsHeader {
pub fn parse(data: &[u8]) -> Result<AdtsHeader> {
let mut br = BitReader::new(data);
if br.read_bits(12)? != 0xFFF {
return Err(Error::Bitstream("missing ADTS syncword"));
}
br.skip(1)?; // ID
if br.read_bits(2)? != 0 {
return Err(Error::Bitstream("ADTS layer must be 0"));
}
let protection_absent = br.read_bits(1)? == 1;
let object_type = br.read_bits(2)? as u8 + 1;
let sf_index = br.read_bits(4)? as u8;
if sf_index as usize >= SAMPLE_RATES.len() {
return Err(Error::Bitstream("invalid ADTS sampling frequency index"));
}
br.skip(1)?;
let channel_config = br.read_bits(3)? as u8;
br.skip(4)?;
let frame_length = br.read_bits(13)? as usize;
let buffer_fullness = br.read_bits(11)? as u16;
let raw_data_blocks = br.read_bits(2)? as u8 + 1;
let header_length = if protection_absent { 7 } else { 9 };
if frame_length < header_length {
return Err(Error::Bitstream("ADTS frame shorter than its header"));
}
Ok(AdtsHeader { object_type, sf_index, channel_config, frame_length, buffer_fullness, header_length, raw_data_blocks })
}
/// The equivalent `AudioSpecificConfig`. Channel configuration 0 (PCE in-band) is resolved by the
/// decoder from the first frame; this returns stereo-less layout in that case.
pub fn to_asc(&self) -> AudioSpecificConfig {
AudioSpecificConfig { object_type: self.object_type, ..AudioSpecificConfig::lc(self.sf_index, self.channel_config) }
}
/// Write a 7-byte header (no CRC) for a frame whose raw payload is `payload_len` bytes.
pub fn write(object_type: u8, sf_index: u8, channel_config: u8, payload_len: usize, buffer_fullness: u16) -> [u8; 7] {
let len = (payload_len + 7) as u32;
let mut bw = BitWriter::new();
bw.write_bits(0xFFF, 12);
bw.write_bits(0, 1); // MPEG-4
bw.write_bits(0, 2);
bw.write_bits(1, 1); // protection absent
bw.write_bits((object_type.max(1) - 1) as u32 & 3, 2);
bw.write_bits(sf_index as u32, 4);
bw.write_bits(0, 1);
bw.write_bits(channel_config as u32 & 7, 3);
bw.write_bits(0, 4);
bw.write_bits(len & 0x1FFF, 13);
bw.write_bits(buffer_fullness as u32 & 0x7FF, 11);
bw.write_bits(0, 2);
let v = bw.finish();
let mut out = [0u8; 7];
out.copy_from_slice(&v);
out
}
}
/// Split an ADTS stream into `(header, raw payload)` frames. Stops at the first malformed or
/// truncated frame (returning what was parsed so far); errors only if nothing parses.
pub fn split_adts(mut data: &[u8]) -> Result<Vec<(AdtsHeader, &[u8])>> {
let mut out = Vec::new();
while data.len() >= 7 {
let h = match AdtsHeader::parse(data) {
Ok(h) => h,
Err(e) => {
if out.is_empty() {
return Err(e);
}
break;
}
};
if h.frame_length > data.len() {
break;
}
out.push((h, &data[h.header_length..h.frame_length]));
data = &data[h.frame_length..];
}
if out.is_empty() {
return Err(Error::Bitstream("no ADTS frames"));
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn asc_roundtrip() {
let a = AudioSpecificConfig::lc(4, 2);
let b = a.to_bytes();
assert_eq!(b, vec![0x12, 0x10]);
assert_eq!(AudioSpecificConfig::parse(&b).unwrap(), a);
// 48 kHz 5.1
let a = AudioSpecificConfig::lc(3, 6);
assert_eq!(AudioSpecificConfig::parse(&a.to_bytes()).unwrap().channels(), 6);
}
#[test]
fn asc_with_pce() {
let pce = ProgramConfig {
sf_index: 3,
object_type: 1,
front: vec![(false, 0), (true, 0)],
side: vec![(true, 1)],
back: vec![(false, 1)],
lfe: vec![0],
..Default::default()
};
let a = AudioSpecificConfig { pce: Some(pce), ..AudioSpecificConfig::lc(3, 0) };
let p = AudioSpecificConfig::parse(&a.to_bytes()).unwrap();
assert_eq!(p.channels(), 7);
assert_eq!(p, a);
}
#[test]
fn he_aac_asc_parses_core() {
// AOT 5, 24 kHz core, stereo, ext 48 kHz, core AOT 2
let mut bw = BitWriter::new();
bw.write_bits(5, 5);
bw.write_bits(6, 4);
bw.write_bits(2, 4);
bw.write_bits(3, 4);
bw.write_bits(2, 5);
bw.write_bits(0, 3);
let a = AudioSpecificConfig::parse(&bw.finish()).unwrap();
assert!(a.sbr);
assert_eq!(a.sample_rate, 24000);
assert_eq!(a.extension_sample_rate, Some(48000));
}
#[test]
fn adts_roundtrip() {
let h = AdtsHeader::write(2, 4, 2, 100, 0x7FF);
let mut frame = h.to_vec();
frame.extend(std::iter::repeat_n(0u8, 100));
let p = AdtsHeader::parse(&frame).unwrap();
assert_eq!(p.frame_length, 107);
assert_eq!((p.object_type, p.sf_index, p.channel_config), (2, 4, 2));
let frames = split_adts(&frame).unwrap();
assert_eq!(frames.len(), 1);
assert_eq!(frames[0].1.len(), 100);
}
}

544
crates/aac/src/decoder.rs Normal file
View File

@ -0,0 +1,544 @@
//! AAC-LC decoder: raw access units (`raw_data_block`, ISO/IEC 14496-3 §4.4.2.1) → planar f32.
use filmcraft_bitstream::BitReader;
use crate::config::{AudioSpecificConfig, ElementType, ProgramConfig};
use crate::huffman::{self, INTENSITY_HCB, INTENSITY_HCB2, NOISE_HCB, ZERO_HCB};
use crate::ics::{IcsInfo, WindowSequence};
use crate::mdct::{WindowShape, mdct_long, mdct_short, window};
use crate::tables::{pow43, sf_gain};
use crate::tns::{self, TnsData};
use crate::{Error, Result};
const ID_SCE: u32 = 0;
const ID_CPE: u32 = 1;
const ID_CCE: u32 = 2;
const ID_LFE: u32 = 3;
const ID_DSE: u32 = 4;
const ID_PCE: u32 = 5;
const ID_FIL: u32 = 6;
const ID_END: u32 = 7;
/// One decoded `individual_channel_stream` before stereo processing and synthesis.
struct Ics {
info: IcsInfo,
/// Band type per (group, sfb).
band_type: [[u8; 64]; 8],
/// Scalefactor / noise energy / intensity position per (group, sfb).
sf: [[i32; 64]; 8],
tns: Option<TnsData>,
/// Dequantised spectrum (short windows at stride 128).
spec: Vec<f32>,
}
#[derive(Clone)]
struct ChannelState {
overlap: Vec<f32>,
prev_shape: WindowShape,
}
/// AAC-LC decoder.
pub struct Decoder {
asc: AudioSpecificConfig,
layout: Vec<(ElementType, u8)>,
/// First output channel of each layout entry.
offsets: Vec<usize>,
channels: Vec<ChannelState>,
noise_state: u32,
}
impl Decoder {
/// Create a decoder from an `AudioSpecificConfig`.
pub fn new(asc: &[u8]) -> Result<Decoder> {
Self::from_config(AudioSpecificConfig::parse(asc)?)
}
/// Create a decoder for an ADTS stream from its first header. With channel configuration 0 the
/// channel layout is taken from the program config element carried in the first frame.
pub fn from_adts(header: &crate::config::AdtsHeader) -> Result<Decoder> {
Self::from_config(header.to_asc())
}
pub fn from_config(asc: AudioSpecificConfig) -> Result<Decoder> {
if asc.object_type != 2 {
return Err(Error::Unsupported("audio object type other than AAC LC"));
}
let layout = asc.layout();
if layout.is_empty() && asc.channel_config != 0 {
return Err(Error::InvalidConfig("unsupported channel configuration"));
}
let mut d = Decoder { asc, layout: Vec::new(), offsets: Vec::new(), channels: Vec::new(), noise_state: 0x2545_F491 };
d.set_layout(layout);
Ok(d)
}
fn set_layout(&mut self, layout: Vec<(ElementType, u8)>) {
let mut offsets = Vec::with_capacity(layout.len());
let mut n = 0;
for e in &layout {
offsets.push(n);
n += e.0.channels();
}
self.layout = layout;
self.offsets = offsets;
self.channels = vec![ChannelState { overlap: vec![0.0; 1024], prev_shape: WindowShape::Sine }; n];
}
pub fn config(&self) -> &AudioSpecificConfig {
&self.asc
}
pub fn sample_rate(&self) -> u32 {
self.asc.sample_rate
}
pub fn channels(&self) -> usize {
self.channels.len()
}
/// Element layout in output channel order (C, L/R, surround pairs, LFE for configurations 1–7).
pub fn layout(&self) -> &[(ElementType, u8)] {
&self.layout
}
/// Clear the overlap state (e.g. after a seek).
pub fn reset(&mut self) {
for c in &mut self.channels {
c.overlap.fill(0.0);
c.prev_shape = WindowShape::Sine;
}
}
fn sf_index(&self) -> u8 {
self.asc.sf_index
}
/// Decode one access unit (one `raw_data_block`) into 1024 samples per channel.
pub fn decode(&mut self, au: &[u8]) -> Result<Vec<Vec<f32>>> {
self.decode_block(au, true)
}
fn decode_block(&mut self, au: &[u8], allow_reconfig: bool) -> Result<Vec<Vec<f32>>> {
let mut br = BitReader::new(au);
let nch = self.channels.len();
let mut saw_element = false;
let mut out: Vec<Option<Vec<f32>>> = vec![None; nch];
let mut used = vec![false; self.layout.len()];
loop {
let id = br.read_bits(3)?;
saw_element |= matches!(id, ID_SCE | ID_CPE | ID_LFE);
match id {
ID_SCE | ID_LFE => {
let tag = br.read_bits(4)? as u8;
let ty = if id == ID_SCE { ElementType::Sce } else { ElementType::Lfe };
let slot = self.find_slot(ty, tag, &used)?;
used[slot] = true;
let mut ics = self.decode_ics(&mut br, None)?;
let ch = self.offsets[slot];
self.fill_noise(&mut ics, None);
out[ch] = Some(self.finish_channel(ch, &mut ics));
}
ID_CPE => {
let tag = br.read_bits(4)? as u8;
let slot = self.find_slot(ElementType::Cpe, tag, &used)?;
used[slot] = true;
let ch = self.offsets[slot];
let [a, b] = self.decode_cpe(&mut br, ch)?;
out[ch] = Some(a);
out[ch + 1] = Some(b);
}
ID_CCE => return Err(Error::Unsupported("coupling channel element")),
ID_DSE => {
br.skip(4)?;
let align = br.read_bits(1)? == 1;
let mut count = br.read_bits(8)? as usize;
if count == 255 {
count += br.read_bits(8)? as usize;
}
if align {
br.byte_align();
}
br.skip(count * 8)?;
}
ID_PCE => {
let pce = ProgramConfig::parse(&mut br)?;
if allow_reconfig && !saw_element && self.asc.channel_config == 0 && self.asc.pce.as_ref() != Some(&pce) {
let layout = pce.layout();
if !layout.is_empty() && layout.iter().map(|e| e.0.channels()).sum::<usize>() <= 64 {
self.asc.pce = Some(pce);
self.set_layout(layout);
return self.decode_block(au, false);
}
}
}
ID_FIL => {
let mut count = br.read_bits(4)? as usize;
if count == 15 {
count += br.read_bits(8)? as usize;
count -= 1;
}
br.skip(count * 8)?;
}
_ => break, // ID_END
}
if id == ID_END {
break;
}
}
if nch == 0 && !saw_element {
return Err(Error::InvalidConfig("channel configuration 0 without a program config element"));
}
let mut res = Vec::with_capacity(nch);
for o in out {
res.push(o.ok_or(Error::Bitstream("access unit is missing a channel element"))?);
}
Ok(res)
}
fn find_slot(&self, ty: ElementType, tag: u8, used: &[bool]) -> Result<usize> {
let compatible = |t: ElementType| t == ty || (ty != ElementType::Cpe && t != ElementType::Cpe);
if let Some(i) = self.layout.iter().enumerate().position(|(i, &(t, g))| t == ty && g == tag && !used[i]) {
return Ok(i);
}
if let Some(i) = self.layout.iter().enumerate().position(|(i, &(t, _))| t == ty && !used[i]) {
return Ok(i);
}
self.layout
.iter()
.enumerate()
.position(|(i, &(t, _))| compatible(t) && !used[i])
.ok_or(Error::Bitstream("unexpected channel element for the channel configuration"))
}
fn decode_cpe(&mut self, br: &mut BitReader, ch: usize) -> Result<[Vec<f32>; 2]> {
let common_window = br.read_bits(1)? == 1;
let mut ms_mask_present = 0;
let mut ms_used = [[false; 64]; 8];
let mut info = None;
if common_window {
let i = IcsInfo::parse(br, self.sf_index())?;
ms_mask_present = br.read_bits(2)?;
if ms_mask_present == 3 {
return Err(Error::Bitstream("reserved ms_mask_present"));
}
if ms_mask_present == 1 {
for g in 0..i.num_groups {
for sfb in 0..i.max_sfb {
ms_used[g][sfb] = br.read_bits(1)? == 1;
}
}
} else if ms_mask_present == 2 {
ms_used = [[true; 64]; 8];
}
info = Some(i);
}
let mut l = self.decode_ics(br, info)?;
let mut r = self.decode_ics(br, info)?;
let ms = if ms_mask_present > 0 { Some(&ms_used) } else { None };
self.fill_noise(&mut l, None);
self.fill_noise(&mut r, if common_window { Some((&l, ms)) } else { None });
if common_window {
let swb = l.info.swb_offsets(self.sf_index());
for g in 0..l.info.num_groups {
let w0 = l.info.group_start(g);
for sfb in 0..l.info.max_sfb {
let (bl, br_) = (l.band_type[g][sfb], r.band_type[g][sfb]);
let range = swb[sfb] as usize..swb[sfb + 1] as usize;
if br_ == INTENSITY_HCB || br_ == INTENSITY_HCB2 {
let mut c: f32 = if br_ == INTENSITY_HCB { 1.0 } else { -1.0 };
if ms_mask_present > 0 && ms_used[g][sfb] {
c = -c;
}
let scale = c * (-0.25 * r.sf[g][sfb].clamp(-400, 400) as f64).exp2() as f32;
for w in w0..w0 + l.info.group_len[g] as usize {
for k in range.clone() {
r.spec[w * 128 + k] = l.spec[w * 128 + k] * scale;
}
}
} else if ms_mask_present > 0 && ms_used[g][sfb] && bl < NOISE_HCB && br_ < NOISE_HCB {
for w in w0..w0 + l.info.group_len[g] as usize {
for k in range.clone() {
let (m, s) = (l.spec[w * 128 + k], r.spec[w * 128 + k]);
l.spec[w * 128 + k] = m + s;
r.spec[w * 128 + k] = m - s;
}
}
}
}
}
}
Ok([self.finish_channel(ch, &mut l), self.finish_channel(ch + 1, &mut r)])
}
fn decode_ics(&mut self, br: &mut BitReader, common: Option<IcsInfo>) -> Result<Ics> {
let sfi = self.sf_index();
let global_gain = br.read_bits(8)? as i32;
let info = match common {
Some(i) => i,
None => IcsInfo::parse(br, sfi)?,
};
let swb = info.swb_offsets(sfi);
let num_swb = swb.len() - 1;
let mut ics = Ics { info, band_type: [[0; 64]; 8], sf: [[0; 64]; 8], tns: None, spec: vec![0.0; 1024] };
// section_data
let (sect_bits, esc) = if info.is_short() { (3, 7) } else { (5, 31) };
for g in 0..info.num_groups {
let mut k = 0;
while k < info.max_sfb {
let cb = br.read_bits(4)? as u8;
if cb == 12 {
return Err(Error::Bitstream("reserved codebook 12"));
}
let mut len = 0usize;
loop {
let inc = br.read_bits(sect_bits)?;
len += inc as usize;
if inc != esc {
break;
}
}
if k + len > info.max_sfb {
return Err(Error::Bitstream("section exceeds max_sfb"));
}
if len == 0 {
// zero-length sections are legal but useless; guard against endless loops
if br.bits_left() == 0 {
return Err(Error::Eof);
}
}
for sfb in k..k + len {
ics.band_type[g][sfb] = cb;
}
k += len;
}
}
// scale_factor_data
let mut sf = global_gain;
let mut noise = global_gain - 90;
let mut is_pos = 0i32;
let mut noise_first = true;
for g in 0..info.num_groups {
for sfb in 0..info.max_sfb {
match ics.band_type[g][sfb] {
ZERO_HCB => ics.sf[g][sfb] = 0,
INTENSITY_HCB | INTENSITY_HCB2 => {
is_pos += huffman::decode_sf(br)?;
ics.sf[g][sfb] = is_pos;
}
NOISE_HCB => {
if noise_first {
noise += br.read_bits(9)? as i32 - 256;
noise_first = false;
} else {
noise += huffman::decode_sf(br)?;
}
ics.sf[g][sfb] = noise;
}
_ => {
sf += huffman::decode_sf(br)?;
if !(0..=255).contains(&sf) {
return Err(Error::Bitstream("scalefactor out of range"));
}
ics.sf[g][sfb] = sf;
}
}
}
}
// pulse_data
let mut pulses: Vec<(usize, i32)> = Vec::new();
if br.read_bits(1)? == 1 {
if info.is_short() {
return Err(Error::Bitstream("pulse data in short window"));
}
let n = br.read_bits(2)? as usize + 1;
let start = br.read_bits(6)? as usize;
if start >= num_swb {
return Err(Error::Bitstream("pulse_start_sfb out of range"));
}
let mut k = swb[start] as usize;
for _ in 0..n {
k += br.read_bits(5)? as usize;
let amp = br.read_bits(4)? as i32;
if k >= 1024 {
return Err(Error::Bitstream("pulse position out of range"));
}
pulses.push((k, amp));
}
}
if br.read_bits(1)? == 1 {
ics.tns = Some(TnsData::parse(br, &info)?);
}
if br.read_bits(1)? == 1 {
return Err(Error::Unsupported("gain control data (AAC SSR)"));
}
// spectral_data
let mut q = vec![0i32; 1024];
let mut vals = [0i32; 4];
for g in 0..info.num_groups {
let w0 = info.group_start(g);
let glen = info.group_len[g] as usize;
for sfb in 0..info.max_sfb {
let cb = ics.band_type[g][sfb];
if cb == ZERO_HCB || cb >= NOISE_HCB {
continue;
}
let dim = huffman::book(cb).dim;
let (lo, hi) = (swb[sfb] as usize, swb[sfb + 1] as usize);
for w in w0..w0 + glen {
let base = w * 128;
let mut k = lo;
while k < hi {
huffman::decode_spectral(br, cb, &mut vals)?;
q[base + k..base + k + dim].copy_from_slice(&vals[..dim]);
k += dim;
}
}
}
}
for (k, amp) in pulses {
if q[k] > 0 {
q[k] += amp;
} else {
q[k] -= amp;
}
}
// dequantise
for g in 0..info.num_groups {
let w0 = info.group_start(g);
for sfb in 0..info.max_sfb {
let cb = ics.band_type[g][sfb];
if cb == ZERO_HCB || cb >= NOISE_HCB {
continue;
}
let gain = sf_gain(ics.sf[g][sfb]);
for w in w0..w0 + info.group_len[g] as usize {
for k in swb[sfb] as usize..swb[sfb + 1] as usize {
let v = q[w * 128 + k];
let m = pow43(v.unsigned_abs()) * gain;
ics.spec[w * 128 + k] = if v < 0 { -m } else { m };
}
}
}
}
Ok(ics)
}
fn next_random(&mut self) -> f32 {
self.noise_state = self.noise_state.wrapping_mul(1_664_525).wrapping_add(1_013_904_223);
(self.noise_state as i32) as f32
}
/// Perceptual noise substitution. `pair` = (left channel, ms mask) for the right channel of a CPE:
/// bands that are noise in both channels with `ms_used` reuse the left channel's noise vector.
fn fill_noise(&mut self, ics: &mut Ics, pair: Option<(&Ics, Option<&[[bool; 64]; 8]>)>) {
let info = ics.info;
let swb = info.swb_offsets(self.sf_index());
for g in 0..info.num_groups {
let w0 = info.group_start(g);
for sfb in 0..info.max_sfb {
if ics.band_type[g][sfb] != NOISE_HCB {
continue;
}
let (lo, hi) = (swb[sfb] as usize, swb[sfb + 1] as usize);
let target = (0.25 * ics.sf[g][sfb].clamp(-400, 400) as f64).exp2();
let correlated = match pair {
Some((l, Some(ms))) => ms[g][sfb] && l.band_type[g][sfb] == NOISE_HCB,
_ => false,
};
for w in w0..w0 + info.group_len[g] as usize {
let band = w * 128 + lo..w * 128 + hi;
if correlated && let Some((l, _)) = pair {
ics.spec[band.clone()].copy_from_slice(&l.spec[band.clone()]);
} else {
for k in band.clone() {
ics.spec[k] = self.next_random();
}
}
let energy: f64 = ics.spec[band.clone()].iter().map(|&v| v as f64 * v as f64).sum();
let scale = if energy > 0.0 { (target / energy.sqrt()) as f32 } else { 0.0 };
for k in band {
ics.spec[k] *= scale;
}
}
}
}
}
/// TNS + filterbank for a single channel element; returns output samples.
fn finish_channel(&mut self, ch: usize, ics: &mut Ics) -> Vec<f32> {
if let Some(t) = &ics.tns {
tns::apply_decoder(&mut ics.spec, &ics.info, self.sf_index(), t);
}
synth(&mut self.channels[ch], &ics.info, &ics.spec)
}
}
/// Inverse MDCT, windowing and overlap-add (§4.6.18), output scaled to ±1.0 full scale.
fn synth(state: &mut ChannelState, info: &IcsInfo, spec: &[f32]) -> Vec<f32> {
let mut buf = vec![0f32; 2048];
let prev = state.prev_shape;
let cur = info.window_shape;
let norm = 1.0 / 32768.0;
match info.window_sequence {
WindowSequence::EightShort => {
let t = mdct_short();
let mut y = vec![0f32; 256];
let wc = window(cur, false);
for w in 0..8 {
t.inverse(&spec[w * 128..(w + 1) * 128], &mut y);
let wl = window(if w == 0 { prev } else { cur }, false);
let off = 448 + 128 * w;
let s = norm / 128.0;
for n in 0..128 {
buf[off + n] += y[n] * wl[n] * s;
buf[off + 128 + n] += y[128 + n] * wc[127 - n] * s;
}
}
}
seq => {
let mut y = vec![0f32; 2048];
mdct_long().inverse(&spec[..1024], &mut y);
let s = norm / 1024.0;
match seq {
WindowSequence::LongStop => {
let ws = window(prev, false);
for n in 448..576 {
buf[n] = y[n] * ws[n - 448] * s;
}
for n in 576..1024 {
buf[n] = y[n] * s;
}
}
_ => {
let wl = window(prev, true);
for n in 0..1024 {
buf[n] = y[n] * wl[n] * s;
}
}
}
match seq {
WindowSequence::LongStart => {
let ws = window(cur, false);
for n in 1024..1472 {
buf[n] = y[n] * s;
}
for n in 1472..1600 {
buf[n] = y[n] * ws[127 - (n - 1472)] * s;
}
}
_ => {
let wl = window(cur, true);
for n in 1024..2048 {
buf[n] = y[n] * wl[2047 - n] * s;
}
}
}
}
}
let out: Vec<f32> = (0..1024).map(|n| state.overlap[n] + buf[n]).collect();
state.overlap.copy_from_slice(&buf[1024..]);
state.prev_shape = cur;
out
}

831
crates/aac/src/encoder/mod.rs Normal file
View File

@ -0,0 +1,831 @@
//! AAC-LC encoder.
//!
//! Pipeline per 1024-sample frame and channel element:
//! 1. transient detection on high-passed 128-sample sub-blocks (one frame of look-ahead) →
//! window sequence (`ONLY_LONG` / `LONG_START` / `EIGHT_SHORT` / `LONG_STOP`) and short-window grouping;
//! 2. windowed MDCT (sine or KBD);
//! 3. psychoacoustic model → per-band masking thresholds and perceptual entropy;
//! 4. TNS analysis (long windows; Levinson–Durbin, applied when the prediction gain is high);
//! 5. per-band M/S decision for channel pairs (common window);
//! 6. rate–distortion loop: a single multiplier λ scales every band's allowed noise; each band gets the
//! coarsest scalefactor meeting its noise target, sections/codebooks are chosen by dynamic
//! programming, and λ is bisected until the frame fits its bit budget (CBR) or fixed (VBR);
//! 7. bit reservoir (CBR): budget follows the frame's PE, bounded by the decoder buffer
//! (6144 bits/channel); overflow is padded with fill elements so the rate stays constant.
mod psy;
mod quant;
use filmcraft_bitstream::BitWriter;
use crate::config::{AdtsHeader, AudioSpecificConfig, ElementType, ProgramConfig, config_layout};
use crate::ics::{IcsInfo, WindowSequence};
use crate::mdct::{WindowShape, mdct_long, mdct_short, window};
use crate::tables::{sample_rate_index, swb_offsets_long, swb_offsets_short, tns_max_bands};
use crate::tns::{self, TnsData, TnsFilter};
use crate::{Error, FRAME_LEN, Result};
use psy::Psy;
use quant::{ChannelInput, QuantChannel, quantise_channel, write_ics};
/// Rate control mode.
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum BitrateMode {
/// Constant bitrate in bits per second (bit reservoir + fill elements).
Cbr(u32),
/// Constant quality, 1.0 (small) ..= 5.0 (transparent-ish). 3.0 targets the masking threshold.
Vbr(f32),
}
/// Encoder settings.
#[derive(Debug, Clone, PartialEq)]
pub struct EncoderConfig {
/// One of the 13 AAC sampling rates (8–96 kHz).
pub sample_rate: u32,
/// 1–8. Channel order is the AAC element order: C; L R; C L R; C L R Cs; C L R Ls Rs;
/// C L R Ls Rs LFE; (7) C L R Ls Rs Cs LFE; (8) C Lc Rc L R Ls Rs LFE.
pub channels: usize,
pub mode: BitrateMode,
/// Audio bandwidth in Hz (`None` = chosen from the bitrate / quality).
pub bandwidth: Option<u32>,
/// Temporal noise shaping on long windows.
pub tns: bool,
/// Per-band mid/side stereo for channel pairs.
pub ms_stereo: bool,
pub window_shape: WindowShape,
/// Prefix every access unit with an ADTS header.
pub adts: bool,
}
impl EncoderConfig {
pub fn cbr(sample_rate: u32, channels: usize, bitrate_bps: u32) -> EncoderConfig {
EncoderConfig {
sample_rate,
channels,
mode: BitrateMode::Cbr(bitrate_bps),
bandwidth: None,
tns: true,
ms_stereo: true,
window_shape: WindowShape::Sine,
adts: false,
}
}
pub fn vbr(sample_rate: u32, channels: usize, quality: f32) -> EncoderConfig {
EncoderConfig { mode: BitrateMode::Vbr(quality), ..Self::cbr(sample_rate, channels, 0) }
}
}
/// Running statistics.
#[derive(Debug, Clone, Copy, Default, PartialEq)]
pub struct EncoderStats {
pub frames: u64,
/// Raw access-unit bytes (without ADTS headers).
pub bytes: u64,
/// Element-frames coded with eight short windows.
pub short_blocks: u64,
pub ms_bands: u64,
pub stereo_bands: u64,
pub tns_filters: u64,
/// Bits spent on fill elements (CBR padding).
pub fill_bits: u64,
}
impl EncoderStats {
/// Average bitrate of the raw access units.
pub fn bitrate(&self, sample_rate: u32) -> f64 {
if self.frames == 0 { 0.0 } else { self.bytes as f64 * 8.0 * sample_rate as f64 / (self.frames as f64 * FRAME_LEN as f64) }
}
}
struct ElementState {
ty: ElementType,
tag: u8,
first_ch: usize,
prev_seq: WindowSequence,
short_next: Option<bool>,
/// Previous long-window thresholds per channel (pre-echo control).
prev_thr: Vec<Vec<f32>>,
}
/// Per-frame analysis of one element.
struct ElementFrame {
info: IcsInfo,
chans: Vec<ChannelInput>,
ms_mask: Option<(u32, Vec<bool>)>,
}
const HISTORY: usize = 2048;
/// AAC-LC encoder.
pub struct Encoder {
cfg: EncoderConfig,
sf_index: u8,
asc: AudioSpecificConfig,
psy: Psy,
elements: Vec<ElementState>,
/// Input history per channel; `hist[c][0]` is absolute sample `origin`.
hist: Vec<Vec<f32>>,
origin: i64,
total_in: u64,
frames_done: u64,
flushed: bool,
bandwidth: u32,
// rate control
mean_acc: u64,
level: i64,
res_max: i64,
max_frame_bits: i64,
pe_avg: f32,
lambda: f32,
stats: EncoderStats,
}
impl Encoder {
pub fn new(cfg: EncoderConfig) -> Result<Encoder> {
let sf_index = sample_rate_index(cfg.sample_rate).ok_or(Error::InvalidConfig("sample rate must be one of the 13 AAC rates (7350–96000 Hz)"))?;
if !(1..=8).contains(&cfg.channels) {
return Err(Error::InvalidConfig("channels must be 1..=8"));
}
use ElementType::*;
let (channel_config, layout) = match cfg.channels {
7 => (0u8, vec![(Sce, 0u8), (Cpe, 0), (Cpe, 1), (Sce, 1), (Lfe, 0)]),
8 => (7, config_layout(7).unwrap_or_default()),
n => (n as u8, config_layout(n as u8).unwrap_or_default()),
};
let mut asc = AudioSpecificConfig::lc(sf_index, channel_config);
if channel_config == 0 {
asc.pce = Some(ProgramConfig {
element_instance_tag: 0,
object_type: 1,
sf_index,
front: vec![(false, 0), (true, 0)],
side: vec![(true, 1)],
back: vec![(false, 1)],
lfe: vec![0],
..Default::default()
});
}
let mut elements = Vec::new();
let mut ch = 0;
for (ty, tag) in layout {
elements.push(ElementState {
ty,
tag,
first_ch: ch,
prev_seq: WindowSequence::OnlyLong,
short_next: None,
prev_thr: vec![Vec::new(); ty.channels()],
});
ch += ty.channels();
}
let nfull = cfg.channels - elements.iter().filter(|e| e.ty == Lfe).count();
let nyq = cfg.sample_rate / 2;
let auto_bw = match cfg.mode {
BitrateMode::Cbr(b) => {
let per = b as f32 / nfull.max(1) as f32 / 1000.0;
let pts = [
(12.0f32, 4000.0f32),
(16.0, 5500.0),
(24.0, 8000.0),
(32.0, 11000.0),
(48.0, 14000.0),
(64.0, 16000.0),
(80.0, 17500.0),
(96.0, 19000.0),
(112.0, 20000.0),
(144.0, 22000.0),
];
interp(&pts, per) as u32
}
BitrateMode::Vbr(q) => interp(&[(1.0, 13000.0), (2.0, 15000.0), (3.0, 17000.0), (4.0, 19000.0), (5.0, 20000.0)], q) as u32,
};
let bandwidth = cfg.bandwidth.unwrap_or(auto_bw).clamp(1000, nyq.saturating_sub(nyq / 50).max(1000));
let max_frame_bits = 6144 * cfg.channels as i64;
let mean = match cfg.mode {
BitrateMode::Cbr(b) => b as i64 * FRAME_LEN as i64 / cfg.sample_rate as i64,
BitrateMode::Vbr(_) => 0,
};
if let BitrateMode::Cbr(b) = cfg.mode
&& (b < 1000 * cfg.channels as u32 || mean > max_frame_bits)
{
return Err(Error::InvalidConfig("bitrate out of range for the channel count and sample rate"));
}
let res_max = (max_frame_bits - mean).max(0);
let hist = vec![vec![0f32; HISTORY]; cfg.channels];
Ok(Encoder {
psy: Psy::new(cfg.sample_rate, sf_index),
sf_index,
asc,
elements,
hist,
origin: -(HISTORY as i64),
total_in: 0,
frames_done: 0,
flushed: false,
bandwidth,
mean_acc: 0,
level: 0,
res_max,
max_frame_bits,
pe_avg: 0.0,
lambda: 1.0,
stats: EncoderStats::default(),
cfg,
})
}
pub fn config(&self) -> &EncoderConfig {
&self.cfg
}
/// `AudioSpecificConfig` bytes for the MP4 `esds` decoder-specific info.
pub fn audio_specific_config(&self) -> Vec<u8> {
self.asc.to_bytes()
}
/// Encoder delay: the first 1024 decoded samples are priming and should be skipped (MP4 edit
/// list `media_time`, or iTunSMPB).
pub fn priming_samples(&self) -> u32 {
FRAME_LEN as u32
}
/// Audio bandwidth in Hz.
pub fn bandwidth(&self) -> u32 {
self.bandwidth
}
pub fn stats(&self) -> EncoderStats {
self.stats
}
/// Feed planar samples (±1.0 full scale); returns any complete access units. Missing channels are
/// treated as silence; all channels are truncated to the shortest slice.
pub fn encode(&mut self, planar: &[&[f32]]) -> Vec<Vec<u8>> {
if self.flushed {
return Vec::new();
}
let n = planar.iter().take(self.cfg.channels).map(|p| p.len()).min().unwrap_or(0);
for c in 0..self.cfg.channels {
match planar.get(c) {
Some(p) => self.hist[c].extend(p[..n].iter().map(|v| if v.is_finite() { v.clamp(-8.0, 8.0) } else { 0.0 })),
None => self.hist[c].extend(std::iter::repeat_n(0.0, n)),
}
}
self.total_in += n as u64;
let mut out = Vec::new();
while self.available() >= self.needed_for(self.frames_done) {
out.push(self.encode_frame(false));
}
out
}
/// Encode the remaining input (zero-padded). Afterwards the decoded stream, minus
/// [`priming_samples`](Self::priming_samples), covers every input sample.
pub fn flush(&mut self) -> Vec<Vec<u8>> {
if self.flushed {
return Vec::new();
}
let frames = if self.total_in == 0 { 0 } else { self.total_in.div_ceil(FRAME_LEN as u64) + 1 };
let mut out = Vec::new();
while self.frames_done < frames {
let need = self.needed_for(self.frames_done);
let have = self.available();
if have < need {
for h in &mut self.hist {
h.extend(std::iter::repeat_n(0.0, (need - have) as usize));
}
}
let last = self.frames_done + 1 == frames;
out.push(self.encode_frame(last));
}
self.flushed = true;
out
}
/// Absolute sample index one past the buffered input.
fn available(&self) -> i64 {
self.origin + self.hist[0].len() as i64
}
/// Input needed (absolute end) before frame `t` can be encoded (window + transient look-ahead).
fn needed_for(&self, t: u64) -> i64 {
(t as i64 + 1) * FRAME_LEN as i64 + 640
}
fn sample(&self, ch: usize, abs: i64) -> f32 {
let i = abs - self.origin;
if i < 0 { 0.0 } else { self.hist[ch].get(i as usize).copied().unwrap_or(0.0) }
}
fn subblock_energy(&self, ch: usize, j: i64) -> f32 {
let s = j * 128;
let mut e = 0f32;
let mut prev = self.sample(ch, s - 1);
for n in s..s + 128 {
let x = self.sample(ch, n);
let d = x - prev;
e += d * d;
prev = x;
}
e
}
/// Transient in the region covered by frame `t`'s short windows.
fn transient(&self, first_ch: usize, nch: usize, t: i64) -> bool {
for ch in first_ch..first_ch + nch {
let base = 8 * t;
let mut hist: Vec<f32> = (base - 13..base - 5).map(|j| self.subblock_energy(ch, j)).collect();
for j in base - 5..=base + 4 {
let e = self.subblock_energy(ch, j);
let mean = hist.iter().sum::<f32>() / hist.len() as f32;
if e > 1.5e-4 && e > 10.0 * mean {
return true;
}
hist.remove(0);
hist.push(e);
}
}
false
}
fn max_sfb(&self, short: bool, lfe: bool) -> usize {
let bw = if lfe { 240.min(self.bandwidth) } else { self.bandwidth };
let (swb, m) = if short { (swb_offsets_short(self.sf_index), 128f64) } else { (swb_offsets_long(self.sf_index), 1024f64) };
let bin = (bw as f64 / (self.cfg.sample_rate as f64 / 2.0) * m).ceil() as u16;
swb.iter().take(swb.len() - 1).filter(|&&o| o < bin).count().max(1)
}
fn mdct(&self, ch: usize, seq: WindowSequence) -> Vec<f32> {
let shape = self.cfg.window_shape;
let seg = &self.hist[ch][HISTORY - FRAME_LEN..HISTORY + FRAME_LEN];
let mut spec = vec![0f32; 1024];
let scale = 32768.0 * 2.0;
if seq == WindowSequence::EightShort {
let ws = window(shape, false);
let mut buf = vec![0f32; 256];
for w in 0..8 {
let off = 448 + 128 * w;
for n in 0..128 {
buf[n] = seg[off + n] * ws[n] * scale;
buf[128 + n] = seg[off + 128 + n] * ws[127 - n] * scale;
}
mdct_short().forward(&buf, &mut spec[w * 128..(w + 1) * 128]);
}
return spec;
}
let wl = window(shape, true);
let ws = window(shape, false);
let mut buf = vec![0f32; 2048];
for n in 0..1024 {
let w = match seq {
WindowSequence::LongStop => {
if n < 448 {
0.0
} else if n < 576 {
ws[n - 448]
} else {
1.0
}
}
_ => wl[n],
};
buf[n] = seg[n] * w * scale;
}
for n in 1024..2048 {
let w = match seq {
WindowSequence::LongStart => {
if n < 1472 {
1.0
} else if n < 1600 {
ws[127 - (n - 1472)]
} else {
0.0
}
}
_ => wl[2047 - n],
};
buf[n] = seg[n] * w * scale;
}
mdct_long().forward(&buf, &mut spec);
spec
}
fn tns_analyse(&self, spec: &mut [f32], info: &IcsInfo) -> Option<TnsData> {
let swb = swb_offsets_long(self.sf_index);
let lim = tns_max_bands(self.sf_index, false).min(info.max_sfb);
let hz = self.cfg.sample_rate as f64 / 2048.0;
let start = swb.iter().position(|&o| o as f64 * hz >= 1400.0).unwrap_or(swb.len() - 1);
if lim < start + 4 {
return None;
}
let (lo, hi) = (swb[start] as usize, swb[lim] as usize);
let x = &spec[lo..hi];
const ORDER: usize = 8;
let mut r = [0f64; ORDER + 1];
for (lag, rv) in r.iter_mut().enumerate() {
let mut acc = 0f64;
for n in lag..x.len() {
acc += x[n] as f64 * x[n - lag] as f64;
}
// Gaussian lag window for robustness
*rv = acc * (-0.5 * (0.02 * lag as f64).powi(2)).exp();
}
if r[0] <= 1e-3 {
return None;
}
let (ks, gain) = tns::levinson(&r, ORDER);
if gain < 1.4 {
return None;
}
let mut coef = [0i8; 12];
let mut order = 0;
for (i, &k) in ks.iter().enumerate() {
coef[i] = tns::quant_coef(k, true);
if coef[i] != 0 {
order = i + 1;
}
}
if order == 0 {
return None;
}
let mut t = TnsData::default();
t.windows[0].n_filt = 1;
t.windows[0].coef_res = true;
t.windows[0].filt[0] = TnsFilter { length: (swb.len() - 1 - start) as u8, order: order as u8, direction: false, coef_compress: false, coef };
tns::apply_encoder(spec, info, self.sf_index, &t);
Some(t)
}
fn analyse_element(&mut self, ei: usize, t: i64) -> (ElementFrame, f32) {
let (ty, first, nch, prev_seq, cached) = {
let e = &self.elements[ei];
(e.ty, e.first_ch, e.ty.channels(), e.prev_seq, e.short_next)
};
let lfe = ty == ElementType::Lfe;
let (short_now, short_next) =
if lfe { (false, false) } else { (cached.unwrap_or_else(|| self.transient(first, nch, t)), self.transient(first, nch, t + 1)) };
use WindowSequence::*;
let seq = if short_now {
EightShort
} else if prev_seq == EightShort || prev_seq == LongStart {
if short_next { EightShort } else { LongStop }
} else if short_next {
LongStart
} else {
OnlyLong
};
let specs: Vec<Vec<f32>> = (first..first + nch).map(|c| self.mdct(c, seq)).collect();
let short = seq == EightShort;
let mut info = IcsInfo { window_sequence: seq, window_shape: self.cfg.window_shape, max_sfb: self.max_sfb(short, lfe), ..Default::default() };
if short {
// group windows of similar energy; an attack starts a new group
let en: Vec<f32> = (0..8).map(|w| specs.iter().map(|s| s[w * 128..(w + 1) * 128].iter().map(|v| v * v).sum::<f32>()).sum::<f32>() + 1.0).collect();
info.num_groups = 1;
info.group_len = [1, 0, 0, 0, 0, 0, 0, 0];
for w in 1..8 {
let r = en[w] / en[w - 1];
if !(1.0 / 6.0..=3.0).contains(&r) {
info.num_groups += 1;
info.group_len[info.num_groups - 1] = 1;
} else {
info.group_len[info.num_groups - 1] += 1;
}
}
}
let mut pe = 0f32;
let mut specs = specs;
let mut thrs: Vec<Vec<Vec<f32>>> = Vec::with_capacity(nch);
let mut tnss = Vec::with_capacity(nch);
for (i, spec) in specs.iter_mut().enumerate() {
let mut r = self.psy.analyse(spec, short);
pe += r.pe;
let e = &mut self.elements[ei];
if !short {
if prev_seq != EightShort && e.prev_thr[i].len() == r.thr[0].len() {
for (t, p) in r.thr[0].iter_mut().zip(&e.prev_thr[i]) {
*t = t.min(2.0 * p.max(1e-9));
}
}
e.prev_thr[i] = r.thr[0].clone();
} else {
e.prev_thr[i].clear();
}
// group thresholds: the most sensitive window times the group length
let mut thr_g = vec![vec![0f32; 64]; info.num_groups];
for g in 0..info.num_groups {
let w0 = info.group_start(g);
let gl = info.group_len[g] as usize;
for sfb in 0..info.max_sfb {
let m = (w0..w0 + gl).map(|w| r.thr[w][sfb]).fold(f32::MAX, f32::min);
thr_g[g][sfb] = m * gl as f32;
}
}
thrs.push(thr_g);
let tns = if self.cfg.tns && !short && !lfe { self.tns_analyse(spec, &info) } else { None };
if let Some(t) = &tns {
self.stats.tns_filters += t.windows[0].n_filt as u64;
}
tnss.push(tns);
}
let swb = info.swb_offsets(self.sf_index);
let mut ms_mask = None;
if ty == ElementType::Cpe {
let mut used = Vec::with_capacity(info.num_groups * info.max_sfb);
let (l, r) = specs.split_at_mut(1);
let (l, r) = (&mut l[0], &mut r[0]);
for g in 0..info.num_groups {
let w0 = info.group_start(g);
let gl = info.group_len[g] as usize;
for sfb in 0..info.max_sfb {
let (mut el, mut er, mut em, mut es) = (0f32, 0f32, 0f32, 0f32);
for w in w0..w0 + gl {
for k in w * 128 + swb[sfb] as usize..w * 128 + swb[sfb + 1] as usize {
let (a, b) = (l[k], r[k]);
el += a * a;
er += b * b;
em += 0.25 * (a + b) * (a + b);
es += 0.25 * (a - b) * (a - b);
}
}
let (tl, tr) = (thrs[0][g][sfb].max(1e-9), thrs[1][g][sfb].max(1e-9));
let tm = 0.5 * tl.min(tr);
let c = |e: f32, t: f32| (e / t).max(1.0).log2();
let use_ms = self.cfg.ms_stereo && c(em, tm) + c(es, tm) < c(el, tl) + c(er, tr);
used.push(use_ms);
self.stats.stereo_bands += 1;
if use_ms {
self.stats.ms_bands += 1;
thrs[0][g][sfb] = tm;
thrs[1][g][sfb] = tm;
for w in w0..w0 + gl {
for k in w * 128 + swb[sfb] as usize..w * 128 + swb[sfb + 1] as usize {
let (a, b) = (l[k], r[k]);
l[k] = 0.5 * (a + b);
r[k] = 0.5 * (a - b);
}
}
}
}
}
let n_ms = used.iter().filter(|&&u| u).count();
let present = if n_ms == 0 {
0
} else if n_ms == used.len() {
2
} else {
1
};
ms_mask = Some((present, used));
}
let chans = specs.iter().zip(thrs.iter()).zip(tnss).map(|((s, th), tn)| ChannelInput::new(info, swb, s, th, tn)).collect();
let e = &mut self.elements[ei];
e.prev_seq = seq;
e.short_next = Some(short_next);
if short {
self.stats.short_blocks += 1;
}
(ElementFrame { info, chans, ms_mask }, pe)
}
fn element_overhead(&self, ei: usize, f: &ElementFrame) -> u32 {
let info_bits = if f.info.is_short() { 15 } else { 11 };
match self.elements[ei].ty {
ElementType::Cpe => {
let ms = match &f.ms_mask {
Some((1, used)) => used.len() as u32,
_ => 0,
};
3 + 4 + 1 + info_bits + 2 + ms
}
_ => 3 + 4 + info_bits,
}
}
fn quantise_all(frames: &[ElementFrame], lambda: f32) -> Vec<Vec<QuantChannel>> {
#[cfg(feature = "rayon")]
{
use rayon::prelude::*;
frames.par_iter().map(|f| f.chans.par_iter().map(|c| quantise_channel(c, lambda)).collect()).collect()
}
#[cfg(not(feature = "rayon"))]
{
frames.iter().map(|f| f.chans.iter().map(|c| quantise_channel(c, lambda)).collect()).collect()
}
}
/// `last`: final frame of the stream — a CBR stream spends (or pads away) the whole reservoir so
/// the total size is exactly `frames × bitrate × 1024 / rate`.
fn encode_frame(&mut self, last: bool) -> Vec<u8> {
let t = self.frames_done as i64;
let mut frames = Vec::with_capacity(self.elements.len());
let mut pe = 0f32;
for ei in 0..self.elements.len() {
let (f, p) = self.analyse_element(ei, t);
pe += p;
frames.push(f);
}
let pce_bits = if self.cfg.adts && self.asc.channel_config == 0 {
let mut bw = BitWriter::new();
if let Some(p) = &self.asc.pce {
p.write(&mut bw);
}
3 + bw.bit_len() as u32 + 8
} else {
0
};
let overhead: u32 = (0..frames.len()).map(|i| self.element_overhead(i, &frames[i])).sum::<u32>() + 3 + 7 + pce_bits;
let total_bits = |q: &Vec<Vec<QuantChannel>>| overhead + q.iter().flatten().map(|c| c.bits).sum::<u32>();
let any_short = frames.iter().any(|f| f.info.is_short());
let (mean, target) = match self.cfg.mode {
BitrateMode::Cbr(b) => {
self.mean_acc += b as u64 * FRAME_LEN as u64;
let mean = (self.mean_acc / self.cfg.sample_rate as u64) as i64;
self.mean_acc %= self.cfg.sample_rate as u64;
self.pe_avg = if self.pe_avg == 0.0 { pe.max(1.0) } else { 0.95 * self.pe_avg + 0.05 * pe.max(1.0) };
let mut ratio = (pe.max(1.0) / self.pe_avg).clamp(0.7, 1.6);
if any_short {
ratio = (ratio * 1.3).min(2.0);
}
let mut desired = mean as f32 * ratio + (self.level as f32 - 0.5 * self.res_max as f32) * 0.15;
desired = desired.min((mean + self.level) as f32).max((mean - (self.res_max - self.level)) as f32);
desired = desired.min(self.max_frame_bits as f32);
if last {
desired = (mean + self.level) as f32;
}
(mean, desired.max(0.0) as u32)
}
BitrateMode::Vbr(_) => (0, self.max_frame_bits as u32),
};
let quant = match self.cfg.mode {
BitrateMode::Vbr(q) => {
let d = q.clamp(0.5, 6.0) - 1.0;
let mut lambda = 10f32.powf(-0.2 - 1.2 * d + 0.09 * d * d);
let mut res = Self::quantise_all(&frames, lambda);
while total_bits(&res) > target && lambda < 1e9 {
lambda *= 2.0;
res = Self::quantise_all(&frames, lambda);
}
res
}
BitrateMode::Cbr(_) => {
let mut lam = self.lambda.clamp(1e-6, 1e9);
let mut cur = Self::quantise_all(&frames, lam);
let (mut lo, mut hi, mut best);
if total_bits(&cur) <= target {
hi = lam;
best = cur;
lo = lam;
loop {
lo /= 4.0;
if lo < 1e-6 {
break;
}
cur = Self::quantise_all(&frames, lo);
if total_bits(&cur) > target {
break;
}
hi = lo;
best = cur;
}
} else {
lo = lam;
loop {
lam *= 4.0;
cur = Self::quantise_all(&frames, lam);
if total_bits(&cur) <= target || lam > 1e9 {
break;
}
lo = lam;
}
hi = lam;
best = cur;
}
for _ in 0..6 {
if hi / lo < 1.03 {
break;
}
let mid = (lo * hi).sqrt();
let r = Self::quantise_all(&frames, mid);
if total_bits(&r) <= target {
hi = mid;
best = r;
} else {
lo = mid;
}
}
self.lambda = hi;
best
}
};
// assemble the raw_data_block
let mut bw = BitWriter::new();
if pce_bits > 0
&& let Some(p) = &self.asc.pce
{
bw.write_bits(5, 3);
p.write(&mut bw);
}
for (ei, (f, q)) in frames.iter().zip(&quant).enumerate() {
let e = &self.elements[ei];
match e.ty {
ElementType::Cpe => {
bw.write_bits(1, 3);
bw.write_bits(e.tag as u32, 4);
bw.write_bits(1, 1); // common_window
f.info.write(&mut bw);
let (present, used) = f.ms_mask.clone().unwrap_or((0, Vec::new()));
bw.write_bits(present, 2);
if present == 1 {
for &u in &used {
bw.write_bits(u as u32, 1);
}
}
write_ics(&mut bw, &f.chans[0], &q[0], false);
write_ics(&mut bw, &f.chans[1], &q[1], false);
}
ty => {
bw.write_bits(if ty == ElementType::Lfe { 3 } else { 0 }, 3);
bw.write_bits(e.tag as u32, 4);
write_ics(&mut bw, &f.chans[0], &q[0], true);
}
}
}
if let BitrateMode::Cbr(_) = self.cfg.mode {
let projected = ((bw.bit_len() as i64 + 3 + 7) / 8) * 8;
let excess = self.level + mean - projected - if last { 0 } else { self.res_max };
if excess > 0 {
self.stats.fill_bits += write_fill(&mut bw, excess as u64);
}
}
bw.write_bits(7, 3); // ID_END
let au = bw.finish();
let used = au.len() as i64 * 8;
if let BitrateMode::Cbr(_) = self.cfg.mode {
self.level = (self.level + mean - used).clamp(0, self.res_max);
}
self.stats.frames += 1;
self.stats.bytes += au.len() as u64;
// slide the input window
for h in &mut self.hist {
h.drain(..FRAME_LEN);
}
self.origin += FRAME_LEN as i64;
self.frames_done += 1;
if self.cfg.adts {
let fullness = match self.cfg.mode {
BitrateMode::Cbr(_) => ((self.level / (32 * self.cfg.channels as i64)) as u16).min(0x7FE),
BitrateMode::Vbr(_) => 0x7FF,
};
let mut out = AdtsHeader::write(2, self.sf_index, self.asc.channel_config, au.len(), fullness).to_vec();
out.extend_from_slice(&au);
out
} else {
au
}
}
}
/// Write fill elements totalling at least `bits - 7` bits (at most `bits + 8`); returns bits written.
fn write_fill(bw: &mut BitWriter, bits: u64) -> u64 {
let start = bw.bit_len() as u64;
let mut left = bits as i64;
while left >= 7 {
let n = (((left - 7) / 8) as usize).min(269);
let n = if n >= 15 && (left - 15) / 8 < 15 { 14 } else { n };
bw.write_bits(6, 3);
if n >= 15 {
bw.write_bits(15, 4);
bw.write_bits((n - 14) as u32, 8);
} else {
bw.write_bits(n as u32, 4);
}
if n > 0 {
bw.write_bits(0, 8); // extension_type EXT_FILL + fill_nibble
for _ in 1..n {
bw.write_bits(0xA5, 8);
}
}
let el = 7 + if n >= 15 { 8 } else { 0 } + 8 * n as i64;
left -= el;
if n == 0 {
break;
}
}
bw.bit_len() as u64 - start
}
fn interp(pts: &[(f32, f32)], x: f32) -> f32 {
if x <= pts[0].0 {
return pts[0].1;
}
for w in pts.windows(2) {
if x <= w[1].0 {
let t = (x - w[0].0) / (w[1].0 - w[0].0);
return w[0].1 + t * (w[1].1 - w[0].1);
}
}
pts[pts.len() - 1].1
}

173
crates/aac/src/encoder/psy.rs Normal file
View File

@ -0,0 +1,173 @@
//! Psychoacoustic model operating on MDCT coefficients.
//!
//! Per scalefactor band: energy; tonality from the spectral flatness measure (SFM) of a
//! neighbourhood of ≥ 16 lines; masking-threshold PSD by spreading the band PSDs with Schroeder's
//! spreading function in the Bark domain; a tonality-dependent masking offset (≈18 dB for tonal,
//! ≈6 dB for noise-like maskers); the absolute threshold of hearing (Terhardt) as a floor; and
//! perceptual entropy (PE) as an estimate of the bits the frame needs.
use crate::tables::{swb_offsets_long, swb_offsets_short};
/// Precomputed per-band data for one window length.
pub struct BandModel {
pub offsets: &'static [u16],
/// Linear spreading factor `[maskee][masker]` (power).
spread: Vec<Vec<f32>>,
/// ATH as band energy in encoder spectrum units.
ath: Vec<f32>,
/// Neighbourhood `[lo, hi)` in lines used for the tonality estimate.
tonal_span: Vec<(usize, usize)>,
}
fn bark(f: f64) -> f64 {
13.0 * (0.00076 * f).atan() + 3.5 * (f / 7500.0).powi(2).atan()
}
/// Terhardt's threshold in quiet (dB SPL), capped at 40 dB so content near the band edge is not
/// discarded wholesale (the bandwidth setting limits the coded range instead).
fn ath_db(f: f64) -> f64 {
let f = (f / 1000.0).max(0.02);
(3.64 * f.powf(-0.8) - 6.5 * (-0.6 * (f - 3.3).powi(2)).exp() + 1e-3 * f.powi(4)).min(40.0)
}
impl BandModel {
fn new(sample_rate: u32, offsets: &'static [u16], m: usize) -> BandModel {
let nb = offsets.len() - 1;
let hz_per_line = sample_rate as f64 / (2.0 * m as f64);
let centre: Vec<f64> = (0..nb).map(|b| bark((offsets[b] as f64 + offsets[b + 1] as f64) * 0.5 * hz_per_line)).collect();
let spread = (0..nb)
.map(|i| {
(0..nb)
.map(|j| {
let dz = centre[i] - centre[j] + 0.474;
let db = 15.81 + 7.5 * dz - 17.5 * (1.0 + dz * dz).sqrt();
if db < -60.0 { 0.0 } else { 10f64.powf(db / 10.0) as f32 }
})
.collect()
})
.collect();
// A full-scale (16-bit) sine of power P_ref = 32768²/2 has MDCT energy ≈ 2·M²·P per frame
// with the encoder's scaling; it is taken as 96 dB SPL. 3 dB of safety margin.
let p_ref = 32768f64 * 32768.0 / 2.0;
let ath = (0..nb)
.map(|b| {
let mut lo = f64::MAX;
for k in offsets[b]..offsets[b + 1] {
lo = lo.min(ath_db((k as f64 + 0.5) * hz_per_line));
}
(2.0 * (m * m) as f64 * p_ref * 10f64.powf((lo - 96.0 - 3.0) / 10.0)) as f32
})
.collect();
let tonal_span = (0..nb)
.map(|b| {
let (lo, hi) = (offsets[b] as usize, offsets[b + 1] as usize);
if hi - lo >= 16 {
(lo, hi)
} else {
let c = (lo + hi) / 2;
let lo = c.saturating_sub(8);
let hi = (lo + 16).min(m);
(hi.saturating_sub(16), hi)
}
})
.collect();
BandModel { offsets, spread, ath, tonal_span }
}
pub fn num_bands(&self) -> usize {
self.offsets.len() - 1
}
}
pub struct Psy {
pub long: BandModel,
pub short: BandModel,
}
/// Per-window, per-band analysis of one channel.
#[derive(Clone, Default)]
pub struct PsyResult {
/// `[window][band]` energy.
pub en: Vec<Vec<f32>>,
/// `[window][band]` masking threshold (allowed noise energy).
pub thr: Vec<Vec<f32>>,
pub pe: f32,
}
impl Psy {
pub fn new(sample_rate: u32, sf_index: u8) -> Psy {
Psy { long: BandModel::new(sample_rate, swb_offsets_long(sf_index), 1024), short: BandModel::new(sample_rate, swb_offsets_short(sf_index), 128) }
}
/// Analyse a spectrum: 1024 lines (long) or 8×128 (short).
pub fn analyse(&self, spec: &[f32], short: bool) -> PsyResult {
let (model, nwin, wlen) = if short { (&self.short, 8, 128) } else { (&self.long, 1, 1024) };
let nb = model.num_bands();
let mut res = PsyResult { en: Vec::with_capacity(nwin), thr: Vec::with_capacity(nwin), pe: 0.0 };
let mut psd = vec![0f32; nb];
let mut tonality = vec![0f32; nb];
for w in 0..nwin {
let s = &spec[w * wlen..(w + 1) * wlen];
let mut en = vec![0f32; nb];
for b in 0..nb {
let (lo, hi) = (model.offsets[b] as usize, model.offsets[b + 1] as usize);
en[b] = s[lo..hi].iter().map(|v| v * v).sum();
psd[b] = en[b] / (hi - lo) as f32;
// spectral flatness over the neighbourhood
let (tlo, thi) = model.tonal_span[b];
let n = (thi - tlo) as f64;
let mut sum = 0f64;
let mut log_sum = 0f64;
for &v in &s[tlo..thi] {
let p = v as f64 * v as f64 + 1e-3;
sum += p;
log_sum += p.ln();
}
let sfm_db = 10.0 * ((log_sum / n).exp() / (sum / n)).log10();
// real-valued Gaussian noise gives ≈ -5.5 dB; a pure tone ≤ -25 dB
tonality[b] = ((-sfm_db - 6.0) / 19.0).clamp(0.0, 1.0) as f32;
}
let mut thr = vec![0f32; nb];
for b in 0..nb {
let row = &model.spread[b];
let mut sp = 0f32;
for j in 0..nb {
sp += psd[j] * row[j];
}
let width = (model.offsets[b + 1] - model.offsets[b]) as f32;
let offset_db = tonality[b] * 30.0 + (1.0 - tonality[b]) * 6.0;
let t = sp * width * 10f32.powf(-offset_db / 10.0);
thr[b] = t.max(model.ath[b]);
if en[b] > thr[b] {
res.pe += width * 0.5 * (en[b] / thr[b]).log2();
}
}
res.en.push(en);
res.thr.push(thr);
}
res
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn tone_is_masked_less_than_noise() {
let psy = Psy::new(44100, 4);
let tone: Vec<f32> = (0..1024).map(|k| if k == 100 { 1e6 } else { 0.0 }).collect();
let mut seed = 1u32;
let noise: Vec<f32> = (0..1024)
.map(|_| {
seed = seed.wrapping_mul(1664525).wrapping_add(1013904223);
((seed >> 8) as f32 / (1 << 24) as f32 - 0.5) * 1e4
})
.collect();
let rt = psy.analyse(&tone, false);
let rn = psy.analyse(&noise, false);
let b = psy.long.offsets.iter().position(|&o| o > 100).unwrap() - 1;
assert!(rt.thr[0][b] < rt.en[0][b] / 30.0);
assert!(rn.thr[0][b] > rn.en[0][b] / 10.0);
}
}

382
crates/aac/src/encoder/quant.rs Normal file
View File

@ -0,0 +1,382 @@
//! Quantisation, scalefactor selection, codebook/section selection, bit counting and writing of one
//! `individual_channel_stream`.
//!
//! Coefficients are kept in bitstream order ("interleaved": group → band → window → line), so every
//! (group, band) is a contiguous slice and short-window grouping needs no special casing.
use filmcraft_bitstream::BitWriter;
use crate::huffman::{self, ZERO_HCB};
use crate::ics::IcsInfo;
use crate::tables::pow43;
use crate::tns::TnsData;
/// Magic rounding offset for AAC quantisation (`0.5 - 0.0946`), minimising the error for the
/// `x^(3/4)` companding.
const ROUND: f32 = 0.4054;
const MAX_Q: i32 = 8191;
/// One (group, band) of a channel, in bitstream order.
#[derive(Clone, Copy, Debug)]
pub struct Band {
/// Range in the interleaved coefficient array.
pub start: usize,
pub end: usize,
/// Allowed noise energy at λ = 1.
pub thr: f32,
/// Coefficient energy.
pub en: f32,
/// Largest `|x|^(3/4)`.
pub max34: f32,
}
/// Everything the rate loop needs for one channel.
pub struct ChannelInput {
pub info: IcsInfo,
/// Interleaved (bitstream-order) coefficients.
pub x: Vec<f32>,
pub xabs: Vec<f32>,
pub x34: Vec<f32>,
pub bands: Vec<Band>,
pub tns: Option<TnsData>,
}
impl ChannelInput {
/// `spec` is window-major (short windows at stride 128); `thr[g][sfb]` and the grouping come from
/// `info`.
pub fn new(info: IcsInfo, swb: &[u16], spec: &[f32], thr: &[Vec<f32>], tns: Option<TnsData>) -> ChannelInput {
let mut x = Vec::with_capacity(1024);
let mut bands = Vec::new();
for g in 0..info.num_groups {
let w0 = info.group_start(g);
for sfb in 0..info.max_sfb {
let start = x.len();
for w in w0..w0 + info.group_len[g] as usize {
x.extend_from_slice(&spec[w * 128 + swb[sfb] as usize..w * 128 + swb[sfb + 1] as usize]);
}
bands.push(Band { start, end: x.len(), thr: thr[g][sfb], en: 0.0, max34: 0.0 });
}
}
let xabs: Vec<f32> = x.iter().map(|v| v.abs()).collect();
let x34: Vec<f32> = xabs.iter().map(|v| v.powf(0.75)).collect();
for b in &mut bands {
b.en = x[b.start..b.end].iter().map(|v| v * v).sum();
b.max34 = x34[b.start..b.end].iter().cloned().fold(0.0, f32::max);
}
ChannelInput { info, x, xabs, x34, bands, tns }
}
}
/// A quantised channel ready to be written.
#[derive(Clone, Default)]
pub struct QuantChannel {
pub q: Vec<i32>,
/// Scalefactor per band (only meaningful where `cb != 0`).
pub sf: Vec<i32>,
pub cb: Vec<u8>,
/// Per group: (codebook, number of bands).
pub sections: Vec<Vec<(u8, usize)>>,
pub global_gain: i32,
/// Bits of the ICS excluding `ics_info`.
pub bits: u32,
}
#[inline]
fn gain34(sf: i32) -> f32 {
(-0.1875 * (sf as f32 - 100.0)).exp2()
}
#[inline]
fn gain(sf: i32) -> f32 {
(0.25 * (sf as f32 - 100.0)).exp2()
}
/// Quantise a slice at `sf`, returning the distortion; writes the (signed) values to `out` if given.
fn quantise(x: &[f32], xabs: &[f32], x34: &[f32], sf: i32, out: Option<&mut [i32]>) -> f32 {
let g34 = gain34(sf);
let g = gain(sf);
let mut d = 0f32;
match out {
Some(out) => {
for i in 0..x.len() {
let q = ((x34[i] * g34 + ROUND) as i32).min(MAX_Q);
let e = xabs[i] - pow43(q as u32) * g;
d += e * e;
out[i] = if x[i] < 0.0 { -q } else { q };
}
}
None => {
for i in 0..x.len() {
let q = ((x34[i] * g34 + ROUND) as i32).min(MAX_Q);
let e = xabs[i] - pow43(q as u32) * g;
d += e * e;
}
}
}
d
}
/// Largest scalefactor whose distortion stays within `allowed`.
fn search_sf(ch: &ChannelInput, b: &Band, allowed: f32) -> i32 {
let r = b.start..b.end;
let (x, xa, x34) = (&ch.x[r.clone()], &ch.xabs[r.clone()], &ch.x34[r]);
let l2 = b.max34.log2();
let lo = ((100.0 - ((MAX_Q as f32 + 0.5).log2() - l2) / 0.1875).ceil() as i32).clamp(0, 255);
let zero = ((100.0 - ((1.0 - ROUND).log2() - l2) / 0.1875).floor() as i32 + 1).clamp(0, 256);
let mut a = lo;
let mut hi = (zero - 1).clamp(lo, 255);
if quantise(x, xa, x34, a, None) > allowed {
return a;
}
while a < hi {
let mid = (a + hi + 1) / 2;
if quantise(x, xa, x34, mid, None) <= allowed {
a = mid;
} else {
hi = mid - 1;
}
}
a
}
/// Candidate codebooks for a band whose largest magnitude is `m` (0 for all-zero bands).
fn candidates(m: i32) -> &'static [u8] {
match m {
0 => &[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
1 => &[1, 2, 3, 4, 5, 6],
2 => &[3, 4, 5, 6, 7, 8],
3..=4 => &[5, 6, 7, 8, 9, 10],
5..=7 => &[7, 8, 9, 10, 11],
8..=12 => &[9, 10, 11],
_ => &[11],
}
}
/// Quantise one channel at rate-distortion multiplier `lambda` and select sections.
pub fn quantise_channel(ch: &ChannelInput, lambda: f32) -> QuantChannel {
let nb = ch.bands.len();
let mut sf = vec![-1i32; nb];
for (i, b) in ch.bands.iter().enumerate() {
let allowed = lambda * b.thr;
if b.max34 <= 0.0 || b.en <= allowed {
continue;
}
sf[i] = search_sf(ch, b, allowed);
}
// Scalefactor differences between coded bands must stay within ±60: lower (refine) offenders,
// then quantise; repeat if a band unexpectedly quantised to all zeros (it leaves the chain).
let mut q = vec![0i32; ch.x.len()];
let mut maxq = vec![0i32; nb];
loop {
for _ in 0..8 {
let mut changed = false;
let coded: Vec<usize> = (0..nb).filter(|&i| sf[i] >= 0).collect();
for w in coded.windows(2) {
if sf[w[1]] > sf[w[0]] + 60 {
sf[w[1]] = sf[w[0]] + 60;
changed = true;
}
}
for w in coded.windows(2).rev() {
if sf[w[0]] > sf[w[1]] + 60 {
sf[w[0]] = sf[w[1]] + 60;
changed = true;
}
}
if !changed {
break;
}
}
let mut removed = false;
for (i, b) in ch.bands.iter().enumerate() {
let r = b.start..b.end;
if sf[i] < 0 {
q[r].fill(0);
maxq[i] = 0;
continue;
}
quantise(&ch.x[r.clone()], &ch.xabs[r.clone()], &ch.x34[r.clone()], sf[i], Some(&mut q[r.clone()]));
maxq[i] = q[r].iter().map(|v| v.abs()).max().unwrap_or(0);
if maxq[i] == 0 {
sf[i] = -1;
removed = true;
}
}
if !removed {
break;
}
}
// Section selection: dynamic programming per group over codebooks.
let info = &ch.info;
let (sect_bits, esc) = if info.is_short() { (3u32, 7usize) } else { (5u32, 31usize) };
let mut cb = vec![ZERO_HCB; nb];
let mut sections = Vec::with_capacity(info.num_groups);
let mut spectral_bits = 0u32;
let mut section_bits = 0u32;
let per_group = info.max_sfb;
for g in 0..info.num_groups {
let base = g * per_group;
let n = per_group;
if n == 0 {
sections.push(Vec::new());
continue;
}
// cost[i][c]
const INF: u32 = u32::MAX / 4;
let mut cost = vec![[INF; 12]; n];
for i in 0..n {
let bi = base + i;
let b = &ch.bands[bi];
let vals = &q[b.start..b.end];
for &c in candidates(maxq[bi]) {
cost[i][c as usize] = if c == 0 {
0
} else {
// all-zero band inside a coded section also needs a (zero) scalefactor difference
huffman::count_bits(c, vals) + if maxq[bi] == 0 { 1 } else { 0 }
};
}
}
let new_sec = 4 + sect_bits;
let mut best = [INF; 12];
let mut back = vec![[0u8; 12]; n];
for c in 0..12 {
if cost[0][c] < INF {
best[c] = new_sec + cost[0][c];
}
}
for i in 1..n {
let (mut min_c, mut min_v) = (0usize, INF);
for c in 0..12 {
if best[c] < min_v {
min_v = best[c];
min_c = c;
}
}
let mut nb_ = [INF; 12];
for c in 0..12 {
if cost[i][c] >= INF {
continue;
}
let stay = best[c];
let switch = min_v + new_sec;
if stay <= switch {
nb_[c] = stay + cost[i][c];
back[i][c] = c as u8;
} else {
nb_[c] = switch + cost[i][c];
back[i][c] = min_c as u8;
}
}
best = nb_;
}
let mut c = (0..12).min_by_key(|&c| best[c]).unwrap_or(0);
for i in (0..n).rev() {
cb[base + i] = c as u8;
c = back[i][c] as usize;
}
// run-length sections and exact costs
let mut secs: Vec<(u8, usize)> = Vec::new();
for i in 0..n {
let c = cb[base + i];
match secs.last_mut() {
Some((lc, len)) if *lc == c => *len += 1,
_ => secs.push((c, 1)),
}
if c != ZERO_HCB {
let b = &ch.bands[base + i];
spectral_bits += huffman::count_bits(c, &q[b.start..b.end]);
}
}
for &(_, len) in &secs {
section_bits += 4 + sect_bits * (len / esc + 1) as u32;
}
sections.push(secs);
}
// Scalefactors: global gain is the first coded band's value; bands coded with a non-zero
// codebook but all-zero values reuse the previous value (difference 0).
let first = (0..nb).find(|&i| sf[i] >= 0).map(|i| sf[i]).unwrap_or(100);
let mut prev = first;
let mut sf_bits = 0u32;
for i in 0..nb {
if cb[i] == ZERO_HCB {
continue;
}
if sf[i] < 0 {
sf[i] = prev;
}
sf_bits += huffman::sf_bits(sf[i] - prev);
prev = sf[i];
}
let tns_bits = 1 + ch.tns.as_ref().map(|t| t.bit_count(info)).unwrap_or(0);
let bits = 8 + section_bits + sf_bits + 1 + tns_bits + 1 + spectral_bits;
QuantChannel { q, sf, cb, sections, global_gain: first, bits }
}
/// Write an `individual_channel_stream`; `write_info` is false inside a common-window CPE.
pub fn write_ics(bw: &mut BitWriter, ch: &ChannelInput, qc: &QuantChannel, write_info: bool) {
let info = &ch.info;
bw.write_bits(qc.global_gain as u32, 8);
if write_info {
info.write(bw);
}
let (sect_bits, esc) = if info.is_short() { (3u32, 7usize) } else { (5u32, 31usize) };
for secs in &qc.sections {
for &(c, len) in secs {
bw.write_bits(c as u32, 4);
let mut l = len;
while l >= esc {
bw.write_bits(esc as u32, sect_bits);
l -= esc;
}
bw.write_bits(l as u32, sect_bits);
}
}
let mut prev = qc.global_gain;
for i in 0..ch.bands.len() {
if qc.cb[i] == ZERO_HCB {
continue;
}
huffman::write_sf(bw, qc.sf[i] - prev);
prev = qc.sf[i];
}
bw.write_bits(0, 1); // pulse_data_present
match &ch.tns {
Some(t) => {
bw.write_bits(1, 1);
t.write(bw, info);
}
None => bw.write_bits(0, 1),
}
bw.write_bits(0, 1); // gain_control_data_present
for (i, b) in ch.bands.iter().enumerate() {
if qc.cb[i] != ZERO_HCB {
huffman::write_spectral(bw, qc.cb[i], &qc.q[b.start..b.end]);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::ics::WindowSequence;
use crate::mdct::WindowShape;
use crate::tables::swb_offsets_long;
use filmcraft_bitstream::BitWriter;
#[test]
fn bit_count_matches_written_bits() {
let swb = swb_offsets_long(4);
let info = IcsInfo { window_sequence: WindowSequence::OnlyLong, window_shape: WindowShape::Sine, max_sfb: 40, ..Default::default() };
let spec: Vec<f32> = (0..1024).map(|i| ((i * 7919 % 1000) as f32 - 500.0) * (1000.0 / (i as f32 + 10.0))).collect();
let thr = vec![vec![1000.0f32; 64]];
let ch = ChannelInput::new(info, swb, &spec, &thr, None);
for lambda in [0.01f32, 1.0, 100.0, 1e5] {
let qc = quantise_channel(&ch, lambda);
let mut bw = BitWriter::new();
write_ics(&mut bw, &ch, &qc, false);
assert_eq!(bw.bit_len() as u32, qc.bits, "lambda {lambda}");
}
}
}

306
crates/aac/src/huffman.rs Normal file
View File

@ -0,0 +1,306 @@
//! Huffman coding of spectral values and scalefactors (ISO/IEC 14496-3 §4.6.3, Annex 4.A).
use std::sync::OnceLock;
use filmcraft_bitstream::{BitReader, BitWriter};
use crate::huffman_tables::{
SCALEFACTOR, SPECTRUM_1, SPECTRUM_2, SPECTRUM_3, SPECTRUM_4, SPECTRUM_5, SPECTRUM_6, SPECTRUM_7, SPECTRUM_8, SPECTRUM_9, SPECTRUM_10, SPECTRUM_11,
};
use crate::{Error, Result};
/// `ZERO_HCB`: band quantised to zero, nothing transmitted.
pub const ZERO_HCB: u8 = 0;
/// `ESC_HCB`: codebook 11 with escape sequences.
pub const ESC_HCB: u8 = 11;
/// `NOISE_HCB`: perceptual noise substitution.
pub const NOISE_HCB: u8 = 13;
/// `INTENSITY_HCB2`: intensity stereo, out of phase.
pub const INTENSITY_HCB2: u8 = 14;
/// `INTENSITY_HCB`: intensity stereo, in phase.
pub const INTENSITY_HCB: u8 = 15;
/// Parameters of one spectrum codebook.
pub struct Book {
/// Values per codeword (4 = quad, 2 = pair).
pub dim: usize,
/// Signed values in the codeword (no separate sign bits).
pub signed: bool,
/// Largest absolute value (16 = escape for codebook 11).
pub lav: u32,
/// Radix of the index (`2*lav+1` signed, `lav+1` unsigned).
pub modulo: u32,
pub codes: &'static [(u32, u8)],
}
static BOOKS: [Book; 11] = [
Book { dim: 4, signed: true, lav: 1, modulo: 3, codes: &SPECTRUM_1 },
Book { dim: 4, signed: true, lav: 1, modulo: 3, codes: &SPECTRUM_2 },
Book { dim: 4, signed: false, lav: 2, modulo: 3, codes: &SPECTRUM_3 },
Book { dim: 4, signed: false, lav: 2, modulo: 3, codes: &SPECTRUM_4 },
Book { dim: 2, signed: true, lav: 4, modulo: 9, codes: &SPECTRUM_5 },
Book { dim: 2, signed: true, lav: 4, modulo: 9, codes: &SPECTRUM_6 },
Book { dim: 2, signed: false, lav: 7, modulo: 8, codes: &SPECTRUM_7 },
Book { dim: 2, signed: false, lav: 7, modulo: 8, codes: &SPECTRUM_8 },
Book { dim: 2, signed: false, lav: 12, modulo: 13, codes: &SPECTRUM_9 },
Book { dim: 2, signed: false, lav: 12, modulo: 13, codes: &SPECTRUM_10 },
Book { dim: 2, signed: false, lav: 16, modulo: 17, codes: &SPECTRUM_11 },
];
/// Spectrum codebook 1..=11.
#[inline]
pub fn book(cb: u8) -> &'static Book {
&BOOKS[(cb as usize).clamp(1, 11) - 1]
}
const LEAF: u32 = 1 << 31;
/// Binary decoding tree for one codebook.
struct Tree {
nodes: Vec<[u32; 2]>,
}
impl Tree {
fn build(codes: &[(u32, u8)]) -> Tree {
let mut nodes: Vec<[u32; 2]> = vec![[0, 0]];
for (sym, &(code, len)) in codes.iter().enumerate() {
let mut node = 0usize;
for i in (0..len).rev() {
let bit = ((code >> i) & 1) as usize;
if i == 0 {
nodes[node][bit] = LEAF | sym as u32;
} else {
if nodes[node][bit] == 0 {
nodes.push([0, 0]);
nodes[node][bit] = (nodes.len() - 1) as u32;
}
node = nodes[node][bit] as usize;
}
}
}
Tree { nodes }
}
#[inline]
fn decode(&self, br: &mut BitReader) -> Result<u32> {
let mut node = 0usize;
loop {
let bit = br.read_bits(1)? as usize;
let next = self.nodes[node][bit];
if next & LEAF != 0 {
return Ok(next & !LEAF);
}
if next == 0 {
return Err(Error::Bitstream("invalid Huffman codeword"));
}
node = next as usize;
}
}
}
fn trees() -> &'static [Tree; 12] {
static TREES: OnceLock<[Tree; 12]> = OnceLock::new();
TREES.get_or_init(|| std::array::from_fn(|i| if i == 0 { Tree::build(&SCALEFACTOR) } else { Tree::build(BOOKS[i - 1].codes) }))
}
/// Decode one scalefactor difference (`-60..=60`).
#[inline]
pub fn decode_sf(br: &mut BitReader) -> Result<i32> {
Ok(trees()[0].decode(br)? as i32 - 60)
}
/// Write one scalefactor difference (`-60..=60`).
#[inline]
pub fn write_sf(bw: &mut BitWriter, diff: i32) {
let (code, len) = SCALEFACTOR[(diff + 60).clamp(0, 120) as usize];
bw.write_bits(code, len as u32);
}
/// Bits of one scalefactor difference.
#[inline]
pub fn sf_bits(diff: i32) -> u32 {
SCALEFACTOR[(diff + 60).clamp(0, 120) as usize].1 as u32
}
/// Decode one codeword of codebook `cb` (1..=11) into `out[..dim]`, including sign bits and escapes.
pub fn decode_spectral(br: &mut BitReader, cb: u8, out: &mut [i32]) -> Result<()> {
let b = book(cb);
let mut idx = trees()[cb as usize].decode(br)?;
for i in (0..b.dim).rev() {
out[i] = (idx % b.modulo) as i32;
idx /= b.modulo;
}
if b.signed {
for v in out[..b.dim].iter_mut() {
*v -= b.lav as i32;
}
return Ok(());
}
for v in out[..b.dim].iter_mut() {
if *v != 0 && br.read_bits(1)? == 1 {
*v = -*v;
}
}
if cb == ESC_HCB {
for v in out[..b.dim].iter_mut() {
if v.abs() == 16 {
let mut n = 0u32;
while br.read_bits(1)? == 1 {
n += 1;
if n > 8 {
return Err(Error::Bitstream("escape sequence too long"));
}
}
let mag = (1i32 << (n + 4)) + br.read_bits(n + 4)? as i32;
*v = if *v < 0 { -mag } else { mag };
}
}
}
Ok(())
}
/// Codebook index of the (sign-stripped for unsigned books) values.
#[inline]
fn index_of(b: &Book, vals: &[i32]) -> usize {
let mut idx = 0u32;
for &v in &vals[..b.dim] {
let d = if b.signed { (v + b.lav as i32) as u32 } else { v.unsigned_abs().min(b.lav) };
idx = idx * b.modulo + d;
}
idx as usize
}
#[inline]
fn escape_bits(mag: u32) -> u32 {
// mag >= 16: N ones, a zero, N+4 bits where 2^(N+4) <= mag < 2^(N+5).
let n = 31 - mag.leading_zeros() - 4;
2 * n + 5
}
/// Bits needed to code `vals` (a multiple of the codebook dimension) with codebook `cb` (1..=11).
/// The values must fit the codebook (`|v| <= lav`, or `<= 8191` for codebook 11).
pub fn count_bits(cb: u8, vals: &[i32]) -> u32 {
let b = book(cb);
let mut bits = 0u32;
for chunk in vals.chunks_exact(b.dim) {
bits += b.codes[index_of(b, chunk)].1 as u32;
if !b.signed {
for &v in chunk {
if v != 0 {
bits += 1;
if cb == ESC_HCB && v.unsigned_abs() >= 16 {
bits += escape_bits(v.unsigned_abs());
}
}
}
}
}
bits
}
/// Write `vals` with codebook `cb` (1..=11).
pub fn write_spectral(bw: &mut BitWriter, cb: u8, vals: &[i32]) {
let b = book(cb);
for chunk in vals.chunks_exact(b.dim) {
let (code, len) = b.codes[index_of(b, chunk)];
bw.write_bits(code, len as u32);
if b.signed {
continue;
}
for &v in chunk {
if v != 0 {
bw.write_bits((v < 0) as u32, 1);
}
}
if cb == ESC_HCB {
for &v in chunk {
let mag = v.unsigned_abs();
if mag >= 16 {
let n = 31 - mag.leading_zeros() - 4;
for _ in 0..n {
bw.write_bits(1, 1);
}
bw.write_bits(0, 1);
bw.write_bits(mag - (1 << (n + 4)), n + 4);
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn check_complete(codes: &[(u32, u8)]) {
// Kraft sum exactly one and prefix-free.
let max = codes.iter().map(|c| c.1).max().unwrap() as u32;
let mut kraft = 0u64;
for &(_, l) in codes {
kraft += 1u64 << (max - l as u32);
}
assert_eq!(kraft, 1u64 << max);
for (i, &(ci, li)) in codes.iter().enumerate() {
for (j, &(cj, lj)) in codes.iter().enumerate() {
if i != j && li <= lj {
assert_ne!(cj >> (lj - li), ci, "codeword {i} is a prefix of {j}");
}
}
}
}
#[test]
fn codebooks_are_complete_prefix_codes() {
check_complete(&SCALEFACTOR);
assert_eq!(SCALEFACTOR[60], (0, 1));
for b in &BOOKS {
assert_eq!(b.codes.len() as u32, b.modulo.pow(b.dim as u32));
check_complete(b.codes);
}
}
#[test]
fn spectral_roundtrip_every_book() {
for cb in 1..=11u8 {
let b = book(cb);
let lim = if cb == 11 { 8191 } else { b.lav as i32 };
let mut vals = Vec::new();
let mut seed = 12345u32;
for _ in 0..400 {
seed = seed.wrapping_mul(1664525).wrapping_add(1013904223);
let r = (seed >> 8) as i32;
let v = if cb == 11 && r % 3 == 0 { r % (lim + 1) } else { r % (b.lav.min(16) as i32 + 1) };
let v = v.min(lim);
vals.push(if b.signed || r & 1 == 0 { v } else { -v });
}
if b.signed {
for v in vals.iter_mut() {
*v = (*v).clamp(-(b.lav as i32), b.lav as i32);
}
}
let mut bw = BitWriter::new();
write_spectral(&mut bw, cb, &vals);
assert_eq!(bw.bit_len() as u32, count_bits(cb, &vals));
let data = bw.finish();
let mut br = BitReader::new(&data);
let mut out = vec![0i32; vals.len()];
for c in out.chunks_mut(b.dim) {
decode_spectral(&mut br, cb, c).unwrap();
}
assert_eq!(out, vals, "codebook {cb}");
}
}
#[test]
fn scalefactor_roundtrip() {
let mut bw = BitWriter::new();
for d in -60..=60 {
write_sf(&mut bw, d);
}
let data = bw.finish();
let mut br = BitReader::new(&data);
for d in -60..=60 {
assert_eq!(decode_sf(&mut br).unwrap(), d);
}
}
}

1406
crates/aac/src/huffman_tables.rs Normal file
View File

File diff suppressed because it is too large Load Diff

145
crates/aac/src/ics.rs Normal file
View File

@ -0,0 +1,145 @@
//! `ics_info` and helpers shared by the encoder and decoder (ISO/IEC 14496-3 §4.4.2.1, §4.5.2.3).
use filmcraft_bitstream::{BitReader, BitWriter};
use crate::mdct::WindowShape;
use crate::tables::{swb_offsets_long, swb_offsets_short};
use crate::{Error, Result};
/// `window_sequence`.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum WindowSequence {
#[default]
OnlyLong = 0,
LongStart = 1,
EightShort = 2,
LongStop = 3,
}
impl WindowSequence {
pub fn from_bits(v: u32) -> Self {
match v & 3 {
0 => WindowSequence::OnlyLong,
1 => WindowSequence::LongStart,
2 => WindowSequence::EightShort,
_ => WindowSequence::LongStop,
}
}
}
/// Decoded/encoded `ics_info`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct IcsInfo {
pub window_sequence: WindowSequence,
pub window_shape: WindowShape,
pub max_sfb: usize,
pub num_groups: usize,
/// Windows per group (only the first `num_groups` entries are used; long windows: `[1]`).
pub group_len: [u8; 8],
}
impl Default for IcsInfo {
fn default() -> Self {
IcsInfo { window_sequence: WindowSequence::OnlyLong, window_shape: WindowShape::Sine, max_sfb: 0, num_groups: 1, group_len: [1, 0, 0, 0, 0, 0, 0, 0] }
}
}
impl IcsInfo {
#[inline]
pub fn is_short(&self) -> bool {
self.window_sequence == WindowSequence::EightShort
}
#[inline]
pub fn num_windows(&self) -> usize {
if self.is_short() { 8 } else { 1 }
}
/// Band offsets of one window.
pub fn swb_offsets(&self, sf_index: u8) -> &'static [u16] {
if self.is_short() { swb_offsets_short(sf_index) } else { swb_offsets_long(sf_index) }
}
/// Number of bands for this window length (`num_swb`).
pub fn num_swb(&self, sf_index: u8) -> usize {
self.swb_offsets(sf_index).len() - 1
}
/// First window of group `g`.
pub fn group_start(&self, g: usize) -> usize {
self.group_len[..g].iter().map(|&l| l as usize).sum()
}
pub fn parse(br: &mut BitReader, sf_index: u8) -> Result<IcsInfo> {
if br.read_bits(1)? != 0 {
// ics_reserved_bit; tolerated by most decoders
}
let window_sequence = WindowSequence::from_bits(br.read_bits(2)?);
let window_shape = WindowShape::from_bit(br.read_bits(1)? == 1);
let mut info = IcsInfo { window_sequence, window_shape, ..Default::default() };
if info.is_short() {
info.max_sfb = br.read_bits(4)? as usize;
let grouping = br.read_bits(7)?;
info.num_groups = 1;
info.group_len = [1, 0, 0, 0, 0, 0, 0, 0];
for w in 1..8 {
if grouping & (1 << (6 - (w - 1))) != 0 {
info.group_len[info.num_groups - 1] += 1;
} else {
info.num_groups += 1;
info.group_len[info.num_groups - 1] = 1;
}
}
} else {
info.max_sfb = br.read_bits(6)? as usize;
if br.read_bits(1)? == 1 {
return Err(Error::Unsupported("predictor data (AAC Main / LTP)"));
}
}
if info.max_sfb > info.num_swb(sf_index) {
return Err(Error::Bitstream("max_sfb exceeds the number of scalefactor bands"));
}
Ok(info)
}
pub fn write(&self, bw: &mut BitWriter) {
bw.write_bits(0, 1);
bw.write_bits(self.window_sequence as u32, 2);
bw.write_bits(self.window_shape.bit(), 1);
if self.is_short() {
bw.write_bits(self.max_sfb as u32, 4);
let mut grouping = 0u32;
let mut w = 0;
for g in 0..self.num_groups {
for i in 0..self.group_len[g] as usize {
if w > 0 && i > 0 {
grouping |= 1 << (6 - (w - 1));
}
w += 1;
}
}
bw.write_bits(grouping, 7);
} else {
bw.write_bits(self.max_sfb as u32, 6);
bw.write_bits(0, 1);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn grouping_roundtrip() {
let info = IcsInfo {
window_sequence: WindowSequence::EightShort,
window_shape: WindowShape::Kbd,
max_sfb: 12,
num_groups: 3,
group_len: [3, 1, 4, 0, 0, 0, 0, 0],
};
let mut bw = BitWriter::new();
info.write(&mut bw);
let d = bw.finish();
let p = IcsInfo::parse(&mut BitReader::new(&d), 4).unwrap();
assert_eq!(p, info);
assert_eq!(p.group_start(2), 4);
}
}

72
crates/aac/src/lib.rs Normal file
View File

@ -0,0 +1,72 @@
//! Clean-room AAC-LC encoder and decoder (ISO/IEC 14496-3, MPEG-4 Audio, Low Complexity profile).
//!
//! - [`Encoder`]: planar f32 → raw access units (+ [`Encoder::audio_specific_config`] for MP4 `esds`,
//! optional ADTS framing). Psychoacoustic model, window switching, M/S, TNS, bit reservoir, CBR/VBR,
//! 1–8 channels.
//! - [`Decoder`]: `AudioSpecificConfig` + raw access units → planar f32. Supports M/S, intensity
//! stereo, PNS, TNS, pulse data, both window shapes and every window sequence.
//!
//! HE-AAC (SBR/PS), Main/SSR/LTP profiles, coupling channels and 960-sample frames are not supported;
//! for explicitly signalled HE-AAC only the AAC-LC core is decoded (at the core rate).
//!
//! Layer L0: depends only on `filmcraft-bitstream` (+ `thiserror`, optional `rayon`); no `unsafe`;
//! builds for `wasm32-unknown-unknown`.
//!
//! ```
//! use filmcraft_aac::{Decoder, Encoder, EncoderConfig};
//!
//! let mut enc = Encoder::new(EncoderConfig::cbr(48_000, 2, 128_000))?;
//! let left: Vec<f32> = (0..4800).map(|i| (i as f32 * 0.05).sin() * 0.5).collect();
//! let right = left.clone();
//! let mut aus = enc.encode(&[&left, &right]); // 0..n access units
//! aus.extend(enc.flush());
//! let asc = enc.audio_specific_config(); // for the MP4 `esds`
//! assert_eq!(enc.priming_samples(), 1024); // MP4 edit list media_time
//!
//! let mut dec = Decoder::new(&asc)?;
//! for au in &aus {
//! let pcm = dec.decode(au)?; // planar, 1024 samples per channel
//! assert_eq!(pcm.len(), 2);
//! }
//! # Ok::<(), filmcraft_aac::Error>(())
//! ```
mod config;
mod decoder;
mod encoder;
mod huffman;
mod huffman_tables;
mod ics;
mod mdct;
mod tables;
mod tns;
pub use config::{AdtsHeader, AudioSpecificConfig, ElementType, ProgramConfig, config_layout, split_adts};
pub use decoder::Decoder;
pub use encoder::{BitrateMode, Encoder, EncoderConfig, EncoderStats};
pub use mdct::WindowShape;
pub use tables::{SAMPLE_RATES, sample_rate_index};
/// Samples per AAC-LC frame (per channel).
pub const FRAME_LEN: usize = 1024;
/// Errors from configuration parsing, encoding and decoding.
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("invalid configuration: {0}")]
InvalidConfig(&'static str),
#[error("unsupported feature: {0}")]
Unsupported(&'static str),
#[error("invalid bitstream: {0}")]
Bitstream(&'static str),
#[error("unexpected end of access unit")]
Eof,
}
impl From<filmcraft_bitstream::BitError> for Error {
fn from(_: filmcraft_bitstream::BitError) -> Self {
Error::Eof
}
}
pub type Result<T> = std::result::Result<T, Error>;

274
crates/aac/src/mdct.rs Normal file
View File

@ -0,0 +1,274 @@
//! MDCT / IMDCT (ISO/IEC 14496-3 §4.6.18) via a DCT-IV computed with an N/4-point complex FFT,
//! plus the sine and Kaiser–Bessel-derived windows.
use std::f64::consts::PI;
use std::sync::OnceLock;
/// Radix-2 complex FFT (forward, `e^{-2πi nk/P}`).
struct Fft {
n: usize,
twiddle: Vec<[f32; 2]>,
rev: Vec<u32>,
}
impl Fft {
fn new(n: usize) -> Fft {
assert!(n.is_power_of_two());
let bits = n.trailing_zeros();
let rev = (0..n as u32).map(|i| if bits == 0 { 0 } else { i.reverse_bits() >> (32 - bits) }).collect();
let twiddle = (0..n / 2)
.map(|k| {
let a = -2.0 * PI * k as f64 / n as f64;
[a.cos() as f32, a.sin() as f32]
})
.collect();
Fft { n, twiddle, rev }
}
fn run(&self, re: &mut [f32], im: &mut [f32]) {
let n = self.n;
for i in 0..n {
let j = self.rev[i] as usize;
if j > i {
re.swap(i, j);
im.swap(i, j);
}
}
let mut len = 2;
while len <= n {
let half = len / 2;
let step = n / len;
for start in (0..n).step_by(len) {
for k in 0..half {
let [wr, wi] = self.twiddle[k * step];
let a = start + k;
let b = a + half;
let xr = re[b] * wr - im[b] * wi;
let xi = re[b] * wi + im[b] * wr;
re[b] = re[a] - xr;
im[b] = im[a] - xi;
re[a] += xr;
im[a] += xi;
}
}
len <<= 1;
}
}
}
/// MDCT of `2M` inputs to `M` outputs (unscaled sums):
/// `X[k] = Σ x[n] cos(π/M (n + ½ + M/2)(k + ½))`, and the matching unscaled inverse.
pub struct Mdct {
m: usize,
fft: Fft,
pre: Vec<[f32; 2]>,
post: Vec<[f32; 2]>,
}
impl Mdct {
pub fn new(m: usize) -> Mdct {
let q = m / 2;
let pre = (0..q)
.map(|n| {
let a = -PI * n as f64 / m as f64;
[a.cos() as f32, a.sin() as f32]
})
.collect();
let post = (0..q)
.map(|k| {
let a = -PI * (k as f64 + 0.25) / m as f64;
[a.cos() as f32, a.sin() as f32]
})
.collect();
Mdct { m, fft: Fft::new(q), pre, post }
}
/// DCT-IV: `out[k] = Σ u[n] cos(π/M (n+½)(k+½))`.
fn dct4(&self, u: &[f32], out: &mut [f32]) {
let m = self.m;
let q = m / 2;
let mut re = vec![0f32; q];
let mut im = vec![0f32; q];
for n in 0..q {
let (a, b) = (u[2 * n], u[m - 1 - 2 * n]);
let [c, s] = self.pre[n];
re[n] = a * c - b * s;
im[n] = a * s + b * c;
}
self.fft.run(&mut re, &mut im);
for k in 0..q {
let [c, s] = self.post[k];
let wr = re[k] * c - im[k] * s;
let wi = re[k] * s + im[k] * c;
out[2 * k] = wr;
out[m - 1 - 2 * k] = -wi;
}
}
/// Forward MDCT of `x` (length 2M, already windowed) into `out` (length M).
pub fn forward(&self, x: &[f32], out: &mut [f32]) {
let m = self.m;
let h = m / 2;
let mut u = vec![0f32; m];
for n in 0..h {
// quarters a, b, c, d of length M/2
u[n] = -x[m + h - 1 - n] - x[m + h + n];
u[h + n] = x[n] - x[m - 1 - n];
}
self.dct4(&u, out);
}
/// Inverse MDCT (unscaled): `y[n] = Σ X[k] cos(π/M (n + ½ + M/2)(k + ½))`, `y` has length 2M.
pub fn inverse(&self, spec: &[f32], y: &mut [f32]) {
let m = self.m;
let h = m / 2;
let mut u = vec![0f32; m];
self.dct4(spec, &mut u);
for n in 0..h {
y[n] = u[h + n];
y[m - 1 - n] = -u[h + n];
y[m + h - 1 - n] = -u[n];
y[m + h + n] = -u[n];
}
}
}
/// Cached transforms for 1024 and 128 lines.
pub fn mdct_long() -> &'static Mdct {
static M: OnceLock<Mdct> = OnceLock::new();
M.get_or_init(|| Mdct::new(1024))
}
pub fn mdct_short() -> &'static Mdct {
static M: OnceLock<Mdct> = OnceLock::new();
M.get_or_init(|| Mdct::new(128))
}
/// Window shape (`window_shape` bit): 0 = sine, 1 = Kaiser–Bessel derived.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum WindowShape {
#[default]
Sine,
Kbd,
}
impl WindowShape {
pub fn from_bit(b: bool) -> Self {
if b { WindowShape::Kbd } else { WindowShape::Sine }
}
pub fn bit(self) -> u32 {
(self == WindowShape::Kbd) as u32
}
}
fn sine_half(m: usize) -> Vec<f32> {
let n = 2 * m;
(0..m).map(|i| (PI / n as f64 * (i as f64 + 0.5)).sin() as f32).collect()
}
fn bessel_i0(x: f64) -> f64 {
let mut sum = 1.0;
let mut term = 1.0;
let y = x * x / 4.0;
for k in 1..60 {
term *= y / (k as f64 * k as f64);
sum += term;
if term < sum * 1e-17 {
break;
}
}
sum
}
fn kbd_half(m: usize, alpha: f64) -> Vec<f32> {
// w[n] = sqrt(Σ_{j<=n} W(j) / Σ_{j<=M} W(j)), W the Kaiser kernel of length M+1.
let kernel: Vec<f64> = (0..=m)
.map(|j| {
let r = (j as f64 - m as f64 / 2.0) / (m as f64 / 2.0);
bessel_i0(PI * alpha * (1.0 - r * r).max(0.0).sqrt())
})
.collect();
let total: f64 = kernel.iter().sum();
let mut acc = 0.0;
(0..m)
.map(|n| {
acc += kernel[n];
(acc / total).sqrt() as f32
})
.collect()
}
/// Rising half (length N/2) of the window of the given shape; `long` selects N = 2048, else N = 256.
pub fn window(shape: WindowShape, long: bool) -> &'static [f32] {
static W: OnceLock<[Vec<f32>; 4]> = OnceLock::new();
let w = W.get_or_init(|| [sine_half(1024), kbd_half(1024, 4.0), sine_half(128), kbd_half(128, 6.0)]);
let i = match (long, shape) {
(true, WindowShape::Sine) => 0,
(true, WindowShape::Kbd) => 1,
(false, WindowShape::Sine) => 2,
(false, WindowShape::Kbd) => 3,
};
&w[i]
}
#[cfg(test)]
mod tests {
use super::*;
fn direct(x: &[f32]) -> Vec<f64> {
let m = x.len() / 2;
(0..m).map(|k| (0..2 * m).map(|n| x[n] as f64 * (PI / m as f64 * (n as f64 + 0.5 + m as f64 / 2.0) * (k as f64 + 0.5)).cos()).sum()).collect()
}
#[test]
fn forward_matches_definition() {
for m in [128usize, 1024] {
let x: Vec<f32> = (0..2 * m).map(|i| ((i * 7919) % 101) as f32 / 50.0 - 1.0).collect();
let mut out = vec![0f32; m];
Mdct::new(m).forward(&x, &mut out);
let d = direct(&x);
for k in 0..m {
assert!((out[k] as f64 - d[k]).abs() < 1e-3 * (m as f64).sqrt(), "m={m} k={k} {} {}", out[k], d[k]);
}
}
}
#[test]
fn inverse_matches_definition() {
let m = 128;
let spec: Vec<f32> = (0..m).map(|i| ((i * 31) % 17) as f32 - 8.0).collect();
let mut y = vec![0f32; 2 * m];
Mdct::new(m).inverse(&spec, &mut y);
for n in 0..2 * m {
let d: f64 = (0..m).map(|k| spec[k] as f64 * (PI / m as f64 * (n as f64 + 0.5 + m as f64 / 2.0) * (k as f64 + 0.5)).cos()).sum();
assert!((y[n] as f64 - d).abs() < 1e-3, "n={n}");
}
}
#[test]
fn tdac_perfect_reconstruction() {
let m = 1024;
let t = Mdct::new(m);
for shape in [WindowShape::Sine, WindowShape::Kbd] {
let w = window(shape, true);
let win = |n: usize| if n < m { w[n] } else { w[2 * m - 1 - n] };
let x: Vec<f32> = (0..4 * m).map(|i| ((i as f32) * 0.013).sin() + ((i * 13) % 7) as f32 * 0.05).collect();
let mut out = vec![0f32; 4 * m];
for f in 0..3 {
let seg: Vec<f32> = (0..2 * m).map(|n| x[f * m + n] * win(n)).collect();
let mut spec = vec![0f32; m];
t.forward(&seg, &mut spec);
for v in spec.iter_mut() {
*v *= 2.0;
}
let mut y = vec![0f32; 2 * m];
t.inverse(&spec, &mut y);
for n in 0..2 * m {
out[f * m + n] += y[n] * win(n) / m as f32;
}
}
for n in m..3 * m {
assert!((out[n] - x[n]).abs() < 1e-4, "{shape:?} n={n} {} {}", out[n], x[n]);
}
}
}
}

143
crates/aac/src/tables.rs Normal file
View File

@ -0,0 +1,143 @@
//! Sampling-rate dependent tables from ISO/IEC 14496-3 §4.5.4 (scalefactor bands) and §4.6.9 (TNS).
/// Sampling frequencies indexed by `samplingFrequencyIndex`.
pub const SAMPLE_RATES: [u32; 13] = [96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350];
/// Map an arbitrary sample rate to its `samplingFrequencyIndex`, if it is one of the standard rates.
pub fn sample_rate_index(rate: u32) -> Option<u8> {
SAMPLE_RATES.iter().position(|&r| r == rate).map(|i| i as u8)
}
/// The table index used for scalefactor-band tables when a non-standard rate is signalled explicitly
/// (§4.5.4: rates are mapped to the nearest table by frequency ranges).
pub fn table_index_for_rate(rate: u32) -> u8 {
match rate {
92017.. => 0,
75132.. => 1,
55426.. => 2,
46009.. => 3,
37566.. => 4,
27713.. => 5,
23004.. => 6,
18783.. => 7,
13856.. => 8,
11502.. => 9,
9391.. => 10,
_ => 11,
}
}
const SWB_LONG_96: &[u16] = &[
0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 64, 72, 80, 88, 96, 108, 120, 132, 144, 156, 172, 188, 212, 240, 276, 320, 384, 448, 512, 576,
640, 704, 768, 832, 896, 960, 1024,
];
const SWB_LONG_64: &[u16] = &[
0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 64, 72, 80, 88, 100, 112, 124, 140, 156, 172, 192, 216, 240, 268, 304, 344, 384, 424, 464, 504,
544, 584, 624, 664, 704, 744, 784, 824, 864, 904, 944, 984, 1024,
];
const SWB_LONG_48: &[u16] = &[
0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 48, 56, 64, 72, 80, 88, 96, 108, 120, 132, 144, 160, 176, 196, 216, 240, 264, 292, 320, 352, 384, 416, 448, 480,
512, 544, 576, 608, 640, 672, 704, 736, 768, 800, 832, 864, 896, 928, 1024,
];
const SWB_LONG_32: &[u16] = &[
0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 48, 56, 64, 72, 80, 88, 96, 108, 120, 132, 144, 160, 176, 196, 216, 240, 264, 292, 320, 352, 384, 416, 448, 480,
512, 544, 576, 608, 640, 672, 704, 736, 768, 800, 832, 864, 896, 928, 960, 992, 1024,
];
const SWB_LONG_24: &[u16] = &[
0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 52, 60, 68, 76, 84, 92, 100, 108, 116, 124, 136, 148, 160, 172, 188, 204, 220, 240, 260, 284, 308, 336, 364,
396, 432, 468, 508, 552, 600, 652, 704, 768, 832, 896, 960, 1024,
];
const SWB_LONG_16: &[u16] = &[
0, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 100, 112, 124, 136, 148, 160, 172, 184, 196, 212, 228, 244, 260, 280, 300, 320, 344, 368, 396, 424, 456, 492,
532, 572, 616, 664, 716, 772, 832, 896, 960, 1024,
];
const SWB_LONG_8: &[u16] = &[
0, 12, 24, 36, 48, 60, 72, 84, 96, 108, 120, 132, 144, 156, 172, 188, 204, 220, 236, 252, 268, 288, 308, 328, 348, 372, 396, 420, 448, 476, 508, 544, 580,
620, 664, 712, 764, 820, 880, 944, 1024,
];
const SWB_SHORT_96: &[u16] = &[0, 4, 8, 12, 16, 20, 24, 32, 40, 48, 64, 92, 128];
const SWB_SHORT_48: &[u16] = &[0, 4, 8, 12, 16, 20, 28, 36, 44, 56, 68, 80, 96, 112, 128];
const SWB_SHORT_24: &[u16] = &[0, 4, 8, 12, 16, 20, 24, 28, 36, 44, 52, 64, 76, 92, 108, 128];
const SWB_SHORT_16: &[u16] = &[0, 4, 8, 12, 16, 20, 24, 28, 32, 40, 48, 60, 72, 88, 108, 128];
const SWB_SHORT_8: &[u16] = &[0, 4, 8, 12, 16, 20, 24, 28, 36, 44, 52, 60, 72, 88, 108, 128];
/// Scalefactor band offsets (`swb_offset_long_window`, 1024-line frames); `len() == num_swb + 1`.
pub fn swb_offsets_long(sf_index: u8) -> &'static [u16] {
match sf_index {
0 | 1 => SWB_LONG_96,
2 => SWB_LONG_64,
3 | 4 => SWB_LONG_48,
5 => SWB_LONG_32,
6 | 7 => SWB_LONG_24,
8..=10 => SWB_LONG_16,
_ => SWB_LONG_8,
}
}
/// Scalefactor band offsets for one short window (128 lines).
pub fn swb_offsets_short(sf_index: u8) -> &'static [u16] {
match sf_index {
0..=2 => SWB_SHORT_96,
3..=5 => SWB_SHORT_48,
6 | 7 => SWB_SHORT_24,
8..=10 => SWB_SHORT_16,
_ => SWB_SHORT_8,
}
}
/// `TNS_MAX_BANDS` for AAC-LC (long, short).
pub fn tns_max_bands(sf_index: u8, short: bool) -> usize {
const LONG: [u8; 13] = [31, 31, 34, 40, 42, 51, 46, 46, 42, 42, 42, 39, 39];
const SHORT: [u8; 13] = [9, 9, 10, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14];
let i = (sf_index as usize).min(12);
if short { SHORT[i] as usize } else { LONG[i] as usize }
}
/// `TNS_MAX_ORDER` for AAC-LC.
pub fn tns_max_order(short: bool) -> usize {
if short { 7 } else { 12 }
}
/// `x^(4/3)` for the integer quantised magnitudes 0..=8191.
pub fn pow43(q: u32) -> f32 {
use std::sync::OnceLock;
static TABLE: OnceLock<Vec<f32>> = OnceLock::new();
let t = TABLE.get_or_init(|| (0..8192u32).map(|i| (i as f64).powf(4.0 / 3.0) as f32).collect());
match t.get(q as usize) {
Some(v) => *v,
None => (q as f64).powf(4.0 / 3.0) as f32,
}
}
/// `2^(0.25 * (sf - 100))`, the dequantisation gain of scalefactor `sf`.
#[inline]
pub fn sf_gain(sf: i32) -> f32 {
(0.25 * (sf as f64 - 100.0)).exp2() as f32
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn band_tables_are_monotonic_and_complete() {
for i in 0..13u8 {
let l = swb_offsets_long(i);
assert_eq!(*l.last().unwrap(), 1024);
assert!(l.windows(2).all(|w| w[0] < w[1] && (w[1] - w[0]) % 4 == 0));
let s = swb_offsets_short(i);
assert_eq!(*s.last().unwrap(), 128);
assert!(s.windows(2).all(|w| w[0] < w[1] && (w[1] - w[0]) % 4 == 0));
assert!(tns_max_bands(i, false) < l.len());
assert!(tns_max_bands(i, true) < s.len());
}
assert_eq!(swb_offsets_long(3).len() - 1, 49);
assert_eq!(swb_offsets_long(5).len() - 1, 51);
assert_eq!(swb_offsets_long(0).len() - 1, 41);
assert_eq!(swb_offsets_long(2).len() - 1, 47);
assert_eq!(swb_offsets_long(6).len() - 1, 47);
assert_eq!(swb_offsets_long(8).len() - 1, 43);
assert_eq!(swb_offsets_long(11).len() - 1, 40);
}
}

313
crates/aac/src/tns.rs Normal file
View File

@ -0,0 +1,313 @@
//! Temporal noise shaping (ISO/IEC 14496-3 §4.6.9): syntax, coefficient conversion, and the
//! decoder (all-pole) and encoder (all-zero) filters.
use std::f64::consts::FRAC_PI_2;
use filmcraft_bitstream::{BitReader, BitWriter};
use crate::ics::IcsInfo;
use crate::tables::{tns_max_bands, tns_max_order};
use crate::{Error, Result};
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct TnsFilter {
/// Length in scalefactor bands.
pub length: u8,
pub order: u8,
/// false = upward, true = downward.
pub direction: bool,
pub coef_compress: bool,
/// Quantised reflection coefficients (signed, `coef_res + 3 - coef_compress` bits each).
pub coef: [i8; 12],
}
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct TnsWindow {
pub n_filt: u8,
/// false = 3-bit, true = 4-bit coefficient resolution.
pub coef_res: bool,
pub filt: [TnsFilter; 4],
}
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct TnsData {
pub windows: [TnsWindow; 8],
}
impl TnsData {
pub fn parse(br: &mut BitReader, info: &IcsInfo) -> Result<TnsData> {
let short = info.is_short();
let mut t = TnsData::default();
for w in 0..info.num_windows() {
let win = &mut t.windows[w];
win.n_filt = br.read_bits(if short { 1 } else { 2 })? as u8;
if win.n_filt == 0 {
continue;
}
win.coef_res = br.read_bits(1)? == 1;
for f in 0..win.n_filt as usize {
let filt = &mut win.filt[f];
filt.length = br.read_bits(if short { 4 } else { 6 })? as u8;
filt.order = br.read_bits(if short { 3 } else { 5 })? as u8;
if filt.order as usize > tns_max_order(short) {
return Err(Error::Bitstream("TNS order too large"));
}
if filt.order > 0 {
filt.direction = br.read_bits(1)? == 1;
filt.coef_compress = br.read_bits(1)? == 1;
let bits = 3 + win.coef_res as u32 - filt.coef_compress as u32;
for i in 0..filt.order as usize {
let v = br.read_bits(bits)? as i32;
let v = if v & (1 << (bits - 1)) != 0 { v - (1 << bits) } else { v };
filt.coef[i] = v as i8;
}
}
}
}
Ok(t)
}
pub fn write(&self, bw: &mut BitWriter, info: &IcsInfo) {
let short = info.is_short();
for w in 0..info.num_windows() {
let win = &self.windows[w];
bw.write_bits(win.n_filt as u32, if short { 1 } else { 2 });
if win.n_filt == 0 {
continue;
}
bw.write_bits(win.coef_res as u32, 1);
for f in 0..win.n_filt as usize {
let filt = &win.filt[f];
bw.write_bits(filt.length as u32, if short { 4 } else { 6 });
bw.write_bits(filt.order as u32, if short { 3 } else { 5 });
if filt.order > 0 {
bw.write_bits(filt.direction as u32, 1);
bw.write_bits(filt.coef_compress as u32, 1);
let bits = 3 + win.coef_res as u32 - filt.coef_compress as u32;
for i in 0..filt.order as usize {
bw.write_bits(filt.coef[i] as i32 as u32 & ((1 << bits) - 1), bits);
}
}
}
}
}
/// Bits `write` would produce.
pub fn bit_count(&self, info: &IcsInfo) -> u32 {
let short = info.is_short();
let mut n = 0;
for w in 0..info.num_windows() {
let win = &self.windows[w];
n += if short { 1 } else { 2 };
if win.n_filt == 0 {
continue;
}
n += 1;
for f in 0..win.n_filt as usize {
let filt = &win.filt[f];
n += if short { 7 } else { 11 };
if filt.order > 0 {
n += 2 + filt.order as u32 * (3 + win.coef_res as u32 - filt.coef_compress as u32);
}
}
}
n
}
}
/// Dequantise one reflection coefficient.
pub fn dequant_coef(c: i8, coef_res: bool) -> f64 {
let res = 3 + coef_res as i32;
let iqfac = ((1 << (res - 1)) as f64 - 0.5) / FRAC_PI_2;
let iqfac_m = ((1 << (res - 1)) as f64 + 0.5) / FRAC_PI_2;
let c = c as f64;
(c / if c >= 0.0 { iqfac } else { iqfac_m }).sin()
}
/// Quantise a reflection coefficient (inverse of [`dequant_coef`], nearest index).
pub fn quant_coef(k: f64, coef_res: bool) -> i8 {
let res = 3 + coef_res as i32;
let iqfac = ((1 << (res - 1)) as f64 - 0.5) / FRAC_PI_2;
let iqfac_m = ((1 << (res - 1)) as f64 + 0.5) / FRAC_PI_2;
let a = k.clamp(-1.0, 1.0).asin();
let lo = -(1 << (res - 1));
let hi = (1 << (res - 1)) - 1;
let q = if a >= 0.0 { (a * iqfac).round() } else { (a * iqfac_m).round() };
(q as i32).clamp(lo, hi) as i8
}
/// Reflection coefficients → direct-form LPC `a[0..=order]` (`a[0] = 1`).
pub fn parcor_to_lpc(k: &[f64]) -> [f64; 13] {
let mut a = [0f64; 13];
a[0] = 1.0;
let mut b = [0f64; 13];
for m in 1..=k.len() {
for i in 1..m {
b[i] = a[i] + k[m - 1] * a[m - i];
}
a[1..m].copy_from_slice(&b[1..m]);
a[m] = k[m - 1];
}
a
}
/// Filter region `[start, end)` of every TNS filter of window `w`, in spectrum lines of that window.
fn regions(info: &IcsInfo, sf_index: u8, win: &TnsWindow) -> Vec<(usize, usize, usize)> {
let short = info.is_short();
let swb = info.swb_offsets(sf_index);
let num_swb = swb.len() - 1;
let lim = tns_max_bands(sf_index, short).min(info.max_sfb);
let mut out = Vec::new();
let mut bottom = num_swb;
for f in 0..win.n_filt as usize {
let filt = &win.filt[f];
let top = bottom;
bottom = top.saturating_sub(filt.length as usize);
if filt.order == 0 {
continue;
}
let start = swb[bottom.min(lim)] as usize;
let end = swb[top.min(lim)] as usize;
if end > start {
out.push((f, start, end));
}
}
out
}
fn lpc_of(filt: &TnsFilter, coef_res: bool) -> [f64; 13] {
let k: Vec<f64> = filt.coef[..filt.order as usize].iter().map(|&c| dequant_coef(c, coef_res)).collect();
parcor_to_lpc(&k)
}
/// Decoder: apply the all-pole TNS synthesis filters to `spec` (1024 lines; short windows at 128 stride).
pub fn apply_decoder(spec: &mut [f32], info: &IcsInfo, sf_index: u8, tns: &TnsData) {
let wlen = if info.is_short() { 128 } else { 1024 };
for w in 0..info.num_windows() {
let win = &tns.windows[w];
for (f, start, end) in regions(info, sf_index, win) {
let filt = &win.filt[f];
let a = lpc_of(filt, win.coef_res);
let order = filt.order as usize;
let s = &mut spec[w * wlen..(w + 1) * wlen];
let size = end - start;
let mut state = [0f64; 12];
for m in 0..size {
let idx = if filt.direction { end - 1 - m } else { start + m };
let mut y = s[idx] as f64;
for i in 0..order {
y -= a[i + 1] * state[i];
}
for i in (1..order).rev() {
state[i] = state[i - 1];
}
if order > 0 {
state[0] = y;
}
s[idx] = y as f32;
}
}
}
}
/// Encoder: apply the all-zero TNS analysis filters (exact inverse of [`apply_decoder`]).
pub fn apply_encoder(spec: &mut [f32], info: &IcsInfo, sf_index: u8, tns: &TnsData) {
let wlen = if info.is_short() { 128 } else { 1024 };
for w in 0..info.num_windows() {
let win = &tns.windows[w];
for (f, start, end) in regions(info, sf_index, win) {
let filt = &win.filt[f];
let a = lpc_of(filt, win.coef_res);
let order = filt.order as usize;
let s = &mut spec[w * wlen..(w + 1) * wlen];
let size = end - start;
let mut state = [0f64; 12];
for m in 0..size {
let idx = if filt.direction { end - 1 - m } else { start + m };
let x = s[idx] as f64;
let mut y = x;
for i in 0..order {
y += a[i + 1] * state[i];
}
for i in (1..order).rev() {
state[i] = state[i - 1];
}
if order > 0 {
state[0] = x;
}
s[idx] = y as f32;
}
}
}
}
/// Levinson–Durbin on autocorrelation `r[0..=order]`: returns reflection coefficients and the
/// prediction gain `r[0] / error`.
pub fn levinson(r: &[f64], order: usize) -> (Vec<f64>, f64) {
let mut a = vec![0f64; order + 1];
a[0] = 1.0;
let mut err = r[0];
let mut ks = Vec::with_capacity(order);
if err <= 0.0 {
return (vec![0.0; order], 1.0);
}
for m in 1..=order {
let mut acc = r[m];
for i in 1..m {
acc += a[i] * r[m - i];
}
let k = -acc / err;
let prev = a.clone();
for i in 1..m {
a[i] = prev[i] + k * prev[m - i];
}
a[m] = k;
err *= 1.0 - k * k;
ks.push(k);
if err <= r[0] * 1e-9 {
ks.resize(order, 0.0);
break;
}
}
(ks, r[0] / err.max(r[0] * 1e-9))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::ics::WindowSequence;
use crate::mdct::WindowShape;
#[test]
fn coef_quant_roundtrip() {
for res in [false, true] {
let n = if res { 8 } else { 4 };
for c in -n..n {
let k = dequant_coef(c as i8, res);
assert_eq!(quant_coef(k, res), c as i8);
}
}
}
#[test]
fn encoder_decoder_filters_are_inverse() {
let info = IcsInfo { window_sequence: WindowSequence::OnlyLong, window_shape: WindowShape::Sine, max_sfb: 49, ..Default::default() };
let mut t = TnsData::default();
t.windows[0].n_filt = 1;
t.windows[0].coef_res = true;
t.windows[0].filt[0] = TnsFilter { length: 40, order: 4, direction: true, coef_compress: false, coef: [3, -2, 1, 5, 0, 0, 0, 0, 0, 0, 0, 0] };
let orig: Vec<f32> = (0..1024).map(|i| ((i * 37 % 101) as f32 - 50.0) * 0.1).collect();
let mut s = orig.clone();
apply_encoder(&mut s, &info, 4, &t);
assert_ne!(s, orig);
apply_decoder(&mut s, &info, 4, &t);
for i in 0..1024 {
assert!((s[i] - orig[i]).abs() < 1e-3);
}
let mut bw = BitWriter::new();
t.write(&mut bw, &info);
assert_eq!(bw.bit_len() as u32, t.bit_count(&info));
let d = bw.finish();
assert_eq!(TnsData::parse(&mut BitReader::new(&d), &info).unwrap(), t);
}
}

283
crates/aac/tests/common/mod.rs Normal file
View File

@ -0,0 +1,283 @@
#![allow(dead_code)]
use std::path::{Path, PathBuf};
use std::process::Command;
use filmcraft_aac::{Decoder, Encoder, EncoderConfig, EncoderStats, split_adts};
pub fn ffmpeg() -> Option<PathBuf> {
for p in ["/opt/homebrew/bin/ffmpeg", "/usr/local/bin/ffmpeg", "/usr/bin/ffmpeg"] {
if Path::new(p).exists() {
return Some(PathBuf::from(p));
}
}
None
}
pub fn fixtures() -> PathBuf {
let d = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../target/fixtures/aac");
std::fs::create_dir_all(&d).unwrap();
d
}
pub struct Rng(pub u64);
impl Rng {
pub fn next(&mut self) -> f32 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
((self.0 >> 40) as f32 / (1u64 << 24) as f32) * 2.0 - 1.0
}
}
use std::f32::consts::PI;
/// Logarithmic sine sweep 40 Hz → 16 kHz.
pub fn sweep(rate: u32, n: usize, amp: f32) -> Vec<f32> {
let (f0, f1) = (40f64, 16000f64.min(rate as f64 * 0.45));
let t_total = n as f64 / rate as f64;
let k = (f1 / f0).ln();
(0..n)
.map(|i| {
let t = i as f64 / rate as f64;
let phase = 2.0 * std::f64::consts::PI * f0 * t_total / k * ((t / t_total * k).exp() - 1.0);
(phase.sin() as f32) * amp
})
.collect()
}
pub fn noise(n: usize, amp: f32, seed: u64) -> Vec<f32> {
let mut r = Rng(seed);
(0..n).map(|_| r.next() * amp).collect()
}
/// Speech-like: glottal pulse train (gliding pitch) through three resonant formant filters with a
/// syllabic envelope and pauses, plus a little fricative noise.
pub fn speech(rate: u32, n: usize, seed: u64) -> Vec<f32> {
let mut r = Rng(seed);
let fs = rate as f32;
let formants = [(700.0f32, 130.0f32), (1220.0, 70.0), (2600.0, 160.0)];
let mut states = [[0f32; 2]; 3];
let mut phase = 0f32;
let mut out = Vec::with_capacity(n);
for i in 0..n {
let t = i as f32 / fs;
let f0 = 120.0 + 30.0 * (2.0 * PI * 0.7 * t).sin();
phase += f0 / fs;
let pulse = if phase >= 1.0 {
phase -= 1.0;
1.0
} else {
0.0
};
let syll = (2.0 * PI * 3.0 * t).sin().max(0.0);
let voiced = if (t * 1.3).fract() < 0.8 { syll } else { 0.0 };
let fric = if (t * 1.3).fract() > 0.85 { 0.05 * r.next() } else { 0.0 };
let mut s = 0f32;
for (j, &(f, bw)) in formants.iter().enumerate() {
let rr = (-PI * bw / fs).exp();
let a1 = 2.0 * rr * (2.0 * PI * f / fs).cos();
let a2 = -rr * rr;
let y = pulse * voiced + a1 * states[j][0] + a2 * states[j][1];
states[j][1] = states[j][0];
states[j][0] = y;
s += y * [1.0, 0.6, 0.3][j];
}
out.push(s * 0.08 + fric);
}
let peak = out.iter().fold(0f32, |m, v| m.max(v.abs()));
out.iter().map(|v| v / peak * 0.7).collect()
}
/// Music-like: a sequence of decaying harmonic chords.
pub fn chords(rate: u32, n: usize, detune: f32) -> Vec<f32> {
let fs = rate as f32;
let roots = [220.0f32, 261.63, 196.0, 246.94];
let chord_len = (fs * 0.5) as usize;
(0..n)
.map(|i| {
let c = (i / chord_len) % roots.len();
let t = (i % chord_len) as f32 / fs;
let env = (-t * 3.0).exp() * (1.0 - (-t * 200.0).exp());
let mut s = 0f32;
for (m, ratio) in [1.0f32, 1.25, 1.5, 2.0].iter().enumerate() {
let f = roots[c] * ratio * (1.0 + detune);
for h in 1..8 {
s += (2.0 * PI * f * h as f32 * t + m as f32).sin() / (h * h) as f32;
}
}
s * env * 0.18
})
.collect()
}
/// Clicks: sharp impulses every 250 ms over a quiet low tone (transient test).
pub fn clicks(rate: u32, n: usize) -> Vec<f32> {
let fs = rate as f32;
let period = (fs * 0.25) as usize;
(0..n)
.map(|i| {
let t = i as f32 / fs;
let k = i % period;
let click = if k < 256 { 0.8 * (-(k as f32) / 24.0).exp() * (2.0 * PI * 1500.0 * k as f32 / fs).cos() } else { 0.0 };
0.02 * (2.0 * PI * 330.0 * t).sin() + click
})
.collect()
}
pub struct Encoded {
pub adts: Vec<u8>,
pub raw_bytes: usize,
pub frames: usize,
pub stats: EncoderStats,
pub asc: Vec<u8>,
pub bandwidth: u32,
}
pub fn encode(cfg: EncoderConfig, planar: &[Vec<f32>]) -> Encoded {
let mut enc = Encoder::new(EncoderConfig { adts: true, ..cfg }).unwrap();
let mut adts = Vec::new();
// feed in uneven chunks
let n = planar[0].len();
let mut pos = 0;
let mut step = 700;
while pos < n {
let end = (pos + step).min(n);
let chunk: Vec<&[f32]> = planar.iter().map(|c| &c[pos..end]).collect();
for au in enc.encode(&chunk) {
adts.extend(au);
}
pos = end;
step = step * 7 % 3001 + 100;
}
for au in enc.flush() {
adts.extend(au);
}
let stats = enc.stats();
Encoded { raw_bytes: stats.bytes as usize, frames: stats.frames as usize, stats, asc: enc.audio_specific_config(), bandwidth: enc.bandwidth(), adts }
}
/// Decode with ffmpeg → planar f32 (ffmpeg's own channel order), plus stderr.
pub fn ffmpeg_decode(ff: &Path, file: &Path, channels: usize) -> (Vec<Vec<f32>>, String) {
let out = Command::new(ff).args(["-v", "error", "-i"]).arg(file).args(["-f", "f32le", "-"]).output().unwrap();
let data = out.stdout;
let n = data.len() / 4 / channels;
let mut planar = vec![Vec::with_capacity(n); channels];
for i in 0..n {
for c in 0..channels {
let o = (i * channels + c) * 4;
planar[c].push(f32::from_le_bytes([data[o], data[o + 1], data[o + 2], data[o + 3]]));
}
}
(planar, String::from_utf8_lossy(&out.stderr).into_owned())
}
/// Encode with ffmpeg's AAC encoder to ADTS.
pub fn ffmpeg_encode(ff: &Path, planar: &[Vec<f32>], rate: u32, args: &[&str], out: &Path) {
let raw = out.with_extension("f32");
let ch = planar.len();
let mut bytes = Vec::with_capacity(planar[0].len() * ch * 4);
for i in 0..planar[0].len() {
for c in planar {
bytes.extend_from_slice(&c[i].to_le_bytes());
}
}
std::fs::write(&raw, bytes).unwrap();
let layout = match ch {
1 => "mono",
2 => "stereo",
3 => "3.0",
4 => "4.0",
5 => "5.0",
6 => "5.1",
_ => "7.1(wide)",
};
let st = Command::new(ff)
.args(["-v", "error", "-y", "-f", "f32le", "-ar", &rate.to_string(), "-ch_layout", layout, "-i"])
.arg(&raw)
.args(["-c:a", "aac"])
.args(args)
.args(["-f", "adts"])
.arg(out)
.status()
.unwrap();
assert!(st.success());
}
/// Decode an ADTS file with our decoder.
pub fn our_decode(data: &[u8]) -> Vec<Vec<f32>> {
let frames = split_adts(data).unwrap();
let mut dec = Decoder::from_adts(&frames[0].0).unwrap();
let mut out: Vec<Vec<f32>> = Vec::new();
for (_, au) in frames {
let pcm = dec.decode(au).unwrap();
out.resize(pcm.len(), Vec::new());
for (o, p) in out.iter_mut().zip(pcm) {
o.extend(p);
}
}
out
}
/// SNR in dB of `dec` against `refr`.
pub fn snr(refr: &[f32], dec: &[f32]) -> f64 {
let n = refr.len().min(dec.len());
let (mut s, mut e) = (0f64, 0f64);
for i in 0..n {
s += (refr[i] as f64).powi(2);
e += (refr[i] as f64 - dec[i] as f64).powi(2);
}
10.0 * (s / e.max(1e-20)).log10()
}
/// Segmental SNR (1024-sample segments above -50 dBFS, each clamped to [-10, 60] dB).
pub fn seg_snr(refr: &[f32], dec: &[f32]) -> f64 {
let n = refr.len().min(dec.len());
let mut acc = 0f64;
let mut cnt = 0;
for s in (0..n.saturating_sub(1024)).step_by(1024) {
let (mut p, mut e) = (0f64, 0f64);
for i in s..s + 1024 {
p += (refr[i] as f64).powi(2);
e += (refr[i] as f64 - dec[i] as f64).powi(2);
}
if p / 1024.0 < 1e-5 {
continue;
}
acc += (10.0 * (p / e.max(1e-20)).log10()).clamp(-10.0, 60.0);
cnt += 1;
}
if cnt == 0 { 0.0 } else { acc / cnt as f64 }
}
pub fn max_abs_diff(a: &[f32], b: &[f32]) -> f32 {
a.iter().zip(b).fold(0f32, |m, (x, y)| m.max((x - y).abs()))
}
/// Windowed-sinc (Blackman, 401 taps) low-pass at `fc` Hz, zero-phase (delay-compensated).
pub fn lowpass(x: &[f32], fc: f32, fs: f32) -> Vec<f32> {
let taps = 401usize;
let m = (taps - 1) as f32;
let wc = fc / fs;
let h: Vec<f32> = (0..taps)
.map(|i| {
let n = i as f32 - m / 2.0;
let sinc = if n == 0.0 { 2.0 * wc } else { (2.0 * PI * wc * n).sin() / (PI * n) };
let w = 0.42 - 0.5 * (2.0 * PI * i as f32 / m).cos() + 0.08 * (4.0 * PI * i as f32 / m).cos();
sinc * w
})
.collect();
let half = taps / 2;
(0..x.len())
.map(|i| {
let mut acc = 0f32;
for (k, hk) in h.iter().enumerate() {
let j = i as isize + k as isize - half as isize;
if j >= 0 && (j as usize) < x.len() {
acc += hk * x[j as usize];
}
}
acc
})
.collect()
}

175
crates/aac/tests/decoder_oracle.rs Normal file
View File

@ -0,0 +1,175 @@
//! Decoder accuracy against ffmpeg's decoder on streams produced by ffmpeg's AAC encoder (oracle
//! only; skipped when ffmpeg is absent).
mod common;
use common::*;
struct Case {
name: &'static str,
rate: u32,
channels: usize,
args: &'static [&'static str],
}
fn input(rate: u32, channels: usize) -> Vec<Vec<f32>> {
let n = rate as usize * 3 / 2;
(0..channels)
.map(|c| {
let a = chords(rate, n, 0.003 * c as f32);
let b = clicks(rate, n);
let s = speech(rate, n, 5 + c as u64);
a.iter().zip(&b).zip(&s).map(|((x, y), z)| 0.6 * x + 0.3 * y + 0.3 * z * if c % 2 == 0 { 1.0 } else { -0.5 }).collect()
})
.collect()
}
/// Returns (max abs error, samples compared).
fn compare(case: &Case) -> Option<(f32, usize)> {
let ff = ffmpeg()?;
let path = fixtures().join(format!("ffenc_{}.aac", case.name));
ffmpeg_encode(&ff, &input(case.rate, case.channels), case.rate, case.args, &path);
let data = std::fs::read(&path).unwrap();
let (ref_pcm, err) = ffmpeg_decode(&ff, &path, case.channels);
assert!(err.trim().is_empty(), "{err}");
let ours = our_decode(&data);
assert_eq!(ours.len(), case.channels);
// ffmpeg reorders channels to its native layouts; match each of our (AAC element order)
// channels to the closest ffmpeg channel and require a permutation.
let mut worst = 0f32;
let mut taken = vec![false; case.channels];
for c in 0..case.channels {
let (best, err) =
(0..case.channels).filter(|&j| !taken[j]).map(|j| (j, max_abs_diff(&ours[c], &ref_pcm[j]))).min_by(|a, b| a.1.total_cmp(&b.1)).unwrap();
taken[best] = true;
assert_eq!(ours[c].len(), ref_pcm[best].len(), "{}: length", case.name);
worst = worst.max(err);
}
Some((worst, ours[0].len()))
}
#[test]
fn matches_ffmpeg_decoder() {
if ffmpeg().is_none() {
eprintln!("ffmpeg not found; skipping");
return;
}
let cases = [
Case { name: "stereo_44k_128k", rate: 44100, channels: 2, args: &["-b:a", "128k", "-aac_pns", "0"] },
Case { name: "stereo_48k_256k", rate: 48000, channels: 2, args: &["-b:a", "256k", "-aac_pns", "0"] },
Case { name: "mono_48k_64k", rate: 48000, channels: 1, args: &["-b:a", "64k", "-aac_pns", "0"] },
Case {
name: "stereo_44k_96k_is_ms_tns",
rate: 44100,
channels: 2,
args: &["-b:a", "96k", "-aac_pns", "0", "-aac_is", "1", "-aac_ms", "1", "-aac_tns", "1"],
},
Case { name: "stereo_32k_48k_is", rate: 32000, channels: 2, args: &["-b:a", "48k", "-aac_pns", "0", "-aac_is", "1"] },
Case { name: "mono_22k_32k", rate: 22050, channels: 1, args: &["-b:a", "32k", "-aac_pns", "0"] },
Case { name: "mono_16k_24k", rate: 16000, channels: 1, args: &["-b:a", "24k", "-aac_pns", "0"] },
Case { name: "mono_8k_16k", rate: 8000, channels: 1, args: &["-b:a", "16k", "-aac_pns", "0"] },
Case { name: "stereo_96k_256k", rate: 96000, channels: 2, args: &["-b:a", "256k", "-aac_pns", "0"] },
Case { name: "c3_48k", rate: 48000, channels: 3, args: &["-b:a", "192k", "-aac_pns", "0"] },
Case { name: "c51_48k_320k", rate: 48000, channels: 6, args: &["-b:a", "320k", "-aac_pns", "0"] },
Case { name: "c71_48k_448k", rate: 48000, channels: 8, args: &["-b:a", "448k", "-aac_pns", "0"] },
Case { name: "stereo_44k_fast_coder", rate: 44100, channels: 2, args: &["-b:a", "160k", "-aac_coder", "fast", "-aac_pns", "0"] },
];
println!("{:<28} {:>10} {:>12}", "stream", "samples/ch", "max |err|");
let mut fails = Vec::new();
for c in &cases {
let (err, n) = compare(c).unwrap();
println!("{:<28} {:>10} {:>12.3e}", c.name, n, err);
if err >= 1e-3 {
fails.push((c.name, err));
}
}
assert!(fails.is_empty(), "decoder mismatches: {fails:?}");
}
/// PNS output is random by design, and ffmpeg's encoder does not emit it: build noise-substituted
/// access units by hand (mono, and a common-window pair with M/S = correlated noise) and compare
/// per-frame energies and the L/R correlation with ffmpeg's decode.
#[test]
fn pns_matches_ffmpeg() {
use filmcraft_aac::AdtsHeader;
use filmcraft_bitstream::BitWriter;
let Some(ff) = ffmpeg() else { return };
fn ics(bw: &mut BitWriter, with_info: bool, nrg: u32) {
bw.write_bits(100, 8); // global_gain
if with_info {
bw.write_bits(0, 4); // reserved, ONLY_LONG, sine
bw.write_bits(40, 6);
bw.write_bits(0, 1);
}
bw.write_bits(13, 4); // NOISE_HCB for all 40 bands
bw.write_bits(31, 5);
bw.write_bits(9, 5);
bw.write_bits(nrg, 9); // first noise energy (PCM)
for _ in 1..40 {
bw.write_bits(0, 1); // difference 0
}
bw.write_bits(0, 3); // pulse, tns, gain control
}
for channels in [1usize, 2] {
let mut stream = Vec::new();
for f in 0..12u32 {
let mut bw = BitWriter::new();
let nrg = 256 + 20 + (f % 6) * 7;
if channels == 1 {
bw.write_bits(0, 3);
bw.write_bits(0, 4);
ics(&mut bw, true, nrg);
} else {
bw.write_bits(1, 3);
bw.write_bits(0, 4);
bw.write_bits(1, 1); // common_window
bw.write_bits(0, 4);
bw.write_bits(40, 6);
bw.write_bits(0, 1);
bw.write_bits(2, 2); // ms_mask_present = all
ics(&mut bw, false, nrg);
ics(&mut bw, false, nrg - 8);
}
bw.write_bits(7, 3);
let au = bw.finish();
stream.extend_from_slice(&AdtsHeader::write(2, 4, channels as u8, au.len(), 0x7FF));
stream.extend_from_slice(&au);
}
let path = fixtures().join(format!("handmade_pns_{channels}.aac"));
std::fs::write(&path, &stream).unwrap();
let (ref_pcm, err) = ffmpeg_decode(&ff, &path, channels);
assert!(err.trim().is_empty(), "{err}");
let ours = our_decode(&stream);
let mut worst_db = 0f64;
for c in 0..channels {
let e = |v: &[f32]| 10.0 * v.iter().map(|x| (*x as f64).powi(2)).sum::<f64>().log10();
let (eo, ef) = (e(&ours[c]), e(&ref_pcm[c]));
println!("PNS {channels}ch ch{c}: total energy ours {eo:.2} dB, ffmpeg {ef:.2} dB");
assert!((eo - ef).abs() < 0.5);
assert_eq!(ours[c].len(), ref_pcm[c].len());
for s in (1024..ours[c].len() - 1024).step_by(1024) {
let ea: f64 = ours[c][s..s + 1024].iter().map(|v| (*v as f64).powi(2)).sum();
let eb: f64 = ref_pcm[c][s..s + 1024].iter().map(|v| (*v as f64).powi(2)).sum();
worst_db = worst_db.max((10.0 * (ea / eb).log10()).abs());
}
}
let corr = |a: &[f32], b: &[f32]| {
let (mut ab, mut aa, mut bb) = (0f64, 0f64, 0f64);
for (x, y) in a.iter().zip(b) {
ab += *x as f64 * *y as f64;
aa += (*x as f64).powi(2);
bb += (*y as f64).powi(2);
}
ab / (aa * bb).sqrt()
};
let (c_ours, c_ff) = if channels == 2 { (corr(&ours[0], &ours[1]), corr(&ref_pcm[0], &ref_pcm[1])) } else { (0.0, 0.0) };
println!("PNS {channels}ch: per-frame energy max deviation {worst_db:.2} dB; L/R correlation ours {c_ours:.3} ffmpeg {c_ff:.3}");
if channels == 1 {
assert!(worst_db < 1.0);
} else {
// ISO/IEC 14496-3 §4.6.13: noise bands with ms_used share one random vector (correlated
// noise). ffmpeg does not implement this and generates independent noise.
assert!(c_ours > 0.99);
}
}
}

272
crates/aac/tests/encoder_oracle.rs Normal file
View File

@ -0,0 +1,272 @@
//! Encoder quality/bitrate tests using ffmpeg as an external decoding oracle (skipped if absent).
mod common;
use common::*;
use filmcraft_aac::{BitrateMode, EncoderConfig, WindowShape};
struct Case {
name: &'static str,
rate: u32,
signal: fn(u32, usize, usize) -> Vec<Vec<f32>>,
channels: usize,
}
fn mono(v: Vec<f32>, ch: usize) -> Vec<Vec<f32>> {
(0..ch).map(|c| if c == 0 { v.clone() } else { v.iter().map(|x| x * (0.8 - 0.1 * c as f32)).collect() }).collect()
}
fn sig_sweep(r: u32, n: usize, ch: usize) -> Vec<Vec<f32>> {
mono(sweep(r, n, 0.5), ch)
}
fn sig_noise(_r: u32, n: usize, ch: usize) -> Vec<Vec<f32>> {
(0..ch).map(|c| noise(n, 0.3, 7 + c as u64)).collect()
}
fn sig_speech(r: u32, n: usize, ch: usize) -> Vec<Vec<f32>> {
mono(speech(r, n, 3), ch)
}
fn sig_chords(r: u32, n: usize, ch: usize) -> Vec<Vec<f32>> {
(0..ch).map(|c| chords(r, n, 0.002 * c as f32)).collect()
}
fn sig_clicks(r: u32, n: usize, ch: usize) -> Vec<Vec<f32>> {
mono(clicks(r, n), ch)
}
struct Row {
label: String,
target: u32,
actual: f64,
snr: f64,
bw_snr: f64,
bw: u32,
seg: f64,
peak: f32,
shorts: u64,
ours_vs_ff: f32,
}
fn run(case: &Case, bitrate: u32, cfg_mod: impl Fn(&mut EncoderConfig)) -> Option<Row> {
let ff = ffmpeg()?;
let secs = 2.0;
let n = (case.rate as f32 * secs) as usize + 333;
let input = (case.signal)(case.rate, n, case.channels);
let mut cfg = EncoderConfig::cbr(case.rate, case.channels, bitrate);
cfg_mod(&mut cfg);
let enc = encode(cfg.clone(), &input);
let tag = match cfg.mode {
BitrateMode::Cbr(b) => format!("{b}"),
BitrateMode::Vbr(q) => format!("q{q}"),
};
let path = fixtures().join(format!("enc_{}_{}ch_{}_{}.aac", case.name, case.channels, case.rate, tag));
std::fs::write(&path, &enc.adts).unwrap();
let (dec, err) = ffmpeg_decode(&ff, &path, case.channels);
assert!(err.trim().is_empty(), "ffmpeg reported errors for {}: {err}", path.display());
assert_eq!(dec[0].len(), enc.frames * 1024);
assert!(dec[0].len() >= n + 1024, "decoded stream too short");
// ffmpeg reorders channels to its native layouts: map each of our (AAC element order) decoded
// channels to the matching ffmpeg channel.
let ours = our_decode(&enc.adts);
let mut taken = vec![false; case.channels];
let mut snr_sum = 0.0;
let mut seg_sum = 0.0;
let mut bw_sum = 0.0;
let mut peak = 0f32;
let mut ours_vs_ff = 0f32;
let mut counted = 0;
for c in 0..case.channels {
let (j, err) = (0..case.channels).filter(|&j| !taken[j]).map(|j| (j, max_abs_diff(&ours[c], &dec[j]))).min_by(|a, b| a.1.total_cmp(&b.1)).unwrap();
taken[j] = true;
ours_vs_ff = ours_vs_ff.max(err);
let d = &dec[j][1024..1024 + n];
peak = peak.max(d.iter().fold(0f32, |m, v| m.max(v.abs())));
if case.channels >= 6 && c == case.channels - 1 {
continue; // LFE is band-limited to 240 Hz by design
}
snr_sum += snr(&input[c], d);
let fc = enc.bandwidth as f32 * 0.97;
bw_sum += snr(&lowpass(&input[c], fc, case.rate as f32), &lowpass(d, fc, case.rate as f32));
seg_sum += seg_snr(&input[c], d);
counted += 1;
}
let actual = enc.stats.bitrate(case.rate);
Some(Row {
label: format!("{:<7} {}ch {:>5}", case.name, case.channels, case.rate),
target: if matches!(cfg.mode, BitrateMode::Vbr(_)) { 0 } else { bitrate },
actual,
snr: snr_sum / counted as f64,
bw_snr: bw_sum / counted as f64,
bw: enc.bandwidth,
seg: seg_sum / counted as f64,
peak,
shorts: enc.stats.short_blocks,
ours_vs_ff,
})
}
fn print(rows: &[Row]) {
println!(
"{:<22} {:>8} {:>8} {:>7} {:>6} {:>7} {:>7} {:>7} {:>6} {:>6} {:>9}",
"signal", "target", "actual", "dev%", "bw", "SNR", "bwSNR", "segSNR", "peak", "short", "us-vs-ff"
);
for r in rows {
let (target, dev) = if r.target == 0 {
("vbr".to_string(), "-".to_string())
} else {
(r.target.to_string(), format!("{:+.2}", (r.actual / r.target as f64 - 1.0) * 100.0))
};
println!(
"{:<22} {:>8} {:>8.0} {:>7} {:>6} {:>7.2} {:>7.2} {:>7.2} {:>6.3} {:>6} {:>9.2e}",
r.label, target, r.actual, dev, r.bw, r.snr, r.bw_snr, r.seg, r.peak, r.shorts, r.ours_vs_ff
);
}
}
fn check(r: &Row, min_seg: f64) {
let dev = (r.actual / r.target as f64 - 1.0).abs();
assert!(dev < 0.05, "{}: bitrate {} vs target {}", r.label, r.actual, r.target);
assert!(r.peak < 1.0, "{}: clipping (peak {})", r.label, r.peak);
assert!(r.seg > min_seg, "{}: segmental SNR {} too low", r.label, r.seg);
assert!(r.ours_vs_ff < 1e-3, "{}: our decoder differs from ffmpeg by {}", r.label, r.ours_vs_ff);
}
#[test]
fn stereo_matrix_44k_48k() {
if ffmpeg().is_none() {
eprintln!("ffmpeg not found; skipping");
return;
}
let cases = [
Case { name: "sweep", rate: 44100, signal: sig_sweep, channels: 2 },
Case { name: "noise", rate: 44100, signal: sig_noise, channels: 2 },
Case { name: "speech", rate: 48000, signal: sig_speech, channels: 2 },
Case { name: "chords", rate: 44100, signal: sig_chords, channels: 2 },
Case { name: "clicks", rate: 48000, signal: sig_clicks, channels: 2 },
];
let mut rows = Vec::new();
for c in &cases {
for br in [64_000, 128_000, 192_000, 320_000] {
let r = run(c, br, |_| {}).unwrap();
rows.push(r);
}
}
print(&rows);
for r in &rows {
check(
r,
if r.label.starts_with("noise") {
0.5
} else if r.target >= 128_000 {
8.0
} else {
3.0
},
);
}
}
#[test]
fn mono_and_surround() {
if ffmpeg().is_none() {
eprintln!("ffmpeg not found; skipping");
return;
}
let mut rows = Vec::new();
for (c, br) in [
(Case { name: "speech", rate: 44100, signal: sig_speech, channels: 1 }, 64_000),
(Case { name: "chords", rate: 48000, signal: sig_chords, channels: 1 }, 96_000),
(Case { name: "clicks", rate: 44100, signal: sig_clicks, channels: 1 }, 128_000),
(Case { name: "chords", rate: 48000, signal: sig_chords, channels: 6 }, 320_000),
(Case { name: "noise", rate: 48000, signal: sig_noise, channels: 6 }, 384_000),
] {
rows.push(run(&c, br, |_| {}).unwrap());
}
print(&rows);
for r in &rows {
check(r, 3.0);
}
}
#[test]
fn kbd_windows_no_tns_no_ms() {
if ffmpeg().is_none() {
return;
}
let c = Case { name: "kbd", rate: 44100, signal: sig_clicks, channels: 2 };
let r = run(&c, 128_000, |cfg| {
cfg.window_shape = WindowShape::Kbd;
cfg.tns = false;
cfg.ms_stereo = false;
})
.unwrap();
print(std::slice::from_ref(&r));
check(&r, 5.0);
}
#[test]
fn sample_rates_8k_to_96k() {
if ffmpeg().is_none() {
return;
}
let mut rows = Vec::new();
for (rate, br) in [
(8000u32, 24_000u32),
(11025, 32_000),
(12000, 32_000),
(16000, 48_000),
(22050, 64_000),
(24000, 64_000),
(32000, 96_000),
(64000, 192_000),
(88200, 256_000),
(96000, 256_000),
] {
let c = Case { name: "chords", rate, signal: sig_chords, channels: 2 };
rows.push(run(&c, br, |_| {}).unwrap());
}
print(&rows);
for r in &rows {
check(r, 5.0);
}
}
#[test]
fn channel_layouts_3_to_8() {
if ffmpeg().is_none() {
return;
}
let mut rows = Vec::new();
for (ch, br) in [(3usize, 192_000u32), (4, 256_000), (5, 320_000), (7, 384_000), (8, 448_000)] {
let c = Case { name: "chords", rate: 48000, signal: sig_chords, channels: ch };
rows.push(run(&c, br, |_| {}).unwrap());
}
print(&rows);
for r in &rows {
check(r, 5.0);
}
}
#[test]
fn vbr_quality_scale() {
if ffmpeg().is_none() {
return;
}
let mut rows = Vec::new();
for sig in [sig_chords as fn(u32, usize, usize) -> Vec<Vec<f32>>, sig_speech] {
for q in [1.0f32, 2.0, 3.0, 4.0, 5.0] {
let c = Case { name: "vbr", rate: 44100, signal: sig, channels: 2 };
let mut r = run(&c, 1, |cfg| cfg.mode = BitrateMode::Vbr(q)).unwrap();
r.label = format!("vbr q{q} 2ch 44100");
rows.push(r);
}
}
print(&rows);
for pair in rows.chunks(5) {
for w in pair.windows(2) {
assert!(w[1].actual > w[0].actual, "VBR bitrate must grow with quality");
assert!(w[1].seg > w[0].seg - 0.5, "VBR quality must grow with quality");
}
for r in pair {
assert!(r.peak < 1.0 && r.ours_vs_ff < 1e-3);
}
}
}

157
crates/aac/tests/robustness.rs Normal file
View File

@ -0,0 +1,157 @@
//! Fuzz-style robustness: mutated access units, configs and ADTS streams must never panic, and the
//! encoder must accept hostile input (NaN, ±inf, > full scale, silence, odd chunk sizes).
mod common;
use common::*;
use filmcraft_aac::{AudioSpecificConfig, Decoder, Encoder, EncoderConfig, split_adts};
fn corpus() -> Vec<(Vec<u8>, Vec<Vec<u8>>)> {
let mut out = Vec::new();
for (ch, rate, br) in [(1usize, 44100u32, 96_000u32), (2, 48000, 128_000), (6, 48000, 256_000), (7, 48000, 256_000)] {
let n = rate as usize / 2;
let sig: Vec<Vec<f32>> = (0..ch).map(|c| if c % 2 == 0 { clicks(rate, n) } else { chords(rate, n, 0.01) }).collect();
let mut enc = Encoder::new(EncoderConfig::cbr(rate, ch, br)).unwrap();
let refs: Vec<&[f32]> = sig.iter().map(|v| v.as_slice()).collect();
let mut aus = enc.encode(&refs);
aus.extend(enc.flush());
out.push((enc.audio_specific_config(), aus));
}
out
}
#[test]
fn mutated_access_units_never_panic() {
let mut rng = Rng(0x9E37_79B9_7F4A_7C15);
let mut rand = move |n: usize| ((rng.next() * 0.5 + 0.5) * n as f32) as usize % n.max(1);
let mut decoded = 0usize;
let mut errors = 0usize;
for (asc, aus) in corpus() {
let mut dec = Decoder::new(&asc).unwrap();
for round in 0..3000 {
let mut au = aus[rand(aus.len())].clone();
match round % 6 {
0 => {
for _ in 0..1 + rand(8) {
let i = rand(au.len());
au[i] ^= 1 << rand(8);
}
}
1 => {
let i = rand(au.len());
au[i] = rand(256) as u8;
}
2 => au.truncate(rand(au.len() + 1)),
3 => au.extend((0..rand(64)).map(|_| rand(256) as u8)),
4 => au = (0..rand(2000)).map(|_| rand(256) as u8).collect(),
_ => {
// splice two access units
let other = &aus[rand(aus.len())];
let cut = rand(au.len());
au.truncate(cut);
au.extend_from_slice(&other[rand(other.len())..]);
}
}
match dec.decode(&au) {
Ok(pcm) => {
decoded += 1;
assert!(pcm.iter().all(|c| c.len() == 1024));
}
Err(_) => errors += 1,
}
}
}
println!("mutated AUs: {decoded} decoded, {errors} rejected, 0 panics");
}
#[test]
fn random_configs_and_adts_never_panic() {
let mut rng = Rng(12345);
let mut byte = move || ((rng.next() * 0.5 + 0.5) * 256.0) as u8;
for _ in 0..20000 {
let len = (byte() % 12) as usize;
let asc: Vec<u8> = (0..len).map(|_| byte()).collect();
if let Ok(cfg) = AudioSpecificConfig::parse(&asc) {
let _ = cfg.channels();
if let Ok(mut d) = Decoder::from_config(cfg) {
let au: Vec<u8> = (0..(byte() as usize * 3)).map(|_| byte()).collect();
let _ = d.decode(&au);
}
}
}
for _ in 0..2000 {
let mut s = vec![0xFF, 0xF1];
s.extend((0..(byte() as usize * 8)).map(|_| byte()));
if let Ok(frames) = split_adts(&s)
&& let Ok(mut d) = Decoder::from_adts(&frames[0].0)
{
for (_, au) in frames {
let _ = d.decode(au);
}
}
}
}
#[test]
fn encoder_handles_hostile_input() {
for ch in [1usize, 2, 6] {
let mut enc = Encoder::new(EncoderConfig::cbr(44100, ch, 64_000 * ch as u32)).unwrap();
let n = 5000;
let evil: Vec<f32> = (0..n)
.map(|i| match i % 7 {
0 => f32::NAN,
1 => f32::INFINITY,
2 => -f32::INFINITY,
3 => 40.0,
4 => -1e30,
_ => ((i as f32) * 0.37).sin(),
})
.collect();
let chans: Vec<&[f32]> = (0..ch).map(|_| evil.as_slice()).collect();
let mut aus = Vec::new();
for chunk in [0usize, 1, 7, 1023, 1024, 1025, 3000] {
let c: Vec<&[f32]> = chans.iter().map(|s| &s[..chunk.min(s.len())]).collect();
aus.extend(enc.encode(&c));
}
// fewer channels than configured: treated as silence
aus.extend(enc.encode(&[&evil[..100]]));
aus.extend(enc.flush());
let mut dec = Decoder::new(&enc.audio_specific_config()).unwrap();
for au in &aus {
let pcm = dec.decode(au).unwrap();
assert!(pcm.iter().flatten().all(|v| v.is_finite()));
}
let expected = (1 + 7 + 1023 + 1024 + 1025 + 3000 + 100usize).div_ceil(1024) + 1;
assert_eq!(aus.len(), expected);
}
// empty stream: flush yields nothing
let mut enc = Encoder::new(EncoderConfig::cbr(48000, 2, 128_000)).unwrap();
assert!(enc.flush().is_empty());
// bad configs are rejected
assert!(Encoder::new(EncoderConfig::cbr(44000, 2, 128_000)).is_err());
assert!(Encoder::new(EncoderConfig::cbr(44100, 9, 128_000)).is_err());
assert!(Encoder::new(EncoderConfig::cbr(8000, 2, 2_000_000)).is_err());
}
#[test]
fn roundtrip_and_delay() {
// Own encoder → own decoder: priming is exactly 1024 samples and the stream covers the input.
let rate = 48000;
let n = 10_000;
let sig = chords(rate, n, 0.0);
let mut enc = Encoder::new(EncoderConfig::cbr(rate, 1, 128_000)).unwrap();
assert_eq!(enc.priming_samples(), 1024);
let mut aus = enc.encode(&[&sig]);
aus.extend(enc.flush());
let mut dec = Decoder::new(&enc.audio_specific_config()).unwrap();
let mut out = Vec::new();
for au in &aus {
out.extend(dec.decode(au).unwrap().remove(0));
}
assert!(out.len() >= n + 1024);
let s = snr(&sig, &out[1024..1024 + n]);
assert!(s > 15.0, "roundtrip SNR {s}");
// no energy in the priming region beyond pre-echo of the first attack
let e: f32 = out[..512].iter().map(|v| v * v).sum();
assert!(e < 1e-3);
}

128
crates/aac/tools/derive.py Normal file
View File

@ -0,0 +1,128 @@
"""Black-box derivation of the AAC Huffman codebooks by probing the ffmpeg decoder (oracle only)."""
import sys, json, math
from concurrent.futures import ThreadPoolExecutor
from probe import frame, silent, run, fit
def unit(sf):
return 2.0 ** (0.25 * (sf - 100)) / (32768 * 1024)
PARAMS = { # cb: (dim, signed, lav)
1: (4, True, 1), 2: (4, True, 1), 3: (4, False, 2), 4: (4, False, 2),
5: (2, True, 4), 6: (2, True, 4), 7: (2, False, 7), 8: (2, False, 7),
9: (2, False, 12), 10: (2, False, 12), 11: (2, False, 16),
}
def analyse(y, err, sfs):
if err or len(y) < 2048:
return None
x, res, en = fit(y[:2048])
if en == 0:
return [0] * 12 if res == 0 else None
if res > 1e-6 * en:
return None
out = []
for i, v in enumerate(x):
u = unit(sfs[i // 4])
m = abs(v) / u
q = m ** 0.75
r = round(q)
if abs(q - r) > 0.02 * max(1, r):
return None
out.append(int(r) * (1 if v >= 0 else -1))
return out
G = 140
def query_spec(cb, S, k, e):
dim, signed, lav = PARAMS[cb]
ncopies = 12 // dim
bits = ''
for c in range(ncopies):
bits += S + ('0' if c % 2 == 0 else '1') * k + '00000' * e
y, err = run([frame(cb, '000', bits, G=G), silent(), silent()])
vals = analyse(y, err, [G, G, G])
if vals is None:
return None
vecs = [vals[c * dim:(c + 1) * dim] for c in range(ncopies)]
v = vecs[0]
for c in range(ncopies):
exp = v if (signed or c % 2 == 0) else [-a for a in v]
if vecs[c] != exp:
return None
if signed:
if any(abs(a) > lav for a in v):
return None
mod = 2 * lav + 1
idx = 0
for a in v:
idx = idx * mod + (a + lav)
return idx
if any(a < 0 for a in v):
return None
if sum(1 for a in v if a != 0) != k:
return None
if cb == 11:
if sum(1 for a in v if a == 16) != e:
return None
elif e:
return None
if any(a > lav for a in v):
return None
mod = lav + 1
idx = 0
for a in v:
idx = idx * mod + a
return idx
def query_sf(S):
Gs = 128
# spectral: cb1 (-1,0,0,0) = '10001' in each band
y, err = run([frame(1, S + '0' + S, '10001' * 3, G=Gs), silent(), silent()])
for d in range(-60, 61):
sfs = [Gs + d, Gs + d, Gs + 2 * d]
if not all(0 <= s <= 255 for s in sfs):
continue
vals = analyse(y, err, sfs)
if vals == [-1, 0, 0, 0] * 3:
return d + 60
return None
def hyps(cb):
dim, signed, lav = PARAMS[cb]
if signed:
return [(0, 0)]
hs = [(k, 0) for k in range(dim + 1)]
if cb == 11:
hs += [(1, 1), (2, 1), (2, 2)]
return hs
def derive(cb, maxlen=20):
leaves = {}
frontier = ['0', '1']
with ThreadPoolExecutor(14) as ex:
while frontier:
def q(S):
if cb == 0:
return S, query_sf(S)
for (k, e) in hyps(cb):
r = query_spec(cb, S, k, e)
if r is not None:
return S, r
return S, None
nxt = []
for S, r in ex.map(q, frontier):
if r is not None:
leaves[S] = r
elif len(S) < maxlen:
nxt += [S + '0', S + '1']
print(f'cb{cb}: depth {len(frontier[0])} frontier {len(frontier)} leaves {len(leaves)}', file=sys.stderr)
frontier = nxt
return leaves
if __name__ == '__main__':
cb = int(sys.argv[1])
leaves = derive(cb)
kraft = sum(2.0 ** -len(s) for s in leaves)
idxs = sorted(leaves.values())
print(f'cb{cb}: {len(leaves)} leaves, kraft {kraft}, unique idx {len(set(idxs))}, range {idxs[0]}..{idxs[-1]}', file=sys.stderr)
json.dump({s: i for s, i in leaves.items()}, open(f'cb{cb}.json', 'w'))

29
crates/aac/tools/gen.py Normal file
View File

@ -0,0 +1,29 @@
import json
out = ['//! AAC Huffman codebooks (ISO/IEC 14496-3 §4.A.1, spectrum codebooks 1–11 and the scalefactor codebook).',
'//!',
'//! Each table is indexed by the codebook index defined in the standard and stores `(codeword, length)`.',
'//! Generated data: derived by black-box probing of an external decoder used purely as a test oracle',
'//! (every table is a complete prefix code — Kraft sum exactly 1 — covering every index once; see',
'//! `tests::codebooks_are_complete_prefix_codes`).',
'',
'#![allow(clippy::unreadable_literal)]',
'']
for cb in [0] + list(range(1, 12)):
d = json.load(open(f'cb{cb}.json'))
n = len(d)
arr = [None] * n
for s, i in d.items():
arr[i] = (int(s, 2), len(s))
name = 'SCALEFACTOR' if cb == 0 else f'SPECTRUM_{cb}'
out.append(f'pub(crate) static {name}: [(u32, u8); {n}] = [')
line = ' '
for (c, l) in arr:
item = f' (0x{c:x}, {l}),'
if len(line) + len(item) > 100:
out.append(line)
line = ' '
line += item
out.append(line)
out.append('];')
out.append('')
open('huffman_tables.rs', 'w').write('\n'.join(out))

78
crates/aac/tools/probe.py Normal file
View File

@ -0,0 +1,78 @@
"""Codebook recovery by black-box probing of an external AAC decoder (ffmpeg binary, oracle only).
Builds single-frame ADTS streams whose band data repeats a candidate bit string, decodes them with
`ffmpeg -f f32le`, least-squares fits the output to the IMDCT basis functions of the probed lines and
accepts the candidate as a codeword when the fit is exact, the copies agree and the implied sign/escape
bits are consistent. A breadth-first walk over bit strings then recovers each complete prefix code.
Usage: python3 derive.py <1..11 | 0 for scalefactors> (writes cbN.json)
python3 gen.py (writes huffman_tables.rs)
Needs only the Python standard library and ffmpeg at /opt/homebrew/bin.
"""
import subprocess, struct, math, sys, os, tempfile
class BW:
def __init__(self): self.bits=[]
def w(self, v, n):
for i in range(n-1,-1,-1): self.bits.append((v>>i)&1)
def s(self, st):
for c in st: self.bits.append(int(c))
def bytes(self):
b=self.bits+[0]*((-len(self.bits))%8)
return bytes(int(''.join(map(str,b[i:i+8])),2) for i in range(0,len(b),8))
def adts(payload, sfi=3, ch=1):
L=len(payload)+7
w=BW(); w.w(0xFFF,12); w.w(0,1); w.w(0,2); w.w(1,1); w.w(1,2); w.w(sfi,4); w.w(0,1); w.w(ch,3)
w.w(0,4); w.w(L,13); w.w(0x7FF,11); w.w(0,2)
return w.bytes()+payload
def frame(cb, sfbits, specbits, nbands=3, G=100):
w=BW()
w.w(0,3); w.w(0,4); w.w(G,8)
w.w(0,1); w.w(0,2); w.w(0,1); w.w(nbands,6); w.w(0,1)
if nbands:
w.w(cb,4); w.w(nbands,5)
w.s(sfbits)
w.w(0,1); w.w(0,1); w.w(0,1)
w.s(specbits)
w.w(7,3)
return adts(w.bytes())
def silent():
return frame(0,'','',nbands=0)
def run(frames):
fd,path=tempfile.mkstemp(suffix='.aac'); os.write(fd,b''.join(frames)); os.close(fd)
p=subprocess.run(['/opt/homebrew/bin/ffmpeg','-v','error','-f','aac','-i',path,'-f','f32le','-'],capture_output=True)
os.unlink(path)
n=len(p.stdout)//4
return list(struct.unpack('<%df'%n,p.stdout)), p.stderr.decode()
N=2048
n0=(N/2+1)/2
def basis(k):
return [math.sin(math.pi*(n+0.5)/N)*math.cos(2*math.pi/N*(n+n0)*(k+0.5)) for n in range(N)]
B=[basis(k) for k in range(12)]
def solve(A,b):
n=len(A); M=[row[:]+[b[i]] for i,row in enumerate(A)]
for c in range(n):
p=max(range(c,n),key=lambda r:abs(M[r][c])); M[c],M[p]=M[p],M[c]
for r in range(n):
if r!=c:
f=M[r][c]/M[c][c]
for j in range(c,n+1): M[r][j]-=f*M[c][j]
return [M[i][n]/M[i][i] for i in range(n)]
G_=[[sum(B[i][n]*B[j][n] for n in range(N)) for j in range(12)] for i in range(12)]
def fit(y):
rhs=[sum(B[i][n]*y[n] for n in range(N)) for i in range(12)]
x=solve(G_,rhs)
res=sum((y[n]-sum(x[k]*B[k][n] for k in range(12)))**2 for n in range(N))
en=sum(v*v for v in y)
return x,res,en
if __name__=='__main__':
spec=sys.argv[2]; cb=int(sys.argv[1])
y,err=run([frame(cb,'000',spec),silent(),silent()])
print(len(y),repr(err))
x,res,en=fit(y[:2048])
print(["%.4g"%v for v in x],res,en)