Pool trim: free only the float shelf, and only after a standalone export

pool::trim() from Pipeline::drop flushed the whole process-wide pool after every
render-preview segment and proxy, and threw away playback's U8/U16 planes too.
Now Pipeline::drop calls pool::trim_f32() only when !settings.part_of_batch.
PoolStats gains f32_idle_bytes so the tests can check the float shelf alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

crates/export/src/pipeline.rs

@@ -68,15 +68,21 @@ pub(crate) struct Pipeline {
     overlay: Option<Image>,
     start_tc: i64,
     drop_frame: bool,
+    /// Free the pool's float images when the export ends: a standalone export, not one part of a
+    /// batch (`ExportSettings::part_of_batch`).
+    trim_pool: bool,
 }
 
 impl Drop for Pipeline {
     /// The export is over, however it ended (done, failed, cancelled, dropped half way). The float
-    /// images its frames left on the shelves of `filmcraft_frame::pool` are the size of its frames,
-    /// which nothing else asks for, so free them instead of holding up to 320 MiB idle. A job still
-    /// rendering at that moment allocates a few images again; the pool is only a cache.
+    /// images its frames left on the `filmcraft_frame::pool` float shelf are the size of its
+    /// frames, which nothing else asks for, so free them instead of holding up to 320 MiB idle. A
+    /// job still rendering at that moment allocates a few images again; the pool is only a cache.
+    /// Parts of a larger job (render-preview segments, proxies) keep them for the next part.
     fn drop(&mut self) {
-        filmcraft_frame::pool::trim();
+        if self.trim_pool {
+            filmcraft_frame::pool::trim_f32();
+        }
     }
 }
 
@@ -110,6 +116,7 @@ impl Pipeline {
             seq_rate: q.settings.frame_rate,
             start_tc: q.start_timecode,
             drop_frame: q.settings.drop_frame,
+            trim_pool: !settings.part_of_batch,
             project: project.clone(),
             seq,
             rate: r.rate,

crates/export/tests/pool_trim.rs

@@ -1,5 +1,5 @@
-//! An export gives back what it left on the frame pool (`filmcraft_frame::pool`) when it ends,
-//! however it ends. One test in this file on purpose: the pool is shared by every test of a binary,
+//! A standalone export gives back the float images it left on the frame pool
+//! (`filmcraft_frame::pool`) when it ends, however it ends; one part of a batch keeps them. One test in this file on purpose: the pool is shared by every test of a binary,
 //! and its numbers mean something only while nothing else renders.
 
 use std::sync::Arc;
@@ -40,11 +40,11 @@ fn project() -> (Arc<Project>, ItemId, SourceMap) {
 }
 
 fn idle_mb() -> f64 {
-    pool::stats().idle_bytes as f64 / 1e6
+    pool::stats().f32_idle_bytes as f64 / 1e6
 }
 
 #[test]
-fn the_pool_is_empty_when_an_export_ends_however_it_ended() {
+fn the_float_shelf_is_empty_when_a_standalone_export_ends_however_it_ended() {
     let (p, seq, sources) = project();
     let dir = std::env::temp_dir().join(format!("fc-pool-trim-{}", std::process::id()));
     std::fs::create_dir_all(&dir).unwrap();
@@ -54,9 +54,9 @@ fn the_pool_is_empty_when_an_export_ends_however_it_ended() {
     // themselves do too: the checks below fail without the trim)
     let leave_something = || {
         pool::recycle_f32(vec![0.0; 100_000]);
-        assert!(pool::stats().idle_bytes > 0);
+        assert!(pool::stats().f32_idle_bytes > 0);
     };
-    let check = |what: &str| assert_eq!(pool::stats().idle_bytes, 0, "after {what}: {:.1} MB idle", idle_mb());
+    let check = |what: &str| assert_eq!(pool::stats().f32_idle_bytes, 0, "after {what}: {:.1} MB idle", idle_mb());
 
     // images, which run in `export` itself
     leave_something();
@@ -91,5 +91,12 @@ fn the_pool_is_empty_when_an_export_ends_however_it_ended() {
     drop(ex);
     check("an export dropped half way");
 
+    // one part of a batch (a render-preview segment, a proxy): the next part reuses the images
+    leave_something();
+    let part = ExportSettings { part_of_batch: true, ..settings(Format::ProRes, "e.mov") };
+    export(&p, seq, &part, &sources, &Progress::default()).unwrap();
+    assert!(pool::stats().f32_idle_bytes > 0, "a part of a batch keeps the float images");
+    pool::trim_f32();
+
     let _ = std::fs::remove_dir_all(&dir);
 }

crates/frame/src/pool.rs

@@ -11,7 +11,7 @@
 //! frame they cost a zero-fill and fresh page faults every time.
 //!
 //! The shelves are sized for the busiest work, so they are a lot to keep once it is over: whoever
-//! finishes such work (an export) calls [`trim`].
+//! finishes such work (a standalone export) calls [`trim_f32`].
 
 use std::sync::atomic::{AtomicU64, Ordering};
 use std::sync::{Arc, Mutex, PoisonError};
@@ -141,23 +141,22 @@ pub fn recycle(frame: Arc<VideoFrame>) {
     }
 }
 
-/// Free every idle buffer. An export leaves the shelves full (up to [`F32_MAX_BYTES`] of float
-/// images, each the size of that export's frames, and [`MAX_BYTES`] of each plane type) and
-/// nothing else asks for those sizes again, so they would sit idle until the next export. What
-/// is still in use is not touched, and the next job starts cold, for the price of its first
-/// allocations.
-pub fn trim() {
-    // taken under the locks, freed after them: giving back hundreds of megabytes takes a moment
+/// Free the idle float images. A standalone export leaves that shelf full (up to [`F32_MAX_BYTES`]
+/// of images, each the size of that export's frames) and nothing else asks for those sizes again,
+/// so they would sit idle until the next export. Images still in use are not touched. The 8- and
+/// 16-bit plane shelves are left alone: playback recycles decoded frames through them all the time.
+pub fn trim_f32() {
+    // taken under the lock, freed after it: giving back hundreds of megabytes takes a moment
     let floats = std::mem::take(&mut *F32.lock().unwrap_or_else(PoisonError::into_inner));
-    let planes8 = std::mem::replace(&mut *U8.lock().unwrap_or_else(PoisonError::into_inner), Shelf::new());
-    let planes16 = std::mem::replace(&mut *U16.lock().unwrap_or_else(PoisonError::into_inner), Shelf::new());
-    drop((floats, planes8, planes16));
+    drop(floats);
 }
 
 /// Idle buffers held and buffers handed out again so far (`perf.stats`).
 #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
 pub struct PoolStats {
     pub idle_bytes: usize,
+    /// The part of `idle_bytes` held as float images ([`recycle_f32`]).
+    pub f32_idle_bytes: usize,
     pub reused: u64,
 }
 
@@ -166,7 +165,7 @@ pub fn stats() -> PoolStats {
     let a = U8.lock().unwrap_or_else(PoisonError::into_inner).bytes;
     let b = U16.lock().unwrap_or_else(PoisonError::into_inner).bytes;
     let c: usize = F32.lock().unwrap_or_else(PoisonError::into_inner).iter().map(|b| b.len().saturating_mul(4)).sum();
-    PoolStats { idle_bytes: idle(idle(a, b), c), reused: REUSED.load(Ordering::Relaxed) }
+    PoolStats { idle_bytes: idle(idle(a, b), c), f32_idle_bytes: c, reused: REUSED.load(Ordering::Relaxed) }
 }
 
 #[cfg(test)]

crates/frame/tests/pool_trim.rs

@@ -1,6 +1,7 @@
-//! `pool::trim` frees every idle buffer of the process-wide pool. This file holds one test on
-//! purpose: the pool is shared by every test of a binary, and another test recycling or taking
-//! buffers while this one counts them would make the counts meaningless.
+//! `pool::trim_f32` frees the idle float images of the process-wide pool and leaves the plane
+//! shelves alone. This file holds one test on purpose: the pool is shared by every test of a binary,
+//! and another test recycling or taking buffers while this one counts them would make the counts
+//! meaningless.
 
 use std::sync::Arc;
 
@@ -23,31 +24,32 @@ fn recycle_planes() {
 }
 
 #[test]
-fn trim_frees_every_idle_buffer_and_the_pool_keeps_working() {
+fn trim_f32_frees_the_float_images_keeps_the_planes_and_the_pool_keeps_working() {
     assert_eq!(pool::stats().idle_bytes, 0, "nothing has been recycled yet");
 
     // one idle buffer on each of the three shelves
     pool::recycle_f32(vec![f32::NAN; FLOATS]);
     recycle_planes();
-    let held = FLOATS * 4 + BYTES + WORDS * 2;
-    assert!(pool::stats().idle_bytes >= held, "idle {} of {held}", pool::stats().idle_bytes);
+    let planes = BYTES + WORDS * 2;
+    assert_eq!(pool::stats().f32_idle_bytes, FLOATS * 4);
+    assert!(pool::stats().idle_bytes >= FLOATS * 4 + planes, "idle {}", pool::stats().idle_bytes);
 
-    pool::trim();
-    assert_eq!(pool::stats().idle_bytes, 0, "every shelf is empty");
+    pool::trim_f32();
+    assert_eq!(pool::stats().f32_idle_bytes, 0, "the float shelf is empty");
+    assert!(pool::stats().idle_bytes >= planes, "the plane shelves are untouched");
 
-    // nothing comes back from the shelves: the float image is a fresh (zeroed) one and no take
-    // counts as a reuse
+    // the float image is a fresh (zeroed) one and doesn't count as a reuse; the planes come back
     let reused = pool::stats().reused;
     let image = pool::take_f32_overwritten(FLOATS);
     assert!(image.iter().all(|v| *v == 0.0), "the recycled image was freed");
-    drop((pool::take_u8(BYTES), pool::take_u16(WORDS)));
     assert_eq!(pool::stats().reused, reused);
+    drop((pool::take_u8(BYTES), pool::take_u16(WORDS)));
+    assert!(pool::stats().reused > reused, "the planes were kept");
 
-    // the pool works after a trim, and trimming twice (or an empty pool) is fine
+    // the pool works after a trim, and trimming twice (or an empty shelf) is fine
     pool::recycle_f32(image);
-    recycle_planes();
-    assert!(pool::stats().idle_bytes >= held);
-    pool::trim();
-    pool::trim();
-    assert_eq!(pool::stats().idle_bytes, 0);
+    assert_eq!(pool::stats().f32_idle_bytes, FLOATS * 4);
+    pool::trim_f32();
+    pool::trim_f32();
+    assert_eq!(pool::stats().f32_idle_bytes, 0);
 }

docs/architecture.md

@@ -306,8 +306,9 @@ file ──► codecs (MP4/MOV, MKV, audio)        demux + decode, GOP-aware see
   not zero (`VideoFrame::alpha_region`, `to_linear_f32_region`, `blend::composite_at`), the rest
   being transparent anyway; and working images are recycled through `filmcraft_frame::pool`
   (`take_f32_overwritten` / `recycle_f32`: the buffer comes back with its old contents, so there is
-  no zero-fill) by the export pipeline once a frame is converted to 8 bits; the pipeline trims the
-  pool (`pool::trim`) when it goes away, so a finished export leaves no idle images behind. A clip
+  no zero-fill) by the export pipeline once a frame is converted to 8 bits; a standalone export's
+  pipeline frees the idle float images (`pool::trim_f32`) when it goes away, so a finished export
+  leaves none behind (parts of a batch, such as render-preview segments, keep them for the next part). A clip
   with effects, opacity masks, frame blending, colour management, or a picture that is not placed
   one to one on the output takes the general path.
 - **Frame scheduling.** `crates/ui-egui/src/frames.rs` runs a small pool of worker threads with

docs/performance.md

@@ -99,9 +99,10 @@ picture (4.5 ms in the synthetic case) and the conversion to 8 bits (2 ms).
 
 The recycled images stay on the shelves of `filmcraft_frame::pool` (up to 320 MiB of them, and
 192 MiB of each plane type), and nothing else asks for images of an export's size, so a finished
-export left them idle. The export pipeline now trims the pool when it goes away, however the
-export ended (done, failed, cancelled, dropped half way): `perf.stats` reads `decode.planePoolMB`
-0 afterwards. A process that exported a synthetic 3 s 1080p24 matte as PNG frames three times
+export left them idle. A standalone export's pipeline now frees the idle float images when it goes
+away, however the export ended (done, failed, cancelled, dropped half way). The plane shelves are
+kept (playback recycles decoded frames through them), and so are the float images after one part
+of a batch (a render-preview segment, a proxy), which the next part reuses. A process that exported a synthetic 3 s 1080p24 matte as PNG frames three times
 (Apple M1, `footprint`, 10 s after the last export) rested at **575 MB** with the pool holding
 332 MB idle and at **258 MB** with the trim. The memory does not come back at once: macOS returns
 freed pages over a few seconds (892 MB right after the export in both cases, 726 MB after 2 s).