D
Delophy Martin
Guest
Cover photo: Jakob Owens on Unsplash. Illustrative photography; not a screenshot of Timeline Studio.
Long-video export is a pipeline problem. Before changing encoder settings, it is worth checking how much work happens around the encoder: whole-track audio decoding, intermediate frame rasterization, and serial waits between stages.
This article walks through an optimization of an open-source browser video editor, with a controlled before/after benchmark and the limits of each fast path.
We recently optimized these paths in Timeline Studio, an open-source browser video editor. In a controlled comparison against the code before the first optimization round, average export time fell from 192.0 seconds to 131.9 seconds—a 31.3% reduction, saving about 60 seconds.
Source code on GitHub · Try the editor
Both versions ran on the same machine with the same approximately eight-minute synthetic stress-test video. The input was H.264 at 1440 × 1080 and approximately 29.88 fps, with stereo AAC audio at 44.1 kHz.
Output settings were identical:
Timing started immediately before embedded-audio preparation and ended when export completed. Import and timeline-thumbnail preparation were excluded.
We ran old → new → new → old, starting a fresh browser process for each run.
Before optimization: 203.7 s and 180.4 s; mean total 192.0 s.
Current implementation: 141.6 s and 122.3 s; mean total 131.9 s.
Audio preparation averaged 9.3 s → 0.9 s. The video export stage averaged 182.7 s → 131.0 s.
These are two runs per version, with visible runtime variation. The input is synthetic, so the 31.3% result should not be generalized to every camera recording, device, or effect stack. We also checked decoding and compatibility with a native Sony ZV-1 4K camera sample, but that sample is not the source of the eight-minute benchmark result.
The old preparation path used FFmpeg to decode embedded audio into WAV before subsequent processing. For long inputs, this adds whole-track decoding, PCM allocation, and intermediate-data handling.
For eligible AAC tracks, the new path extracts encoded packets and remuxes them instead. Final output audio is still encoded according to export settings; this change removes unnecessary work in the preparation stage.
Timestamp handling matters. AAC can contain preroll and negative timestamps. Simply copying packets while discarding that information can shift audio relative to the picture. The implementation preserves and compensates for the source offset and distinguishes storage duration from logical segment duration.
Other formats retain the fallback path. In this benchmark, mean audio preparation dropped from 9.3 seconds to 0.9 seconds.
At 30 fps, an eight-minute output contains approximately 14,400 frames. Even a small repeated operation is worth examining at that scale.
The new eligible path sends decoded VideoFrame objects directly to the shared compositor, avoiding an additional intermediate Canvas rasterization step.
Eligibility is deliberately narrow: original-size, unrotated, square-pixel frames with no relevant transformations or effects that require the existing rasterization path. Scaling, grading, animation, masks, and other affected cases keep their compatible Canvas path.
This restriction came from validation: different rasterization routes produced small pixel differences when resizing. We therefore did not extend the direct-frame optimization to those cases.
A fully serial loop can leave resources waiting:
The updated pipeline overlaps a limited amount of work:
The queue bounds are essential. Unrestricted prefetching can turn a throughput problem into a memory problem on long videos.
The exporter also probes supported hardware-encoding configurations and falls back when necessary. Actual hardware acceleration depends on the browser, operating system, and device.
Cancellation needs the same discipline: stop submitting new work, let bounded in-flight operations settle, and then release output resources. Closing everything immediately can leave asynchronous waits stranded.
Export was only part of the original problem. Playback and seeking also competed with background timeline work.
We changed scheduling so that:
For live thumbnail updates, confirmed decoded frames are drawn directly rather than repeatedly going through JPEG encoding and decoding. Source timestamps remain aligned with timeline sampling.
For locally imported media processed in the browser, these are client-side resource and coordination issues worth measuring alongside network behavior.
We kept output resolution, frame rate, and target bitrate unchanged. Validation covered:
This does not establish byte-identical output between the original version and the final version, or cover every possible project. It checks the specific paths changed by this optimization.
The main gains came from reducing intermediate work and overlapping bounded tasks while retaining the processing paths required by effects.
If you reproduce a performance issue with your own footage, include your device, browser version, source codec parameters, export settings, and effects in a GitHub issue. Separating preparation time from export time makes the bottleneck much easier to discuss.
This article was drafted with AI assistance using the implementation and measured benchmark results.
Long-video export is a pipeline problem. Before changing encoder settings, it is worth checking how much work happens around the encoder: whole-track audio decoding, intermediate frame rasterization, and serial waits between stages.
This article walks through an optimization of an open-source browser video editor, with a controlled before/after benchmark and the limits of each fast path.
We recently optimized these paths in Timeline Studio, an open-source browser video editor. In a controlled comparison against the code before the first optimization round, average export time fell from 192.0 seconds to 131.9 seconds—a 31.3% reduction, saving about 60 seconds.
Source code on GitHub · Try the editor
Benchmark: what we actually measured
Both versions ran on the same machine with the same approximately eight-minute synthetic stress-test video. The input was H.264 at 1440 × 1080 and approximately 29.88 fps, with stereo AAC audio at 44.1 kHz.
Output settings were identical:
- H.264, 1440 × 1080, 30 fps
- 10 Mbps target video bitrate
- AAC audio at 192 kbps
- No effects or overlays in this benchmark
Timing started immediately before embedded-audio preparation and ended when export completed. Import and timeline-thumbnail preparation were excluded.
We ran old → new → new → old, starting a fresh browser process for each run.
Before optimization: 203.7 s and 180.4 s; mean total 192.0 s.
Current implementation: 141.6 s and 122.3 s; mean total 131.9 s.
Audio preparation averaged 9.3 s → 0.9 s. The video export stage averaged 182.7 s → 131.0 s.
These are two runs per version, with visible runtime variation. The input is synthetic, so the 31.3% result should not be generalized to every camera recording, device, or effect stack. We also checked decoding and compatibility with a native Sony ZV-1 4K camera sample, but that sample is not the source of the eight-minute benchmark result.
1. Avoid decoding the entire audio track just to prepare it
The old preparation path used FFmpeg to decode embedded audio into WAV before subsequent processing. For long inputs, this adds whole-track decoding, PCM allocation, and intermediate-data handling.
For eligible AAC tracks, the new path extracts encoded packets and remuxes them instead. Final output audio is still encoded according to export settings; this change removes unnecessary work in the preparation stage.
Timestamp handling matters. AAC can contain preroll and negative timestamps. Simply copying packets while discarding that information can shift audio relative to the picture. The implementation preserves and compensates for the source offset and distinguishes storage duration from logical segment duration.
Other formats retain the fallback path. In this benchmark, mean audio preparation dropped from 9.3 seconds to 0.9 seconds.
2. Remove an intermediate frame conversion only when it is safe
At 30 fps, an eight-minute output contains approximately 14,400 frames. Even a small repeated operation is worth examining at that scale.
The new eligible path sends decoded VideoFrame objects directly to the shared compositor, avoiding an additional intermediate Canvas rasterization step.
Eligibility is deliberately narrow: original-size, unrotated, square-pixel frames with no relevant transformations or effects that require the existing rasterization path. Scaling, grading, animation, masks, and other affected cases keep their compatible Canvas path.
This restriction came from validation: different rasterization routes produced small pixel differences when resizing. We therefore did not extend the direct-frame optimization to those cases.
3. Overlap work with bounded queues
A fully serial loop can leave resources waiting:
The updated pipeline overlaps a limited amount of work:
- Prepare one decoded frame ahead.
- Compose the next frame while a video submission is in flight, subject to backpressure.
- Process audio and video concurrently.
- Release frames promptly after use.
The queue bounds are essential. Unrestricted prefetching can turn a throughput problem into a memory problem on long videos.
The exporter also probes supported hardware-encoding configurations and falls back when necessary. Actual hardware acceleration depends on the browser, operating system, and device.
Cancellation needs the same discipline: stop submitting new work, let bounded in-flight operations settle, and then release output resources. Closing everything immediately can leave asynchronous waits stranded.
4. Give playback priority over background thumbnails
Export was only part of the original problem. Playback and seeking also competed with background timeline work.
We changed scheduling so that:
- Playback pauses background thumbnail refinement.
- Scrubbing cancels background thumbnail work for the pointer gesture.
- Releasing the pointer resumes viewport-prioritized refinement.
- Starting playback prioritizes the latest seek instead of stale scrub requests.
- The playback clock pauses while decoding is seeking or buffering, reducing cases where the timeline advances while the picture is frozen.
For live thumbnail updates, confirmed decoded frames are drawn directly rather than repeatedly going through JPEG encoding and decoding. Source timestamps remain aligned with timeline sampling.
For locally imported media processed in the browser, these are client-side resource and coordination issues worth measuring alongside network behavior.
Validation: speed with the same output settings
We kept output resolution, frame rate, and target bitrate unchanged. Validation covered:
- Output video/audio tracks, duration, and dimensions.
- AAC packet extraction and source-offset handling.
- Pixel comparisons for eligible direct-frame cases.
- Canvas fallback behavior for scaling, grading, and animation.
- Encoded-packet consistency for a fixed effects case across the pipeline scheduling change.
- Cancellation and resource cleanup.
This does not establish byte-identical output between the original version and the final version, or cover every possible project. It checks the specific paths changed by this optimization.
Explore the implementation
The main gains came from reducing intermediate work and overlapping bounded tasks while retaining the processing paths required by effects.
If you reproduce a performance issue with your own footage, include your device, browser version, source codec parameters, export settings, and effects in a GitHub issue. Separating preparation time from export time makes the bottleneck much easier to discuss.
This article was drafted with AI assistance using the implementation and measured benchmark results.