What the second pass buys under a 10 MB cap

Across 14 clips, Fitbyte landed every file between 96.8% and 100.0% of 10,000,000 bytes. A single-pass encode with the same bitrate budget landed between 80.7% and 100.7%, and on one clip it went over. The quality gap is smaller than I expected. Mean VMAF went from 89.90 to 90.28. Most of the difference is in the worst frames: the 5th-percentile frame VMAF went from 81.26 to 84.65. The cost was 1.05× to 1.49× the encode time.

Fitbyte is the small tool I wrote to squeeze videos under Discord's upload cap. It does two-pass x264 and wraps it in a retry loop. In the Fitbyte post I said the loop "almost always settles in one or two" attempts. I had never checked that, and I had never checked whether the second pass was worth it. So I measured both.

The setup

Clips. Fourteen clips, 15 to 90 seconds each, 592.6 seconds in total, all from freely licensed sources:

• the three Blender open movies (CC BY 3.0)
• three TU München test sequences from the Xiph collection (no restrictions)
• five Wikimedia Commons uploads: beach soccer, street skating, a talking-head interview, a 60 fps screen recording, and a grainy 35 mm trailer scan (CC0, CC BY, CC BY-SA, public domain)

URLs, authors, licenses, and cut points are in the credits at the end. I left out the popular NTIA sports clips because they are licensed for research use only.

Mezzanines. Every clip is cut to a near-lossless mezzanine: x264 CRF 8, FLAC audio. 4K sources are downscaled to 1080p. That one file is both the input to every encoder and the reference for VMAF. So the scores measure loss against the mezzanine, not against the original master.

Box. An 8-vCPU KVM guest on an Intel Xeon (family 6, model 207, AVX-512), with 15 GB of RAM and other things running on the same machine. The software:

• static ffmpeg 7.0.2 (johnvansickle build) with libx264 0.164.3191 and libvmaf 2.3.0
• Fitbyte at commit 6162436

The system ffmpeg is 7.1.5, but Debian's build has no libvmaf, so the static build did all the encoding and all the measuring.

Methods. The bench doesn't re-implement Fitbyte. It imports app.py and calls Fitbyte's own initial_auto_budgets, build_video_encode_command and run_auto_video. The only patch is a wrapper on run_streaming that times each ffmpeg call and saves a copy of each attempt's output. Everything uses -preset slow, AAC at 128k, and +faststart. There were seven methods per clip:

The budget is plain arithmetic: 98.5% of 10,000,000 bytes, spread over the duration, minus 128 kbps of audio when the clip has audio. That gives 5253 kbps of video for the 15-second clip (no audio track) and 747 kbps for the 90-second one. The two-pass command pair is:

ffmpeg -y -i clip.mkv -map 0:v:0 -c:v libx264 -preset slow -b:v 747k \
  -pass 1 -passlogfile X -an -f mp4 /dev/null
ffmpeg -y -i clip.mkv -map 0:v:0 -map 0:a:0 -c:v libx264 -preset slow -b:v 747k \
  -pass 2 -passlogfile X -c:a aac -b:a 128k -movflags +faststart out.mp4

The loop counts as done when the file lands between 96.5% and 100% of target. It allows up to five attempts, and from attempt three it accepts any file under the target.

Metrics. libvmaf vmaf_v0.6.1, with PSNR and float SSIM from the same pass. For each encode I kept the mean, the harmonic mean, the 5th-percentile frame and the minimum. Encode time is perf_counter wall time plus child CPU time. I re-timed four clips three times each, with nothing else in the bench running.

I paired the wrong frames first

My first smoke test on a 4-second Sintel cut gave a harmonic-mean VMAF of 2.2. Every third frame scored exactly 0. The filter graph used setpts=PTS-STARTPTS on both inputs. The mezzanines are Matroska with 1 ms timestamps, the outputs are MP4 with a 1/12288 timebase, and at 24 fps the rounding made libvmaf's framesync pair some frames with their neighbours. Switching to frame-index timestamps fixed it:

[0:v]settb=1/1000,setpts=N,format=yuv420p[d];
[1:v]settb=1/1000,setpts=N,format=yuv420p[r];
[d][r]libvmaf=model=version=vmaf_v0.6.1:feature=name=psnr|name=float_ssim:...

The bench now also counts frames in both files. All 98 measured encodes have equal counts. If your VMAF numbers look strangely bimodal, check the pairing before you blame the encoder.

Landing on the number

"10 MB" here means 10,000,000 bytes, which is what Fitbyte targets. 10 MiB is 10,485,760. I didn't test which one Discord actually enforces. Fitbyte aims under both, so it doesn't matter for Fitbyte.

CRF is useless for hitting a size. CRF 23 ranged from 45% of the cap on the screen recording to 428% on beach soccer. Even CRF 28 went over on 6 of 14 clips.

Single-pass ABR mostly undershoots. It left 19% of the budget unused on bbb_action and 18% on screen_tutorial. Then on skate_tokyo it went over, at 10,066,993 bytes. With the retry loop it does land, but it took three attempts on four clips. On two of those, bbb_action (92.8%) and screen_tutorial (92.2%), it hit the "attempt three, good enough" exit without reaching the window.

The surprise was two-pass overshooting. On its first attempt, screen_tutorial came out at 10,764,908 bytes, 7.6% over and above 10 MiB too. tos_dialog came out at 10,070,397. Fitbyte's loop caught both, scaled the bitrate down, and finished on attempt 2. That's 2 of 14 clips that needed a second attempt. So my "one or two" was right, but only because the loop exists. Without it, two-pass alone would have failed the cap on 2 of 14 clips. I don't know why the 60 fps screen recording overshoots by that much. My guess is the pass-1 stats from fastfirstpass on mostly static UI, but I haven't tested that.

What the bytes buy

Averaged over the 14 clips, the second pass is worth about 1 VMAF point against a bare single pass. Some of that is just extra bytes: one pass leaves budget unused, and two passes spend it. The fairer comparison is Fitbyte against the single-pass loop, since both end up close to the same size. There the mean gap is 0.38 points. The 5th-percentile frame gap is 3.39, and PSNR actually goes slightly the other way (41.74 vs 41.90). Two-pass moves bits from easy frames to hard ones, and an average metric barely notices.

The clearest case is tos_dialog. Fitbyte's file was 12 KB smaller than the single-pass file, and its 5th-percentile frame scored 74.56 against 61.53. That's a gain of 13.0 points, on the shots where a viewer would see blocks. The longest clip, sintel_snow_long at 747 kbps, gained 4.05 at the 5th percentile and lifted its worst frame from 53.2 to 65.4.

Two clips go the other way:

• teacher_interview lost 0.56 mean VMAF with two-pass, even though it got 1.0 pp more bytes. Its 5th percentile still went up by 0.96.
• skate_tokyo is a wash: +0.04 mean, +0.03 at p5, with 1.8 pp fewer bytes.

The cap matters more than the pass count. beach_soccer (fast pans over sand and a crowd) gets 1842 kbps for 40 seconds and scores 75.34 with Fitbyte and 74.95 without. CRF 23 on that clip needs 4.3× the budget to score 98.5. No rate-control mode fixes that. Fewer pixels or fewer seconds would.

One thing I can't fully separate: the two methods rarely finish at exactly the same size. For the seven clips where the CRF points bracket Fitbyte's size, I interpolated the CRF curve at that size. Fitbyte came out 0.12 to 0.39 points ahead. So at these bitrates, two-pass ABR is at least as good as CRF at the same size. Interpolating between three points is rough, though.

What it costs

Mean of three runs, wall seconds. Sizes were byte-identical across repeats.

Two-pass doesn't cost double. Pass 1 uses x264's fastfirstpass and skips audio, so it took 26% to 42% of the two-pass total. Pass 2 was sometimes faster than the equivalent single pass: 15.71 s against 20.02 s on skate_tokyo, and 11.71 against 13.49 on tos_dialog. Standard deviations were 0.2–1.4 s.

Over the whole 14-clip bench (one run each, so noisier), the totals:

The single-pass loop is the slowest way to hit the target. It needs 1.86 attempts on average, and each miss costs a full encode. Fitbyte's two overshoots cost one extra pair each. The encodes alone took about 38 minutes of wall time in total (that's the sum of measured encode wall time across all 98 encodes; VMAF scoring time wasn't recorded).

What I'm changing, and what I'd use

For Fitbyte:

• Keep two-pass as the default. It doesn't buy much on average, but it reliably helps the worst frames, and that's what people notice in a clip squeezed into 10 MB.
• Keep the retry loop. In the old post I called the five-attempt cap paranoid. The loop earned its place: two-pass overshot on 2 of 14 clips, once by 7.6%.
• Drop the cap from five attempts to three. No clip needed more than two.
• Warn on short budgets. I want to add a warning when video bitrate per pixel per frame drops below a threshold, and suggest downscaling instead. The beach soccer result is what a 1080p clip looks like at 1.8 Mbps, and no amount of rate control will hide it.

For anyone else with a hard size cap and ffmpeg: use two-pass ABR, compute the bitrate from duration minus audio, leave 1.5% for the container, and check the output size afterwards. Don't trust either pass to land exactly. Don't use CRF if you need to hit a size. If you don't have a hard cap, CRF is still the simpler tool.

Clip credits

Fourteen clips, cut from nine sources. All clips were trimmed and re-encoded for the bench. Nothing in this post redistributes the clip files. CC BY and CC BY-SA clips are attributed below.

Big Buck Bunny, Sintel, and Tears of Steel are © Blender Foundation, blender.org. The Xiph derf collection lists the three SVT/TUM sequences as no restrictions and no copyright. skate_tokyo and film_grain_1964 were downscaled from 4K to 1080p for the bench. NTIA sports clips on the same derf page are research-use only, so they are not here.