From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 7E11F4C779 for ; Wed, 24 Dec 2025 16:55:51 +0000 (UTC) Authentication-Results: ffbox; dkim=fail (body hash mismatch (got b'uKjO61O/+dz+eqXSZN7pnieXrRnGHa4hmIByJ0+4MvQ=', expected b'x1klHKCDrpVCy/u4KQHoxOAQ5yp7EXoqLGrX35O+LhA=')) header.d=gmail.com header.a=rsa-sha256 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1766595308; h=to : date : message-id : mime-version : reply-to : subject : list-id : list-archive : list-archive : list-help : list-owner : list-post : list-subscribe : list-unsubscribe : from : cc : content-type : content-transfer-encoding : from; bh=R66VIvGcAYaVjw2PF8XNTNoJCtUYOqjDQPS999aWwO4=; b=SSwkhPk6wGA/VCtHKblLaJpEijGz8KnmBM+rZ18lNNaTYQuALIGMRn7WaQAaUbHwW+ad3 EboU01KKBTTWkrCeQpIdhys17TtNULob3AunF1tQSSvNJDlBuX8eDCeLXafjBPg9vEm8uCl alWNU+YsXPH1e6V2etzAnQ3N97TPF/ZXFuzegJz2aLn/Z2UJOTPI1nRg3ijow4LjOqVkeXD bL87FFQCfM5+/SpSfMkocPNDSTXKHxiwmLl18i8epdZpKXuZ2KI5nv7Yc3aV3QHimNxNgra W54Fs7ISLTfp3Z9HMw/UuBgcyumIMX3XftDSxnzzjUhOasF9bl3htgTRsczw== Received: from [172.20.0.4] (unknown [172.20.0.4]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 0C4CE690BCE; Wed, 24 Dec 2025 18:55:08 +0200 (EET) ARC-Seal: i=1; cv=none; a=rsa-sha256; d=ffmpeg.org; s=arc; t=1766595293; b=QDOPeZI79RuiP0Q4E/WTlwrsNB5SIjEtT06dlZlk+GfQnVDeoyAQibB+Zksv8HZH/cPiE WARRUK8VuxkAOvuRmycsmBKJOoWn1hMeBpjjPP4zwfX+ynWsvetXhr+rR9FuxpvYxGqPm8g rA/VcirDbCUv1rBe6ZCk9uvmT5lrG/NW9kJ8NR5q0Fr8pFRqbpl8x3Iyb8UGTL3Zoeweg6M PeTiu6A/T7oJzbN9OyzNW1CA5mT3McKI/bqVy7H8r0qWOzZhJ5WoTfNl/sdSFo9AViAeDVo JgzEArJ21x63T3mBkCenHUj4voBYvmtBJ2ZEg3IM5PzE0wPf/C3+zkK0CC8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=ffmpeg.org; s=arc; t=1766595293; h=from : sender : reply-to : subject : date : message-id : to : cc : mime-version : content-type : content-transfer-encoding : content-id : content-description : resent-date : resent-from : resent-sender : resent-to : resent-cc : resent-message-id : in-reply-to : references : list-id : list-help : list-unsubscribe : list-subscribe : list-post : list-owner : list-archive; bh=uKjO61O/+dz+eqXSZN7pnieXrRnGHa4hmIByJ0+4MvQ=; b=j1ELTXK8UbZYLfALaJqta44EDw+VQG+fFoYv0eo6dASwBggfaGcpVQ+z0amAsuEXnzodU mNrC4+k9VhOezSuC0L+t4+4zrrC3lKLXeTIdQ6VewFwef2OKFlBQ7MCQ4p/tAP5iiJTKu7r WgQ+y3DoGd9JupA6+shhtcbaaCfUi51nXbwR2/MJyEkKqSA2xdjYwNXhtHcaGP4MQhgWZGn OzSS5uvXA+bRHoXVofVW874ZYayZ629Lshtk/bWRCSEMKG0nQja94hjIMimjQoq1g2ViZd6 f11jeX3jP9VGJ3fovFMZ3ZkiB80OsIZTVhvueuRTnzoKuqawTgPD0zkJmdlw== ARC-Authentication-Results: i=1; ffmpeg.org; dkim=pass header.d=gmail.com; arc=none; dmarc=pass header.from=gmail.com policy.dmarc=quarantine Authentication-Results: ffmpeg.org; dkim=pass header.d=gmail.com; arc=none (Message is not ARC signed); dmarc=pass (Used From Domain Record) header.from=gmail.com policy.dmarc=quarantine Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id A76F46908A2 for ; Mon, 22 Dec 2025 06:45:46 +0200 (EET) Received: by mail-pg1-f171.google.com with SMTP id 41be03b00d2f7-c1e4a9033abso2786831a12.3 for ; Sun, 21 Dec 2025 20:45:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1766378743; x=1766983543; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=x1klHKCDrpVCy/u4KQHoxOAQ5yp7EXoqLGrX35O+LhA=; b=DGDTlRkri0mlBoF0KSfSrZn/yb8S9WR/a5+NDYtSaRB6E0vFfyUdu2RxJWy81rz382 IMlXSIxJCpewq219LgxuApxGStVcba4gKtpg0dfSATXMe8N6GEJaTFfpDUJElXMeQ2TR Nh/I8aqmbi3BIn/RPdtGJTR3SF3iVYQqySGifaraZcs+VdeGGzi4qbzWnoo/Mp0ySdRu SEH1Xnjk5EsJjjysIztyHHzP0vF/qV0TrVSdByjR+Lkig9L2Qx7Xuw0812oVIPc9SgoQ +NL0k8r3jm8Yc/5LrB5o6e8vx4u9JAPOdPGYLJtW5kLsTZ9xfa55rk0EUdPg2NIVFCYi HAjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766378743; x=1766983543; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=x1klHKCDrpVCy/u4KQHoxOAQ5yp7EXoqLGrX35O+LhA=; b=r0JGxjDl0HE7zc+uKJ0NpCm9qopYRI/Sr597M2h2mfsaVy3obMp/9y579e0U1mjq44 7oRJ4XzQwAFk+iiQ9MC+V8DKEgoqdkS20Mv2yGvamA0H0o4ca988v6A2jTu/uneGrB4h ljvWn1qS1dZLzWW+ddzFDOvR0z6BzRr2qJDzPIEjY31TiH2u9uvHZOaJlMsMr6ORaCdV GXvcdQmK/jo9bz3hobU+yFNC8idCZN12JgbpYwloep/vfcb6YwCi6aYSxRNL5g4Y8p/J SvSRPKY6+J5mp0wwce36yp9H5/nKgIPY6UHD65Evj5Bvuz2aRLQ0TckUzrE5+/wuZdkf Wk5Q== X-Gm-Message-State: AOJu0YwRWrk/dsg6JTLX+uigCmY/1B/0qXHY/5Urd9ZTaiB+dFJWJ6jF auKQfp1EHAiOwGflJAsWcFHkLRdGlIdmJGvSFN3a7hhIafgcUHfgpwGt4QcC4A== X-Gm-Gg: AY/fxX6XXuFDt4xFw0f4xVFctYpUJHDGnkbEt3+K3IYw9TQnmMw6X0O9H0Ks+Fd1xjX 3t1VI8k1rEIGWzdCIdwvdRhVxaJLvbONZvv7T3h367Ev8MvqNnwHrndwK9yD7ChdijYn7cDVv1q Unhx0E53a6qqBvqbbz16rZ6dcHYdPZt3oE3vXnIpWjL0kKy1buNv2NRgN4AlW2N2x0HRfz11uKI XTL0UNOb2+1qjXPaeAc8RN266yaV41nxWYORHhNQ+S1+3KjtKYaKZGoFKUbH4MVqkwyKVy07YVh xM1ksMvL1ani8Wdo26TRncCRqrI1S6gGCWkB9nYWe/UZEvNnDo+PSKqtP36qBTW3VRtZ113krQc xWrKZaxxLalk9d96BPAUOpAV17ycQTFqlVaGzulg20C+pwAVnRsmPZOwfgvdA0C3tDX4xi/FEG9 sYpg5Dw5IytPwCA6doWyl+zw== X-Google-Smtp-Source: AGHT+IEAGpgNZgHDaN7a9SK1ogwekhpqIoccvm0j2dkKg6nLckc6Lw4aZTyVK1r9614K0KPJqH4ldA== X-Received: by 2002:a05:7022:ead1:b0:119:e56b:c749 with SMTP id a92af1059eb24-121722b277cmr10381047c88.14.1766378743242; Sun, 21 Dec 2025 20:45:43 -0800 (PST) Received: from victor-MS-7C94 ([179.126.163.190]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-121724de25dsm29556748c88.7.2025.12.21.20.45.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Dec 2025 20:45:42 -0800 (PST) To: ffmpeg-devel@ffmpeg.org Date: Mon, 22 Dec 2025 01:45:35 -0300 Message-ID: <20251222044535.252202-1-victormeloasm@gmail.com> X-Mailer: git-send-email 2.51.0 MIME-Version: 1.0 X-MailFrom: SRS0=V8+h=64=gmail.com=victormeloasm@ffmpeg.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-ffmpeg-devel.ffmpeg.org-0; header-match-ffmpeg-devel.ffmpeg.org-1; header-match-ffmpeg-devel.ffmpeg.org-2; header-match-ffmpeg-devel.ffmpeg.org-3; emergency; member-moderation Message-ID-Hash: ZPSNBF3VTJIH6MEGDQRRDRPDUEJL4Q7J X-Message-ID-Hash: ZPSNBF3VTJIH6MEGDQRRDRPDUEJL4Q7J X-Mailman-Approved-At: Wed, 24 Dec 2025 16:54:47 +0000 X-Mailman-Version: 3.3.10 Precedence: list Reply-To: FFmpeg development discussions and patches Subject: [FFmpeg-devel] [PATCH] lavu/base64: optimize base64 encoding hot path List-Id: FFmpeg development discussions and patches Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Victor Duarte Melo via ffmpeg-devel Cc: Victor Duarte Melo Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Archived-At: List-Archive: List-Post: Reduce branches in the encoder fast path and improve throughput while preserving bit-exact behavior and API compatibility. The decoder logic is unchanged. Benchmarks and differential fuzzing were used to validate correctness and performance. --- libavutil/base64.c | 147 ++++++++++++++++++++++++++++++--------------- libavutil/base64.h | 40 +++++++++--- 2 files changed, 131 insertions(+), 56 deletions(-) diff --git a/libavutil/base64.c b/libavutil/base64.c index 69e11e6f5e..6d3c6c0d83 100644 --- a/libavutil/base64.c +++ b/libavutil/base64.c @@ -22,16 +22,40 @@ * @file * @brief Base64 encode/decode * @author Ryan Martell (with lots of Michael) + * + * This is a drop-in compatible implementation of FFmpeg's base64 helpers. + * The decode routine preserves FFmpeg's historical semantics (strict input, + * stops at the first invalid character, supports unpadded input). + * + * Small performance-oriented changes were made to the encoder: + * - The slow "shift loop" tail handling was replaced by a constant-time + * switch on the remaining 1 or 2 bytes, reducing branches and shifts. + * - The main loop now packs 3 bytes into a 24-bit value directly instead of + * reading an overlapping 32-bit word (avoids endian conversions and makes + * the loop easier for compilers to optimize). + * + * The API and output are fully compatible with the original code. */ #include #include +#include #include "base64.h" #include "error.h" #include "intreadwrite.h" -/* ---------------- private code */ +/* ---------------- private code + * + * map2[c] returns: + * - 0..63 : decoded 6-bit value for valid Base64 symbols + * - 0xFE : "stop" symbol (NUL terminator and '=' padding) + * - 0xFF : invalid symbol (produces AVERROR_INVALIDDATA) + * + * The decoder uses: + * - bits & 0x80 to detect "stop/invalid" quickly (both 0xFE and 0xFF have MSB set) + * - bits & 1 to distinguish invalid (0xFF) from stop (0xFE) + */ static const uint8_t map2[256] = { 0xfe, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, @@ -72,58 +96,72 @@ static const uint8_t map2[256] = }; #define BASE64_DEC_STEP(i) do { \ - bits = map2[in[i]]; \ - if (bits & 0x80) \ - goto out ## i; \ - v = i ? (v << 6) + bits : bits; \ -} while(0) + bits = map2[in[i]]; \ + if (bits & 0x80) \ + goto out ## i; \ + v = (i) ? (v << 6) + bits : bits; \ +} while (0) int av_base64_decode(uint8_t *out, const char *in_str, int out_size) { uint8_t *dst = out; uint8_t *end; - // no sign extension - const uint8_t *in = in_str; + /* Cast to unsigned to avoid sign extension on platforms where char is signed. */ + const uint8_t *in = (const uint8_t *)in_str; unsigned bits = 0xff; unsigned v; + /* Validation-only mode: keep FFmpeg's original behavior. */ if (!out) goto validity_check; end = out + out_size; + + /* + * Fast path: decode complete 4-char blocks while we can safely do a 32-bit store. + * We write 4 bytes and advance by 3 (the 4th written byte is overwritten on the next iteration). + */ while (end - dst > 3) { BASE64_DEC_STEP(0); BASE64_DEC_STEP(1); BASE64_DEC_STEP(2); BASE64_DEC_STEP(3); - // Using AV_WB32 directly confuses compiler + + /* Convert to native-endian so a native write yields correct byte order in memory. */ v = av_be2ne32(v << 8); AV_WN32(dst, v); + dst += 3; - in += 4; + in += 4; } + + /* Tail: decode at most one more block without overrunning the output buffer. */ if (end - dst) { BASE64_DEC_STEP(0); BASE64_DEC_STEP(1); BASE64_DEC_STEP(2); BASE64_DEC_STEP(3); + *dst++ = v >> 16; if (end - dst) *dst++ = v >> 8; if (end - dst) *dst++ = v; + in += 4; } + validity_check: + /* + * Strict validation: keep decoding groups of 4 until we hit the first stop/invalid. + * Using BASE64_DEC_STEP(0) ensures we always jump to out0 and never touch out1/out2/out3 + * (important for the out == NULL validation-only mode). + */ while (1) { - BASE64_DEC_STEP(0); - in++; - BASE64_DEC_STEP(0); - in++; - BASE64_DEC_STEP(0); - in++; - BASE64_DEC_STEP(0); - in++; + BASE64_DEC_STEP(0); in++; + BASE64_DEC_STEP(0); in++; + BASE64_DEC_STEP(0); in++; + BASE64_DEC_STEP(0); in++; } out3: @@ -135,49 +173,64 @@ out2: *dst++ = v >> 4; out1: out0: - return bits & 1 ? AVERROR_INVALIDDATA : out ? dst - out : 0; + /* bits==0xFE => stop (NUL or '=') => success. bits==0xFF => invalid => error. */ + return (bits & 1) ? AVERROR_INVALIDDATA : (out ? (int)(dst - out) : 0); } /***************************************************************************** -* b64_encode: Stolen from VLC's http.c. -* Simplified by Michael. -* Fixed edge cases and made it work from data (vs. strings) by Ryan. -*****************************************************************************/ + * b64_encode: Stolen from VLC's http.c. + * Simplified by Michael. + * Fixed edge cases and made it work from data (vs. strings) by Ryan. + * + * Encoder micro-optimizations: + * - Direct 24-bit packing (3 bytes -> 4 symbols) in the main loop. + * - Branchless tail handling via a small switch for 1 or 2 remaining bytes. + *****************************************************************************/ char *av_base64_encode(char *out, int out_size, const uint8_t *in, int in_size) { static const char b64[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; char *ret, *dst; - unsigned i_bits = 0; - int i_shift = 0; - int bytes_remaining = in_size; - if (in_size >= UINT_MAX / 4 || - out_size < AV_BASE64_SIZE(in_size)) + if (in_size >= (int)(UINT_MAX / 4) || out_size < AV_BASE64_SIZE(in_size)) return NULL; + ret = dst = out; - while (bytes_remaining > 3) { - i_bits = AV_RB32(in); - in += 3; bytes_remaining -= 3; - *dst++ = b64[ i_bits>>26 ]; - *dst++ = b64[(i_bits>>20) & 0x3F]; - *dst++ = b64[(i_bits>>14) & 0x3F]; - *dst++ = b64[(i_bits>>8 ) & 0x3F]; - } - i_bits = 0; - while (bytes_remaining) { - i_bits = (i_bits << 8) + *in++; - bytes_remaining--; - i_shift += 8; + + /* Encode full 3-byte blocks. */ + while (in_size >= 3) { + uint32_t v = ((uint32_t)in[0] << 16) | + ((uint32_t)in[1] << 8) | + ((uint32_t)in[2] ); + in += 3; + in_size -= 3; + + dst[0] = b64[ (v >> 18) ]; + dst[1] = b64[ (v >> 12) & 0x3F ]; + dst[2] = b64[ (v >> 6) & 0x3F ]; + dst[3] = b64[ (v ) & 0x3F ]; + dst += 4; } - while (i_shift > 0) { - *dst++ = b64[(i_bits << 6 >> i_shift) & 0x3f]; - i_shift -= 6; + + /* Encode the remaining 1 or 2 bytes (if any) and add '=' padding. */ + if (in_size == 1) { + uint32_t v = (uint32_t)in[0]; + dst[0] = b64[(v >> 2) & 0x3F]; + dst[1] = b64[(v & 0x03) << 4]; + dst[2] = '='; + dst[3] = '='; + dst += 4; + } else if (in_size == 2) { + uint32_t v = ((uint32_t)in[0] << 8) | (uint32_t)in[1]; + dst[0] = b64[(v >> 10) & 0x3F]; + dst[1] = b64[(v >> 4) & 0x3F]; + dst[2] = b64[(v & 0x0F) << 2]; + dst[3] = '='; + dst += 4; } - while ((dst - ret) & 3) - *dst++ = '='; - *dst = '\0'; + /* NUL-terminate. The caller guaranteed enough space via AV_BASE64_SIZE(). */ + *dst = '\0'; return ret; } diff --git a/libavutil/base64.h b/libavutil/base64.h index 2954c12d42..31bd8357e3 100644 --- a/libavutil/base64.h +++ b/libavutil/base64.h @@ -23,6 +23,16 @@ #include +/* + * NOTE: This header intentionally keeps the original FFmpeg API surface + * (function names, macros and semantics). The implementation shipped in + * base64.c is a drop-in replacement with extra tests/benchmarks around it. + */ + +#ifdef __cplusplus +extern "C" { +#endif + /** * @defgroup lavu_base64 Base64 * @ingroup lavu_crypto @@ -32,12 +42,17 @@ /** * Decode a base64-encoded string. * - * @param out buffer for decoded data + * The input must be a NUL-terminated string. This decoder is strict: + * it does not ignore whitespace and it stops at the first invalid byte + * (including the terminating NUL). This matches FFmpeg's historical behavior. + * + * @param out buffer for decoded data, or NULL to only validate input * @param in null-terminated input string * @param out_size size in bytes of the out buffer, must be at - * least 3/4 of the length of in, that is AV_BASE64_DECODE_SIZE(strlen(in)) - * @return number of bytes written, or a negative value in case of - * invalid input + * least 3/4 of the length of in, that is + * AV_BASE64_DECODE_SIZE(strlen(in)) + * @return number of bytes written, 0 for validation-only success, + * or a negative value in case of invalid input */ int av_base64_decode(uint8_t *out, const char *in, int out_size); @@ -50,12 +65,15 @@ int av_base64_decode(uint8_t *out, const char *in, int out_size); /** * Encode data to base64 and null-terminate. * + * The output is padded using '=' and is always NUL-terminated (if out is large + * enough). This matches FFmpeg's av_base64_encode behavior. + * * @param out buffer for encoded data * @param out_size size in bytes of the out buffer (including the * null terminator), must be at least AV_BASE64_SIZE(in_size) * @param in input buffer containing the data to encode * @param in_size size in bytes of the in buffer - * @return out or NULL in case of error + * @return out or NULL in case of error (e.g. output too small) */ char *av_base64_encode(char *out, int out_size, const uint8_t *in, int in_size); @@ -63,10 +81,14 @@ char *av_base64_encode(char *out, int out_size, const uint8_t *in, int in_size); * Calculate the output size needed to base64-encode x bytes to a * null-terminated string. */ -#define AV_BASE64_SIZE(x) (((x)+2) / 3 * 4 + 1) +#define AV_BASE64_SIZE(x) (((x) + 2) / 3 * 4 + 1) + +/** + * @} + */ - /** - * @} - */ +#ifdef __cplusplus +} /* extern "C" */ +#endif #endif /* AVUTIL_BASE64_H */ -- 2.51.0 _______________________________________________ ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org