From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id EB86D47517 for ; Mon, 18 Aug 2025 06:23:09 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id BC78D68D36F; Mon, 18 Aug 2025 09:23:05 +0300 (EEST) Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id 18C6668B978 for ; Mon, 18 Aug 2025 09:22:59 +0300 (EEST) Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-45a1b00f23eso19560485e9.0 for ; Sun, 17 Aug 2025 23:22:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755498178; x=1756102978; h=content-disposition:mime-version:message-id:subject:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yyLr20AHjhHPLhddwgzeSG0w5GwRQkhq3pZPk3IvvwQ=; b=gXHnyoBHtW9Z+/PSWn1aCpRJ4wgR4QGlie0a75ODUfj08MXM/HpPK4/qzFMaxX8dp7 QLPZbWpW73TaOBAAe64XP1m/nmsZaJbf338W0N0KR8v8+LUqW7Q1M5spA6xq0MM8GTdQ eiCEprFNXH7nbcaMfNnRuiNcpdIZ7Vf4L/TIPrvQ8MrB6Pu3GyewV7ZESihlOt4gyaMq l/VdqYgwvszRG0SYfL14aw40O/y7DbKI+8nz15SRTElj45q42hXegWkOevVbeUYDhUo8 DYzC4KVYIHS8/qzt1vwKXkDsxdSPg043NPovtI9glCSnJgytypeluguKNv5hNpZlwcZi F46w== X-Gm-Message-State: AOJu0YzWy3xI0ef3KzKpSDAblGXKl4gRkhnR4Cpa/38PdlnFsHuvVqqf Pz7WtCgzkMwVSPBcgpa6Ub0gvwtzlhg14vrmZIf3km/9djcyuCRI0AKdd9Yju+AH X-Gm-Gg: ASbGncvgI7kxNQCs251g5jmk0lnWRRcdiZaZuVGQplIorCriZQvlNJ0n1YlFIpSDLPC DUBBTBzH35llxpeIghsUCtZ+AQd+TMxgftrI3q8te64ktOsmhEsLKYrTfh7vVymoSnrPE/goI81 FZMikrD7fZEOuTDJ/rg35NKcZYyYXn6PMziJG1Q8V9ljI45cQPYK0Li5TGK6cRGkPRKUcDkkQem q47Ybrwd7ri5yXoY4r2I2hnkoBp0L/dR69Fczj7jkURB2s3c9saVskMRDhJjEcPnUxK8Z4mZiE+ OXWQ+ecOlTCmg+bz6K8fcrVVzBaPeIsMhM/0On2jab5wqHAAPA3nq8QGZeT6zywKYNujNe0BIgI HCn2szgGnCYnGFDOwvQ== X-Google-Smtp-Source: AGHT+IH1A/cYWg2EHf8xImhDCC6pIYBOmIJUS/HfIy4YT6xPQor772fTAWGq1YeLYqoy6gj/z1eP5A== X-Received: by 2002:a05:600c:45c6:b0:459:db80:c2d0 with SMTP id 5b1f17b1804b1-45a217f3096mr69598775e9.7.1755498178127; Sun, 17 Aug 2025 23:22:58 -0700 (PDT) Received: from zazen.home ([212.231.125.195]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-45a22321985sm116624855e9.17.2025.08.17.23.22.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 Aug 2025 23:22:57 -0700 (PDT) Date: Mon, 18 Aug 2025 07:22:56 +0100 To: ffmpeg-devel@ffmpeg.org Message-ID: <20250818062256.s2blarmtwdv6psz4@zazen.home> MIME-Version: 1.0 Content-Disposition: inline Subject: [FFmpeg-devel] [Proposal] drawvg filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Ayose via ffmpeg-devel Reply-To: FFmpeg development discussions and patches Cc: Ayose Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: This is a proposal to add a new filter to FFmpeg: drawvg, to draw vector graphics on top of a video, using libcairo. This is my first contribution to FFmpeg, so I don't know if this is the right channel to make the proposal. I was thinking on how to write it, and I thought that a FAQ-like summary may be the easiest way to describe it. Apologies if this is more confusing than helpful. == What is drawvg == drawvg is a filter to render vector graphics on top of video frames. The render is done by executing a script written in its own language, called VGS (Vector Graphics Script). The script consists of a series of commands to describe 2D graphics, which are rasterized using the libcairo library. VGS is not intended to be used as a general-purpose language. Since its scope is limited, it prioritizes being concise and easy to use. The syntax is heavily inspired by languages like [1]Magick Vector Graphics, or [2]SVG's . [1] https://imagemagick.org/script/magick-vector-graphics.php [2] https://developer.mozilla.org/en-US/docs/Web/SVG/Reference/Element/path Some features of the syntax (like using whitespaces to separate arguments) are also present in languages like TCL or shell scripts. Many command names are taken from PostScript. Scripts can use FFmpeg expressions to describe graphics dynamically, so they can compute coordinates based on frame dimensions, frame metadata, generate random values, read pixel colors, etc. For example, to draw a blue circle in the middle of a frame: circle (w / 2) (h / 2) 100 setcolor blue fill Then: $ ffplay -i example.webm -vf 'drawvg=file=blue-circle.vgs' There is a [3]website with some examples on how to integrate drawvg with other FFmpeg filters, and also a [4]playground to experiment with the capabilities of the language. [3] https://ayosec.github.io/ffmpeg-drawvg/ [4] https://ayosec.github.io/ffmpeg-drawvg/playground/ == Why is it useful for FFmpeg == FFmpeg has some filters to modify video frames, like drawbox, xfade, geq, drawtext, drawgrid, etc. I guess that most users will only need filters like scale or crop, but for some other users, the ability to draw vector graphics makes possible to perform more complex operations, like adding annotations, or creating masks for custom transitions. The implementation is much smaller than I was expecting. The current version of [5]vf_drawvg.c, which contains the whole implementation of drawvg (the parser and the interpreter for VGS, and the class definition for the filter) has 2590 lines of code. For comparison, vf_drawtext.c has 1917 LOC. [5] https://github.com/ayosec/ffmpeg-drawvg/blob/drawvg-filter/libavfilter/vf_drawvg.c == Why not SVG == FFmpeg can load SVG images with librsvg. While SVG is much more powerful than drawvg, the images are static, and can't rely on any information from FFmpeg itself. On the other hand, a drawvg script can use FFmpeg expressions, to compute colors and coordinates by using data available in the frame (like dimensions, timestamp, colors from pixels of the video, frame metadata, etc). == Why libcairo == There are two reasons to choose libcairo: First, it is a well-tested, stable, and portable library. Second, librsvg depends on libcairo, so any installation of FFmpeg that supports SVG is already using libcairo. I expect that, since there are no new dependencies, distro maintainers should not need to make changes on how their FFmpeg package is built in order to support the new filter. == Why a custom language == Before starting to work on the implementation I was considering to reuse an existing interpreter, like Lua or a lightweight JavaScript implementation (QuickJS, Duktape, MuJS, etc.). But this option was discarded for three reasons: First, it adds more external dependencies to FFmpeg. Second, a filter like this does not need the complexity of a general purpose language. Even when both Lua and JavaScript are quite simple, we can have a more minimalistic language to describe what must be drawn by the filter. And, if someone really needs to create a more complex script, they may prefer something different to Lua/JS anyways. Third, by using a custom language, we can have full integration with the existing language for FFmpeg expressions. Thus, users with some experience writing complex filtergraphs can use a familiar syntax. The current version of the language is [6]fully documented. [6] https://ayosec.github.io/ffmpeg-drawvg/playground/docs/langref.html == What is done == The filter is [7]already implemented (except for yet-to-be-found bugs). [7] https://github.com/ayosec/ffmpeg-drawvg/compare/8426622bb..drawvg-filter I'm compiling FFmpeg with "--toolchain=clang-asan", so I hope that the implementation is (mostly) correct, although I'm not a C expert. The patch already has two FATE tests: one to verify the calls to libcairo, and another one to compare the output (CMD = video_filter). There is a [8]repository in GitHub with the changes. The commit messages are mostly noise (I assume that all commits must be squashed, so I didn't spent time writing proper messages), but I left the original commit history, in case it is helpful for reviewing the code. [8] https://github.com/ayosec/ffmpeg-drawvg == What is missing == The most important task that is still missing is to port the documentation to Texinfo. I used Markdown because it is what I'm most used to. If the filter is accepted, I guess than pandoc can do most of the required changes, so it should be easy. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".