From: "François-Simon Fauteux-Chapleau" <francois-simon.fauteux-chapleau@savoirfairelinux.com>
To: ffmpeg-devel@ffmpeg.org
Subject: [FFmpeg-devel] [PATCH] libavfilter: add PipeWire-based grab
Date: Fri, 15 Mar 2024 12:51:34 -0400
Message-ID: <20240315165134.527481-1-francois-simon.fauteux-chapleau@savoirfairelinux.com> (raw)
This is a revised version of the "pipewiregrab" patch submitted by
Abhishek Ojha a few months ago:
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231227162504.690730-1-abhishek.ojha@savoirfairelinux.com/
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231227162504.690730-2-abhishek.ojha@savoirfairelinux.com/
The main change is that the patch is now implemented as a libavfilter
source filter instead of a libavdevice input device, as was requested in
a comment on the previous version.
There are also several small changes meant to fix bugs or simplify the
code, but the overall structure remains the same as before: we use the
ScreenCast interface provided by XDG Desktop Portal to obtain a file
descriptor, which is then used to create a PipeWire stream. The data from
that stream can then be used to generate frames for FFmpeg.
To test the patch:
ffplay -f lavfi -i pipewiregrab
Signed-off-by: François-Simon Fauteux-Chapleau <francois-simon.fauteux-chapleau@savoirfairelinux.com>
---
configure | 16 +
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/vsrc_pipewiregrab.c | 1357 +++++++++++++++++++++++++++++++
4 files changed, 1375 insertions(+)
create mode 100644 libavfilter/vsrc_pipewiregrab.c
diff --git a/configure b/configure
index 2b4c4ec9a2..f31651dce2 100755
--- a/configure
+++ b/configure
@@ -301,6 +301,7 @@ External library support:
--enable-libxcb-shm enable X11 grabbing shm communication [autodetect]
--enable-libxcb-xfixes enable X11 grabbing mouse rendering [autodetect]
--enable-libxcb-shape enable X11 grabbing shape rendering [autodetect]
+ --enable-libpipewire enable screen grabbing using pipewire [autodetect]
--enable-libxvid enable Xvid encoding via xvidcore,
native MPEG-4/Xvid encoder exists [no]
--enable-libxml2 enable XML parsing using the C library libxml2, needed
@@ -1792,6 +1793,8 @@ EXTERNAL_AUTODETECT_LIBRARY_LIST="
libxcb_shm
libxcb_shape
libxcb_xfixes
+ libpipewire
+ libgio_unix
lzma
mediafoundation
metal
@@ -3817,6 +3820,7 @@ pad_opencl_filter_deps="opencl"
pan_filter_deps="swresample"
perspective_filter_deps="gpl"
phase_filter_deps="gpl"
+pipewiregrab_filter_deps="libpipewire libgio_unix pthreads"
pp7_filter_deps="gpl"
pp_filter_deps="gpl postproc"
prewitt_opencl_filter_deps="opencl"
@@ -7126,6 +7130,18 @@ if enabled libxcb; then
enabled libxcb_xfixes && check_pkg_config libxcb_xfixes xcb-xfixes xcb/xfixes.h xcb_xfixes_get_cursor_image
fi
+# Starting with version 0.3.52, PipeWire's spa library uses the __LOCALE_C_ONLY macro to determine
+# whether the locale_t type (introduced in POSIX.1-2008) and some related functions are available (see
+# https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2390 for more information).
+# Unfortunately, this macro is specific to uclibc, which can cause build issues on systems that use a
+# different implementation of libc if POSIX 2008 support isn't enabled (which is the case for FFmpeg currently).
+# As a workaround for this problem, we add a compilation flag to ensure that __LOCALE_C_ONLY is always defined.
+add_cppflags -D__LOCALE_C_ONLY
+enabled libpipewire && check_pkg_config libpipewire "libpipewire-0.3 >= 0.3.40" pipewire/pipewire.h pw_init
+if enabled libpipewire; then
+ enabled libgio_unix && check_pkg_config libgio_unix gio-unix-2.0 gio/gio.h g_main_loop_new
+fi
+
check_func_headers "windows.h" CreateDIBSection "$gdigrab_indev_extralibs"
# check if building for desktop or uwp
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 994d9773ba..982904be2d 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -600,6 +600,7 @@ OBJS-$(CONFIG_NULLSRC_FILTER) += vsrc_testsrc.o
OBJS-$(CONFIG_OPENCLSRC_FILTER) += vf_program_opencl.o opencl.o
OBJS-$(CONFIG_PAL75BARS_FILTER) += vsrc_testsrc.o
OBJS-$(CONFIG_PAL100BARS_FILTER) += vsrc_testsrc.o
+OBJS-$(CONFIG_PIPEWIREGRAB_FILTER) += vsrc_pipewiregrab.o
OBJS-$(CONFIG_QRENCODE_FILTER) += qrencode.o textutils.o
OBJS-$(CONFIG_QRENCODESRC_FILTER) += qrencode.o textutils.o
OBJS-$(CONFIG_RGBTESTSRC_FILTER) += vsrc_testsrc.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 149bf50997..a3c9c891b0 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -567,6 +567,7 @@ extern const AVFilter ff_vsrc_openclsrc;
extern const AVFilter ff_vsrc_qrencodesrc;
extern const AVFilter ff_vsrc_pal75bars;
extern const AVFilter ff_vsrc_pal100bars;
+extern const AVFilter ff_vsrc_pipewiregrab;
extern const AVFilter ff_vsrc_rgbtestsrc;
extern const AVFilter ff_vsrc_sierpinski;
extern const AVFilter ff_vsrc_smptebars;
diff --git a/libavfilter/vsrc_pipewiregrab.c b/libavfilter/vsrc_pipewiregrab.c
new file mode 100644
index 0000000000..870e0f2d30
--- /dev/null
+++ b/libavfilter/vsrc_pipewiregrab.c
@@ -0,0 +1,1357 @@
+/*
+ * PipeWire input grabber (ScreenCast)
+ * Copyright (C) 2024 Savoir-faire Linux, Inc.
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * PipeWireGrab video source
+ * @author Firas Ashkar <firas.ashkar at savoirfairelinux.com>
+ * @author Abhishek Ojha <abhishek.ojha at savoirfairelinux.com>
+ * @author François-Simon Fauteux-Chapleau <francois-simon.fauteux-chapleau at savoirfairelinux.com>
+ */
+
+#include "config.h"
+
+#include <fcntl.h>
+#include <linux/dma-buf.h>
+#include <math.h>
+#include <pthread.h>
+#include <stdatomic.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/queue.h>
+
+#include "libavutil/internal.h"
+#include "libavutil/mathematics.h"
+#include "libavutil/opt.h"
+#include "libavutil/parseutils.h"
+#include "libavutil/time.h"
+#include "libavutil/avstring.h"
+#include "libavformat/avformat.h"
+#include "libavformat/internal.h"
+#include "libavutil/avassert.h"
+#include "avfilter.h"
+#include "formats.h"
+#include "internal.h"
+#include "video.h"
+
+#include <pipewire/pipewire.h>
+#include <pipewire/thread-loop.h>
+#include <spa/debug/types.h>
+#include <spa/param/video/format-utils.h>
+#include <spa/param/video/raw.h>
+#include <spa/param/video/type-info.h>
+
+#include <gio/gio.h>
+#include <gio/gunixfdlist.h>
+
+#ifndef __USE_XOPEN2K8
+#define F_DUPFD_CLOEXEC \
+ 1030 /* Duplicate file descriptor with close-on-exit set. */
+#endif
+
+#define REQUEST_PATH "/org/freedesktop/portal/desktop/request/%s/%s"
+#define BYTES_PER_PIXEL 4 /* currently all formats assume 4 bytes per pixel */
+#define MAX_SPA_PARAM 4 /* max number of params for spa pod */
+
+#define CURSOR_META_SIZE(width, height) \
+ (sizeof(struct spa_meta_cursor) + sizeof(struct spa_meta_bitmap) + \
+ width * height * 4)
+
+/**
+ * PipeWire capture types
+ */
+typedef enum {
+ DESKTOP_CAPTURE = 1,
+ WINDOW_CAPTURE = 2,
+} pw_capture_type;
+
+/**
+ * XDG Desktop Portal supported cursor modes
+ */
+enum PortalCursorMode {
+ PORTAL_CURSOR_MODE_HIDDEN = 1 << 0,
+ PORTAL_CURSOR_MODE_EMBEDDED = 1 << 1,
+};
+
+typedef struct PipewireGrabContext {
+ const AVClass *class;
+
+ pthread_cond_t pipewire_initialization_cond_var;
+ pthread_mutex_t pipewire_initialization_mutex;
+ atomic_int pipewire_initialization_over;
+
+ int pw_init_called;
+
+ pthread_mutex_t current_frame_mutex;
+ AVFrame *current_frame;
+
+ GDBusConnection *connection;
+ GDBusProxy *proxy;
+ GCancellable *cancellable;
+
+ char *sender_name;
+ char *session_handle;
+
+ uint32_t pipewire_node;
+ int pipewire_fd;
+
+ uint32_t available_cursor_modes;
+
+ GMainLoop *glib_main_loop;
+ struct pw_thread_loop *thread_loop;
+ struct pw_context *context;
+
+ struct pw_core *core;
+ struct spa_hook core_listener;
+
+ struct pw_stream *stream;
+ struct spa_hook stream_listener;
+ struct spa_video_info format;
+
+ pw_capture_type capture_type;
+
+ int draw_mouse;
+
+ uint32_t width, height;
+
+ size_t frame_size;
+ uint8_t Bpp;
+ enum AVPixelFormat av_pxl_format;
+
+ int64_t time_frame;
+ int64_t frame_duration;
+
+ AVRational framerate;
+
+ int portal_error;
+ int pipewire_error;
+} PipewireGrabContext;
+
+/**
+ * DBus method/event marshalling structure
+ */
+struct DbusCallData {
+ AVFilterContext *ctx;
+ char *request_path;
+ guint signal_id;
+ gulong cancelled_id;
+};
+
+
+#define OFFSET(x) offsetof(PipewireGrabContext, x)
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
+static const AVOption pipewiregrab_options[] = {
+ { "framerate", "set video frame rate", OFFSET(framerate), AV_OPT_TYPE_VIDEO_RATE, { .str = "ntsc" }, 0, INT_MAX, FLAGS },
+ { "draw_mouse", "draw the mouse pointer", OFFSET(draw_mouse), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, FLAGS },
+ { "capture_type", "set the capture type (1 for screen, 2 for window)", OFFSET(capture_type), AV_OPT_TYPE_INT, { .i64 = 1 }, 1, 2, FLAGS },
+ { "fd", "set file descriptor to be used by PipeWire", OFFSET(pipewire_fd), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS },
+ { NULL },
+};
+
+AVFILTER_DEFINE_CLASS(pipewiregrab);
+
+/**
+ * Helper function to allow portal_init_screencast to stop and return an error
+ * code if a DBus operation/callback fails.
+ *
+ * @param ctx
+ * @param error error code
+ * @param message error message
+ */
+static void portal_abort(AVFilterContext *ctx, int error, const char *message)
+{
+ PipewireGrabContext *pw_ctx = ctx->priv;
+
+ pw_ctx->portal_error = error;
+ av_log(ctx, AV_LOG_ERROR, "Aborting: %s\n", message);
+
+ if (pw_ctx->glib_main_loop &&
+ g_main_loop_is_running(pw_ctx->glib_main_loop))
+ g_main_loop_quit(pw_ctx->glib_main_loop);
+}
+
+/**
+ * Callback to handle PipeWire core info events
+ *
+ * @param user_data pointer to the filter's AVFilterContext
+ * @param info pw_core_info
+ */
+static void on_core_info_callback(void *user_data, const struct pw_core_info *info)
+{
+ AVFilterContext *ctx = user_data;
+ av_log(ctx, AV_LOG_DEBUG, "Server version: %s\n", info->version);
+ av_log(ctx, AV_LOG_INFO, "Library version: %s\n", pw_get_library_version());
+ av_log(ctx, AV_LOG_DEBUG, "Header version: %s\n", pw_get_headers_version());
+}
+
+/**
+ * Callback to handle PipeWire core done events
+ *
+ * @param user_data pointer to the filter's AVFilterContext
+ * @param id PipeWire object id of calling
+ * @param seq PipeWire object sequence
+ */
+static void on_core_done_callback(void *user_data, uint32_t id, int seq)
+{
+ AVFilterContext *ctx = user_data;
+ PipewireGrabContext *pw_ctx;
+
+ if (!ctx || !ctx->priv)
+ return;
+
+ pw_ctx = ctx->priv;
+
+ if (id == PW_ID_CORE)
+ pw_thread_loop_signal(pw_ctx->thread_loop, false);
+}
+
+/**
+ * Callback to handle Pipewire core error events
+ *
+ * @param user_data pointer to the filter's AVFilterContext
+ * @param id PipeWire object id of calling
+ * @param seq PipeWire object sequence
+ * @param res error number
+ * @param message error message
+ */
+static void on_core_error_callback(void *user_data, uint32_t id, int seq,
+ int res, const char *message)
+{
+ AVFilterContext *ctx = user_data;
+ PipewireGrabContext *pw_ctx;
+
+ if (!ctx)
+ return;
+
+ av_log(ctx, AV_LOG_ERROR,
+ "PipeWire core error: %s (id=%u, seq=%d, res=%d: %s)\n",
+ message, id, seq, res, g_strerror(-res));
+
+ pw_ctx = ctx->priv;
+ if (!pw_ctx) {
+ av_log(ctx, AV_LOG_ERROR,
+ "Invalid private context data\n");
+ return;
+ }
+
+ pw_thread_loop_signal(pw_ctx->thread_loop, false);
+
+ pw_ctx->pipewire_error = res;
+ atomic_store(&pw_ctx->pipewire_initialization_over, 1);
+ pthread_cond_signal(&pw_ctx->pipewire_initialization_cond_var);
+}
+
+/**
+ * PipeWire core events callbacks
+ */
+static const struct pw_core_events core_events = {
+ PW_VERSION_CORE_EVENTS,
+ .info = on_core_info_callback,
+ .done = on_core_done_callback,
+ .error = on_core_error_callback,
+};
+
+/**
+ * Helper function: convert spa video format to AVPixelFormat
+ *
+ * @param video_format spa video format to convert
+ * @return the corresponding AVPixelFormat
+ */
+static enum AVPixelFormat
+spa_video_format_to_av_pixel_format(enum spa_video_format video_format)
+{
+ switch (video_format) {
+ case SPA_VIDEO_FORMAT_RGBA:
+ case SPA_VIDEO_FORMAT_RGBx:
+ return AV_PIX_FMT_RGBA;
+
+ case SPA_VIDEO_FORMAT_BGRA:
+ case SPA_VIDEO_FORMAT_BGRx:
+ return AV_PIX_FMT_BGRA;
+
+ default:
+ return AV_PIX_FMT_NONE;
+ }
+}
+
+/**
+ * Callback to free a DbusCallData object's memory and unsubscribe from the
+ * associated dbus signal.
+ *
+ * @param ptr_dbus_call_data DBus marshalling structure
+ */
+static void dbus_call_data_free(struct DbusCallData *ptr_dbus_call_data)
+{
+ AVFilterContext *ctx;
+ PipewireGrabContext *pw_ctx;
+
+ if (!ptr_dbus_call_data)
+ return;
+
+ ctx = ptr_dbus_call_data->ctx;
+ if (!ctx || !ctx->priv)
+ return;
+
+ pw_ctx = ctx->priv;
+
+ if (ptr_dbus_call_data->signal_id)
+ g_dbus_connection_signal_unsubscribe(pw_ctx->connection,
+ ptr_dbus_call_data->signal_id);
+
+ if (ptr_dbus_call_data->cancelled_id > 0)
+ g_signal_handler_disconnect(pw_ctx->cancellable,
+ ptr_dbus_call_data->cancelled_id);
+
+ g_clear_pointer(&ptr_dbus_call_data->request_path, g_free);
+ av_free(ptr_dbus_call_data);
+}
+
+/**
+ * DBus callback of cancelled events
+ *
+ * @param cancellable (not used)
+ * @param user_data DBus marshalling structure
+ */
+static void on_cancelled_callback(GCancellable *cancellable, gpointer user_data)
+{
+ struct DbusCallData *ptr_dbus_call_data = user_data;
+ AVFilterContext *ctx = ptr_dbus_call_data->ctx;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+ if (!pw_ctx)
+ return;
+
+ g_dbus_connection_call(pw_ctx->connection, "org.freedesktop.portal.Desktop",
+ ptr_dbus_call_data->request_path,
+ "org.freedesktop.portal.Request", "Close", NULL,
+ NULL, G_DBUS_CALL_FLAGS_NONE, -1, NULL, NULL, NULL);
+
+ av_log(ctx, AV_LOG_WARNING, "Portal request cancelled\n");
+
+ pw_ctx->portal_error = ECANCELED;
+ g_main_loop_quit(pw_ctx->glib_main_loop);
+}
+
+/**
+ * PipeWire callback of parameters changed events
+ *
+ * @param user_data DBus marshalling structure
+ * @param id contains chan param type
+ * @param param pointer to changed param structure
+ */
+static void on_stream_param_changed_callback(void *user_data, uint32_t id,
+ const struct spa_pod *param)
+{
+ struct spa_pod_builder pod_builder;
+ const struct spa_pod *params[MAX_SPA_PARAM];
+ uint32_t n_params = 0;
+ uint8_t params_buffer[4096];
+ int result;
+ PipewireGrabContext *pw_ctx;
+ AVFilterContext *ctx = user_data;
+
+ if (!ctx || !ctx->priv || !param)
+ return;
+
+ if (id != SPA_PARAM_Format) {
+ av_log(ctx, AV_LOG_WARNING,
+ "Ignoring non-Format param change\n");
+ return;
+ }
+
+ pw_ctx = ctx->priv;
+
+ result = spa_format_parse(param, &pw_ctx->format.media_type,
+ &pw_ctx->format.media_subtype);
+ if (result < 0) {
+ av_log(ctx, AV_LOG_ERROR, "Unable to parse media type\n");
+ pw_ctx->pipewire_error = AVERROR(EINVAL);
+ goto end;
+ }
+
+ if (pw_ctx->format.media_type != SPA_MEDIA_TYPE_video ||
+ pw_ctx->format.media_subtype != SPA_MEDIA_SUBTYPE_raw) {
+ av_log(ctx, AV_LOG_ERROR, "Unexpected media type\n");
+ pw_ctx->pipewire_error = AVERROR(EINVAL);
+ goto end;
+ }
+
+ spa_format_video_raw_parse(param, &pw_ctx->format.info.raw);
+
+ av_log(ctx, AV_LOG_INFO, "Negotiated format:\n");
+
+ av_log(ctx, AV_LOG_INFO, "Format: %d (%s)\n",
+ pw_ctx->format.info.raw.format,
+ spa_debug_type_find_name(spa_type_video_format,
+ pw_ctx->format.info.raw.format));
+ av_log(ctx, AV_LOG_INFO, "Size: %dx%d\n",
+ pw_ctx->format.info.raw.size.width,
+ pw_ctx->format.info.raw.size.height);
+ av_log(ctx, AV_LOG_INFO, "Framerate: %d/%d\n",
+ pw_ctx->format.info.raw.framerate.num,
+ pw_ctx->format.info.raw.framerate.denom);
+
+ pw_ctx->width = pw_ctx->format.info.raw.size.width;
+ pw_ctx->height = pw_ctx->format.info.raw.size.height;
+ pw_ctx->Bpp = BYTES_PER_PIXEL;
+ pw_ctx->frame_size = pw_ctx->width * pw_ctx->height * pw_ctx->Bpp;
+ if (pw_ctx->frame_size + AV_INPUT_BUFFER_PADDING_SIZE > INT_MAX) {
+ av_log(ctx, AV_LOG_ERROR, "Captured area is too large\n");
+ pw_ctx->pipewire_error = AVERROR(EINVAL);
+ goto end;
+ }
+
+ pw_ctx->av_pxl_format =
+ spa_video_format_to_av_pixel_format(pw_ctx->format.info.raw.format);
+ if (pw_ctx->av_pxl_format == AV_PIX_FMT_NONE) {
+ av_log(ctx, AV_LOG_ERROR,
+ "Unsupported buffer format: %d\n", pw_ctx->format.info.raw.format);
+ pw_ctx->pipewire_error = AVERROR(EINVAL);
+ goto end;
+ }
+
+ /* Video crop */
+ pod_builder = SPA_POD_BUILDER_INIT(params_buffer, sizeof(params_buffer));
+ params[n_params++] = spa_pod_builder_add_object(
+ &pod_builder, SPA_TYPE_OBJECT_ParamMeta, SPA_PARAM_Meta,
+ SPA_PARAM_META_type, SPA_POD_Id(SPA_META_VideoCrop),
+ SPA_PARAM_META_size, SPA_POD_Int(sizeof(struct spa_meta_region)));
+
+ /* Cursor */
+ params[n_params++] = spa_pod_builder_add_object(
+ &pod_builder, SPA_TYPE_OBJECT_ParamMeta, SPA_PARAM_Meta,
+ SPA_PARAM_META_type, SPA_POD_Id(SPA_META_Cursor), SPA_PARAM_META_size,
+ SPA_POD_CHOICE_RANGE_Int(CURSOR_META_SIZE(64, 64),
+ CURSOR_META_SIZE(1, 1),
+ CURSOR_META_SIZE(1024, 1024)));
+
+ /* Buffer options */
+ params[n_params++] = spa_pod_builder_add_object(
+ &pod_builder, SPA_TYPE_OBJECT_ParamBuffers, SPA_PARAM_Buffers,
+ SPA_PARAM_BUFFERS_dataType,
+ SPA_POD_Int((1 << SPA_DATA_MemPtr) | (1 << SPA_DATA_MemFd)));
+
+ /* Meta header */
+ params[n_params++] = spa_pod_builder_add_object(
+ &pod_builder, SPA_TYPE_OBJECT_ParamMeta, SPA_PARAM_Meta,
+ SPA_PARAM_META_type, SPA_POD_Id(SPA_META_Header),
+ SPA_PARAM_META_size,
+ SPA_POD_Int(sizeof(struct spa_meta_header)));
+
+ pw_stream_update_params(pw_ctx->stream, params, n_params);
+
+end:
+ // Signal pipewiregrab_init that PipeWire initialization is over (either
+ // because it was completed successfully or because there was an error, in
+ // which case pw_ctx->pipewire_error will have been set to a nonzero value).
+ atomic_store(&pw_ctx->pipewire_initialization_over, 1);
+ pthread_cond_signal(&pw_ctx->pipewire_initialization_cond_var);
+}
+
+/**
+ * PipeWire callback of state changed events
+ *
+ * @param user_data DBus marshalling structure
+ * @param old PipeWire stream old state
+ * @param state PipeWire stream current state
+ * @param error received error information
+ */
+static void on_stream_state_changed_callback(void *user_data,
+ enum pw_stream_state old,
+ enum pw_stream_state state,
+ const char *error)
+{
+ AVFilterContext *ctx = user_data;
+ if (!ctx)
+ return;
+
+ av_log(ctx, AV_LOG_INFO, "stream state: \"%s\"\n",
+ pw_stream_state_as_string(state));
+}
+
+/**
+ * Find most recent buffer received in a PipeWire stream
+ *
+ * @param stream stream to get buffer from
+ * @return most recent buffer in the stream
+ */
+static struct pw_buffer *find_most_recent_buffer_and_recycle_olders(struct pw_stream *stream)
+{
+ struct pw_buffer *pw_buf = NULL;
+ while (1) {
+ struct pw_buffer *aux = pw_stream_dequeue_buffer(stream);
+ if (!aux)
+ break;
+ if (pw_buf)
+ pw_stream_queue_buffer(stream, pw_buf);
+ pw_buf = aux;
+ }
+ return pw_buf;
+}
+
+/**
+ * Our data processing function
+ *
+ * @param user_data DBus marshalling structure
+ */
+static void on_stream_process_callback(void *user_data)
+{
+ struct spa_buffer *spa_buf = NULL;
+ struct pw_buffer *pw_buf = NULL;
+ uint8_t *map = NULL;
+ void *sdata = NULL;
+ struct spa_meta_header *header = NULL;
+ AVFilterContext *ctx = user_data;
+ PipewireGrabContext *pw_ctx = NULL;
+
+ if (!ctx || !ctx->priv)
+ return;
+
+ pw_ctx = ctx->priv;
+
+ // We need to wait for pw_ctx->current_frame to have been allocated before
+ // we can use it to get frames from the PipeWire thread to FFmpeg
+ pthread_mutex_lock(&pw_ctx->current_frame_mutex);
+ if (!pw_ctx->current_frame) {
+ pthread_mutex_unlock(&pw_ctx->current_frame_mutex);
+ return;
+ }
+ pthread_mutex_unlock(&pw_ctx->current_frame_mutex);
+
+ // Get a buffer from PipeWire
+ pw_buf = find_most_recent_buffer_and_recycle_olders(pw_ctx->stream);
+ if (!pw_buf) {
+ av_log(ctx, AV_LOG_ERROR, "Out of buffers\n");
+ return;
+ }
+
+ // Check whether the buffer is corrupted
+ spa_buf = pw_buf->buffer;
+ header = spa_buffer_find_meta_data(spa_buf,
+ SPA_META_Header, sizeof(*header));
+ if (header && (header->flags & SPA_META_HEADER_FLAG_CORRUPTED)) {
+ av_log(ctx, AV_LOG_ERROR, "Corrupted PipeWire buffer");
+ goto end;
+ }
+
+ // Get a pointer to the buffer's data
+ if (spa_buf->datas[0].type == SPA_DATA_MemFd ) {
+ map = mmap(NULL, spa_buf->datas[0].maxsize + spa_buf->datas[0].mapoffset,
+ PROT_READ, MAP_PRIVATE, spa_buf->datas[0].fd, 0);
+ if (map == MAP_FAILED) {
+ av_log(ctx, AV_LOG_ERROR, "mmap failed! error %s\n", g_strerror(errno));
+ goto end;
+ }
+ sdata = SPA_PTROFF(map, spa_buf->datas[0].mapoffset, uint8_t);
+ } else if (spa_buf->datas[0].type == SPA_DATA_MemPtr) {
+ if (spa_buf->datas[0].data == NULL) {
+ av_log(ctx, AV_LOG_ERROR, "No data\n");
+ goto end;
+ }
+ map = NULL;
+ sdata = spa_buf->datas[0].data;
+ } else {
+ av_log(ctx, AV_LOG_ERROR, "Buffer is not valid\n");
+ goto end;
+ }
+
+ // Update current_frame with the new data
+ pthread_mutex_lock(&pw_ctx->current_frame_mutex);
+ memcpy(pw_ctx->current_frame->data[0], sdata, pw_ctx->frame_size);
+ pthread_mutex_unlock(&pw_ctx->current_frame_mutex);
+
+ // Cleanup
+ if (spa_buf->datas[0].type == SPA_DATA_MemFd)
+ munmap(map, spa_buf->datas[0].maxsize + spa_buf->datas[0].mapoffset);
+end:
+ pw_stream_queue_buffer(pw_ctx->stream, pw_buf);
+ return;
+}
+
+static const struct pw_stream_events stream_events = {
+ PW_VERSION_STREAM_EVENTS,
+ .state_changed = on_stream_state_changed_callback,
+ .param_changed = on_stream_param_changed_callback,
+ .process = on_stream_process_callback,
+};
+
+static struct DbusCallData *subscribe_to_signal(AVFilterContext *ctx,
+ const char *path,
+ GDBusSignalCallback callback)
+{
+ struct DbusCallData *ptr_dbus_call_data;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+
+ ptr_dbus_call_data = (struct DbusCallData *)av_mallocz(sizeof(struct DbusCallData));
+ if (!ptr_dbus_call_data)
+ return NULL;
+
+ ptr_dbus_call_data->ctx = ctx;
+ ptr_dbus_call_data->request_path = g_strdup(path);
+ ptr_dbus_call_data->cancelled_id =
+ g_signal_connect(pw_ctx->cancellable, "cancelled",
+ G_CALLBACK(on_cancelled_callback),
+ ptr_dbus_call_data /* user_data */);
+ ptr_dbus_call_data->signal_id = g_dbus_connection_signal_subscribe(
+ pw_ctx->connection, "org.freedesktop.portal.Desktop" /*sender*/,
+ "org.freedesktop.portal.Request" /*interface_name*/,
+ "Response" /*member: dbus signal name*/,
+ ptr_dbus_call_data->request_path /*object_path*/, NULL,
+ G_DBUS_SIGNAL_FLAGS_NO_MATCH_RULE, callback, ptr_dbus_call_data, NULL);
+
+ return ptr_dbus_call_data;
+}
+
+static int play_pipewire_stream(AVFilterContext *ctx)
+{
+ int ret;
+ const struct spa_pod *ptr_spa_pod;
+ uint8_t buffer[4096];
+ struct spa_pod_builder spa_pod_bldr = {
+ 0,
+ };
+
+ PipewireGrabContext *pw_ctx = ctx->priv;
+
+ pw_init(NULL, NULL);
+ pw_ctx->pw_init_called = 1;
+
+ pw_ctx->thread_loop =
+ pw_thread_loop_new("thread loop", NULL);
+ if (!pw_ctx->thread_loop) {
+ av_log(ctx, AV_LOG_ERROR, "pw_thread_loop_new failed\n");
+ return AVERROR(ENOMEM);
+ }
+
+ pw_ctx->context =
+ pw_context_new(pw_thread_loop_get_loop(pw_ctx->thread_loop), NULL, 0);
+ if (!pw_ctx->context) {
+ av_log(ctx, AV_LOG_ERROR, "pw_context_new failed\n");
+ ret = AVERROR(ENOMEM);
+ goto fail;
+ }
+
+ if (pw_thread_loop_start(pw_ctx->thread_loop) < 0) {
+ av_log(ctx, AV_LOG_ERROR, "pw_thread_loop_start failed\n");
+ ret = AVERROR(EFAULT);
+ goto fail;
+ }
+
+ pw_thread_loop_lock(pw_ctx->thread_loop);
+
+ // Core
+ pw_ctx->core =
+ pw_context_connect_fd(pw_ctx->context,
+ fcntl(pw_ctx->pipewire_fd, F_DUPFD_CLOEXEC, 3),
+ NULL, 0);
+ if (!pw_ctx->core) {
+ ret = AVERROR(errno);
+ av_log(ctx, AV_LOG_ERROR, "pw_context_connect_fd failed\n");
+ pw_thread_loop_unlock(pw_ctx->thread_loop);
+ goto fail;
+ }
+
+ pw_core_add_listener(pw_ctx->core, &pw_ctx->core_listener, &core_events,
+ ctx /* user_data */);
+
+ // Stream
+ pw_ctx->stream = pw_stream_new(
+ pw_ctx->core, "wayland grab",
+ pw_properties_new(PW_KEY_MEDIA_TYPE, "Video", PW_KEY_MEDIA_CATEGORY,
+ "Capture", PW_KEY_MEDIA_ROLE, "Screen", NULL));
+
+ if (!pw_ctx->stream) {
+ av_log(ctx, AV_LOG_ERROR, "pw_stream_new failed\n");
+ ret = AVERROR(ENOMEM);
+ pw_thread_loop_unlock(pw_ctx->thread_loop);
+ goto fail;
+ }
+
+ pw_stream_add_listener(pw_ctx->stream, &pw_ctx->stream_listener,
+ &stream_events, ctx /* user_data */);
+
+ // Stream parameters
+ spa_pod_bldr = SPA_POD_BUILDER_INIT(buffer, sizeof(buffer));
+ ptr_spa_pod = spa_pod_builder_add_object(
+ &spa_pod_bldr, SPA_TYPE_OBJECT_Format, SPA_PARAM_EnumFormat,
+ SPA_FORMAT_mediaType, SPA_POD_Id(SPA_MEDIA_TYPE_video),
+ SPA_FORMAT_mediaSubtype, SPA_POD_Id(SPA_MEDIA_SUBTYPE_raw),
+ SPA_FORMAT_VIDEO_format,
+ SPA_POD_CHOICE_ENUM_Id(4, SPA_VIDEO_FORMAT_RGBA, SPA_VIDEO_FORMAT_RGBx,
+ SPA_VIDEO_FORMAT_BGRx, SPA_VIDEO_FORMAT_BGRA),
+ SPA_FORMAT_VIDEO_size,
+ SPA_POD_CHOICE_RANGE_Rectangle(&SPA_RECTANGLE(320, 240),
+ &SPA_RECTANGLE(1, 1),
+ &SPA_RECTANGLE(4096, 4096)),
+ SPA_FORMAT_VIDEO_framerate,
+ SPA_POD_CHOICE_RANGE_Fraction(
+ &SPA_FRACTION(pw_ctx->framerate.num,
+ pw_ctx->framerate.den),
+ &SPA_FRACTION(0, 1), &SPA_FRACTION(144, 1)));
+
+ ret = pw_stream_connect(
+ pw_ctx->stream, PW_DIRECTION_INPUT, pw_ctx->pipewire_node,
+ PW_STREAM_FLAG_AUTOCONNECT | PW_STREAM_FLAG_MAP_BUFFERS, &ptr_spa_pod, 1);
+ if (ret != 0) {
+ av_log(ctx, AV_LOG_ERROR, "pw_stream_connect failed\n");
+ pw_thread_loop_unlock(pw_ctx->thread_loop);
+ goto fail;
+ }
+
+ av_log(ctx, AV_LOG_INFO, "Starting screen capture ...\n");
+ pw_thread_loop_unlock(pw_ctx->thread_loop);
+ return 0;
+
+fail:
+ if (pw_ctx->core) {
+ pw_core_disconnect(pw_ctx->core);
+ pw_ctx->core = NULL;
+ }
+ if (pw_ctx->context) {
+ pw_context_destroy(pw_ctx->context);
+ pw_ctx->context = NULL;
+ }
+ if (pw_ctx->thread_loop) {
+ pw_thread_loop_destroy(pw_ctx->thread_loop);
+ pw_ctx->thread_loop = NULL;
+ }
+
+ return ret;
+}
+
+static void portal_open_pipewire_remote(AVFilterContext *ctx)
+{
+ GUnixFDList* fd_list = NULL;
+ GVariant* result = NULL;
+ GError* error = NULL;
+ int fd_index;
+ GVariantBuilder builder;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+
+ g_variant_builder_init(&builder, G_VARIANT_TYPE_VARDICT);
+
+ result = g_dbus_proxy_call_with_unix_fd_list_sync(
+ pw_ctx->proxy, "OpenPipeWireRemote",
+ g_variant_new("(oa{sv})", pw_ctx->session_handle, &builder),
+ G_DBUS_CALL_FLAGS_NONE, -1, NULL, &fd_list, pw_ctx->cancellable,
+ &error);
+ if (error)
+ goto fail;
+
+ g_variant_get(result, "(h)", &fd_index);
+ g_variant_unref(result);
+
+ pw_ctx->pipewire_fd = g_unix_fd_list_get(fd_list, fd_index, &error);
+ g_object_unref(fd_list);
+ if (error)
+ goto fail;
+
+ g_main_loop_quit(pw_ctx->glib_main_loop);
+ return;
+
+fail:
+ av_log(ctx, AV_LOG_ERROR,
+ "Error retrieving PipeWire fd: %s\n", error->message);
+ g_error_free(error);
+ portal_abort(ctx, EIO, "Failed to open PipeWire remote");
+}
+
+static void on_start_response_received_callback(
+ GDBusConnection *connection, const char *sender_name,
+ const char *object_path, const char *interface_name,
+ const char *signal_name, GVariant *parameters, gpointer user_data)
+{
+ GVariant* stream_properties = NULL;
+ GVariant* streams = NULL;
+ GVariant* result = NULL;
+ GVariantIter iter;
+ uint32_t response;
+
+ struct DbusCallData *ptr_dbus_call_data = user_data;
+ AVFilterContext *ctx = ptr_dbus_call_data->ctx;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+ if (!pw_ctx) {
+ portal_abort(ctx, EINVAL, "Invalid private context data");
+ return;
+ }
+
+ g_clear_pointer(&ptr_dbus_call_data, dbus_call_data_free);
+
+ g_variant_get(parameters, "(u@a{sv})", &response, &result);
+
+ if (response) {
+ g_variant_unref(result);
+ portal_abort(
+ ctx, EACCES, "Failed to start screencast, denied or cancelled by user");
+ return;
+ }
+
+ streams = g_variant_lookup_value(result, "streams", G_VARIANT_TYPE_ARRAY);
+
+ g_variant_iter_init(&iter, streams);
+ av_assert0(g_variant_iter_n_children(&iter) == 1);
+
+ g_variant_iter_loop(&iter, "(u@a{sv})", &pw_ctx->pipewire_node,
+ &stream_properties);
+
+ av_log(ctx, AV_LOG_INFO, "Monitor selected, setting up screencast\n\n");
+
+ g_variant_unref(result);
+ g_variant_unref(streams);
+ g_variant_unref(stream_properties);
+
+ portal_open_pipewire_remote(ctx);
+}
+
+static int portal_call_dbus_method(AVFilterContext *ctx,
+ const gchar *method_name, GVariant *parameters)
+{
+ GVariant* result;
+ GError* error = NULL;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+
+ result = g_dbus_proxy_call_sync(pw_ctx->proxy, method_name, parameters,
+ G_DBUS_CALL_FLAGS_NONE, -1,
+ pw_ctx->cancellable, &error);
+ if (error) {
+ av_log(ctx, AV_LOG_ERROR,
+ "Call to DBus method '%s' failed: %s\n",
+ method_name, error->message);
+ g_error_free(error);
+ return EIO;
+ }
+ g_variant_unref(result);
+ return 0;
+}
+
+static void portal_start(AVFilterContext *ctx)
+{
+ int ret;
+ const char *request_token;
+ g_autofree char *request_path;
+ GVariantBuilder builder;
+ GVariant *parameters;
+ struct DbusCallData *ptr_dbus_call_data;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+ if (!pw_ctx) {
+ portal_abort(ctx, EINVAL, "Invalid private context data");
+ return;
+ }
+
+ request_token = "pipewiregrabStart";
+ request_path = g_strdup_printf(REQUEST_PATH, pw_ctx->sender_name, request_token);
+
+ av_log(ctx, AV_LOG_INFO, "Asking for monitor…\n");
+
+ ptr_dbus_call_data = subscribe_to_signal(ctx, request_path,
+ on_start_response_received_callback);
+ if (!ptr_dbus_call_data) {
+ portal_abort(ctx, ENOMEM, "Failed to allocate DBus call data");
+ return;
+ }
+
+ g_variant_builder_init(&builder, G_VARIANT_TYPE_VARDICT);
+ g_variant_builder_add(&builder, "{sv}", "handle_token",
+ g_variant_new_string(request_token));
+ parameters = g_variant_new("(osa{sv})", pw_ctx->session_handle, "", &builder);
+
+ ret = portal_call_dbus_method(ctx, "Start", parameters);
+ if (ret != 0)
+ portal_abort(ctx, ret, "Failed to start screen cast session");
+}
+
+static void on_select_sources_response_received_callback(
+ GDBusConnection *connection, const char *sender_name,
+ const char *object_path, const char *interface_name,
+ const char *signal_name, GVariant *parameters, gpointer user_data)
+{
+ GVariant* ret = NULL;
+ uint32_t response;
+ struct DbusCallData *ptr_dbus_call_data = user_data;
+ AVFilterContext *ctx = ptr_dbus_call_data->ctx;
+
+ av_log(ctx, AV_LOG_INFO,
+ "Response to select source received\n");
+
+ g_clear_pointer(&ptr_dbus_call_data, dbus_call_data_free);
+
+ g_variant_get(parameters, "(u@a{sv})", &response, &ret);
+ g_variant_unref(ret);
+ if (response) {
+ portal_abort(
+ ctx, EACCES, "Failed to select screencast sources, denied or cancelled by user");
+ return;
+ }
+
+ portal_start(ctx);
+}
+
+static void portal_select_sources(AVFilterContext *ctx)
+{
+ int ret;
+ const char *request_token;
+ g_autofree char *request_path;
+ GVariantBuilder builder;
+ GVariant *parameters;
+ struct DbusCallData *ptr_dbus_call_data;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+
+ request_token = "pipewiregrabSelectSources";
+ request_path = g_strdup_printf(REQUEST_PATH, pw_ctx->sender_name, request_token);
+
+ ptr_dbus_call_data = subscribe_to_signal(ctx, request_path,
+ on_select_sources_response_received_callback);
+ if (!ptr_dbus_call_data) {
+ portal_abort(ctx, ENOMEM, "Failed to allocate DBus call data");
+ return;
+ }
+
+ g_variant_builder_init(&builder, G_VARIANT_TYPE_VARDICT);
+ g_variant_builder_add(&builder, "{sv}", "types",
+ g_variant_new_uint32(pw_ctx->capture_type));
+ g_variant_builder_add(&builder, "{sv}", "multiple",
+ g_variant_new_boolean(FALSE));
+ g_variant_builder_add(&builder, "{sv}", "handle_token",
+ g_variant_new_string(request_token));
+
+ if ((pw_ctx->available_cursor_modes & PORTAL_CURSOR_MODE_EMBEDDED)
+ && pw_ctx->draw_mouse)
+ g_variant_builder_add(&builder, "{sv}", "cursor_mode",
+ g_variant_new_uint32(PORTAL_CURSOR_MODE_EMBEDDED));
+ else
+ g_variant_builder_add(&builder, "{sv}", "cursor_mode",
+ g_variant_new_uint32(PORTAL_CURSOR_MODE_HIDDEN));
+ parameters = g_variant_new("(oa{sv})", pw_ctx->session_handle, &builder);
+
+ ret = portal_call_dbus_method(ctx, "SelectSources", parameters);
+ if (ret != 0)
+ portal_abort(ctx, ret, "Failed to select sources for screen cast session");
+}
+
+static void on_create_session_response_received_callback(
+ GDBusConnection *connection, const char *sender_name,
+ const char *object_path, const char *interface_name,
+ const char *signal_name, GVariant *parameters, gpointer user_data)
+{
+ uint32_t response;
+ GVariant* result = NULL;
+ struct DbusCallData *ptr_dbus_call_data = user_data;
+ AVFilterContext *ctx = ptr_dbus_call_data->ctx;
+
+ PipewireGrabContext *pw_ctx = ctx->priv;
+ if (!pw_ctx) {
+ portal_abort(ctx, EINVAL, "Invalid private context data");
+ return;
+ }
+
+ g_clear_pointer(&ptr_dbus_call_data, dbus_call_data_free);
+
+ g_variant_get(parameters, "(u@a{sv})", &response, &result);
+
+ if (response != 0) {
+ g_variant_unref(result);
+ portal_abort(
+ ctx, EACCES, "Failed to create screencast session, denied or cancelled by user");
+ return;
+ }
+
+ av_log(ctx, AV_LOG_DEBUG, "Screencast session created\n");
+
+ g_variant_lookup(result, "session_handle", "s", &pw_ctx->session_handle);
+ g_variant_unref(result);
+
+ portal_select_sources(ctx);
+}
+
+/**
+ * Function to create a screen cast session
+ *
+ * @param ctx
+ */
+static void portal_create_session(AVFilterContext *ctx)
+{
+ int ret;
+ GVariantBuilder builder;
+ GVariant *parameters;
+ const char *request_token;
+ g_autofree char *request_path;
+ struct DbusCallData *ptr_dbus_call_data;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+
+ request_token = "pipewiregrabCreateSession";
+ request_path = g_strdup_printf(REQUEST_PATH, pw_ctx->sender_name, request_token);
+
+ ptr_dbus_call_data = subscribe_to_signal(ctx, request_path,
+ on_create_session_response_received_callback);
+ if (!ptr_dbus_call_data) {
+ portal_abort(ctx, ENOMEM, "Failed to allocate DBus call data");
+ return;
+ }
+
+ g_variant_builder_init(&builder, G_VARIANT_TYPE_VARDICT);
+ g_variant_builder_add(&builder, "{sv}", "handle_token",
+ g_variant_new_string(request_token));
+ g_variant_builder_add(&builder, "{sv}", "session_handle_token",
+ g_variant_new_string("pipewiregrab"));
+ parameters = g_variant_new("(a{sv})", &builder);
+
+ ret = portal_call_dbus_method(ctx, "CreateSession", parameters);
+ if (ret != 0)
+ portal_abort(ctx, ret, "Failed to create screen cast session");
+}
+
+/**
+ * Helper function: get available cursor modes and update the
+ * PipewireGrabContext accordingly
+ *
+ * @param ctx
+ */
+static void portal_update_available_cursor_modes(AVFilterContext *ctx)
+{
+ GVariant* cached_cursor_modes = NULL;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+
+ cached_cursor_modes =
+ g_dbus_proxy_get_cached_property(pw_ctx->proxy, "AvailableCursorModes");
+
+ pw_ctx->available_cursor_modes =
+ cached_cursor_modes ? g_variant_get_uint32(cached_cursor_modes) : 0;
+
+ g_variant_unref(cached_cursor_modes);
+}
+
+static int create_dbus_proxy(AVFilterContext *ctx)
+{
+ GError* error = NULL;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+
+ pw_ctx->proxy = g_dbus_proxy_new_sync(
+ pw_ctx->connection, G_DBUS_PROXY_FLAGS_NONE, NULL,
+ "org.freedesktop.portal.Desktop",
+ "/org/freedesktop/portal/desktop",
+ "org.freedesktop.portal.ScreenCast", NULL, &error);
+ if (error) {
+ av_log(ctx, AV_LOG_ERROR,
+ "Error creating proxy: %s\n", error->message);
+ g_error_free(error);
+ return EPERM;
+ }
+ return 0;
+}
+
+/**
+ * Create DBus connection and related objects
+ *
+ * @param ctx
+ */
+static int create_dbus_connection(AVFilterContext *ctx)
+{
+ char *aux;
+ GError* error = NULL;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+
+ pw_ctx->cancellable = g_cancellable_new();
+
+ pw_ctx->connection = g_bus_get_sync(G_BUS_TYPE_SESSION, NULL, &error);
+ if (error) {
+ av_log(ctx, AV_LOG_ERROR,
+ "Error getting session bus: %s\n", error->message);
+ g_error_free(error);
+ return EPERM;
+ }
+
+ aux = g_strdup(g_dbus_connection_get_unique_name(pw_ctx->connection) + 1);
+ pw_ctx->sender_name = av_strireplace(aux, ".", "_");
+ av_log(ctx, AV_LOG_DEBUG, "Initialized (sender name: %s)\n",
+ pw_ctx->sender_name);
+ return 0;
+}
+
+
+/**
+ * Use XDG Desktop Portal's ScreenCast interface to open a file descriptor that
+ * can be used by PipeWire to access the screen cast streams.
+ * (https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html)
+ *
+ * @param ctx
+ */
+static int portal_init_screencast(AVFilterContext *ctx)
+{
+ int ret = 0;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+ GMainContext *glib_main_context;
+
+ // Create a new GLib context and set it as the default for the current thread.
+ // This ensures that the callbacks from DBus operations started in this thread are
+ // handled by the GLib main loop defined below, even if pipewiregrab_init was
+ // called by a program which also uses GLib and already had its own main loop running.
+ glib_main_context = g_main_context_new();
+ g_main_context_push_thread_default(glib_main_context);
+ pw_ctx->glib_main_loop = g_main_loop_new(glib_main_context, FALSE);
+ if (!pw_ctx->glib_main_loop) {
+ av_log(ctx, AV_LOG_ERROR, "g_main_loop_new failed\n");
+ ret = ENOMEM;
+ }
+
+ ret = create_dbus_connection(ctx);
+ if (ret != 0)
+ goto exit_glib_loop;
+
+ ret = create_dbus_proxy(ctx);
+ if (ret != 0)
+ goto exit_glib_loop;
+
+ portal_update_available_cursor_modes(ctx);
+ portal_create_session(ctx);
+ if (pw_ctx->portal_error) {
+ ret = pw_ctx->portal_error;
+ goto exit_glib_loop;
+ }
+
+ g_main_loop_run(pw_ctx->glib_main_loop);
+ // The main loop will run until it's stopped by portal_open_pipewire_remote (if
+ // all DBus method calls were successfully), portal_abort (in case of error) or
+ // on_cancelled_callback (if a DBus request is cancelled).
+ // In the latter two cases, pw_ctx->portal_error gets set to a nonzero value.
+ if (pw_ctx->portal_error)
+ ret = pw_ctx->portal_error;
+
+exit_glib_loop:
+ g_main_loop_unref(pw_ctx->glib_main_loop);
+ pw_ctx->glib_main_loop = NULL;
+ g_main_context_pop_thread_default(glib_main_context);
+ g_main_context_unref(glib_main_context);
+
+ return AVERROR(ret);
+}
+
+static av_cold int pipewiregrab_init(AVFilterContext *ctx)
+{
+ int ret;
+ PipewireGrabContext *pw_ctx = ctx->priv;
+ if (!pw_ctx) {
+ av_log(ctx, AV_LOG_ERROR,
+ "Invalid private context data!\n");
+ return AVERROR(EINVAL);
+ }
+
+ atomic_init(&pw_ctx->pipewire_initialization_over, 0);
+ pthread_cond_init(&pw_ctx->pipewire_initialization_cond_var, NULL);
+ pthread_mutex_init(&pw_ctx->pipewire_initialization_mutex, NULL);
+ pthread_mutex_init(&pw_ctx->current_frame_mutex, NULL);
+
+ if (pw_ctx->pipewire_fd == 0) {
+ ret = portal_init_screencast(ctx);
+ if (ret != 0) {
+ atomic_store(&pw_ctx->pipewire_initialization_over, 1);
+ pthread_cond_signal(&pw_ctx->pipewire_initialization_cond_var);
+ return ret;
+ }
+ }
+
+ ret = play_pipewire_stream(ctx);
+ if (ret != 0)
+ return ret;
+
+ // Wait until PipeWire initialization is over
+ pthread_mutex_lock(&pw_ctx->pipewire_initialization_mutex);
+ while (!atomic_load(&pw_ctx->pipewire_initialization_over)) {
+ pthread_cond_wait(&pw_ctx->pipewire_initialization_cond_var,
+ &pw_ctx->pipewire_initialization_mutex);
+ }
+ pthread_mutex_unlock(&pw_ctx->pipewire_initialization_mutex);
+
+ if (pw_ctx->pipewire_error)
+ return pw_ctx->pipewire_error;
+
+ return 0;
+}
+
+static void pipewiregrab_uninit(AVFilterContext *ctx)
+{
+ PipewireGrabContext *pw_ctx = ctx->priv;
+ if (!pw_ctx)
+ return;
+
+ if (pw_ctx->glib_main_loop &&
+ g_main_loop_is_running(pw_ctx->glib_main_loop)) {
+ // Cancel ongoing DBus operation, if any
+ g_cancellable_cancel(pw_ctx->cancellable);
+ pthread_mutex_lock(&pw_ctx->pipewire_initialization_mutex);
+ while (!atomic_load(&pw_ctx->pipewire_initialization_over)) {
+ pthread_cond_wait(&pw_ctx->pipewire_initialization_cond_var,
+ &pw_ctx->pipewire_initialization_mutex);
+ }
+ pthread_mutex_unlock(&pw_ctx->pipewire_initialization_mutex);
+ }
+ g_clear_object(&pw_ctx->cancellable);
+
+ // PipeWire cleanup
+ if (pw_ctx->thread_loop) {
+ pw_thread_loop_signal(pw_ctx->thread_loop, false);
+ pw_thread_loop_unlock(pw_ctx->thread_loop);
+ pw_thread_loop_stop(pw_ctx->thread_loop);
+ }
+ if (pw_ctx->stream) {
+ pw_stream_disconnect(pw_ctx->stream);
+ g_clear_pointer(&pw_ctx->stream, pw_stream_destroy);
+ pw_ctx->stream = NULL;
+ }
+ if (pw_ctx->core){
+ pw_core_disconnect(pw_ctx->core);
+ pw_ctx->core = NULL;
+ }
+ if (pw_ctx->context) {
+ pw_context_destroy(pw_ctx->context);
+ pw_ctx->context = NULL;
+ }
+ if (pw_ctx->thread_loop) {
+ pw_thread_loop_destroy(pw_ctx->thread_loop);
+ pw_ctx->thread_loop = NULL;
+ }
+ if (pw_ctx->pw_init_called) {
+ pw_deinit();
+ pw_ctx->pw_init_called = 0;
+ }
+ if (pw_ctx->pipewire_fd > 0) {
+ close(pw_ctx->pipewire_fd);
+ pw_ctx->pipewire_fd = 0;
+ }
+ av_frame_free(&pw_ctx->current_frame);
+
+ // DBus cleanup
+ if (pw_ctx->session_handle) {
+ g_dbus_connection_call(
+ pw_ctx->connection, "org.freedesktop.portal.Desktop",
+ pw_ctx->session_handle, "org.freedesktop.portal.Session", "Close",
+ NULL, NULL, G_DBUS_CALL_FLAGS_NONE, -1, NULL, NULL, NULL);
+
+ g_clear_pointer(&pw_ctx->session_handle, g_free);
+ }
+ g_clear_object(&pw_ctx->connection);
+ g_clear_object(&pw_ctx->proxy);
+ g_clear_pointer(&pw_ctx->sender_name, g_free);
+}
+
+static int pipewiregrab_config_props(AVFilterLink *outlink)
+{
+ AVFrame *frame;
+ PipewireGrabContext *pw_ctx = outlink->src->priv;
+
+ AVRational time_base = av_inv_q(pw_ctx->framerate);
+ pw_ctx->frame_duration = av_rescale_q(1, time_base, AV_TIME_BASE_Q);
+ pw_ctx->time_frame = av_gettime_relative();
+
+ outlink->w = pw_ctx->width;
+ outlink->h = pw_ctx->height;
+ outlink->time_base = AV_TIME_BASE_Q;
+ outlink->frame_rate = pw_ctx->framerate;
+
+ frame = ff_get_video_buffer(outlink, pw_ctx->width, pw_ctx->height);
+ if (!frame)
+ return AVERROR(ENOMEM);
+ pthread_mutex_lock(&pw_ctx->current_frame_mutex);
+ pw_ctx->current_frame = frame;
+ pthread_mutex_unlock(&pw_ctx->current_frame_mutex);
+
+ return 0;
+}
+
+/**
+ * Helper function: calculate the wait time based
+ * on the frame duration
+ *
+ * @param pw_ctx
+ * @return current time
+ */
+static int64_t wait_frame(PipewireGrabContext *pw_ctx)
+{
+ int64_t curtime, delay;
+
+ /* Calculate the time of the next frame */
+ pw_ctx->time_frame += pw_ctx->frame_duration;
+
+ /* wait based on the frame rate */
+ while (1) {
+ curtime = av_gettime_relative();
+ delay = pw_ctx->time_frame - curtime;
+ if (delay <= 0)
+ break;
+ av_usleep(delay);
+ }
+
+ return curtime;
+}
+
+static int pipewiregrab_request_frame(AVFilterLink *outlink)
+{
+ int ret;
+ PipewireGrabContext *pw_ctx = outlink->src->priv;
+ AVFrame *frame = av_frame_alloc();
+ if (!frame)
+ return AVERROR(ENOMEM);
+
+ wait_frame(pw_ctx);
+
+ pthread_mutex_lock(&pw_ctx->current_frame_mutex);
+ ret = av_frame_ref(frame, pw_ctx->current_frame);
+ pthread_mutex_unlock(&pw_ctx->current_frame_mutex);
+ if (ret < 0) {
+ av_frame_free(&frame);
+ return ret;
+ }
+
+ frame->pts = av_gettime();
+ frame->duration = pw_ctx->frame_duration;
+ frame->sample_aspect_ratio = (AVRational) {1, 1};
+ frame->format = pw_ctx->av_pxl_format;
+
+ return ff_filter_frame(outlink, frame);
+}
+
+static int pipewiregrab_query_formats(AVFilterContext *ctx)
+{
+ PipewireGrabContext *pw_ctx = ctx->priv;
+ enum AVPixelFormat pix_fmts[] = {pw_ctx->av_pxl_format, AV_PIX_FMT_NONE};
+
+ return ff_set_common_formats_from_list(ctx, pix_fmts);
+}
+
+static const AVFilterPad pipewiregrab_outputs[] = {
+ {
+ .name = "default",
+ .type = AVMEDIA_TYPE_VIDEO,
+ .request_frame = pipewiregrab_request_frame,
+ .config_props = pipewiregrab_config_props,
+ },
+};
+
+const AVFilter ff_vsrc_pipewiregrab= {
+ .name = "pipewiregrab",
+ .description = NULL_IF_CONFIG_SMALL("Capture screen or window using PipeWire."),
+ .priv_size = sizeof(struct PipewireGrabContext),
+ .priv_class = &pipewiregrab_class,
+ .init = pipewiregrab_init,
+ .uninit = pipewiregrab_uninit,
+ .inputs = NULL,
+ FILTER_OUTPUTS(pipewiregrab_outputs),
+ FILTER_QUERY_FUNC(pipewiregrab_query_formats),
+};
--
2.34.1
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
reply other threads:[~2024-03-15 16:51 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240315165134.527481-1-francois-simon.fauteux-chapleau@savoirfairelinux.com \
--to=francois-simon.fauteux-chapleau@savoirfairelinux.com \
--cc=ffmpeg-devel@ffmpeg.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
This inbox may be cloned and mirrored by anyone:
git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
ffmpegdev@gitmailbox.com
public-inbox-index ffmpegdev
Example config snippet for mirrors.
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git