From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id CECCF42306 for ; Mon, 4 Apr 2022 11:41:06 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DBDA368B34D; Mon, 4 Apr 2022 14:38:01 +0300 (EEST) Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E3F1F68B359 for ; Mon, 4 Apr 2022 14:37:46 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id A0D2B24050B for ; Mon, 4 Apr 2022 13:37:46 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id 8zjOjZSNdjwP for ; Mon, 4 Apr 2022 13:37:45 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id B8E1D24056A for ; Mon, 4 Apr 2022 13:37:41 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id 52FF93A0E88; Mon, 4 Apr 2022 13:32:12 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Mon, 4 Apr 2022 13:30:36 +0200 Message-Id: <20220404113037.13070-49-anton@khirnov.net> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220404113037.13070-1-anton@khirnov.net> References: <20220404113037.13070-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 48/49] fftools: add a multistream thread-safe queue X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: It is similar to AVThreadMessageQueue, but supports multiple streams, each with its own EOF state. --- fftools/Makefile | 1 + fftools/thread_queue.c | 232 +++++++++++++++++++++++++++++++++++++++++ fftools/thread_queue.h | 37 +++++++ 3 files changed, 270 insertions(+) create mode 100644 fftools/thread_queue.c create mode 100644 fftools/thread_queue.h diff --git a/fftools/Makefile b/fftools/Makefile index f015df6846..b0de4f4c61 100644 --- a/fftools/Makefile +++ b/fftools/Makefile @@ -15,6 +15,7 @@ OBJS-ffmpeg += \ fftools/ffmpeg_mux.o \ fftools/ffmpeg_opt.o \ fftools/sync_queue.o \ + fftools/thread_queue.o \ define DOFFTOOL OBJS-$(1) += fftools/cmdutils.o fftools/opt_common.o fftools/$(1).o $(OBJS-$(1)-yes) diff --git a/fftools/thread_queue.c b/fftools/thread_queue.c new file mode 100644 index 0000000000..9ec02ca772 --- /dev/null +++ b/fftools/thread_queue.c @@ -0,0 +1,232 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include "libavutil/avassert.h" +#include "libavutil/error.h" +#include "libavutil/fifo.h" +#include "libavutil/mem.h" +#include "libavutil/thread.h" + +#include "thread_queue.h" + +enum { + FINISHED_SEND = (1 << 0), + FINISHED_RECV = (1 << 1), +}; + +typedef struct ThreadQueueStream { + AVFifo *fifo; + int finished; +} ThreadQueueStream; + +struct ThreadQueue { + ThreadQueueStream *streams; + unsigned int nb_streams; + + void (*elem_free)(void *); + + pthread_mutex_t lock; + pthread_cond_t cond; +}; + +void tq_free(ThreadQueue **ptq) +{ + ThreadQueue *tq = *ptq; + + if (!tq) + return; + + for (unsigned int i = 0; i < tq->nb_streams; i++) { + ThreadQueueStream *st = &tq->streams[i]; + + if (st->fifo && tq->elem_free) { + void *elem; + while (av_fifo_read(st->fifo, &elem, 1) >= 0) + tq->elem_free(elem); + } + av_fifo_freep2(&tq->streams[i].fifo); + } + av_freep(&tq->streams); + + pthread_cond_destroy(&tq->cond); + pthread_mutex_destroy(&tq->lock); + + av_freep(ptq); +} + +ThreadQueue *tq_alloc(unsigned int nb_streams, + size_t nb_elems, size_t elem_size, + void (*elem_free)(void *)) +{ + ThreadQueue *tq; + int ret; + + tq = av_mallocz(sizeof(*tq)); + if (!tq) + return NULL; + + tq->elem_free = elem_free; + + ret = pthread_cond_init(&tq->cond, NULL); + if (ret) { + av_freep(&tq); + return NULL; + } + + ret = pthread_mutex_init(&tq->lock, NULL); + if (ret) { + pthread_cond_destroy(&tq->cond); + av_freep(&tq); + return NULL; + } + + tq->streams = av_calloc(nb_streams, sizeof(*tq->streams)); + if (!tq->streams) + goto fail; + tq->nb_streams = nb_streams; + + for (unsigned int i = 0; i < nb_streams; i++) { + ThreadQueueStream *st = &tq->streams[i]; + + st->fifo = av_fifo_alloc2(nb_elems, elem_size, 0); + if (!st->fifo) + goto fail; + } + + return tq; +fail: + tq_free(&tq); + return NULL; +} + +int tq_send(ThreadQueue *tq, unsigned int stream_idx, void *data) +{ + ThreadQueueStream *st; + int ret; + + av_assert0(stream_idx < tq->nb_streams); + st = &tq->streams[stream_idx]; + + pthread_mutex_lock(&tq->lock); + + if (st->finished & FINISHED_SEND) { + ret = AVERROR(EINVAL); + goto finish; + } + + while (!(st->finished & FINISHED_RECV) && !av_fifo_can_write(st->fifo)) + pthread_cond_wait(&tq->cond, &tq->lock); + + if (st->finished & FINISHED_RECV) + ret = AVERROR_EOF; + else { + ret = av_fifo_write(st->fifo, data, 1); + if (ret >= 0) + pthread_cond_broadcast(&tq->cond); + } + +finish: + pthread_mutex_unlock(&tq->lock); + + return ret; +} + +static int receive_locked(ThreadQueue *tq, int *stream_idx, + void *data) +{ + unsigned int nb_finished = 0; + + for (unsigned int i = 0; i < tq->nb_streams; i++) { + ThreadQueueStream *st = &tq->streams[i]; + + if (av_fifo_read(st->fifo, data, 1) >= 0) { + *stream_idx = i; + return 0; + } + + if (st->finished & FINISHED_SEND) { + /* return EOF to the consumer at most once for each stream */ + if (!(st->finished & FINISHED_RECV)) { + st->finished |= FINISHED_RECV; + *stream_idx = i; + return AVERROR_EOF; + } + + nb_finished++; + } + } + + return nb_finished == tq->nb_streams ? AVERROR_EOF : AVERROR(EAGAIN); +} + +int tq_receive(ThreadQueue *tq, int *stream_idx, void *data) +{ + int ret; + + *stream_idx = -1; + + pthread_mutex_lock(&tq->lock); + + while (1) { + ret = receive_locked(tq, stream_idx, data); + if (ret == AVERROR(EAGAIN)) { + pthread_cond_wait(&tq->cond, &tq->lock); + continue; + } + + break; + } + + if (ret == 0) + pthread_cond_broadcast(&tq->cond); + + pthread_mutex_unlock(&tq->lock); + + return ret; +} + +void tq_send_finish(ThreadQueue *tq, unsigned int stream_idx) +{ + av_assert0(stream_idx < tq->nb_streams); + + pthread_mutex_lock(&tq->lock); + + /* mark the stream as send-finished; + * the next time the consumer thread tries to read this stream it will get + * and EOF and recv-finished flag will be set */ + tq->streams[stream_idx].finished |= FINISHED_SEND; + pthread_cond_broadcast(&tq->cond); + + pthread_mutex_unlock(&tq->lock); +} + +void tq_receive_finish(ThreadQueue *tq, unsigned int stream_idx) +{ + av_assert0(stream_idx < tq->nb_streams); + + pthread_mutex_lock(&tq->lock); + + /* mark the stream as send/recv-finished; + * both send AND recvs for this stream will now return EOF */ + tq->streams[stream_idx].finished |= FINISHED_SEND | FINISHED_RECV; + pthread_cond_broadcast(&tq->cond); + + pthread_mutex_unlock(&tq->lock); +} diff --git a/fftools/thread_queue.h b/fftools/thread_queue.h new file mode 100644 index 0000000000..aaf491fe3c --- /dev/null +++ b/fftools/thread_queue.h @@ -0,0 +1,37 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef FFTOOLS_THREAD_QUEUE_H +#define FFTOOLS_THREAD_QUEUE_H + +#include + +typedef struct ThreadQueue ThreadQueue; + +ThreadQueue *tq_alloc(unsigned int nb_streams, + size_t nb_elems, size_t elem_size, + void (*elem_free)(void *)); +void tq_free(ThreadQueue **tq); + +int tq_send(ThreadQueue *tq, unsigned int stream_idx, void *data); +void tq_send_finish(ThreadQueue *tq, unsigned int stream_idx); + +int tq_receive(ThreadQueue *tq, int *stream_idx, void *data); +void tq_receive_finish(ThreadQueue *tq, unsigned int stream_idx); + +#endif // FFTOOLS_THREAD_QUEUE_H -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".