From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id E178A47C57 for ; Sun, 15 Oct 2023 09:44:45 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EBBFD68C8CB; Sun, 15 Oct 2023 12:44:42 +0300 (EEST) Received: from EUR03-AM7-obe.outbound.protection.outlook.com (mail-am7eur03olkn2045.outbound.protection.outlook.com [40.92.59.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 577BD68C6BF for ; Sun, 15 Oct 2023 12:44:37 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bmCc43yeOHhxtu8mmqa3xWQOSfw9ksyk6B0qj9tDlrOEyNN1yPopMr/Hj1BraIuhG7OMh4sceFHXPiK/4ma2YgrmY0fNacF4eTBg4xzPbPzsgGPanuhx/+pvlZ+yZKLxFD7rLR9e/Zr9TW+X7PzqtNlsKHb786FiGFl9y/HuKrALMT4lEOp8JoB4eCstDpwdPRe2i3ulXBdmcCaiiLhqoPBfH56fA0h/Qzfvd+Ej6RFBnRBJPfBLKDD5UbF5PpRTwiwKcFqNS+9v/v5ux5kHQfqo6wnRp3uDGyaLv38xLDSKV7NW/uqm9Qr/+IjD/yMn8UJzBnSLW2s1wbubFju26g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8QoWmJaPboGdJ8pR/ZWGs0rFrJQ2rc6oJkcKmwpfqq8=; b=W0xci1AtzfkzOhASPER+dnmZvheQBsgbcjH+cEFLH+OW+BxCqPaSW9zLW8JwkokSVLy7UbpBBWyANXerQPjKurp0Lwi4Mi1Zp90AQavX3LJ8d4sbZzpmRFm82ktY41pP0qXZtRPfE72Y8ZTmpX3UwxpkDub60RZuLp4fYlfWlAPMkDXN7fCw/3BgqfmD+OUfVm3Ztr+LAOLbyf44uKfAiqTBy9Q+ty3FSPioNuAPDUiQVrbwapDIbp8xWUt9QBX3kvoNxAgNWfjw7T2kN1RdHowbsevI7CdSWGVsZJzjXloxCCeGXERynwTTNH471zvOZ0zJvl0w68itCqMX6H16TQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8QoWmJaPboGdJ8pR/ZWGs0rFrJQ2rc6oJkcKmwpfqq8=; b=lIsdeQEryfiC8NKKy/jv89LYwfz9mwexh9W8QjOKqXNLYuFHEjfWiQ/iMmLzcn6vVrfkLwGw3IvIpjpB8fPYr31FbI0o2WOvqVVufSoAsm/imVRo9TrJ4rwKeDhX/q+e73L4FxN0TxlZd7DAVkpsFb3yxpVb9Rzbt0UrOhGTo8bEJz5pNZjavTZpxFyRl0+9Ltft0rF9tjmwado5/C0Bgts7vGYOOV+muNF79v31XXFHO3v+k/kYbKpXaUNa8Qd92MbN5sE2hTUMQx/g8mv5sQDLEtB0U13xpF1bjEV19MDOk7Q4Ku/+H7UHyAvaZ7yUpjO+zMnhcO0AbB3kXQTxEA== Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) by AM8P250MB0044.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:3db::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.34; Sun, 15 Oct 2023 09:44:35 +0000 Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::3126:4823:194e:6f86]) by AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::3126:4823:194e:6f86%4]) with mapi id 15.20.6863.043; Sun, 15 Oct 2023 09:44:35 +0000 Message-ID: Date: Sun, 15 Oct 2023 11:45:50 +0200 User-Agent: Mozilla Thunderbird Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: <20230902160741.19197-1-nuomi2021@gmail.com> From: Andreas Rheinhardt In-Reply-To: X-TMN: [ailiUavuAz2RBGTNtZwK45HYYU7MokWkTTmz55jGkss=] X-ClientProxiedBy: ZR2P278CA0036.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:47::14) To AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) X-Microsoft-Original-Message-ID: MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AS8P250MB0744:EE_|AM8P250MB0044:EE_ X-MS-Office365-Filtering-Correlation-Id: a84cd91c-3fff-4dcd-e36b-08dbcd635744 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4wu43CSOjfvNrMd/ng9Jmp2zedM0S4LMDtDyCxuh8slIrRrPLfEhCcGnWcVwkee95RzfgqkeB7XyD+HLh0NrodT9xWQAg23O743kPHErD3LzD881RNta4jdoOptawp7ywtoAHnpdRfXEfx9cMrqysPT8xZiJRA1FX52om6gGjpaO2DcDS/Fm88N76BBubNwvUqhr66eyGCM/XpA3xr0vx4PsuvJR1fbGh5a3dQQiNKieM7s8mlo5pSZIfMflA+IFmMZ5jhD1rGljitsTBGewqRdAci4RwtFsFVIV/DYb4x4Aphv+TiNBJ93qmKxDsg8OCo6w9GLm6C9j2FQJQMEEN9AnVvDXT9JSNjp4aKIGkRTJFd8dQnUyhU+PmTDhHeX32PUYmx8Ajozf1hwHcv1ryNOL3KzgRXeSp30qAI3scz15I9EUCCaTQ6g8SMxD0+mQfsNNd1PEEmyTmEkEzP/BarrihMfD1r6oXy1wemA27XSpg+klfTOVlrNyFCaZaL8NY+GDNgQISYtmDfuc8UsawiJ4An5/JDb6CDGpG+5u7HFeRDlRLL1wSYKN6Cby+vOq X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Yll6SXpiVENqSFYyK29RbmRLRXlvWWZJV2hoOTVSaVdUNUEwbXFoeFdITmNM?= =?utf-8?B?Z2JXeTFiUy9MWDBMUlRDYloxVmVVQnAvc1lyai9ucUtLRlJWMHc5dEFnVWZ1?= =?utf-8?B?ZUYvS013VlQyOFdPK3JTQXgyRkF3SEtnQzd4SW50WUl3OE55RmZsN0c3dW9j?= =?utf-8?B?eHAyZVRXTGprSEJXVGNLeGduSTJmUC8wcWZyZnVXaEkzbWx0YWhFMEFUSUJO?= =?utf-8?B?eWNkY1IzMHNHVlRvT3Y0TXlQanRsY0ZlMFhFdGtYcFFpZk90L2h1dzI5aDBN?= =?utf-8?B?R2NmbXBIZFVLVUlBeEFFMXZualN0YjlCb2VMYm9PUjVhY3RhSUd5cDlrMU9y?= =?utf-8?B?bVlBdTNJRmRvSVE4bnI1OTAzUzdLbHV6NjRBai85ZHUwMEprNEVQd2p3UUh3?= =?utf-8?B?NE5EUjNRZDNVbmxveCs4eEdTQmtESndLVnNYV2loeUw3WGNrUGErMzlaM1VK?= =?utf-8?B?ZUl1MFM0SHh0S1NqV0txTFVqcGVIcy9MTGl4L0owZzNHSjh1V0FCNWdSNEto?= =?utf-8?B?RXgwbDh6Qm1BMi9sRk1EYW43ZVRGcy80QlhUUWk0Y3hBdzRKNkhLUWxheXV3?= =?utf-8?B?U1ZuNG9KUkw2OVB2anpPVCt2cFp6SE9TcXNkNjNVL2JsRDlnRkxyUnR4R0VB?= =?utf-8?B?M2NkUzA3RlBWYWlxeHo2QndnbGhTTWhtNlIvR0wvL2sxcUhwYm94K2RxRFpF?= =?utf-8?B?V3R0QVEvU2Evd1JxbjRBSExMTDh3bTQ5TzQzREQ0VTF4bDNsMDFRRUlGSkdN?= =?utf-8?B?c1JaUktaclRKZFhzZEg4OE43ZFZPQ0hqTmdEMTIrcUdsRzZpbUZCbUJCQUVG?= =?utf-8?B?NWVjMnZod04rUkFnd0s2aEFsZ0xiWXQ5VTR6Y05yZzQrS3dFdFcyaGhwQ21C?= =?utf-8?B?akZvL3NDZm0za0ZlN05EZUdUenI0cHJGaUVSL0pLakxXV2xSTGNvQTEvaFpv?= =?utf-8?B?a1ExT3NTU0RDR0IvRjhWV1dFdklpaEoxbFRsZjd2TnJRWlk5Vkh3bkRWRTJY?= =?utf-8?B?WCthU2gyU0lQKzFEVEkwbXZFVyt4cGREVlVSOVJoMUtRaGtFaFVwbkk2MVNl?= =?utf-8?B?V1cyYTlnSFZNSlpzam1yME5qSkk5TEtVb0lwYXNQUmpHK0lLbU9ZQWEvNTJr?= =?utf-8?B?MlhadTBRSEpPTCtRT0cyMUN3eC9zdlVpWVdWK2xkdVYvYnkzNTRIWTdhdGZx?= =?utf-8?B?UlRKeE83TGRZdXdzNzJkWWFXaE9ESFE0UGd4Y2paeFE2UWtqMjdVYUl6dm5J?= =?utf-8?B?UktuTHNMaWVCbTdCK1lGWC9qVDlDOWY0cm9KS3dQMUNoQkoxSHdWdUtsbTF0?= =?utf-8?B?emZRWk9wVEVneWxCenlLNTFFcFNJcmRhbWxlY2xacW9DNTVxaGVRMUgvTFhO?= =?utf-8?B?dXk5bmdOaklYMkpVYVZTVnBKMUZ3OTlOSUxOdGtESHRSazNNT1RsTjZENFJD?= =?utf-8?B?MG5POG9BRmU1aHV3RkF4VE0ydm53SnFtYnRuQzhrSk9QS0xxZVZpL2hCQ0pn?= =?utf-8?B?YnJzd1NvV0FFTy9KSkEwMWRrUlpPeWgrNjhKOFZPNldGOGZUN1I0V3hBNXRY?= =?utf-8?B?dFJoRk5UU1lpMkFrNU8rMzVlUEFOVTJWekpEeis2UWZvekZQd0MxdDRPaHVM?= =?utf-8?B?aHEwT2ZJazltYVN1WVNUaXNMMFprUGdVdUw2dE16Witaa24zQjZDZlZNZzhF?= =?utf-8?Q?CbRGPNKHGQXMM7u+cRr2?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a84cd91c-3fff-4dcd-e36b-08dbcd635744 X-MS-Exchange-CrossTenant-AuthSource: AS8P250MB0744.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Oct 2023 09:44:35.4903 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8P250MB0044 Subject: Re: [FFmpeg-devel] [PATCH v3 05/14] vvcdec: add reference management X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Nuo Mi: > --- > libavcodec/vvc/Makefile | 3 +- > libavcodec/vvc/vvc_refs.c | 526 ++++++++++++++++++++++++++++++++++++++ > libavcodec/vvc/vvc_refs.h | 47 ++++ > 3 files changed, 575 insertions(+), 1 deletion(-) > create mode 100644 libavcodec/vvc/vvc_refs.c > create mode 100644 libavcodec/vvc/vvc_refs.h > > diff --git a/libavcodec/vvc/Makefile b/libavcodec/vvc/Makefile > index 64c9ea9688..6eb181784b 100644 > --- a/libavcodec/vvc/Makefile > +++ b/libavcodec/vvc/Makefile > @@ -5,4 +5,5 @@ OBJS-$(CONFIG_VVC_DECODER) += vvc/vvcdec.o \ > vvc/vvc_cabac.o \ > vvc/vvc_ctu.o \ > vvc/vvc_data.o \ > - vvc/vvc_ps.o > + vvc/vvc_ps.o \ > + vvc/vvc_refs.o > diff --git a/libavcodec/vvc/vvc_refs.c b/libavcodec/vvc/vvc_refs.c > new file mode 100644 > index 0000000000..010f1892ed > --- /dev/null > +++ b/libavcodec/vvc/vvc_refs.c > @@ -0,0 +1,526 @@ > +/* > + * VVC reference management > + * > + * Copyright (C) 2023 Nuo Mi > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include > + > +#include "libavutil/thread.h" > + > +#include "vvc_refs.h" > + > +#define VVC_FRAME_FLAG_OUTPUT (1 << 0) > +#define VVC_FRAME_FLAG_SHORT_REF (1 << 1) > +#define VVC_FRAME_FLAG_LONG_REF (1 << 2) > +#define VVC_FRAME_FLAG_BUMPING (1 << 3) > + > +typedef struct FrameProgress { > + atomic_int progress[VVC_PROGRESS_LAST]; > + AVMutex lock; > +} FrameProgress; > + > +void ff_vvc_unref_frame(VVCFrameContext *fc, VVCFrame *frame, int flags) > +{ > + /* frame->frame can be NULL if context init failed */ > + if (!frame->frame || !frame->frame->buf[0]) > + return; > + > + frame->flags &= ~flags; > + if (!frame->flags) { > + ff_thread_release_ext_buffer(fc->avctx, &frame->tf); > + > + av_buffer_unref(&frame->progress_buf); > + > + av_buffer_unref(&frame->tab_dmvr_mvf_buf); > + frame->tab_dmvr_mvf = NULL; > + > + av_buffer_unref(&frame->rpl_buf); > + av_buffer_unref(&frame->rpl_tab_buf); > + frame->rpl_tab = NULL; > + > + frame->collocated_ref = NULL; > + } > +} > + > +const RefPicList *ff_vvc_get_ref_list(const VVCFrameContext *fc, const VVCFrame *ref, int x0, int y0) > +{ > + int x_cb = x0 >> fc->ps.sps->ctb_log2_size_y; > + int y_cb = y0 >> fc->ps.sps->ctb_log2_size_y; > + int pic_width_cb = fc->ps.pps->ctb_width; > + int ctb_addr_rs = y_cb * pic_width_cb + x_cb; > + > + return (const RefPicList *)ref->rpl_tab[ctb_addr_rs]; > +} > + > +void ff_vvc_clear_refs(VVCFrameContext *fc) > +{ > + int i; > + for (i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) > + ff_vvc_unref_frame(fc, &fc->DPB[i], > + VVC_FRAME_FLAG_SHORT_REF | VVC_FRAME_FLAG_LONG_REF); > +} > + > +static void free_progress(void *opaque, uint8_t *data) > +{ > + ff_mutex_destroy(&((FrameProgress*)data)->lock); > + av_free(data); > +} > + > +static AVBufferRef *alloc_progress(void) > +{ > + int ret; > + AVBufferRef *buf; > + FrameProgress *p = av_mallocz(sizeof(FrameProgress)); > + > + if (!p) > + return NULL; > + > + ret = ff_mutex_init(&p->lock, NULL); > + if (ret) { > + av_free(p); > + return NULL; > + } > + buf = av_buffer_create((void*)p, sizeof(*p), free_progress, NULL, 0); > + if (!buf) > + free_progress(NULL, (void*)p); > + return buf; > +} > + > +static VVCFrame *alloc_frame(VVCContext *s, VVCFrameContext *fc) > +{ > + const VVCPPS *pps = fc->ps.pps; > + int i, j, ret; > + for (i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) { > + VVCFrame *frame = &fc->DPB[i]; > + if (frame->frame->buf[0]) > + continue; > + > + ret = ff_thread_get_ext_buffer(fc->avctx, &frame->tf, > + AV_GET_BUFFER_FLAG_REF); > + if (ret < 0) > + return NULL; > + > + frame->rpl_buf = av_buffer_allocz(s->current_frame.nb_units * sizeof(RefPicListTab)); > + if (!frame->rpl_buf) > + goto fail; > + > + frame->tab_dmvr_mvf_buf = av_buffer_pool_get(fc->tab_dmvr_mvf_pool); > + if (!frame->tab_dmvr_mvf_buf) > + goto fail; > + frame->tab_dmvr_mvf = (MvField *)frame->tab_dmvr_mvf_buf->data; > + //fixme: remove this > + memset(frame->tab_dmvr_mvf, 0, frame->tab_dmvr_mvf_buf->size); This stuff here would benefit a lot from the (not yet merged) RefStruct pool API. There is even a flag to zero pool entries automatically so you don't have to. > + > + frame->rpl_tab_buf = av_buffer_pool_get(fc->rpl_tab_pool); > + if (!frame->rpl_tab_buf) > + goto fail; > + frame->rpl_tab = (RefPicListTab **)frame->rpl_tab_buf->data; > + frame->ctb_count = pps->ctb_width * pps->ctb_height; > + for (j = 0; j < frame->ctb_count; j++) > + frame->rpl_tab[j] = (RefPicListTab *)frame->rpl_buf->data; > + > + > + frame->progress_buf = alloc_progress(); > + if (!frame->progress_buf) > + goto fail; > + > + return frame; > +fail: > + ff_vvc_unref_frame(fc, frame, ~0); > + return NULL; > + } > + av_log(s->avctx, AV_LOG_ERROR, "Error allocating frame, DPB full.\n"); > + return NULL; > +} > + > +int ff_vvc_set_new_ref(VVCContext *s, VVCFrameContext *fc, AVFrame **frame) > +{ > + const VVCPH *ph= &fc->ps.ph; > + const int poc = ph->poc; > + VVCFrame *ref; > + int i; > + > + /* check that this POC doesn't already exist */ > + for (i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) { > + VVCFrame *frame = &fc->DPB[i]; > + > + if (frame->frame->buf[0] && frame->sequence == s->seq_decode && > + frame->poc == poc) { > + av_log(s->avctx, AV_LOG_ERROR, "Duplicate POC in a sequence: %d.\n", > + poc); > + return AVERROR_INVALIDDATA; > + } > + } > + > + ref = alloc_frame(s, fc); > + if (!ref) > + return AVERROR(ENOMEM); > + > + *frame = ref->frame; > + fc->ref = ref; > + > + if (s->no_output_before_recovery_flag && (IS_RASL(s) || !GDR_IS_RECOVERED(s))) > + ref->flags = 0; > + else if (ph->r->ph_pic_output_flag) > + ref->flags = VVC_FRAME_FLAG_OUTPUT; > + > + if (!ph->r->ph_non_ref_pic_flag) > + ref->flags |= VVC_FRAME_FLAG_SHORT_REF; > + > + ref->poc = poc; > + ref->sequence = s->seq_decode; > + ref->frame->crop_left = fc->ps.pps->r->pps_conf_win_left_offset; > + ref->frame->crop_right = fc->ps.pps->r->pps_conf_win_right_offset; > + ref->frame->crop_top = fc->ps.pps->r->pps_conf_win_top_offset; > + ref->frame->crop_bottom = fc->ps.pps->r->pps_conf_win_bottom_offset; > + > + return 0; > +} > + > +int ff_vvc_output_frame(VVCContext *s, VVCFrameContext *fc, AVFrame *out, const int no_output_of_prior_pics_flag, int flush) > +{ > + const VVCSPS *sps = fc->ps.sps; > + do { > + int nb_output = 0; > + int min_poc = INT_MAX; > + int i, min_idx, ret; > + > + if (no_output_of_prior_pics_flag) { > + for (i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) { > + VVCFrame *frame = &fc->DPB[i]; > + if (!(frame->flags & VVC_FRAME_FLAG_BUMPING) && frame->poc != fc->ps.ph.poc && > + frame->sequence == s->seq_output) { > + ff_vvc_unref_frame(fc, frame, VVC_FRAME_FLAG_OUTPUT); > + } > + } > + } > + > + for (i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) { > + VVCFrame *frame = &fc->DPB[i]; > + if ((frame->flags & VVC_FRAME_FLAG_OUTPUT) && > + frame->sequence == s->seq_output) { > + nb_output++; > + if (frame->poc < min_poc || nb_output == 1) { > + min_poc = frame->poc; > + min_idx = i; > + } > + } > + } > + > + /* wait for more frames before output */ > + if (!flush && s->seq_output == s->seq_decode && sps && > + nb_output <= sps->r->sps_dpb_params.dpb_max_dec_pic_buffering_minus1[sps->r->sps_max_sublayers_minus1] + 1) > + return 0; > + > + if (nb_output) { > + VVCFrame *frame = &fc->DPB[min_idx]; > + > + ret = av_frame_ref(out, frame->frame); > + if (frame->flags & VVC_FRAME_FLAG_BUMPING) > + ff_vvc_unref_frame(fc, frame, VVC_FRAME_FLAG_OUTPUT | VVC_FRAME_FLAG_BUMPING); > + else > + ff_vvc_unref_frame(fc, frame, VVC_FRAME_FLAG_OUTPUT); > + if (ret < 0) > + return ret; > + > + av_log(s->avctx, AV_LOG_DEBUG, > + "Output frame with POC %d.\n", frame->poc); > + return 1; > + } > + > + if (s->seq_output != s->seq_decode) > + s->seq_output = (s->seq_output + 1) & 0xff; > + else > + break; > + } while (1); > + return 0; > +} > + > +void ff_vvc_bump_frame(VVCContext *s, VVCFrameContext *fc) > +{ > + const VVCSPS *sps = fc->ps.sps; > + const int poc = fc->ps.ph.poc; > + int dpb = 0; > + int min_poc = INT_MAX; > + int i; > + > + for (i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) { > + VVCFrame *frame = &fc->DPB[i]; > + if ((frame->flags) && > + frame->sequence == s->seq_output && > + frame->poc != poc) { > + dpb++; > + } > + } > + > + if (sps && dpb >= sps->r->sps_dpb_params.dpb_max_dec_pic_buffering_minus1[sps->r->sps_max_sublayers_minus1] + 1) { > + for (i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) { > + VVCFrame *frame = &fc->DPB[i]; > + if ((frame->flags) && > + frame->sequence == s->seq_output && > + frame->poc != poc) { > + if (frame->flags == VVC_FRAME_FLAG_OUTPUT && frame->poc < min_poc) { > + min_poc = frame->poc; > + } > + } > + } > + > + for (i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) { > + VVCFrame *frame = &fc->DPB[i]; > + if (frame->flags & VVC_FRAME_FLAG_OUTPUT && > + frame->sequence == s->seq_output && > + frame->poc <= min_poc) { > + frame->flags |= VVC_FRAME_FLAG_BUMPING; > + } > + } > + > + dpb--; > + } > +} > + > +static VVCFrame *find_ref_idx(VVCContext *s, VVCFrameContext *fc, int poc, uint8_t use_msb) > +{ > + int mask = use_msb ? ~0 : fc->ps.sps->max_pic_order_cnt_lsb - 1; > + int i; > + > + for (i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) { > + VVCFrame *ref = &fc->DPB[i]; > + if (ref->frame->buf[0] && ref->sequence == s->seq_decode) { > + if ((ref->poc & mask) == poc) > + return ref; > + } > + } > + return NULL; > +} > + > +static void mark_ref(VVCFrame *frame, int flag) > +{ > + frame->flags &= ~(VVC_FRAME_FLAG_LONG_REF | VVC_FRAME_FLAG_SHORT_REF); > + frame->flags |= flag; > +} > + > +static VVCFrame *generate_missing_ref(VVCContext *s, VVCFrameContext *fc, int poc) > +{ > + const VVCSPS *sps = fc->ps.sps; > + VVCFrame *frame; > + int i, y; > + > + frame = alloc_frame(s, fc); > + if (!frame) > + return NULL; > + > + if (!s->avctx->hwaccel) { > + if (!sps->pixel_shift) { > + for (i = 0; frame->frame->buf[i]; i++) > + memset(frame->frame->buf[i]->data, 1 << (sps->bit_depth - 1), > + frame->frame->buf[i]->size); > + } else { > + for (i = 0; frame->frame->data[i]; i++) > + for (y = 0; y < (sps->height >> sps->vshift[i]); y++) { > + uint8_t *dst = frame->frame->data[i] + y * frame->frame->linesize[i]; > + AV_WN16(dst, 1 << (sps->bit_depth - 1)); > + av_memcpy_backptr(dst + 2, 2, 2*(sps->width >> sps->hshift[i]) - 2); > + } > + } > + } > + > + frame->poc = poc; > + frame->sequence = s->seq_decode; > + frame->flags = 0; > + > + ff_vvc_report_frame_finished(frame); > + > + return frame; > +} > + > +/* add a reference with the given poc to the list and mark it as used in DPB */ > +static int add_candidate_ref(VVCContext *s, VVCFrameContext *fc, RefPicList *list, > + int poc, int ref_flag, uint8_t use_msb) > +{ > + VVCFrame *ref = find_ref_idx(s, fc, poc, use_msb); > + > + if (ref == fc->ref || list->nb_refs >= VVC_MAX_REF_ENTRIES) > + return AVERROR_INVALIDDATA; > + > + if (!ref) { > + ref = generate_missing_ref(s, fc, poc); > + if (!ref) > + return AVERROR(ENOMEM); > + } > + > + list->list[list->nb_refs] = poc; > + list->ref[list->nb_refs] = ref; > + list->isLongTerm[list->nb_refs] = ref_flag & VVC_FRAME_FLAG_LONG_REF; > + list->nb_refs++; > + > + mark_ref(ref, ref_flag); > + return 0; > +} > + > +static int init_slice_rpl(const VVCFrameContext *fc, SliceContext *sc) > +{ > + VVCFrame *frame = fc->ref; > + const VVCSH *sh = &sc->sh; > + > + if (sc->slice_idx >= frame->rpl_buf->size / sizeof(RefPicListTab)) > + return AVERROR_INVALIDDATA; > + > + for (int i = 0; i < sh->num_ctus_in_curr_slice; i++) { > + const int rs = sh->ctb_addr_in_curr_slice[i]; > + frame->rpl_tab[rs] = (RefPicListTab *)frame->rpl_buf->data + sc->slice_idx; > + } > + > + sc->rpl = (RefPicList *)frame->rpl_tab[sh->ctb_addr_in_curr_slice[0]]; > + > + return 0; > +} > + > +static int delta_poc_st(const H266RefPicListStruct *rpls, > + const int lx, const int i, const VVCSPS *sps) > +{ > + int abs_delta_poc_st = rpls->abs_delta_poc_st[i]; > + if (!((sps->r->sps_weighted_pred_flag || > + sps->r->sps_weighted_bipred_flag) && i != 0)) > + abs_delta_poc_st++; > + return (1 - 2 * rpls->strp_entry_sign_flag[i]) * abs_delta_poc_st; > +} > + > +static int poc_lt(int *prev_delta_poc_msb, const int poc, const H266RefPicLists *ref_lists, > + const int lx, const int j, const int max_poc_lsb) > +{ > + const H266RefPicListStruct *rpls = ref_lists->rpl_ref_list + lx; > + int lt_poc = rpls->ltrp_in_header_flag ? ref_lists->poc_lsb_lt[lx][j] : rpls->rpls_poc_lsb_lt[j]; > + > + if (ref_lists->delta_poc_msb_cycle_present_flag[lx][j]) { > + const uint32_t delta = ref_lists->delta_poc_msb_cycle_lt[lx][j] + *prev_delta_poc_msb; > + lt_poc += poc - delta * max_poc_lsb - (poc & (max_poc_lsb - 1)); > + *prev_delta_poc_msb = delta; > + } > + return lt_poc; > +} > + > +int ff_vvc_slice_rpl(VVCContext *s, VVCFrameContext *fc, SliceContext *sc) > +{ > + const VVCSPS *sps = fc->ps.sps; > + const H266RawPPS *pps = fc->ps.pps->r; > + const VVCPH *ph = &fc->ps.ph; > + const H266RawSliceHeader *rsh = sc->sh.r; > + const int max_poc_lsb = sps->max_pic_order_cnt_lsb; > + const H266RefPicLists *ref_lists = > + pps->pps_rpl_info_in_ph_flag ? &ph->r->ph_ref_pic_lists : &rsh->sh_ref_pic_lists; > + int ret = 0; > + > + ret = init_slice_rpl(fc, sc); > + if (ret < 0) > + return ret; > + > + for (int lx = L0; lx <= L1; lx++) { > + const H266RefPicListStruct *rpls = ref_lists->rpl_ref_list + lx; > + RefPicList *rpl = sc->rpl + lx; > + int poc_base = ph->poc; > + int prev_delta_poc_msb = 0; > + > + rpl->nb_refs = 0; > + for (int i = 0, j = 0; i < rpls->num_ref_entries; i++) { > + int poc; > + if (!rpls->inter_layer_ref_pic_flag[i]) { > + int use_msb = 1; > + int ref_flag; > + if (rpls->st_ref_pic_flag[i]) { > + poc = poc_base + delta_poc_st(rpls, lx, i, sps); > + poc_base = poc; > + ref_flag = VVC_FRAME_FLAG_SHORT_REF; > + } else { > + use_msb = ref_lists->delta_poc_msb_cycle_present_flag[lx][j]; > + poc = poc_lt(&prev_delta_poc_msb, ph->poc, ref_lists, lx, j, max_poc_lsb); > + ref_flag = VVC_FRAME_FLAG_LONG_REF; > + j++; > + } > + ret = add_candidate_ref(s, fc, rpl, poc, ref_flag, use_msb); > + if (ret < 0) > + goto fail; > + } else { > + avpriv_request_sample(fc->avctx, "Inter layer ref"); > + ret = AVERROR_PATCHWELCOME; > + goto fail; > + } > + } > + if ((!rsh->sh_collocated_from_l0_flag) == lx && > + rsh->sh_collocated_ref_idx < rpl->nb_refs) > + fc->ref->collocated_ref = rpl->ref[rsh->sh_collocated_ref_idx]; > + } > +fail: > + return ret; > +} > + > +int ff_vvc_frame_rpl(VVCContext *s, VVCFrameContext *fc, SliceContext *sc) > +{ > + int i, ret = 0; > + > + /* clear the reference flags on all frames except the current one */ > + for (i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) { > + VVCFrame *frame = &fc->DPB[i]; > + > + if (frame == fc->ref) > + continue; > + > + mark_ref(frame, 0); > + } > + > + if ((ret = ff_vvc_slice_rpl(s, fc, sc)) < 0) > + goto fail; > + > +fail: > + /* release any frames that are now unused */ > + for (i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) > + ff_vvc_unref_frame(fc, &fc->DPB[i], 0); > + return ret; > +} > + > +void ff_vvc_report_frame_finished(VVCFrame *frame) > +{ > + ff_vvc_report_progress(frame, VVC_PROGRESS_MV, INT_MAX); > + ff_vvc_report_progress(frame, VVC_PROGRESS_PIXEL, INT_MAX); > +} > + > +void ff_vvc_report_progress(VVCFrame *frame, const VVCProgress vp, const int y) > +{ > + FrameProgress *p = (FrameProgress*)frame->progress_buf->data; > + > + ff_mutex_lock(&p->lock); > + > + av_assert0(p->progress[vp] < y || p->progress[vp] == INT_MAX); > + p->progress[vp] = y; > + > + ff_mutex_unlock(&p->lock); > +} > + > +int ff_vvc_check_progress(VVCFrame *frame, const VVCProgress vp, const int y) > +{ > + int ready ; > + FrameProgress *p = (FrameProgress*)frame->progress_buf->data; > + > + ff_mutex_lock(&p->lock); > + > + ready = p->progress[vp] > y + 1; > + > + ff_mutex_unlock(&p->lock); > + return ready; > +} > diff --git a/libavcodec/vvc/vvc_refs.h b/libavcodec/vvc/vvc_refs.h > new file mode 100644 > index 0000000000..1bba2ca34a > --- /dev/null > +++ b/libavcodec/vvc/vvc_refs.h > @@ -0,0 +1,47 @@ > +/* > + * VVC reference management > + * > + * Copyright (C) 2023 Nuo Mi > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#ifndef AVCODEC_VVC_REFS_H > +#define AVCODEC_VVC_REFS_H > + > +#include "vvcdec.h" > + > +int ff_vvc_output_frame(VVCContext *s, VVCFrameContext *fc, AVFrame *out, int no_output_of_prior_pics_flag, int flush); > +void ff_vvc_bump_frame(VVCContext *s, VVCFrameContext *fc); > +int ff_vvc_set_new_ref(VVCContext *s, VVCFrameContext *fc, AVFrame **frame); > +const RefPicList *ff_vvc_get_ref_list(const VVCFrameContext *fc, const VVCFrame *ref, int x0, int y0); > +int ff_vvc_frame_rpl(VVCContext *s, VVCFrameContext *fc, SliceContext *sc); > +int ff_vvc_slice_rpl(VVCContext *s, VVCFrameContext *fc, SliceContext *sc); > +void ff_vvc_unref_frame(VVCFrameContext *fc, VVCFrame *frame, int flags); > +void ff_vvc_clear_refs(VVCFrameContext *fc); > + > +typedef enum VVCProgress{ > + VVC_PROGRESS_MV, > + VVC_PROGRESS_PIXEL, > + VVC_PROGRESS_LAST, > +} VVCProgress; > + > +void ff_vvc_report_frame_finished(VVCFrame *frame); > +void ff_vvc_report_progress(VVCFrame *frame, VVCProgress vp, int y); > +int ff_vvc_check_progress(VVCFrame *frame, VVCProgress vp, int y); > + > +#endif // AVCODEC_VVC_REFS_H _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".