From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id ACD8E45A80 for ; Tue, 14 Mar 2023 11:51:11 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5893868BBCD; Tue, 14 Mar 2023 13:51:09 +0200 (EET) Received: from mail-ot1-f48.google.com (mail-ot1-f48.google.com [209.85.210.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5B8E568BBCD for ; Tue, 14 Mar 2023 13:51:03 +0200 (EET) Received: by mail-ot1-f48.google.com with SMTP id b26-20020a9d755a000000b006972b28ae37so823760otl.4 for ; Tue, 14 Mar 2023 04:51:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678794661; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id:from :to:cc:subject:date:message-id:reply-to; bh=hFCO+cHUTrk7PA6nz1YkpPUgF6jwZ9nf2Vig1Xxw1us=; b=ZzkY2v/qlSW34COTMvALliiJTAP+Aidntfox1rmo1ctfMvnSb7gurQaTDUU+v7lhl+ ZMQpKDH7RmvsTOcU756tSDI1woT/ldxKgUB9ukyxXfyhFW1KFbJQpIt1dJXHxdFg+O04 nXR2qysxfY4AXdMHCwPUBEeiGJNf/0HhCaKLePbhq8XidDPoMWxZZToj+lVnxxrksw0X kGvgev4L2BW3HnvTM6YuzMjv3J1yaWwhb6EQowG80XMkA10STQ+H31/+0KHGImkjO4Zy 3FEmPOFEgDgM/VPkQ5/qxHkMM48mvxqBbTguiB4oU+cCuaDrF7wEhGq7SMdCC9JVVwbO Oggw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678794661; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hFCO+cHUTrk7PA6nz1YkpPUgF6jwZ9nf2Vig1Xxw1us=; b=Mg26M7xiwhoMMxiPyBcBNuVmhCAYiFovJybtCikuD65cF7PWIPWnceSlDDKS2mdgax DVLJ0vP92DsbwrcqX47FR6iFExO5K9/5RFpX9s1pvNpHHPL06YoDePKJgQCfk3Y2kzKg i9bDRhTNECfrDvE7Cu+/SH+9Xg6dAzBxlxYV1P2tES48/8X1RiCkN3kaVzE2lMnPSJSI KHz/oXXK/pePXyriVruhXnQZYU+U7wjO3obGQPK1JelNnsrBj4q8neieMXGdBimXghSh tt8tKYoroQSGgW+RsvzofIpxeHH0LzBstcj8d9OQ/2cxDz/lzRsAnqZwkJWwygWmZcMk 0sMg== X-Gm-Message-State: AO0yUKWthwNUwtPdWSnY6QB/hRwL1Bh2USFd4cATG7Lk/K1cnpLwtc0s ytATFweBWAOc83qyJtUgy80TBG/LgvY= X-Google-Smtp-Source: AK7set9p7NGywm0lAwtRQkN/i5iUSx2VCjoUb5SVDkiWAdSPL1JwD2F3+iYipb9oRTU1ayKtvGvSQQ== X-Received: by 2002:a9d:4909:0:b0:68c:1b5f:96de with SMTP id e9-20020a9d4909000000b0068c1b5f96demr10851392otf.2.1678794661494; Tue, 14 Mar 2023 04:51:01 -0700 (PDT) Received: from [192.168.0.14] (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id i17-20020a9d6251000000b0069457b86060sm1043699otk.47.2023.03.14.04.50.59 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 Mar 2023 04:51:00 -0700 (PDT) Message-ID: Date: Tue, 14 Mar 2023 08:51:00 -0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 To: ffmpeg-devel@ffmpeg.org References: Content-Language: en-US From: James Almer In-Reply-To: Subject: Re: [FFmpeg-devel] [PATCH 31/92] Vulkan patchset part 1 - common code changes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 3/14/2023 3:33 AM, Lynne wrote: > The attached patchset is all the common code changes that my Vulkan patchset needs. > > In total lines of code, this part has 425 additions and 131 deletions. > Most of that is additions to HEVC parsing. Excluding them, the patchset is > 200 lines of code added, which is manageable. > > Apart from the parser changes, the following other changes have been > made to the API: > > AVHWAccel.free_frame_priv exists due to Vulkan's way of using VkImageView > objects to wrap VkImage objects, which we need to free once they're no longer > in use. Every other API uses the direct objects in decoding, but with Vulkan, > they have to be represented by other objects. > We also use it to free the slice offsets buffer. > > AVHWAccel.flush exists due to Vulkan keeping decoder state, despite being > stateless in theory. The decoder has to be notified of flushes in order to reset > decoding slots and other data it needs, such as motion vectors and reference > lists for AV1. Otherwise, inferring whether a flush has happened can be codec > dependent, and hacky. > > hwaccel_params_buf exists due to Vulkan's way of compiling SPS/PPS data > into objects, making updating expensive. The change allows for hardware > to only upload new parameters if they have been changed. > It's insignificant for H264 and AV1, but HEVC's structures can reach 114 > megabytes of data that has to be uploaded, for a specially crafted input, > which is enough to DDOS an ingest. > The data is set and managed by the hwaccel, but does need to be synchronized > between different decoding threads, which this patch performs. > > Finally, the HWACCEL_CAP_THREAD_SAFE flag is added due to Vulkan being > actually threadsafe, and requiring no serialization. It does work and it > does actually make a difference, on average, it can increase performance > by 20% for an average B-frame using HEVC stream, depending on the > number of threads and the number of decode queues. > While hardware decoders are fast in general, certain vendors such as AMD > can choke up while playing 8k video, and this patch can significantly help > increase throughput. > > In context, the changes can be viewed here: > https://github.com/cyanreg/FFmpeg/tree/vulkan > > The rest of the whole patchset is either rewrites, filter code, or > the actual hardware accel code. > > The patchset will not be pushed standalone, but as part of the greater > Vulkan patchset. > > 31 patches attached. [...] > diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c > index a80e37e33f..5a3c51e94a 100644 > --- a/libavcodec/av1dec.c > +++ b/libavcodec/av1dec.c > @@ -591,6 +591,8 @@ static void av1_frame_unref(AVCodecContext *avctx, AV1Frame *f) > f->spatial_id = f->temporal_id = 0; > memset(f->skip_mode_frame_idx, 0, > 2 * sizeof(uint8_t)); > + memset(f->ref_order_hint, 0, > + 7 * sizeof(uint8_t)); > memset(&f->film_grain, 0, sizeof(f->film_grain)); > f->coded_lossless = 0; > } > @@ -633,6 +635,9 @@ static int av1_frame_ref(AVCodecContext *avctx, AV1Frame *dst, const AV1Frame *s > memcpy(dst->skip_mode_frame_idx, > src->skip_mode_frame_idx, > 2 * sizeof(uint8_t)); > + memcpy(dst->ref_order_hint, > + src->ref_order_hint, > + 7 * sizeof(uint8_t)); > memcpy(&dst->film_grain, > &src->film_grain, > sizeof(dst->film_grain)); > @@ -1267,6 +1272,10 @@ static int av1_decode_frame(AVCodecContext *avctx, AVFrame *frame, > s->cur_frame.spatial_id = header->spatial_id; > s->cur_frame.temporal_id = header->temporal_id; > > + for (int i = 0; i < 7; i++) > + s->cur_frame.ref_order_hint[i] = > + s->raw_frame_header->ref_order_hint[s->raw_frame_header->ref_frame_idx[i]]; Why do you need this in cur_frame? It's not a derived value, and the AV1RawFrameHeader struct is accessible in all AVHWaccel callbacks. And i think you should be looking at s->ref[s->raw_frame_header->ref_frame_idx[i]].raw_frame_header->order_hint instead, too, which is the decoder state vs the raw values in the current frame header (Although they should match in theory). > + > if (avctx->hwaccel && s->cur_frame.f->buf[0]) { > ret = avctx->hwaccel->start_frame(avctx, unit->data, _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".