From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id A402042B5A for ; Thu, 26 May 2022 08:09:02 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1397E68B4FB; Thu, 26 May 2022 11:08:59 +0300 (EEST) Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B237E68B2A1 for ; Thu, 26 May 2022 11:08:52 +0300 (EEST) Received: by mail-pj1-f52.google.com with SMTP id cs3-20020a17090af50300b001e0808b5838so1125525pjb.1 for ; Thu, 26 May 2022 01:08:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:from:date:subject:fcc:content-transfer-encoding :mime-version:to:cc; bh=GOFPsGXRAR0xVCtuaMQ2IC6RLyhdV7Zq0QElMXfzrRk=; b=oBkKP5j23yqUp87oddbSqn/2ps67f0rGL1UlABUrY5ZmSyEd3A95zzttYIujGvxzKh 6eZady/vaHSdNujjtS/mKtQwy5ny4/6wPqbtm+6+TvOiaQO3tHuEh1/2VaAgcCIPuVnM ECko25a5cdrlmEB+jWLyPQpKU0EbhwexjC8diLWVpwx16udjMzfWapmlmgXpvj2X5+Wj iKMJU4RqwRca1po26evoxF0ddM/KXE68aHopHMSev6OHQJQIS6nj9XREGtr9rREp94Vq R0JwhSqXnkPgIB7z4cYsCP0YtKqD0DlklQKt2qshhQxijZX55N8kKVT6xUl26wHbhbaX Hovw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=GOFPsGXRAR0xVCtuaMQ2IC6RLyhdV7Zq0QElMXfzrRk=; b=hJJVpEixShX+jYAOfPXsZxZsmHsPiEYYkGZSErfVcD3xXzx8PYiIgxWgcD6PYmZRL9 IYf+4kc0cOHzsLU8zv/KkVnC+WGzjgUyE+feTBQ26xvJmrOvYDJDAWUdXdtQPyJvxPhy 0a67QSvgejqZfJhawtwF27su5+TvtZI0lpj5KM5neN41JurDuCsPhKWlqB2uIWv+2yCz 6lZUDAUI8xaYGjS/T/SIhrdAkWD0Kp1fdS+3nXa80mZriAxvRUyDFmCIhmTXgxjbNBDo sfPqdfBnkZ29m8XJPmIYBubuYnTkTi5ysb/mC9a9oRKCjPR6JJhcHSnv/qqmQiUwp80u lQjg== X-Gm-Message-State: AOAM530cD34yHGErVAoec1aJA5YbC/aPQoRIuLUjGUxlIMf5AxjZmloq fH6mcT5pRQJE3QaOupDXWWcKp6Qoc0bEoA== X-Google-Smtp-Source: ABdhPJwdyY8OE2qx4PxhmH18n/ZyQGpOGVaq3YhjaI8lt63FDJQ+bg8LqHXe4NPZCh7QB8kRr1cVbQ== X-Received: by 2002:a17:902:d549:b0:163:7f27:f748 with SMTP id z9-20020a170902d54900b001637f27f748mr974011plf.116.1653552531106; Thu, 26 May 2022 01:08:51 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id z6-20020aa79f86000000b0051851077bcfsm799363pfr.33.2022.05.26.01.08.50 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 May 2022 01:08:50 -0700 (PDT) Message-Id: From: ffmpegagent Date: Thu, 26 May 2022 08:08:43 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 0/6] Implement SEI parsing for QSV decoders X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Missing SEI information has always been a major drawback when using the QSV decoders. I used to think that there's no chance to get at the data without explicit implementation from the MSDK side (or doing something weird like parsing in parallel). It turned out that there's a hardly known api method that provides access to all SEI (h264/hevc) or user data (mpeg2video). This allows to get things like closed captions, frame packing, display orientation, HDR data (mastering display, content light level, etc.) without having to rely on those data being provided by the MSDK as extended buffers. The commit "Implement SEI parsing for QSV decoders" includes some hard-coded workarounds for MSDK bugs which I reported: https://github.com/Intel-Media-SDK/MediaSDK/issues/2597#issuecomment-1072795311 But that doesn't help. Those bugs exist and I'm sharing my workarounds, which are empirically determined by testing a range of files. If someone is interested, I can provide private access to a repository where we have been testing this. Alternatively, I could also leave those workarounds out, and just skip those SEI types. In a previous version of this patchset, there was a concern that payload data might need to be re-ordered. Meanwhile I have researched this carefully and the conclusion is that this is not required. My detailed analysis can be found here: https://gist.github.com/softworkz/36c49586a8610813a32270ee3947a932 softworkz (6): avutil/frame: Add av_frame_copy_side_data() and av_frame_remove_all_side_data() avcodec/vpp_qsv: Copy side data from input to output frame avcodec/mpeg12dec: make mpeg_decode_user_data() accessible avcodec/hevcdec: make set_side_data() accessible avcodec/h264dec: make h264_export_frame_props() accessible avcodec/qsvdec: Implement SEI parsing for QSV decoders doc/APIchanges | 4 + libavcodec/h264_slice.c | 98 +++++++-------- libavcodec/h264dec.h | 2 + libavcodec/hevcdec.c | 117 +++++++++--------- libavcodec/hevcdec.h | 2 + libavcodec/mpeg12.h | 28 +++++ libavcodec/mpeg12dec.c | 40 +----- libavcodec/qsvdec.c | 233 +++++++++++++++++++++++++++++++++++ libavfilter/qsvvpp.c | 6 + libavfilter/vf_overlay_qsv.c | 19 ++- libavutil/frame.c | 67 ++++++---- libavutil/frame.h | 32 +++++ libavutil/version.h | 2 +- 13 files changed, 477 insertions(+), 173 deletions(-) base-commit: b033913d1c5998a29dfd13e9906dd707ff6eff12 Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-31%2Fsoftworkz%2Fsubmit_qsv_sei-v1 Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-31/softworkz/submit_qsv_sei-v1 Pull-Request: https://github.com/ffstaging/FFmpeg/pull/31 -- ffmpeg-codebot _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".