From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 701E94063B for ; Sun, 26 Jun 2022 23:42:10 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3DF2F68B877; Mon, 27 Jun 2022 02:41:48 +0300 (EEST) Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 663FC68B41C for ; Mon, 27 Jun 2022 02:41:32 +0300 (EEST) Received: by mail-pj1-f43.google.com with SMTP id b12-20020a17090a6acc00b001ec2b181c98so10676482pjm.4; Sun, 26 Jun 2022 16:41:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=t62x7cV1zvVNr2mnFGKkOwvvTMihNVFtOBEB4J6vHqQ=; b=DtdNKeKU2tqDodpZJiD+A7gU2wD2Z8emIpiQrcvxJFDrJWcBOxcJo/UFlUIVB+dA7c X7WfDOy3pYghiXmZZCiZg0jGYfXgakgxw37PDsaORfeOb5N2StH5LB9NHSeKbQUZm/zX U1txpP2nBLk0QdEa50714FTf20Sfh/SoxBv0buWjUZXiLDnRs7Wmi+yiRqPKN79T3COG rE0KmNtkn14uk3mZFM350QC3SCHioEWPfwaYvjAaGTP6gMqQO5vl0Qd53X4itG00wHGP +Mmvu7SMYM8ftE9SyhayaU10AAKNLdEemKCFC1NLat2gOsvRdwhN+GX5IXG7Pk8FgZWD +tsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=t62x7cV1zvVNr2mnFGKkOwvvTMihNVFtOBEB4J6vHqQ=; b=Fk04MHkTaSMmNSHY0LA7YEUDPUkpg23FXK+cWeYVk7uhkxvfNtneMiwSrNqGsD+awL L0W38dag0Qnftxkss3QCMRKzgHf4G+KAUEORTYXX3pQ/5ispyIXI38lwP4HW3ZPcS89T dwtESYYFVplS+ZM+z5vsM1Keiyto+KiDyNV01QegMPnsJidicex4oS4KRxA5udH8vxie iZsgNWBNE2X6dHOuowVTUr+Rb2B+QT3WQ5GM61My5jDXzrDjPIwox/PJ8Ootg3tSg1qH 2YnCpGglE14Y0uix+vfDRCU0+sef+q2L/hpgIS79QoL5SDO1alqrIiZ70LG6DEsX4F5Z EEaw== X-Gm-Message-State: AJIora9rmLpXM8L2dfA64GJsYSjJ9ugHiohg7251u5liBOSKnG6p/3hP my8d69VJ9i/t6TjuQ6Jy5XAIHBgpjIemnA== X-Google-Smtp-Source: AGRyM1txmgYFJLoRaVGCMhHVhhuWlrSd4PMbU/8qQam0Ns4wjJ6fukLHcs9xAIU+snyKPahKo5RUUA== X-Received: by 2002:a17:902:cecc:b0:16a:416c:3d14 with SMTP id d12-20020a170902cecc00b0016a416c3d14mr11426254plg.73.1656286889771; Sun, 26 Jun 2022 16:41:29 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id u2-20020a1709026e0200b0016a106cb221sm5675378plk.243.2022.06.26.16.41.29 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 26 Jun 2022 16:41:29 -0700 (PDT) Message-Id: In-Reply-To: References: From: ffmpegagent Date: Sun, 26 Jun 2022 23:41:22 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v4 0/6] Implement SEI parsing for QSV decoders X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Kieran Kunhya , softworkz , "Xiang, Haihao" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Missing SEI information has always been a major drawback when using the QSV decoders. I used to think that there's no chance to get at the data without explicit implementation from the MSDK side (or doing something weird like parsing in parallel). It turned out that there's a hardly known api method that provides access to all SEI (h264/hevc) or user data (mpeg2video). This allows to get things like closed captions, frame packing, display orientation, HDR data (mastering display, content light level, etc.) without having to rely on those data being provided by the MSDK as extended buffers. The commit "Implement SEI parsing for QSV decoders" includes some hard-coded workarounds for MSDK bugs which I reported: https://github.com/Intel-Media-SDK/MediaSDK/issues/2597#issuecomment-1072795311 But that doesn't help. Those bugs exist and I'm sharing my workarounds, which are empirically determined by testing a range of files. If someone is interested, I can provide private access to a repository where we have been testing this. Alternatively, I could also leave those workarounds out, and just skip those SEI types. In a previous version of this patchset, there was a concern that payload data might need to be re-ordered. Meanwhile I have researched this carefully and the conclusion is that this is not required. My detailed analysis can be found here: https://gist.github.com/softworkz/36c49586a8610813a32270ee3947a932 v3 * frame.h: clarify doc text for av_frame_copy_side_data() v2 * qsvdec: make error handling consistent and clear * qsvdec: remove AV_CODEC_ID_MPEG1VIDEO constants * hevcdec: rename function to ff_hevc_set_side_data(), add doc text v3 * qsvdec: fix c/p error softworkz (6): avutil/frame: Add av_frame_copy_side_data() and av_frame_remove_all_side_data() avcodec/vpp_qsv: Copy side data from input to output frame avcodec/mpeg12dec: make mpeg_decode_user_data() accessible avcodec/hevcdec: make set_side_data() accessible avcodec/h264dec: make h264_export_frame_props() accessible avcodec/qsvdec: Implement SEI parsing for QSV decoders doc/APIchanges | 4 + libavcodec/h264_slice.c | 98 ++++++++------- libavcodec/h264dec.h | 2 + libavcodec/hevcdec.c | 117 +++++++++--------- libavcodec/hevcdec.h | 9 ++ libavcodec/mpeg12.h | 28 +++++ libavcodec/mpeg12dec.c | 40 +----- libavcodec/qsvdec.c | 234 +++++++++++++++++++++++++++++++++++ libavfilter/qsvvpp.c | 6 + libavfilter/vf_overlay_qsv.c | 19 ++- libavutil/frame.c | 67 ++++++---- libavutil/frame.h | 32 +++++ libavutil/version.h | 2 +- 13 files changed, 485 insertions(+), 173 deletions(-) base-commit: 6a82412bf33108111eb3f63076fd5a51349ae114 Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-31%2Fsoftworkz%2Fsubmit_qsv_sei-v4 Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-31/softworkz/submit_qsv_sei-v4 Pull-Request: https://github.com/ffstaging/FFmpeg/pull/31 Range-diff vs v3: 1: c442597a35 ! 1: 7656477360 avutil/frame: Add av_frame_copy_side_data() and av_frame_remove_all_side_data() @@ doc/APIchanges: libavutil: 2021-04-27 API changes, most recent first: -+2022-05-26 - xxxxxxxxx - lavu 57.26.100 - frame.h ++2022-05-26 - xxxxxxxxx - lavu 57.28.100 - frame.h + Add av_frame_remove_all_side_data(), av_frame_copy_side_data(), + AV_FRAME_TRANSFER_SD_COPY, and AV_FRAME_TRANSFER_SD_FILTER. + - 2022-05-23 - xxxxxxxxx - lavu 57.25.100 - avutil.h - Deprecate av_fopen_utf8() without replacement. - + 2022-06-12 - xxxxxxxxxx - lavf 59.25.100 - avio.h + Add avio_vprintf(), similar to avio_printf() but allow to use it + from within a function taking a variable argument list as input. ## libavutil/frame.c ## @@ libavutil/frame.c: FF_ENABLE_DEPRECATION_WARNINGS @@ libavutil/frame.h: int av_frame_copy(AVFrame *dst, const AVFrame *src); + * @param src a frame from which to copy the side data. + * @param flags a combination of AV_FRAME_TRANSFER_SD_* + * -+ * @return >= 0 on success, a negative AVERROR on error. ++ * @return 0 on success, a negative AVERROR on error. + * + * @note This function will create new references to side data buffers in src, + * unless the AV_FRAME_TRANSFER_SD_COPY flag is passed. @@ libavutil/version.h */ #define LIBAVUTIL_VERSION_MAJOR 57 --#define LIBAVUTIL_VERSION_MINOR 25 -+#define LIBAVUTIL_VERSION_MINOR 26 +-#define LIBAVUTIL_VERSION_MINOR 27 ++#define LIBAVUTIL_VERSION_MINOR 28 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ 2: 6f50d0bd57 = 2: 06976606c5 avcodec/vpp_qsv: Copy side data from input to output frame 3: f682b1d695 = 3: 320a8a535c avcodec/mpeg12dec: make mpeg_decode_user_data() accessible 4: 995d835233 = 4: e58ad6564f avcodec/hevcdec: make set_side_data() accessible 5: ac8dc06395 = 5: a57bfaebb9 avcodec/h264dec: make h264_export_frame_props() accessible 6: 27c3dded4d = 6: 3f2588563e avcodec/qsvdec: Implement SEI parsing for QSV decoders -- ffmpeg-codebot _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".