From patchwork Wed Feb 8 06:26:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 40322 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5494:b0:bf:7b3a:fd32 with SMTP id i20csp5005146pzk; Tue, 7 Feb 2023 22:26:55 -0800 (PST) X-Google-Smtp-Source: AK7set+bcnkX5Bn+ZHJaTSRqGWlnV026Oxl71sEoaPfRDaKmTG4fku54jazH4WGCMUUlnpOQKt/n X-Received: by 2002:a17:906:da1b:b0:86d:7c2d:f65b with SMTP id fi27-20020a170906da1b00b0086d7c2df65bmr6623259ejb.27.1675837615744; Tue, 07 Feb 2023 22:26:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675837615; cv=none; d=google.com; s=arc-20160816; b=y99Hz5V4PnWKPsW3U1Q5x2iN5+jIpJTAR46NRbACHZw3w7+/C4H6zL5lFNjjYc3wbB 6PaGMpZ8//2WZgFl+EIrwDK+I9X92yNwXHceSmMXLFGAVS3q6WfN0jdB4uzqpAEmHPoT sVksudQ1mUZUDQZp+7Z3oTfDo8jvs7xTuNDqAYKX57J1Izt8BZ9Bdu47ZJPCkUy8tU6J zmyX3iDF+GDdhQVIsAT8Y0InWozHXj+vbIfvYrj87Fdq8Fcd89eK1hwTMLt5/lVrs+8j Y3I0uMynPk2fkT14R2XNgr1i75rLadkD+d2saORVFmcMm3hiYhb/b7a8Teh6cK1VhpUL 8mEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=hx/S+Dm4ZvAI6jjV03JhsGvxG1gmwbhWi39Xo5mMIG4=; b=cQvlWzC1hPZYENsDszotGtQdBY6ImS4CFWnWycoHiNmNz8W8PLQ0kLRmmqDqHeHNzJ laSZkZBHvvurB7tQRhd0ttrUto9+7t7NKjDpBezfOKxPbcdxFQOk4wm7yhPgkQVpK3B4 KdbSPYQItnIbnqG/ovZTd14+pWjuBtPxjvVRa7T0XMT6/95U4dsvYoUdiuG2OWyyFfoq WTo4hvSB+P0l4ZNphR2bAAeGccyIQqJPtJJ3oO27F5v8CyQuo/78NV7q4XZZncuYHLYG qy/iHLgy20tT44f956BVMeWzV3CA9R1VTjvGyjAB1Xg3AjedQallKrGIT/nKKN5y3o2a lKVg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=A2TnVkXu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s12-20020a170906220c00b008ac8f6ad2c8si1276497ejs.57.2023.02.07.22.26.55; Tue, 07 Feb 2023 22:26:55 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=A2TnVkXu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 88E4468BDFA; Wed, 8 Feb 2023 08:26:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7BF3D68BDFA for ; Wed, 8 Feb 2023 08:26:39 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675837604; x=1707373604; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=PH8xHn067kETga56Ii/O4/bgHa8fEaqPfSfj/cOGXBk=; b=A2TnVkXuR2yM6WZ5DR9MZDWECeqn+Ycy5B2Xq1tR+iVhqoFL5/BWJRDs AE1JzDYU+EedR97IMrivBzBTD8GYZdaHKjAa+QarnmzZ2vmJdMQTktqGZ 0sjrT6hgYEemxI6/7YTy/dyx02vXR9VGmS+fInqUMPTAXf6V0yX6qBjJ4 +Jn3L7YBJn8SsKvk2ww6BNa8y0FuIGoLu6xFSGj5sD7QgjV+08I1LlaPB +DftoJNhO0/xn16/50wd++c69kFNeeQXoEQebwnW7aQfaqKR0FTGGXlZ+ cEDSanUB9Df8haVKO9KQUDbC5+V9FpiyWuSQdR8/h0ESo8hQVtTnoJT63 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10614"; a="328378425" X-IronPort-AV: E=Sophos;i="5.97,280,1669104000"; d="scan'208";a="328378425" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2023 22:26:31 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10614"; a="755924663" X-IronPort-AV: E=Sophos;i="5.97,280,1669104000"; d="scan'208";a="755924663" Received: from xhh-dg164.sh.intel.com ([10.238.5.169]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2023 22:26:30 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 8 Feb 2023 14:26:19 +0800 Message-Id: <20230208062619.19496-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230208062619.19496-1-haihao.xiang@intel.com> References: <20230208062619.19496-1-haihao.xiang@intel.com> Subject: [FFmpeg-devel] [PATCH 2/2] lavc/qsvenc_{h264, hevc}: import user data unregistered SEIs if available X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: vIZ/4MGcP7I9 From: Haihao Xiang option udu_sei is added, user should set udu_sei to true|on|1 if user data unregistered SEI is expected. Verify user data unregistered SEI with commands below: $ ffmpeg -y -f lavfi -i testsrc -vf "format=nv12" -c:v libx264 -frames:v 1 a.h264 $ ffmpeg -y -init_hw_device qsv -i a.h264 -c:v hevc_qsv -udu_sei 1 b.h265 $ ffmpeg -y -init_hw_device qsv -i a.h264 -c:v hevc_qsv -udu_sei 0 c.h265 $ ffmpeg -i b.h265 -vf showinfo -f null - $ ffmpeg -i c.h265 -vf showinfo -f null - Signed-off-by: Haihao Xiang --- doc/encoders.texi | 6 +++++ libavcodec/qsvenc.c | 47 ++++++++++++++++++++++++++++++++++++++++ libavcodec/qsvenc.h | 1 + libavcodec/qsvenc_h264.c | 1 + libavcodec/qsvenc_hevc.c | 1 + 5 files changed, 56 insertions(+) diff --git a/doc/encoders.texi b/doc/encoders.texi index 727f12a59d..a9972d8b9c 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3582,6 +3582,9 @@ skip_frame metadata indicates the number of missed frames before the current frame. @end table +@item udu_sei @var{boolean} +Import user data unregistered SEI if available into output. Default is 0 (off). + @end table @subsection HEVC Options @@ -3787,6 +3790,9 @@ skip_frame metadata indicates the number of missed frames before the current frame. @end table +@item udu_sei @var{boolean} +Import user data unregistered SEI if available into output. Default is 0 (off). + @end table @subsection MPEG2 Options diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 2f0e94a914..c0c40e706f 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -2046,6 +2046,49 @@ static void set_skip_frame_encode_ctrl(AVCodecContext *avctx, const AVFrame *fra return; } +static int set_udu_encode_ctrl(AVCodecContext *avctx, QSVEncContext *q, + const AVFrame *frame, mfxEncodeCtrl *enc_ctrl) +{ + if (!frame || !q->udu_sei) + return 0; + + for (int i = 0; i < frame->nb_side_data && enc_ctrl->NumPayload < QSV_MAX_ENC_PAYLOAD; i++) { + AVFrameSideData *sd = NULL; + mfxPayload *payload = NULL; + mfxU8* sei_data; + int j; + + sd = frame->side_data[i]; + if (sd->type != AV_FRAME_DATA_SEI_UNREGISTERED) + continue; + + /* SEI type: 1 byte, SEI size: sd->size / 255 + 1 bytes, SEI data: sd->size bytes */ + payload = av_malloc(sizeof(*payload) + sd->size / 255 + 2 + sd->size); + if (!payload) + return AVERROR(ENOMEM); + + memset(payload, 0, sizeof(*payload)); + sei_data = (mfxU8 *)(payload + 1); + // SEI header + sei_data[0] = 5; + for (j = 0; j < sd->size / 255; j++) + sei_data[j + 1] = 0xff; + sei_data[j + 1] = sd->size % 255; + // SEI data + memcpy(&sei_data[sd->size / 255 + 2], sd->data, sd->size); + + payload->BufSize = sd->size + sd->size / 255 + 2; + payload->NumBit = payload->BufSize * 8; + payload->Type = 5; + payload->Data = sei_data; + + enc_ctrl->Payload[enc_ctrl->NumPayload] = payload; + enc_ctrl->NumPayload++; + } + + return 0; +} + static int update_qp(AVCodecContext *avctx, QSVEncContext *q) { int updated = 0, new_qp = 0; @@ -2416,6 +2459,10 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, ret = set_roi_encode_ctrl(avctx, frame, enc_ctrl); if (ret < 0) goto free; + + ret = set_udu_encode_ctrl(avctx, q, frame, enc_ctrl); + if (ret < 0) + goto free; } if ((avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id == AV_CODEC_ID_H265) && diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 4a6fa2caed..5b66d31053 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -255,6 +255,7 @@ typedef struct QSVEncContext { int transform_skip; int a53_cc; + int udu_sei; #if QSV_HAVE_MF int mfmode; diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c index 071a9a79e9..31de8d7073 100644 --- a/libavcodec/qsvenc_h264.c +++ b/libavcodec/qsvenc_h264.c @@ -166,6 +166,7 @@ static const AVOption options[] = { #endif { "repeat_pps", "repeat pps for every frame", OFFSET(qsv.repeat_pps), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, + { "udu_sei", "Use user data unregistered SEI if available", OFFSET(qsv.udu_sei), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, { NULL }, }; diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index 5e23ca9647..5941c332cf 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -362,6 +362,7 @@ static const AVOption options[] = { { "int_ref_cycle_size", "Number of frames in the intra refresh cycle", OFFSET(qsv.int_ref_cycle_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, UINT16_MAX, VE }, { "int_ref_qp_delta", "QP difference for the refresh MBs", OFFSET(qsv.int_ref_qp_delta), AV_OPT_TYPE_INT, { .i64 = INT16_MIN }, INT16_MIN, INT16_MAX, VE }, { "int_ref_cycle_dist", "Distance between the beginnings of the intra-refresh cycles in frames", OFFSET(qsv.int_ref_cycle_dist), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT16_MAX, VE }, + { "udu_sei", "Use user data unregistered SEI if available", OFFSET(qsv.udu_sei), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, { NULL }, };