From patchwork Sat Apr 2 09:35:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 35133 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c05:b0:7a:e998:b410 with SMTP id bw5csp1023306pzb; Sat, 2 Apr 2022 02:36:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxn0HT1QSwrSn1vqE1MLuTu3dWRjguJB4MXx5EyUb2qRrX5uyUnizvjQsFic8C/N5fTcZqh X-Received: by 2002:a05:6402:2743:b0:419:4edd:c487 with SMTP id z3-20020a056402274300b004194eddc487mr24839096edd.325.1648892189929; Sat, 02 Apr 2022 02:36:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1648892189; cv=none; d=google.com; s=arc-20160816; b=aHLMh+tdfaizZBPxpP7XxuO1CIqmxteBPXZVH9jAfmvrLle8nGxjEda3C41mTQp5Ee 4qnXjK/YvQ5fIoRCRTpaGK2Y52A03ZKIFzIsodHGcomXyR+C7Z5NlvP9BR3+bbcIXnOS FZ9ie6EFbouhTONczteDidVS0t/wwJU3bISdNGCr7yIF7lfs/e8y1WaXdymg8JoynowV WZLtMFDraLX0h1lieOOml55IJDBi9SS/TwJSFj1qWbCS9CC0//ERavNIB2kTAtZfR72b iLhTF8QxuaDtCmPEXFl3rjiS3lQO2rPgUvL/DaIk7osmdibXHA8Okr8ayQDE93HKz3RW I6Jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=MvuA336khmupdv5PC1hk5FBG1InijIClFM2jxrvW6cQ=; b=brSMbvo/3dNVddjDxEKT/OUnKpv/cCb2Y4h3EEc6wuUdKM7OaKF9+lSbFBnmXJmbqp 4UvZjpIaQMb5SyX9U9WlcldOHGOTuFymXqrm7RIhB55axOXNr4WoNuzW43PZ7PLn4UxM u3m+O88Jab1DDE2NlD4iWreozBLSyzMXve/1KymkH9ZHQBfFGLFtr3jo6n/lZHulXu0/ Syb0OZG39RA9aOKxaH9so+PQKLADPIPgEsNoqaCtTnLdGavyqBtpnszHi80ppdauqAxq y+vajn8jssPjtXVrRdpqhLj9O4LZDBKNw/pq814iRmFr0qw25BFKrEYc+BuM9Xi1uTkc RfXg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Tr0fXUgx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f9-20020a50e089000000b00418f9b49b63si2882112edl.475.2022.04.02.02.36.29; Sat, 02 Apr 2022 02:36:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Tr0fXUgx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C7D5E68B21E; Sat, 2 Apr 2022 12:36:26 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0170968AE6D for ; Sat, 2 Apr 2022 12:36:19 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648892185; x=1680428185; h=from:to:subject:date:message-id:mime-version: content-transfer-encoding; bh=RVeW5K0CZ/OR4RGVvOFwg3zwW22v5cYjzSBLFJ823fw=; b=Tr0fXUgxludm5jNkODqVAa/zr+SUHV2HIaBSlu9t5UKl73mv+HDreXwf FVn5fLBUGmgP6SH6iTLvB8BttH+xDcbVa2gpAMbZcux35na+EcUFD9G/u H4DrycOFQrIF+AFyy+40WQSrqiP4VacByjGCwzAfsFslKF9YU3WA/S7jW 35oEgCUjo+Ny7jRHPSx+VzAgm0FxTFDVEScZVoT6J9rA4slitrRPz5Xyr PPFyjt/cmZgMymqVVDuhExsGCJJnP4SAiLMVgr6aZ0JPeJagH+TFVhxaS GeuE0h9rUNRVOwtsNgkzhW9vNSgvJOn9QWw4fR+y5yCH+Ku+Ucj3VsexK A==; X-IronPort-AV: E=McAfee;i="6200,9189,10304"; a="285228667" X-IronPort-AV: E=Sophos;i="5.90,229,1643702400"; d="scan'208";a="285228667" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Apr 2022 02:36:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,229,1643702400"; d="scan'208";a="789051168" Received: from wenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.4]) by fmsmga006.fm.intel.com with ESMTP; 02 Apr 2022 02:36:17 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Sat, 2 Apr 2022 17:35:52 +0800 Message-Id: <20220402093553.965954-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/2] libavcodec/qsvdec: Add more pixel format support to qsvdec X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: eFMoMx1LDN5E Qsv decoder only supports directly output nv12 and p010 to system memory. For other format, we need to download frame from qsv format to system memory. Now add other supported format to qsvdec. Signed-off-by: Wenbin Chen --- libavcodec/qsv.c | 36 ++++++++++++++++++++++++++++++++++++ libavcodec/qsv_internal.h | 3 +++ libavcodec/qsvdec.c | 23 +++++++++++++++++------ 3 files changed, 56 insertions(+), 6 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index b75877e698..cc1352aa2a 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -244,6 +244,42 @@ int ff_qsv_map_pixfmt(enum AVPixelFormat format, uint32_t *fourcc) } } +int ff_qsv_map_frame_to_surface(const AVFrame *frame, mfxFrameSurface1 *surface) +{ + switch (frame->format) { + case AV_PIX_FMT_NV12: + case AV_PIX_FMT_P010: + surface->Data.Y = frame->data[0]; + surface->Data.UV = frame->data[1]; + /* The SDK checks Data.V when using system memory for VP9 encoding */ + surface->Data.V = surface->Data.UV + 1; + break; + case AV_PIX_FMT_X2RGB10LE: + case AV_PIX_FMT_BGRA: + surface->Data.B = frame->data[0]; + surface->Data.G = frame->data[0] + 1; + surface->Data.R = frame->data[0] + 2; + surface->Data.A = frame->data[0] + 3; + break; + case AV_PIX_FMT_YUYV422: + surface->Data.Y = frame->data[0]; + surface->Data.U = frame->data[0] + 1; + surface->Data.V = frame->data[0] + 3; + break; + + case AV_PIX_FMT_Y210: + surface->Data.Y16 = (mfxU16 *)frame->data[0]; + surface->Data.U16 = (mfxU16 *)frame->data[0] + 1; + surface->Data.V16 = (mfxU16 *)frame->data[0] + 3; + break; + default: + return AVERROR(ENOSYS); + } + surface->Data.PitchLow = frame->linesize[0]; + + return 0; +} + int ff_qsv_find_surface_idx(QSVFramesContext *ctx, QSVFrame *frame) { int i; diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 58186ea7ca..e2aecdcbd6 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -147,4 +147,7 @@ int ff_qsv_find_surface_idx(QSVFramesContext *ctx, QSVFrame *frame); void ff_qsv_frame_add_ext_param(AVCodecContext *avctx, QSVFrame *frame, mfxExtBuffer *param); +int ff_qsv_map_frame_to_surface(const AVFrame *frame, mfxFrameSurface1 *surface); + + #endif /* AVCODEC_QSV_INTERNAL_H */ diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 6236391357..f1d56b2af3 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -129,21 +129,28 @@ static int qsv_get_continuous_buffer(AVCodecContext *avctx, AVFrame *frame, frame->linesize[0] = FFALIGN(avctx->width, 128); break; case AV_PIX_FMT_P010: + case AV_PIX_FMT_YUYV422: frame->linesize[0] = 2 * FFALIGN(avctx->width, 128); break; + case AV_PIX_FMT_Y210: + frame->linesize[0] = 4 * FFALIGN(avctx->width, 128); + break; default: av_log(avctx, AV_LOG_ERROR, "Unsupported pixel format.\n"); return AVERROR(EINVAL); } - frame->linesize[1] = frame->linesize[0]; frame->buf[0] = av_buffer_pool_get(pool); if (!frame->buf[0]) return AVERROR(ENOMEM); frame->data[0] = frame->buf[0]->data; - frame->data[1] = frame->data[0] + - frame->linesize[0] * FFALIGN(avctx->height, 64); + if (avctx->pix_fmt == AV_PIX_FMT_NV12 || + avctx->pix_fmt == AV_PIX_FMT_P010) { + frame->linesize[1] = frame->linesize[0]; + frame->data[1] = frame->data[0] + + frame->linesize[0] * FFALIGN(avctx->height, 64); + } ret = ff_attach_decode_data(frame); if (ret < 0) @@ -423,9 +430,11 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext *q, QSVFrame *frame) if (frame->frame->format == AV_PIX_FMT_QSV) { frame->surface = *(mfxFrameSurface1*)frame->frame->data[3]; } else { - frame->surface.Data.PitchLow = frame->frame->linesize[0]; - frame->surface.Data.Y = frame->frame->data[0]; - frame->surface.Data.UV = frame->frame->data[1]; + ret = ff_qsv_map_frame_to_surface(frame->frame, &frame->surface); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "map frame to surface failed.\n"); + return ret; + } } frame->surface.Info = q->frame_info; @@ -990,6 +999,8 @@ const AVCodec ff_##x##_qsv_decoder = { \ .priv_class = &x##_qsv_class, \ .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_NV12, \ AV_PIX_FMT_P010, \ + AV_PIX_FMT_YUYV422, \ + AV_PIX_FMT_Y210, \ AV_PIX_FMT_QSV, \ AV_PIX_FMT_NONE }, \ .hw_configs = qsv_hw_configs, \ From patchwork Sat Apr 2 09:35:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 35134 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c05:b0:7a:e998:b410 with SMTP id bw5csp1023342pzb; Sat, 2 Apr 2022 02:36:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxlrDDoqV91aBqnuGDwtMbpFc82ydiUXy91x/oFEVN1yVUvSKBOiX4Yk2YSG8VXj4cJpPZT X-Received: by 2002:a17:907:2ce1:b0:6df:85e4:d65d with SMTP id hz1-20020a1709072ce100b006df85e4d65dmr3233115ejc.209.1648892198442; Sat, 02 Apr 2022 02:36:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1648892198; cv=none; d=google.com; s=arc-20160816; b=vyo4v06mvvwjOnhe2Qw3rwYZgpAIxDkoLHqyUxbGxcKDy401Tsfy7unGhgkWNXpJ6x Bg+vLeyB8Dc2AF2t1/kSk7shlAKLoFNNqwTh0X5NNl7939W9UEBnK6ZByz8KrO5RxMHf 5WQnVufrX6DHr6ctooRISyugxvDhkdZglOjr8Q9zI3gRN73UnAvPRcJfae5Ir81H12/j oNLhK1tsetsZZPeYrDNqgW+KaMrehvTLcLTvDRi9wyxnGl6J7gA3HDGWPB6X5lKixJ5I 3rbJQgnc0K7Vx6KG4DMSYvQ1uUE/u9gsZvt55Tp2BFv2ppzmbL7CJSRCJUOze6WOn+DX Fg2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=iQ92lM0Ys/aLSCiLojbwXGtmeMinVzX16IB4upLYbus=; b=DmAjYNsPtlF+aqn++Y2c/8eOzw1Esde/7Z9ZGO+o/QAhfI/OIqQRQVOFrK9sIqAX3u 5K9uO8f/8DEaJqovhBxARrjdynPWrhMd34pVbuDVQ10iPgNPRqjnyGcea7Ch/0OoydaP /ANXFI7ESuDpePxsp+o7BmUImLWHxFWliPOKQodbAzMYF+Heij3/OqV01k8ZGuwYV6La NrsfvT+9cgAzTFy+tGJn/QXtZnfsrZl/4TK0UlAcOmzlUmVuXDkkhk3V3qUUefDToLQx SrcAKhPyc26I3jBw/kvie5i6lM28Zunh9U/M8M/+yIZZ4fMR8fxIn5oxY5Vd0L1OLWHu Bvbg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=TTjHsb5r; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i4-20020a1709061e4400b006df8927010fsi2709205ejj.777.2022.04.02.02.36.38; Sat, 02 Apr 2022 02:36:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=TTjHsb5r; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B6AD668B20C; Sat, 2 Apr 2022 12:36:32 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DA75568B20C for ; Sat, 2 Apr 2022 12:36:25 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648892191; x=1680428191; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=JZO532CCTP/6TLgrxFOL4RCh5ooeQRkaQVezB1X9BXo=; b=TTjHsb5r9aKDU13Yl7zoitQlaapFxlFRmED5kQpTK9tiWq/0w8lY20mh 8xYXYKQe7SyGJI7JE5rk6CerBpB+jjRbMRR0hzwqz0ItStNezKn3W/+3l emBUe+VI8AkJj4/ctYQo3pcDW8s8Qn3J0DarksUCIcy6ZKXv3jlQ8VTHB f0/S6NcPlrsJcnqju2jjmpGhGDkca2A1rrj82wBT2wMxKQa45JQ6+U8vI a5TkYm2RNYZKfjethMT9ok/Sy69Mk/ATHfw9+7aFzXq34XR7kqNMuhcs8 mhleqdzlh97z3DrE8QLoQW7L1IlwvmrQ2Gzl2sFDW4tD2q34WFkM6uI0F w==; X-IronPort-AV: E=McAfee;i="6200,9189,10304"; a="285228668" X-IronPort-AV: E=Sophos;i="5.90,229,1643702400"; d="scan'208";a="285228668" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Apr 2022 02:36:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,229,1643702400"; d="scan'208";a="789051171" Received: from wenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.4]) by fmsmga006.fm.intel.com with ESMTP; 02 Apr 2022 02:36:18 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Sat, 2 Apr 2022 17:35:53 +0800 Message-Id: <20220402093553.965954-2-wenbin.chen@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220402093553.965954-1-wenbin.chen@intel.com> References: <20220402093553.965954-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 2/2] libavcodec/qsvenc: Add more pixel format support to qsvenc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: mMF5z8QwyEhL Qsv encoder only support input P010 and nv12 format directly from system memory. For other format, we need to upload frame to device memory and input qsv format to encoder. Now add other system memory format support to qsv encoder. Signed-off-by: Wenbin Chen --- libavcodec/qsvenc.c | 30 ++++-------------------------- libavcodec/qsvenc_hevc.c | 2 ++ 2 files changed, 6 insertions(+), 26 deletions(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 40d60cde3c..55cce96022 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -1602,32 +1602,10 @@ static int submit_frame(QSVEncContext *q, const AVFrame *frame, else if (frame->repeat_pict == 4) qf->surface.Info.PicStruct |= MFX_PICSTRUCT_FRAME_TRIPLING; - qf->surface.Data.PitchLow = qf->frame->linesize[0]; - qf->surface.Data.Y = qf->frame->data[0]; - qf->surface.Data.UV = qf->frame->data[1]; - - /* The SDK checks Data.V when using system memory for VP9 encoding */ - switch (frame->format) { - case AV_PIX_FMT_NV12: - qf->surface.Data.V = qf->surface.Data.UV + 1; - break; - - case AV_PIX_FMT_P010: - qf->surface.Data.V = qf->surface.Data.UV + 2; - break; - - case AV_PIX_FMT_X2RGB10: - case AV_PIX_FMT_BGRA: - qf->surface.Data.B = qf->frame->data[0]; - qf->surface.Data.G = qf->frame->data[0] + 1; - qf->surface.Data.R = qf->frame->data[0] + 2; - qf->surface.Data.A = qf->frame->data[0] + 3; - break; - - default: - /* should not reach here */ - av_assert0(0); - break; + ret = ff_qsv_map_frame_to_surface(qf->frame, &qf->surface); + if (ret < 0) { + av_log(q->avctx, AV_LOG_ERROR, "map frame to surface failed.\n"); + return ret; } } qf->surface.Data.TimeStamp = av_rescale_q(frame->pts, q->avctx->time_base, (AVRational){1, 90000}); diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index ade546d4ca..c8d58f9247 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -303,6 +303,8 @@ const AVCodec ff_hevc_qsv_encoder = { .capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HYBRID, .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_NV12, AV_PIX_FMT_P010, + AV_PIX_FMT_YUYV422, + AV_PIX_FMT_Y210, AV_PIX_FMT_QSV, #if QSV_VERSION_ATLEAST(1, 17) AV_PIX_FMT_BGRA,