From patchwork Wed Apr 6 08:48:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 35224 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c05:b0:7a:e998:b410 with SMTP id bw5csp761809pzb; Wed, 6 Apr 2022 01:48:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzNwfzjfSnlcGaOVtDypI9ETuURNsLyF14HisUGf/od4659GtxdwecvsWxUJDgsaE/YoYRF X-Received: by 2002:a05:6402:1941:b0:413:2822:9c8 with SMTP id f1-20020a056402194100b00413282209c8mr7618955edz.13.1649234921439; Wed, 06 Apr 2022 01:48:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649234921; cv=none; d=google.com; s=arc-20160816; b=Za+Be6Q1FhjOfdAvozv5Q+dmB8ceGvvQcbPNUuMvF1VPLCNnsbzj9g6WJH5mq1GNEJ 05L8mZJwP+5EaP21GHeNkOv6FSSVJOwvW5PAfhpPZcQ8dI3MDiVfy+ey1SBNngmbz5FJ 0AmHX+OyFhbh0Zkpk/8Iz1ALJRuSolZuwmOHtTL50Lu+2jmUocZW7Hn7jXkEv5B5fJ6P pmOqMcSVHMTihwPX1d3+hB7O9QJ+qxntFjUKGy+dlZ5vq1sRPKsA6rnj9eil36xceUNX kZfzDsRKnChZ+e070h7HVzzSKZ4fkGA3UrWk/lxPckKCG57txyVFA/ZHb4X0MDQ5EmCx nSGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=78qWd+R7bpRF34SLSlCgrxQbrlaAboz3T/uaSwYXirM=; b=I/4dAg0d7XFZCaeyPHFMLmXhD8tJtMwTE/R+jEBY0JjFaj2liw+AwNKiGClGcOdk5Y 01QTcy9a6k1ryXGXGqnZ+rEycqRTbT/nzu+20Zd0EyHDTea3KU/Ow5ttmo+HrwTxVfUm JTE+C47FRdPRqwX798iL1+fdd7uBsovCAI8TjCW3pANHMIhvvJjj6RnKPiTqqWTyoioQ PIam7OlNOXD5aRcrVQ0hjA4rD7UWifx50+gWMhdHS7JxC8Su4wIDXwNoTbAlFwfDt8iE UJUBWdJiCVEZuEI/ReBTvZqL3fzbEPhBfpcDEzYTl8Qcd0rl7uVm/2XJq3DTnpUjIi5k eVIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=JtTbTBEw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e22-20020a170906c01600b006e7e6c83ea0si6955467ejz.157.2022.04.06.01.48.41; Wed, 06 Apr 2022 01:48:41 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=JtTbTBEw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 17DE968B277; Wed, 6 Apr 2022 11:48:38 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0F37868A467 for ; Wed, 6 Apr 2022 11:48:30 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649234916; x=1680770916; h=from:to:subject:date:message-id:mime-version: content-transfer-encoding; bh=bje8hYCtYUSL1/KUVmyMbhAFlOKvhdY66SPoLB8rzdI=; b=JtTbTBEwvCO/l+C0/n7bVMoSUifPalC1n69EMvsizjV5sLvWX8eMjB/l kyGgaAYOVpLUCxRnI9kkg+r5QAlxyLGA1/RJjQJYRCkSQeOmcDSd/+dpE 36OaSQH1iIw49p4rzTkHWS5yHdVOe+zEyNLK8W+tHoxbapSM0czidDhU4 9K7EO9TPWyHSCX+oqqvSLbuL4NgejOwZUd18PHcCVcwLl2qP86CXyCB+i Id1C5bq8CvVAJKtVeZml09e8nTFqeDpyurwUXrNA3/CD9VtSYYe10/dWd 02X+ltgAAhMWrVxvtH53M8swLzadcnjX1q9OOLIABB1HTxbf6xBcvhEuv Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10308"; a="260982944" X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="260982944" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Apr 2022 01:48:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="652266485" Received: from wenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.4]) by fmsmga002.fm.intel.com with ESMTP; 06 Apr 2022 01:48:27 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Wed, 6 Apr 2022 16:48:03 +0800 Message-Id: <20220406084804.1043196-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 1/2] libavcodec/qsvdec: Add more pixel format support to qsvdec X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ziYPYSb+ILeu Qsv decoder only supports directly output nv12 and p010 to system memory. For other format, we need to download frame from qsv format to system memory. Now add other supported format to qsvdec. Signed-off-by: Wenbin Chen --- libavcodec/qsv.c | 36 ++++++++++++++++++++++++++++++++++++ libavcodec/qsv_internal.h | 3 +++ libavcodec/qsvdec.c | 23 +++++++++++++++++------ 3 files changed, 56 insertions(+), 6 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index 67d0e3934a..b86c20b153 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -244,6 +244,42 @@ int ff_qsv_map_pixfmt(enum AVPixelFormat format, uint32_t *fourcc) } } +int ff_qsv_map_frame_to_surface(const AVFrame *frame, mfxFrameSurface1 *surface) +{ + switch (frame->format) { + case AV_PIX_FMT_NV12: + case AV_PIX_FMT_P010: + surface->Data.Y = frame->data[0]; + surface->Data.UV = frame->data[1]; + /* The SDK checks Data.V when using system memory for VP9 encoding */ + surface->Data.V = surface->Data.UV + 1; + break; + case AV_PIX_FMT_X2RGB10LE: + case AV_PIX_FMT_BGRA: + surface->Data.B = frame->data[0]; + surface->Data.G = frame->data[0] + 1; + surface->Data.R = frame->data[0] + 2; + surface->Data.A = frame->data[0] + 3; + break; + case AV_PIX_FMT_YUYV422: + surface->Data.Y = frame->data[0]; + surface->Data.U = frame->data[0] + 1; + surface->Data.V = frame->data[0] + 3; + break; + + case AV_PIX_FMT_Y210: + surface->Data.Y16 = (mfxU16 *)frame->data[0]; + surface->Data.U16 = (mfxU16 *)frame->data[0] + 1; + surface->Data.V16 = (mfxU16 *)frame->data[0] + 3; + break; + default: + return AVERROR(ENOSYS); + } + surface->Data.PitchLow = frame->linesize[0]; + + return 0; +} + int ff_qsv_find_surface_idx(QSVFramesContext *ctx, QSVFrame *frame) { int i; diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 58186ea7ca..e2aecdcbd6 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -147,4 +147,7 @@ int ff_qsv_find_surface_idx(QSVFramesContext *ctx, QSVFrame *frame); void ff_qsv_frame_add_ext_param(AVCodecContext *avctx, QSVFrame *frame, mfxExtBuffer *param); +int ff_qsv_map_frame_to_surface(const AVFrame *frame, mfxFrameSurface1 *surface); + + #endif /* AVCODEC_QSV_INTERNAL_H */ diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index de4af1754d..c4296f80d7 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -132,21 +132,28 @@ static int qsv_get_continuous_buffer(AVCodecContext *avctx, AVFrame *frame, frame->linesize[0] = FFALIGN(avctx->width, 128); break; case AV_PIX_FMT_P010: + case AV_PIX_FMT_YUYV422: frame->linesize[0] = 2 * FFALIGN(avctx->width, 128); break; + case AV_PIX_FMT_Y210: + frame->linesize[0] = 4 * FFALIGN(avctx->width, 128); + break; default: av_log(avctx, AV_LOG_ERROR, "Unsupported pixel format.\n"); return AVERROR(EINVAL); } - frame->linesize[1] = frame->linesize[0]; frame->buf[0] = av_buffer_pool_get(pool); if (!frame->buf[0]) return AVERROR(ENOMEM); frame->data[0] = frame->buf[0]->data; - frame->data[1] = frame->data[0] + - frame->linesize[0] * FFALIGN(avctx->height, 64); + if (avctx->pix_fmt == AV_PIX_FMT_NV12 || + avctx->pix_fmt == AV_PIX_FMT_P010) { + frame->linesize[1] = frame->linesize[0]; + frame->data[1] = frame->data[0] + + frame->linesize[0] * FFALIGN(avctx->height, 64); + } ret = ff_attach_decode_data(frame); if (ret < 0) @@ -426,9 +433,11 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext *q, QSVFrame *frame) if (frame->frame->format == AV_PIX_FMT_QSV) { frame->surface = *(mfxFrameSurface1*)frame->frame->data[3]; } else { - frame->surface.Data.PitchLow = frame->frame->linesize[0]; - frame->surface.Data.Y = frame->frame->data[0]; - frame->surface.Data.UV = frame->frame->data[1]; + ret = ff_qsv_map_frame_to_surface(frame->frame, &frame->surface); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "map frame to surface failed.\n"); + return ret; + } } frame->surface.Info = q->frame_info; @@ -992,6 +1001,8 @@ const FFCodec ff_##x##_qsv_decoder = { \ .p.priv_class = &x##_qsv_class, \ .p.pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_NV12, \ AV_PIX_FMT_P010, \ + AV_PIX_FMT_YUYV422, \ + AV_PIX_FMT_Y210, \ AV_PIX_FMT_QSV, \ AV_PIX_FMT_NONE }, \ .hw_configs = qsv_hw_configs, \ From patchwork Wed Apr 6 08:48:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 35225 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c05:b0:7a:e998:b410 with SMTP id bw5csp761846pzb; Wed, 6 Apr 2022 01:48:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzHXVIAAe7+KCvH7t0bULDYwZVwVmPVQSpIMBSZeoHwvNRIWlCdsz4CVEWZirafLERs1QTe X-Received: by 2002:a17:907:9622:b0:6e0:b38b:df18 with SMTP id gb34-20020a170907962200b006e0b38bdf18mr7181027ejc.182.1649234930563; Wed, 06 Apr 2022 01:48:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649234930; cv=none; d=google.com; s=arc-20160816; b=phxX1xWfZvFGqBiLfvsekMwr0BoTHuLG1l8vq9QVIK5Q907VQlgIP8ylcHed1POJPB 5pOMVKMuCWSzFcApual+2FaYurC9inJOnt6/a2wro+kUIgjoSIS0B3ahC1G1aEFbtptp uCAkRJwy0WkySSbPwhKt1RFbCzUAAKfQ3wqSTv5Z1pT/OFpuB/nCex4pFFjDYsVZCZo+ OlS+mm9vJ0PCOteaPZ3kXMUE8SirSkTE5Zb3wUQQ3YGrgCm3zXYJp4Vcc8BIWk4PB4vu H/gbYW/TObMOKU8v7XPWaSIvRl/n4h1752k0VZKL+AbT1idZIOBMY4xiT4YgCagBirXm xukg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=yKnu7hx41LBN+4SEJJNgJj8fOvkTpNYmSGEM4clgSvI=; b=yVKCBr0Zr+pBfivAanbqNmsV7RsWKppb4g2dxQASEjXmjPvCAmbW1m3Fi7B4JW/6Ce DE/Ce/oFlo3i+7t+y7M3uvyR2dJ2HPHjwBDWuopIBazwPBJ3f2Nv/T/1hzNo65FVCrBK NGvvnI98o93C55sXcC8GWR85MSU+K/v+tLom5RkT5SI0hrB+NKwyxbodPOx4QfYNbXww XrG7yL/8jCw48pD3ICFnp4I3CeTUAkEEyF13u00dRRK3Jqaw4mBtXbrjdQ7W1I5PGupP VkdwNvorN7Mmd6q1YtmJKiFw3vh1VW+qMcrE/A1K6DklTt7VCTS9+kGY8Qt5ta/80OW0 pV3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=AgvqTNST; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k1-20020a1709060ca100b006df76385e27si10781373ejh.711.2022.04.06.01.48.50; Wed, 06 Apr 2022 01:48:50 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=AgvqTNST; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1600D68B254; Wed, 6 Apr 2022 11:48:44 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EFD8A68B27B for ; Wed, 6 Apr 2022 11:48:36 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649234922; x=1680770922; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=4G0h0u+AlW0GHBU+vo9A6wwbNai9u6Q6o9lZi2cSV4Q=; b=AgvqTNSTsS+HuO3P132dOGsNgfyrLmRKzYh2+fhvIvFw8wxpawOX8lYw DNdSGMjtI9JutMK2Lw3ti0ZdA5oCPxH3bT6fWpE1yUtbB2ZRH7fR4RDHA vda2d8plyqOTdIRqk43piebU5fTSB0CR0VkDbWU8D1hQ69sQDwgbXg1xg g4TGG7xsxgE+BHTi5RFrLdO0mOGv/OGDnWSvIdfcodn9Px1An6Zz12Pu3 pAv3SZX52trangxKIkLdMws9WXr6aa1lM9pNTrEccxNU3KFP30Yh3RqiJ nc8AmZoC4Z0+FcjwvqQregb/K1Cm0T00e9iD/q0WCdc2KKAN1etwNiawf Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10308"; a="260982945" X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="260982945" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Apr 2022 01:48:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,239,1643702400"; d="scan'208";a="652266491" Received: from wenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.4]) by fmsmga002.fm.intel.com with ESMTP; 06 Apr 2022 01:48:28 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Wed, 6 Apr 2022 16:48:04 +0800 Message-Id: <20220406084804.1043196-2-wenbin.chen@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220406084804.1043196-1-wenbin.chen@intel.com> References: <20220406084804.1043196-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 2/2] libavcodec/qsvenc: Add more pixel format support to qsvenc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: aA6etRbvlIO4 Qsv encoder only support input P010 and nv12 format directly from system memory. For other format, we need to upload frame to device memory and input qsv format to encoder. Now add other system memory format support to qsv encoder. Signed-off-by: Wenbin Chen --- libavcodec/qsvenc.c | 30 ++++-------------------------- libavcodec/qsvenc_hevc.c | 2 ++ 2 files changed, 6 insertions(+), 26 deletions(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 55ce3d2499..345df0190c 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -1606,32 +1606,10 @@ static int submit_frame(QSVEncContext *q, const AVFrame *frame, else if (frame->repeat_pict == 4) qf->surface.Info.PicStruct |= MFX_PICSTRUCT_FRAME_TRIPLING; - qf->surface.Data.PitchLow = qf->frame->linesize[0]; - qf->surface.Data.Y = qf->frame->data[0]; - qf->surface.Data.UV = qf->frame->data[1]; - - /* The SDK checks Data.V when using system memory for VP9 encoding */ - switch (frame->format) { - case AV_PIX_FMT_NV12: - qf->surface.Data.V = qf->surface.Data.UV + 1; - break; - - case AV_PIX_FMT_P010: - qf->surface.Data.V = qf->surface.Data.UV + 2; - break; - - case AV_PIX_FMT_X2RGB10: - case AV_PIX_FMT_BGRA: - qf->surface.Data.B = qf->frame->data[0]; - qf->surface.Data.G = qf->frame->data[0] + 1; - qf->surface.Data.R = qf->frame->data[0] + 2; - qf->surface.Data.A = qf->frame->data[0] + 3; - break; - - default: - /* should not reach here */ - av_assert0(0); - break; + ret = ff_qsv_map_frame_to_surface(qf->frame, &qf->surface); + if (ret < 0) { + av_log(q->avctx, AV_LOG_ERROR, "map frame to surface failed.\n"); + return ret; } } qf->surface.Data.TimeStamp = av_rescale_q(frame->pts, q->avctx->time_base, (AVRational){1, 90000}); diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index 36c2d484ad..3a2d50e332 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -303,6 +303,8 @@ const FFCodec ff_hevc_qsv_encoder = { .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HYBRID, .p.pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_NV12, AV_PIX_FMT_P010, + AV_PIX_FMT_YUYV422, + AV_PIX_FMT_Y210, AV_PIX_FMT_QSV, #if QSV_VERSION_ATLEAST(1, 17) AV_PIX_FMT_BGRA,