From patchwork Thu Mar 24 06:12:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 34945 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:ab0:5fda:0:0:0:0:0 with SMTP id g26csp1059401uaj; Wed, 23 Mar 2022 23:13:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyJdH8QEZLhACd9JGKIbyLYNeXheItFdP1xbfYIY1omColK6xKTyC18IF+TVer2LuFGZi4c X-Received: by 2002:a05:6402:c81:b0:410:a329:e27a with SMTP id cm1-20020a0564020c8100b00410a329e27amr4721625edb.142.1648102390672; Wed, 23 Mar 2022 23:13:10 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v24-20020a509558000000b00418c2b5bf05si18674559eda.487.2022.03.23.23.13.09; Wed, 23 Mar 2022 23:13:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@nxp.com header.s=selector2 header.b=IHIl9glk; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 48C5E680226; Thu, 24 Mar 2022 08:13:05 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60055.outbound.protection.outlook.com [40.107.6.55]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1F5A4680226 for ; Thu, 24 Mar 2022 08:12:58 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dxw7En2t+8nnDQpxQu8t64BkGwOU9W6QpXnZzW2n7WWM+M3FzsSl/5bjJtN54g4fKZ/v8WB2VjU3kqsZDYwa62HGjvUgIb9o6DDxjUbnbG9oQ903X4O7UNUdrIWFUgq6XAZLMB2Ng1Srn0xL5YgmpUda+qX7e6u8DXXq+yOtneRpZzzPNzIARukhaOqlGy5tRh0ux2k77UH1JuwbhgU64X1C8EQdHCOmYcddcDIsb3aE9uHuR3Wxw19geBDLsn/GGCO/4uLv8TBK8aIddCotiHtgxeSiyG8B5dAfd6bFen02KCehIU6anVjgDLFTDvLBL71coReiuHLamgT31OWFTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qHQr9R3fdGqmntRhUdbbGVDnj91ogac29gFJBNKquEQ=; b=TEPxEP5vxekHCwAzOBOwJTIxZjDyvUfsrbi7e5YfMc5L9CAv+OLpZzD4Cx5g6ZPMQ3VGbrX5ELToR6meNsSCLhtHe26MCcvokmFOqNtABUMgvalZD3dzaKz9V6uBMMTz24cTS33ZStS68JPUW3BJGyQCmgu4W21e7t8EW9sac10nb/RiorbiClC1iMEkf9+h1lP0SMO4EUqxNl9TcWrp8NCbU7gaNi0GwHtGy0+hH+gDUVvYH7sZYt51mE7GhfsWsQ8Z1Xy7DF/PVIopMOimZcj0m4bnA1AuMdQCvuISi/MIYkMcJvKL4j7jZtxuWJAp/emycohMmmEtlpSzOJengA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qHQr9R3fdGqmntRhUdbbGVDnj91ogac29gFJBNKquEQ=; b=IHIl9glkZwOB6mbOa/uAYFXaKt6Eqtn73VGjiFmJ7GtpKkBG5yEg3pR/HJeTbct2frCyXaPzx4atuqbp8pd8yqp1MupJvGc1PPR7yc+onPk3WdEeA93sUKtFPB+wUGq/O8vf4p5TO/FysOSjhiMTH/34aJbemJzWTnSEVLJYw9o= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by VI1PR04MB2960.eurprd04.prod.outlook.com (2603:10a6:802:9::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.17; Thu, 24 Mar 2022 06:12:53 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::c39:69cf:c4ea:967]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::c39:69cf:c4ea:967%5]) with mapi id 15.20.5102.018; Thu, 24 Mar 2022 06:12:53 +0000 From: Ming Qian To: ffmpeg-devel@ffmpeg.org Date: Thu, 24 Mar 2022 14:12:23 +0800 Message-Id: <20220324061223.18493-1-ming.qian@nxp.com> X-Mailer: git-send-email 2.33.0 X-ClientProxiedBy: SI2PR02CA0029.apcprd02.prod.outlook.com (2603:1096:4:195::6) To AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 81d0a955-62d8-42ad-d8f1-08da0d5d541f X-MS-TrafficTypeDiagnostic: VI1PR04MB2960:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YI7LnWLsvCVcZv/jG4oB5D1F/rr/UKV9f3AbI0hq2dV6OK8Zb4YZzF3UbLRenJWaewe/Acdx6aSyqQat1hKLwVA5jTeAaLILM/iIy3FWkwX8IK1m1HUEVogjv1R5ApkrNfZ7mPa4gB6HQ+z0coHXzY9mZpnt1Cb0OSI8lOUi8OUpYhGb1lHwz48c8+o0urV4Y1tfvL/mm+DB9lv3u6V2/9THZm7Oh8+xZFvq0UvJQ1MRiRJRYryt4PGkOHeGHmBpnE1JPJMbzZwkBgMQrBKG2zpHmsG2kUC5vgdOUVrzEXrczW+iaIKZm2qb496o6gq65d84E5625EiDhS+8SNjg2xUdEu+VD8WXiVXD4pQDobxpHsIZKTwOtzwpxmcenOBrBfOu9++bq5TK1TUIM06Dpcb2bGtNbOSxnaAtqapr/QIIgzVhb/P0IaBNiJFhGVkZTAaBsWv3oiMDkj1EZbOaXYVAtJWe4zl0k57qK9qMsNkfvttv5rOKxG5MrHDUrdAC4hTkWiKoUZuVawLM9foGi37hWLYZPId+VTBuZ4HN/Bpc2uv+1ylRnx38/zkaxaRxMjLs2o3Hju0AP4kaS5rXGiPDTEQnvBZLcb2Cx4ekrKF+iZsc3VyBSaO7zJSQ8PGw3AG3ZqOLdAG4fI17i1sp5httzP3BpNsrg3I87k2vGQMalnu4knUxFfoHZNyBWZeeavo9wcFgJYSJdpH0Ur3uGVKZBcBmhXy4SStIRn7+4/0= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(86362001)(1076003)(8676002)(66556008)(8936002)(38100700002)(38350700002)(44832011)(30864003)(66476007)(5660300002)(66946007)(6512007)(6506007)(6666004)(52116002)(83380400001)(36756003)(2906002)(508600001)(316002)(6486002)(2616005)(6916009)(186003)(26005)(70780200001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: kwXbiY5VCUTwPODicxDidcbqPTowAhMbE0vGtQd4R2UYMgJw54g8wD8zMKSuU5DPBlIBf8DQoVVNVzu0JoLbjnJSGfsuWof2Ms7+k/5wVao2JG9qAJ5cRqMWGOrREX12jJFp6kULsnsEHjn66pgMjrAG5DI5gadEWzFrRPNVlqxeSgwP7h7sXzdP/U3EmdMFdDDKQZMdR9aUBW2c2ARBTMubv8YMhcKh3VZ3aW/6+9hvFBkFpjPbXmHINx/zGOJUxL797xcqzTBJDoiF0/fn4NroaKIOhQjE2X0D7pKv7ZxTAGR8uopiw3hvARTw/C+ekG5eAiBo85pkhDItf6cK33pCBMTY3ogb1o5N81z1vEqo3XV3F6nuv4lbBhQPsDR0jx/igU7EFqxKcNHS88yw4Ny0dtqxRpiBUNP3BpQuxS7/OLamuNKl33tZT51lMkzd0H+r5E0gz0rea0Zvr4yChRRJZDU/wjr7eKp0IehqxyR9UempNMq3E+xs+RdNoFmd5pON7qDcY9jIaUJ72+oxiQPoSbx1SrkAVaiUooDqYt338XKrg8U7pWCoxLdeh6BcLeOFIAP47ZEOZivdWa038cRjtAN9+uMheiAr6FvHuF5R6eT/46NOpMD0UWb/8h9O+gBvniSXVz++p3q+fGARWoPSdCkR15H5QuYmYd8j3FFqxAdfdnGs/W49XPQnNkimjViim9M7glejkEZo6wHNpADiGk6sJxbyyj63j/tPJ0wwNaAM/NQiriBKuaO2OWfXZiBatYCoAHG0kT6MrHSmCt9uC8SD7jnaNVJCSlglEncK35sPXHAr1Ia+UM2tGuSCzojq770kFliGw7HJ5qUC0xXSV8afxIv+3xOwU0COFOSYXl0PARq9s62et5oTwLQi2vNNY3LRk9T9jh0oCRHEjKQveaZCydTxQBENYNAsAk2vHCcHLF4XW4eGvWeCQ5lLmiiOIauHTCN70gCJ6/rwfki+N/NXs8Kc4n4YbiXf4gbM+1/i8Ll891inVtAyuHzLCRY0csGxjmb8cdFdFG3YgZRNn1te6JaiNvzWzhESEIgHASYB0yB5pkRmkju/hR7XH5JVyiGoVWHmDkhT9ru9dXTGxMsA4vxzGvIKo8xJvAIF44Kji1SWQPHqXYVWTkzOqmFTTM51asv+a5MwlvC1AMtBWQr9+NbNgM+bbBv/CJqnE2Rji8Yt2iAqFFTQ7XzZB+PATA2MfkFIElDUVL4i8zcdNApq567JqU21vCYay+P8J3oSy1FNSUpezXOKHrLogwkUonswT14PqjpYd++CsAXrbGtJi6oHx8DgKC90WAXZCGdofVdRIUDeHSeVYtzxQ80TfZAga1+GZtXJsavhYPqCqVBzLmqxZoyKNGxrbkFLGQjQ/8JkkHe6L5fsNwkdzvXeF53caZaBeHBZCwzscfi9HDlxLijfLHvDR5Wa7EYUpiL0pAhHP2Wi+xv6EVwI X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 81d0a955-62d8-42ad-d8f1-08da0d5d541f X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Mar 2022 06:12:52.9276 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aIAtRmcCmjGw8BoC/cCFPlvfHIhRlZo1IBG7bG6PpRhMoiKTVJhLLG9xlPHBOeZsQGq2M3WqKDvAzcLoOZ8DGA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2960 Subject: [FFmpeg-devel] [PATCH v2] avcodec/v4l2_m2m_dec: export v4l2 buffer dma-buf X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 8+gTMGPR9tue if the v4l2 buffer is supported to export dma-buf, then we can report it to AV_PIX_FMT_DRM_PRIME, so the caller can pass it to other hardware device, such as display it directly without copy frame data. Signed-off-by: Ming Qian --- libavcodec/v4l2_buffers.c | 115 +++++++++++++++++++++++++++++++++++++- libavcodec/v4l2_buffers.h | 2 + libavcodec/v4l2_context.c | 53 ++++++++++++++++++ libavcodec/v4l2_context.h | 17 ++++++ libavcodec/v4l2_fmt.c | 96 ++++++++++++++++++------------- libavcodec/v4l2_fmt.h | 1 + libavcodec/v4l2_m2m_dec.c | 21 +++++++ 7 files changed, 264 insertions(+), 41 deletions(-) diff --git a/libavcodec/v4l2_buffers.c b/libavcodec/v4l2_buffers.c index 3f5471067a1a..18f17d871b8c 100644 --- a/libavcodec/v4l2_buffers.c +++ b/libavcodec/v4l2_buffers.c @@ -21,17 +21,23 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif #include #include #include #include #include #include +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_drm.h" #include "libavcodec/avcodec.h" #include "libavutil/pixdesc.h" #include "v4l2_context.h" #include "v4l2_buffers.h" #include "v4l2_m2m.h" +#include "v4l2_fmt.h" #define USEC_PER_SEC 1000000 static AVRational v4l2_timebase = { 1, USEC_PER_SEC }; @@ -209,7 +215,7 @@ static enum AVColorTransferCharacteristic v4l2_get_color_trc(V4L2Buffer *buf) return AVCOL_TRC_UNSPECIFIED; } -static void v4l2_free_buffer(void *opaque, uint8_t *unused) +static void v4l2_free_buffer(void *opaque, uint8_t *data) { V4L2Buffer* avbuf = opaque; V4L2m2mContext *s = buf_to_m2mctx(avbuf); @@ -229,6 +235,12 @@ static void v4l2_free_buffer(void *opaque, uint8_t *unused) ff_v4l2_buffer_enqueue(avbuf); } + if (avbuf->hwctx_ref) { + AVDRMFrameDescriptor *desc = (AVDRMFrameDescriptor *)data; + + av_buffer_unref(&avbuf->hwctx_ref); + av_free(desc); + } av_buffer_unref(&avbuf->context_ref); } } @@ -337,6 +349,90 @@ static int v4l2_buffer_buf_to_swframe(AVFrame *frame, V4L2Buffer *avbuf) return 0; } +static int v4l2_buffer_buf_to_hwframe(AVFrame *frame, V4L2Buffer *avbuf) +{ + V4L2Context *ctx = avbuf->context; + AVDRMFrameDescriptor *desc = NULL; + AVDRMLayerDescriptor *layer = NULL; + int i; + int ret; + + if (!ctx->hwframes) + return AVERROR(EINVAL); + + for (i = 0; i < avbuf->num_planes; i++) { + if (avbuf->plane_info[i].dmafd < 0) + return AVERROR(EINVAL); + } + + desc = av_mallocz(sizeof(AVDRMFrameDescriptor)); + if (!desc) + return AVERROR(ENOMEM); + + for (i = 0; i < avbuf->num_planes; i++) { + desc->objects[i].fd = avbuf->plane_info[i].dmafd; + desc->objects[i].size = avbuf->plane_info[i].length; + } + desc->nb_objects = avbuf->num_planes; + + desc->nb_layers = 1; + layer = &desc->layers[0]; + layer->format = ff_v4l2_format_avfmt_to_drm(avbuf->context->av_pix_fmt); + layer->nb_planes = avbuf->num_planes; + for (i = 0; i < avbuf->num_planes; i++) { + layer->planes[i].object_index = i; + layer->planes[i].offset = 0; + layer->planes[i].pitch = avbuf->plane_info[i].bytesperline; + } + + /* fixup special cases */ + switch (avbuf->context->av_pix_fmt) { + case AV_PIX_FMT_NV12: + case AV_PIX_FMT_NV21: + if (avbuf->num_planes > 1) + break; + layer->nb_planes = 2; + layer->planes[1].object_index = 0; + layer->planes[1].offset = avbuf->plane_info[0].bytesperline * avbuf->context->format.fmt.pix_mp.height; + layer->planes[1].pitch = avbuf->plane_info[0].bytesperline; + break; + + case AV_PIX_FMT_YUV420P: + if (avbuf->num_planes > 1) + break; + layer->nb_planes = 3; + layer->planes[1].object_index = 0; + layer->planes[2].object_index = 0; + layer->planes[1].offset = avbuf->plane_info[0].bytesperline * avbuf->context->format.fmt.pix_mp.height; + layer->planes[2].offset = layer->planes[1].offset + ((avbuf->plane_info[0].bytesperline * avbuf->context->format.fmt.pix_mp.height) >> 2); + layer->planes[1].pitch = avbuf->plane_info[0].bytesperline >> 1; + layer->planes[2].pitch = avbuf->plane_info[0].bytesperline >> 1; + break; + + default: + break; + } + + avbuf->hwctx_ref = av_buffer_ref(ctx->hwdevice); + frame->buf[0] = av_buffer_create((uint8_t *)desc, sizeof(*desc), v4l2_free_buffer, avbuf, 0); + if (!frame->buf[0]) { + av_free(desc); + av_buffer_unref(&avbuf->hwctx_ref); + return AVERROR(ENOMEM); + } + frame->data[0] = (uint8_t *)desc; + frame->format = AV_PIX_FMT_DRM_PRIME; + frame->hw_frames_ctx = av_buffer_ref(ctx->hwframes); + + ret = v4l2_buf_increase_ref(avbuf); + if (ret) { + av_buffer_unref(&frame->buf[0]); + return ret; + } + + return 0; +} + static int v4l2_buffer_swframe_to_buf(const AVFrame *frame, V4L2Buffer *out) { int i, ret; @@ -417,12 +513,17 @@ int ff_v4l2_buffer_avframe_to_buf(const AVFrame *frame, V4L2Buffer *out) int ff_v4l2_buffer_buf_to_avframe(AVFrame *frame, V4L2Buffer *avbuf) { + V4L2m2mContext *s = buf_to_m2mctx(avbuf); int ret; av_frame_unref(frame); /* 1. get references to the actual data */ - ret = v4l2_buffer_buf_to_swframe(frame, avbuf); + if (s->avctx->pix_fmt == AV_PIX_FMT_DRM_PRIME) { + ret = v4l2_buffer_buf_to_hwframe(frame, avbuf); + } else { + ret = v4l2_buffer_buf_to_swframe(frame, avbuf); + } if (ret) return ret; @@ -493,6 +594,7 @@ int ff_v4l2_buffer_avpkt_to_buf(const AVPacket *pkt, V4L2Buffer *out) int ff_v4l2_buffer_initialize(V4L2Buffer* avbuf, int index) { V4L2Context *ctx = avbuf->context; + struct v4l2_exportbuffer exp; int ret, i; avbuf->buf.memory = V4L2_MEMORY_MMAP; @@ -538,6 +640,15 @@ int ff_v4l2_buffer_initialize(V4L2Buffer* avbuf, int index) if (avbuf->plane_info[i].mm_addr == MAP_FAILED) return AVERROR(ENOMEM); + + exp.type = ctx->type; + exp.index = avbuf->buf.index; + exp.plane = i; + exp.fd = -1; + exp.flags = O_CLOEXEC | O_RDWR; + if (ioctl(buf_to_m2mctx(avbuf)->fd, VIDIOC_EXPBUF, &exp)) + avbuf->context->support_dma_buf = 0; + avbuf->plane_info[i].dmafd = exp.fd; } avbuf->status = V4L2BUF_AVAILABLE; diff --git a/libavcodec/v4l2_buffers.h b/libavcodec/v4l2_buffers.h index 3d2ff1b9a5d7..04250cda175e 100644 --- a/libavcodec/v4l2_buffers.h +++ b/libavcodec/v4l2_buffers.h @@ -55,6 +55,7 @@ typedef struct V4L2Buffer { int bytesperline; void * mm_addr; size_t length; + int dmafd; } plane_info[VIDEO_MAX_PLANES]; int num_planes; @@ -66,6 +67,7 @@ typedef struct V4L2Buffer { int flags; enum V4L2Buffer_status status; + AVBufferRef *hwctx_ref; } V4L2Buffer; /** diff --git a/libavcodec/v4l2_context.c b/libavcodec/v4l2_context.c index 8910ae08d3a5..685de60690e0 100644 --- a/libavcodec/v4l2_context.c +++ b/libavcodec/v4l2_context.c @@ -455,6 +455,10 @@ static int v4l2_release_buffers(V4L2Context* ctx) if (p->mm_addr && p->length) if (munmap(p->mm_addr, p->length) < 0) av_log(logger(ctx), AV_LOG_ERROR, "%s unmap plane (%s))\n", ctx->name, av_err2str(AVERROR(errno))); + if (p->dmafd >= 0) { + close(p->dmafd); + p->dmafd = -1; + } } } @@ -696,6 +700,53 @@ int ff_v4l2_context_set_format(V4L2Context* ctx) return ioctl(ctx_to_m2mctx(ctx)->fd, VIDIOC_S_FMT, &ctx->format); } + +int ff_v4l2_context_init_hw_ctx(V4L2Context *ctx) +{ + AVHWFramesContext *hwframes; + int ret; + + if (!ctx->support_dma_buf) + return AVERROR(EINVAL); + + ctx->hwdevice = av_hwdevice_ctx_alloc(AV_HWDEVICE_TYPE_DRM); + if (!ctx->hwdevice) { + ret = AVERROR(ENOMEM); + goto fail; + } + + ret = av_hwdevice_ctx_init(ctx->hwdevice); + if (ret < 0) + goto fail; + + ctx->hwframes = av_hwframe_ctx_alloc(ctx->hwdevice); + if (!ctx->hwframes) { + ret = AVERROR(ENOMEM); + goto fail; + } + + hwframes = (AVHWFramesContext*)ctx->hwframes->data; + hwframes->format = AV_PIX_FMT_DRM_PRIME; + hwframes->sw_format = ctx->av_pix_fmt; + hwframes->width = ctx->width; + hwframes->height = ctx->height; + ret = av_hwframe_ctx_init(ctx->hwframes); + if (ret < 0) + goto fail; + + return 0; +fail: + ff_v4l2_context_uninit_hw_ctx(ctx); + ctx->support_dma_buf = 0; + return ret; +} + +void ff_v4l2_context_uninit_hw_ctx(V4L2Context *ctx) +{ + av_buffer_unref(&ctx->hwframes); + av_buffer_unref(&ctx->hwdevice); +} + void ff_v4l2_context_release(V4L2Context* ctx) { int ret; @@ -708,6 +759,7 @@ void ff_v4l2_context_release(V4L2Context* ctx) av_log(logger(ctx), AV_LOG_WARNING, "V4L2 failed to unmap the %s buffers\n", ctx->name); av_freep(&ctx->buffers); + ff_v4l2_context_uninit_hw_ctx(ctx); } int ff_v4l2_context_init(V4L2Context* ctx) @@ -742,6 +794,7 @@ int ff_v4l2_context_init(V4L2Context* ctx) return AVERROR(ENOMEM); } + ctx->support_dma_buf = 1; for (i = 0; i < req.count; i++) { ctx->buffers[i].context = ctx; ret = ff_v4l2_buffer_initialize(&ctx->buffers[i], i); diff --git a/libavcodec/v4l2_context.h b/libavcodec/v4l2_context.h index 6f7460c89a9d..723d622e38c3 100644 --- a/libavcodec/v4l2_context.h +++ b/libavcodec/v4l2_context.h @@ -93,6 +93,9 @@ typedef struct V4L2Context { */ int done; + int support_dma_buf; + AVBufferRef *hwdevice; + AVBufferRef *hwframes; } V4L2Context; /** @@ -184,4 +187,18 @@ int ff_v4l2_context_enqueue_packet(V4L2Context* ctx, const AVPacket* pkt); */ int ff_v4l2_context_enqueue_frame(V4L2Context* ctx, const AVFrame* f); +/** + * Initializes the hw context of V4L2Context. + * + * @param[in] ctx A pointer to a V4L2Context. See V4L2Context description for required variables. + * @return 0 in case of success, a negative value representing the error otherwise. + */ +int ff_v4l2_context_init_hw_ctx(V4L2Context *ctx); + +/** + * Releases the hw context of V4L2Context. + * + * @param[in] ctx A pointer to a V4L2Context. + */ +void ff_v4l2_context_uninit_hw_ctx(V4L2Context *ctx); #endif // AVCODEC_V4L2_CONTEXT_H diff --git a/libavcodec/v4l2_fmt.c b/libavcodec/v4l2_fmt.c index 6df47e3f5a3c..a64b6d530283 100644 --- a/libavcodec/v4l2_fmt.c +++ b/libavcodec/v4l2_fmt.c @@ -29,83 +29,91 @@ #define AV_CODEC(x) AV_CODEC_ID_##x #define AV_FMT(x) AV_PIX_FMT_##x +#if CONFIG_LIBDRM +#include +#define DRM_FMT(x) DRM_FORMAT_##x +#else +#define DRM_FMT(x) 0 +#endif + static const struct fmt_conversion { enum AVPixelFormat avfmt; enum AVCodecID avcodec; uint32_t v4l2_fmt; + uint32_t drm_fmt; } fmt_map[] = { - { AV_FMT(RGB555LE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB555) }, - { AV_FMT(RGB555BE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB555X) }, - { AV_FMT(RGB565LE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB565) }, - { AV_FMT(RGB565BE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB565X) }, - { AV_FMT(BGR24), AV_CODEC(RAWVIDEO), V4L2_FMT(BGR24) }, - { AV_FMT(RGB24), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB24) }, - { AV_FMT(BGR0), AV_CODEC(RAWVIDEO), V4L2_FMT(BGR32) }, - { AV_FMT(0RGB), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB32) }, - { AV_FMT(GRAY8), AV_CODEC(RAWVIDEO), V4L2_FMT(GREY) }, - { AV_FMT(YUV420P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV420) }, - { AV_FMT(YUYV422), AV_CODEC(RAWVIDEO), V4L2_FMT(YUYV) }, - { AV_FMT(UYVY422), AV_CODEC(RAWVIDEO), V4L2_FMT(UYVY) }, - { AV_FMT(YUV422P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV422P) }, - { AV_FMT(YUV411P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV411P) }, - { AV_FMT(YUV410P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV410) }, - { AV_FMT(YUV410P), AV_CODEC(RAWVIDEO), V4L2_FMT(YVU410) }, - { AV_FMT(NV12), AV_CODEC(RAWVIDEO), V4L2_FMT(NV12) }, - { AV_FMT(NONE), AV_CODEC(MJPEG), V4L2_FMT(MJPEG) }, - { AV_FMT(NONE), AV_CODEC(MJPEG), V4L2_FMT(JPEG) }, + { AV_FMT(RGB555LE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB555), DRM_FMT(XRGB1555) }, + { AV_FMT(RGB555BE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB555X), DRM_FMT(XRGB1555) | DRM_FMT(BIG_ENDIAN) }, + { AV_FMT(RGB565LE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB565), DRM_FMT(RGB565) }, + { AV_FMT(RGB565BE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB565X), DRM_FMT(RGB565) | DRM_FMT(BIG_ENDIAN) }, + { AV_FMT(BGR24), AV_CODEC(RAWVIDEO), V4L2_FMT(BGR24), DRM_FMT(BGR888) }, + { AV_FMT(RGB24), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB24), DRM_FMT(RGB888) }, + { AV_FMT(BGR0), AV_CODEC(RAWVIDEO), V4L2_FMT(BGR32), DRM_FMT(XRGB8888) }, + { AV_FMT(0RGB), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB32), DRM_FMT(BGRX8888) }, + { AV_FMT(GRAY8), AV_CODEC(RAWVIDEO), V4L2_FMT(GREY), DRM_FMT(R8) }, + { AV_FMT(YUV420P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV420), DRM_FMT(YUV420) }, + { AV_FMT(YUYV422), AV_CODEC(RAWVIDEO), V4L2_FMT(YUYV), DRM_FMT(YUYV) }, + { AV_FMT(UYVY422), AV_CODEC(RAWVIDEO), V4L2_FMT(UYVY), DRM_FMT(UYVY) }, + { AV_FMT(YUV422P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV422P), DRM_FMT(YUV422) }, + { AV_FMT(YUV411P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV411P), DRM_FMT(YUV411) }, + { AV_FMT(YUV410P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV410), DRM_FMT(YUV410) }, + { AV_FMT(YUV410P), AV_CODEC(RAWVIDEO), V4L2_FMT(YVU410), DRM_FMT(YVU410) }, + { AV_FMT(NV12), AV_CODEC(RAWVIDEO), V4L2_FMT(NV12), DRM_FMT(NV12) }, + { AV_FMT(NONE), AV_CODEC(MJPEG), V4L2_FMT(MJPEG), DRM_FMT(INVALID) }, + { AV_FMT(NONE), AV_CODEC(MJPEG), V4L2_FMT(JPEG), DRM_FMT(INVALID) }, #ifdef V4L2_PIX_FMT_SRGGB8 - { AV_FMT(BAYER_BGGR8), AV_CODEC(RAWVIDEO), V4L2_FMT(SBGGR8) }, - { AV_FMT(BAYER_GBRG8), AV_CODEC(RAWVIDEO), V4L2_FMT(SGBRG8) }, - { AV_FMT(BAYER_GRBG8), AV_CODEC(RAWVIDEO), V4L2_FMT(SGRBG8) }, - { AV_FMT(BAYER_RGGB8), AV_CODEC(RAWVIDEO), V4L2_FMT(SRGGB8) }, + { AV_FMT(BAYER_BGGR8), AV_CODEC(RAWVIDEO), V4L2_FMT(SBGGR8), DRM_FMT(INVALID) }, + { AV_FMT(BAYER_GBRG8), AV_CODEC(RAWVIDEO), V4L2_FMT(SGBRG8), DRM_FMT(INVALID) }, + { AV_FMT(BAYER_GRBG8), AV_CODEC(RAWVIDEO), V4L2_FMT(SGRBG8), DRM_FMT(INVALID) }, + { AV_FMT(BAYER_RGGB8), AV_CODEC(RAWVIDEO), V4L2_FMT(SRGGB8), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_Y16 - { AV_FMT(GRAY16LE), AV_CODEC(RAWVIDEO), V4L2_FMT(Y16) }, + { AV_FMT(GRAY16LE), AV_CODEC(RAWVIDEO), V4L2_FMT(Y16), DRM_FMT(R16) }, #endif #ifdef V4L2_PIX_FMT_NV12M - { AV_FMT(NV12), AV_CODEC(RAWVIDEO), V4L2_FMT(NV12M) }, + { AV_FMT(NV12), AV_CODEC(RAWVIDEO), V4L2_FMT(NV12M), DRM_FMT(NV12) }, #endif #ifdef V4L2_PIX_FMT_NV21M - { AV_FMT(NV21), AV_CODEC(RAWVIDEO), V4L2_FMT(NV21M) }, + { AV_FMT(NV21), AV_CODEC(RAWVIDEO), V4L2_FMT(NV21M), DRM_FMT(NV21) }, #endif #ifdef V4L2_PIX_FMT_YUV420M - { AV_FMT(YUV420P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV420M) }, + { AV_FMT(YUV420P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV420M), DRM_FMT(YUV420) }, #endif #ifdef V4L2_PIX_FMT_NV16M - { AV_FMT(NV16), AV_CODEC(RAWVIDEO), V4L2_FMT(NV16M) }, + { AV_FMT(NV16), AV_CODEC(RAWVIDEO), V4L2_FMT(NV16M), DRM_FMT(NV16) }, #endif #ifdef V4L2_PIX_FMT_H263 - { AV_FMT(NONE), AV_CODEC(H263), V4L2_FMT(H263) }, + { AV_FMT(NONE), AV_CODEC(H263), V4L2_FMT(H263), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_H264 - { AV_FMT(NONE), AV_CODEC(H264), V4L2_FMT(H264) }, + { AV_FMT(NONE), AV_CODEC(H264), V4L2_FMT(H264), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_MPEG4 - { AV_FMT(NONE), AV_CODEC(MPEG4), V4L2_FMT(MPEG4) }, + { AV_FMT(NONE), AV_CODEC(MPEG4), V4L2_FMT(MPEG4), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_CPIA1 - { AV_FMT(NONE), AV_CODEC(CPIA), V4L2_FMT(CPIA1) }, + { AV_FMT(NONE), AV_CODEC(CPIA), V4L2_FMT(CPIA1), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_DV - { AV_FMT(NONE), AV_CODEC(DVVIDEO), V4L2_FMT(DV) }, + { AV_FMT(NONE), AV_CODEC(DVVIDEO), V4L2_FMT(DV), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_MPEG1 - { AV_FMT(NONE), AV_CODEC(MPEG1VIDEO), V4L2_FMT(MPEG1) }, + { AV_FMT(NONE), AV_CODEC(MPEG1VIDEO), V4L2_FMT(MPEG1), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_MPEG2 - { AV_FMT(NONE), AV_CODEC(MPEG2VIDEO), V4L2_FMT(MPEG2) }, + { AV_FMT(NONE), AV_CODEC(MPEG2VIDEO), V4L2_FMT(MPEG2), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_VP8 - { AV_FMT(NONE), AV_CODEC(VP8), V4L2_FMT(VP8) }, + { AV_FMT(NONE), AV_CODEC(VP8), V4L2_FMT(VP8), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_VP9 - { AV_FMT(NONE), AV_CODEC(VP9), V4L2_FMT(VP9) }, + { AV_FMT(NONE), AV_CODEC(VP9), V4L2_FMT(VP9), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_HEVC - { AV_FMT(NONE), AV_CODEC(HEVC), V4L2_FMT(HEVC) }, + { AV_FMT(NONE), AV_CODEC(HEVC), V4L2_FMT(HEVC), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_VC1_ANNEX_G - { AV_FMT(NONE), AV_CODEC(VC1), V4L2_FMT(VC1_ANNEX_G) }, + { AV_FMT(NONE), AV_CODEC(VC1), V4L2_FMT(VC1_ANNEX_G), DRM_FMT(INVALID) }, #endif }; @@ -139,3 +147,13 @@ enum AVPixelFormat ff_v4l2_format_v4l2_to_avfmt(uint32_t v4l2_fmt, enum AVCodecI } return AV_PIX_FMT_NONE; } + +uint32_t ff_v4l2_format_avfmt_to_drm(enum AVPixelFormat avfmt) +{ + int i; + for (i = 0; i < FF_ARRAY_ELEMS(fmt_map); i++) { + if (fmt_map[i].avfmt == avfmt) + return fmt_map[i].drm_fmt; + } + return DRM_FMT(INVALID); +} diff --git a/libavcodec/v4l2_fmt.h b/libavcodec/v4l2_fmt.h index 577e03a7a76c..f705fe9c0b30 100644 --- a/libavcodec/v4l2_fmt.h +++ b/libavcodec/v4l2_fmt.h @@ -31,5 +31,6 @@ enum AVPixelFormat ff_v4l2_format_v4l2_to_avfmt(uint32_t v4l2_fmt, enum AVCodecID avcodec); uint32_t ff_v4l2_format_avcodec_to_v4l2(enum AVCodecID avcodec); uint32_t ff_v4l2_format_avfmt_to_v4l2(enum AVPixelFormat avfmt); +uint32_t ff_v4l2_format_avfmt_to_drm(enum AVPixelFormat avfmt); #endif /* AVCODEC_V4L2_FMT_H*/ diff --git a/libavcodec/v4l2_m2m_dec.c b/libavcodec/v4l2_m2m_dec.c index 5793fca6d6dc..91b9dc7b474c 100644 --- a/libavcodec/v4l2_m2m_dec.c +++ b/libavcodec/v4l2_m2m_dec.c @@ -29,6 +29,10 @@ #include "libavcodec/avcodec.h" #include "codec_internal.h" #include "libavcodec/decode.h" +#include "libavcodec/internal.h" +#include "hwconfig.h" +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_drm.h" #include "v4l2_context.h" #include "v4l2_m2m.h" @@ -99,6 +103,17 @@ static int v4l2_try_start(AVCodecContext *avctx) return ret; } + if (capture->support_dma_buf) { + ff_v4l2_context_uninit_hw_ctx(capture); + ret = ff_v4l2_context_init_hw_ctx(capture); + if (!ret) { + enum AVPixelFormat pix_fmts[3] = { AV_PIX_FMT_DRM_PRIME, + capture->av_pix_fmt, + AV_PIX_FMT_NONE }; + avctx->pix_fmt = ff_get_format(s->avctx, pix_fmts); + } + } + return 0; } @@ -228,6 +243,11 @@ static const AVOption options[] = { { NULL}, }; +static const AVCodecHWConfigInternal *const v4l2_m2m_dec_hw_configs[] = { + HW_CONFIG_INTERNAL(DRM_PRIME), + NULL +}; + #define M2MDEC_CLASS(NAME) \ static const AVClass v4l2_m2m_ ## NAME ## _dec_class = { \ .class_name = #NAME "_v4l2m2m_decoder", \ @@ -252,6 +272,7 @@ static const AVOption options[] = { .p.capabilities = AV_CODEC_CAP_HARDWARE | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AVOID_PROBING, \ .caps_internal = FF_CODEC_CAP_SETS_PKT_DTS | FF_CODEC_CAP_INIT_CLEANUP, \ .p.wrapper_name = "v4l2m2m", \ + .hw_configs = v4l2_m2m_dec_hw_configs, \ } M2MDEC(h264, "H.264", AV_CODEC_ID_H264, "h264_mp4toannexb");