From patchwork Fri Nov 19 09:11:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 31487 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp2084955iob; Fri, 19 Nov 2021 01:11:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJzI5DeYeULlxzH4q4YYYGtFqyKX2EaDYRDDWn9DFj9GENGJNarHNLesD2xsL3XywjIvHvqv X-Received: by 2002:a50:c31c:: with SMTP id a28mr21898684edb.4.1637313113765; Fri, 19 Nov 2021 01:11:53 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id z1si7118461edc.600.2021.11.19.01.11.52; Fri, 19 Nov 2021 01:11:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@nxp.com header.s=selector2 header.b=WbDihoL2; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D305E68A13F; Fri, 19 Nov 2021 11:11:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80084.outbound.protection.outlook.com [40.107.8.84]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E1FBE6883DB for ; Fri, 19 Nov 2021 11:11:40 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q9FVumQvkiqRqDXbFmfpVYcKHgEVP+UZcwJReYE2b3dg3qZA78T069kqTnU9F6THSkhO6PhNuzpe9oZz1qPXuGeKDr1Hcb4DV7EkDycylcZX6c5r383a0JYl5rsaFVEhPdKXOl0FGMmQWM/14RJ9GS8Av9zDSKtddw9vT+7E/PkhsplWnv87+wOrr9ey50CWHv3TgIpMqYE7qEzO4cE/nr990DrA504XfY4Q5ep1wvnIg12dupT7uH+ln0SAHRlGjntafgG4o6KTKzyAtsn1tej2vGgDi63u4SD6rfw99M53YSPCh3rmUVPGd1i2GsrID4HKJ0yropyRD0w2xYqrdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=c7IeHckrUhRGMasMewppwmLKvmypHTzxVuWOwTX6lYc=; b=oUXJgXJPEe3wzQAFMxMs/YmIGVR70JgdcNJW0PcYp3zqd4LWPnoYUofN/t2lb1fNn+qS2HJvDO8j/KEiqFS+4Q8vN/6rt7OqcdJyJVgucAnw5b29z6ltxLnAVrubFmoZHDEPO9I4H8fre1XdyhmACxyoJ8/IY7OEGvABwoqK4iSDCmtburnUXPuqC/Kx8j536x7IWBvfxb4cv8oLPsJbLP4aHVk1PuOQYXXwAMm3Gae6OORpX4Zn/ML3SGHWCnTfiZ7apbym10JUfZNMuqZYPQvrHU3AZyNhNSjb1qlar9r+nlgoQT7SAw1CPN3lv91cWJEi1HkH6I+ZyUSiClmlng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=c7IeHckrUhRGMasMewppwmLKvmypHTzxVuWOwTX6lYc=; b=WbDihoL2SoCFsCx8DU0qKjSXL5tSwtTETOYKQ3Jien1Ua/dVi1yPjfytNX3DqwWPKbD5iDIUZ7WjKw0RhLP/dElBJMoV5wmSA0BIYBRWDOfT8AnPN4y2a7+RCNyBx+3Z4XSGgykz00uGMUyCfLoF4nMiRk7FMHC5xqCs5nw5Oag= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by AM6PR04MB5752.eurprd04.prod.outlook.com (2603:10a6:20b:a7::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4713.22; Fri, 19 Nov 2021 09:11:37 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::fc3a:8aea:2c92:3886]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::fc3a:8aea:2c92:3886%5]) with mapi id 15.20.4713.022; Fri, 19 Nov 2021 09:11:37 +0000 From: Ming Qian To: ffmpeg-devel@ffmpeg.org Date: Fri, 19 Nov 2021 17:11:06 +0800 Message-Id: <20211119091106.23046-1-ming.qian@nxp.com> X-Mailer: git-send-email 2.33.0 X-ClientProxiedBy: SI2PR06CA0007.apcprd06.prod.outlook.com (2603:1096:4:186::9) To AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) MIME-Version: 1.0 Received: from lsv11149.swis.cn-sha01.nxp.com (119.31.174.70) by SI2PR06CA0007.apcprd06.prod.outlook.com (2603:1096:4:186::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4713.22 via Frontend Transport; Fri, 19 Nov 2021 09:11:36 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c6e80709-d55c-4608-30a2-08d9ab3c973a X-MS-TrafficTypeDiagnostic: AM6PR04MB5752: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:843; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: t0X6O7rNQbx7kSc6SweCBx/8SrMJYU00idwzIdF6hd9KW6Ji+Zh+MoBj7eU8IgR3nMxA4VW9DGa/xHjhQUBujGR3v08LjFOIYOGVhQiaLL6+2OKwy980OSowr6pa7hkdjh6frdGJx8AnOD74LZbG8SCDXSv1iPD+34zGPJBaaRpEnHFUffblwDcFIBWDzTvExLfVFy0/JkNuRPi9RrYF21KNYVjIVBo1jXJK2Kg7El22hWtnjoggvdT+Qwnn7rKG4DrhhMYlOwCtR3N5QwfQp/nbDqFEBhRxRTnctoUVlO1CyQ4nBmRQcxbtTV1X/muY3pLO2oKbOCIWTj11XXVKQBVig8auL/G6RJPMIZjDnFBxho9yqZ4AacDRWpLMd7iJzmS0i0j6P1vN24dZt0EzzkhIXf/9JFGeJL36TBwnJ/n9qAmxnshkZVhyxGZnCp3UtqLX5p/xr5i67wdnV2vqSrHKZ8PK3wV6heJy1XIyc1CqCmPA+O21+Hk7fOgbxac7LNKbqYBivIndAIG7hvgaDcU2QlW0cbGulMjAqlCOUTvil8lj9DyWxZRaAPdXQC09JccuxabD+67x5nerjAWqAwOebC+a5plaZijgRPM8P2QrumvPWoieAQEMt7Pw/keyMNXLbRj+t3bBPAbz22hAQkwPAP4+1ab+LWtyArclhjtDiBsPuUx9qzeDCxiGXNavfiEmxa9vbOCW9zcKSQbvKg50++UfZgjkGE4fs3M4ZHks0LDhkhOfmPPv4uEZBeIx X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(66556008)(2616005)(66476007)(36756003)(956004)(66946007)(83380400001)(8676002)(5660300002)(186003)(52116002)(8936002)(7696005)(86362001)(2906002)(26005)(30864003)(6916009)(38350700002)(1076003)(4326008)(508600001)(6666004)(316002)(6486002)(44832011)(38100700002)(70780200001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: RnSMYj95VfBh7K3MJRc3X+fMLoHq4BrQQWCDZWJvresf6XFn0sRftdlx5mJApNqGDkQDM8t00XiGJEUm+ucPhhPmc+Y3oMsLBm7u+XY4oqgDDCjppRkz2R2gxqEc+xWIciZyb9XWZe1BxuJ+y1lrEQEA/1r9KeS9kRzlQ9xglipuSFJ2rZx+rEUILeMS4wWckP4P1PP/wrobHpHDm+7QIREK6BIkL8GFFTl/v1bW1Z/076NLaCzzuEW/i1OQBqUEpG4+CgjJF4ifgxcyrxTXpc4SSDKVv4n/BjXwDgs5hJc9ZPsmb5imGbLom2llmR+TOkYXYKgDS/0eIyBSvyoBVWbDkeN1HtkSa4ODyAFeBILf/aJ6TGIq5L982yKjSrlalhp8vJEDC0Js5R/btk3EsUKt1n2CNZdNhFv5P3Oi68J/mEbUd4W4iTJ3yBzSg7J+QqLP8JNcvsJUd8FOcg+Ua8qWZpszVT0eS1wJyUqyCSHwLQUkIh7O1IFUXrWKJXycd/0V48jRmqPcpIad3j+nTwSeEXf2HdjNbfYdz17y8UD+YPwLrQqFHYGGTd2FEymSuE8ZiFiXI74+jxgpYi1KJre0qwFECUDgQ2r1dtGJEziyAmVlFT2Gk3c2Vk9n95n3YxAdLuXtGEdrdNPlhpl942XB4uZsPCBOc9aDQ0Q2Qg0HJQKbGooTrIpImCyQ2DyokI9aFuoAW9+LKdEyZaqVsItlnqMJpwkSEiSV+1BvNm6pGSvdZOHCU5ihhjBQN8DwkvT+JHQahS1a+D+ivZMKXKsagmIhM83z/IgKM3/Vw59SLNK9jMagGz1sI2tz/s945UL0Vz6O9MRKtrBws/RblEbzm57jL8eeqZXdqw/xLj2sH3l5eqYJZ71+vV0tCOenIsMxg4aVXmLiCWep5BeU82hsG5KApZx3T1H+9VR8nmm1ij7hWftaan1qrkQau1I4Z3FZtrq6l119duIpdtERydkGoUpLbY7kl3ero/U8IqCEhOtdbgv8LFO2SIWqJVX2tUdfUloxlko0/WY7H7bISwNVL2+ErxK9u8K9wwuYUnjSv4wkI3GTLEt9ckhSrG2fb4iVOZvqhSLQFPyiZvAE5YrM99LjXAhCWAfg2ZLNs0nbM7ZTA4/y/scqjT1AR9ErHnOgPmidOceaFW6MSuGl5/tafV7gCrqKUCLSQuMl+gE3/lZGaSz+7iLbiqs2tVYB5zMsf1wGtkYfeBqaVYD07RuilEhkvpL7LC46UNKg7IjPq7my3zqdZ1gusdEO9giJRINYcNdMAXQb9TSmZXC8eKzWuGVezPJt3v4ezdMmMN0vkHf2F2Dd39ccEfLdtJm7S/3gKubd8adMKW/f6eEPtCy+nrhg5Jpn0wVZO0N5JWISoUvwzou9w4YsozJDth1OuF531Nw7LY7msiB/t5Rv1qOSNB2clIHNdMBdgVdu7FNUL58C6dgDeAQFd3l1EJnxI1wV2cbDVuMepl1rXQG5fnRl3IFO+J2DT8/ir0yViT4piSMZXZziw5IkAKQBoeNTNBjNysGmMT1lfDJpW/7KTc+uWc0iioNrv1d16lZumONENrMg16ScWMbqYNav0WKPLzuD8Btgf9iHYOm1npnOUwHMsVlSQuQurrOaFGHA1jg= X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: c6e80709-d55c-4608-30a2-08d9ab3c973a X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Nov 2021 09:11:37.6704 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: OV+BrUibw+uswD3It1gdU/q1G/Rc28H826KvQiQYfESlgH2vhQNtWftXLE8SDEMerNnDQnOr/YiXAqr5P+c8sA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB5752 Subject: [FFmpeg-devel] [PATCH v2] avcodec/v4l2_m2m_dec: export v4l2 buffer dma-buf X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: shijie.qin@nxp.com, eagle.zhou@nxp.com Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 1eglcs8nX5um if the v4l2 buffer is supported to export dma-buf, then we can report it to AV_PIX_FMT_DRM_PRIME, so the caller can pass it to other hardware device, such as display it directly without copy frame data. Signed-off-by: Ming Qian --- libavcodec/v4l2_buffers.c | 115 +++++++++++++++++++++++++++++++++++++- libavcodec/v4l2_buffers.h | 2 + libavcodec/v4l2_context.c | 53 ++++++++++++++++++ libavcodec/v4l2_context.h | 17 ++++++ libavcodec/v4l2_fmt.c | 96 ++++++++++++++++++------------- libavcodec/v4l2_fmt.h | 1 + libavcodec/v4l2_m2m_dec.c | 20 +++++++ 7 files changed, 263 insertions(+), 41 deletions(-) diff --git a/libavcodec/v4l2_buffers.c b/libavcodec/v4l2_buffers.c index 4b2679eb3814..2f695fa269df 100644 --- a/libavcodec/v4l2_buffers.c +++ b/libavcodec/v4l2_buffers.c @@ -21,18 +21,24 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif #include #include #include #include #include #include +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_drm.h" #include "libavcodec/avcodec.h" #include "libavcodec/internal.h" #include "libavutil/pixdesc.h" #include "v4l2_context.h" #include "v4l2_buffers.h" #include "v4l2_m2m.h" +#include "v4l2_fmt.h" #define USEC_PER_SEC 1000000 static AVRational v4l2_timebase = { 1, USEC_PER_SEC }; @@ -210,7 +216,7 @@ static enum AVColorTransferCharacteristic v4l2_get_color_trc(V4L2Buffer *buf) return AVCOL_TRC_UNSPECIFIED; } -static void v4l2_free_buffer(void *opaque, uint8_t *unused) +static void v4l2_free_buffer(void *opaque, uint8_t *data) { V4L2Buffer* avbuf = opaque; V4L2m2mContext *s = buf_to_m2mctx(avbuf); @@ -230,6 +236,12 @@ static void v4l2_free_buffer(void *opaque, uint8_t *unused) ff_v4l2_buffer_enqueue(avbuf); } + if (avbuf->hwctx_ref) { + AVDRMFrameDescriptor *desc = (AVDRMFrameDescriptor *)data; + + av_buffer_unref(&avbuf->hwctx_ref); + av_free(desc); + } av_buffer_unref(&avbuf->context_ref); } } @@ -338,6 +350,90 @@ static int v4l2_buffer_buf_to_swframe(AVFrame *frame, V4L2Buffer *avbuf) return 0; } +static int v4l2_buffer_buf_to_hwframe(AVFrame *frame, V4L2Buffer *avbuf) +{ + V4L2Context *ctx = avbuf->context; + AVDRMFrameDescriptor *desc = NULL; + AVDRMLayerDescriptor *layer = NULL; + int i; + int ret; + + if (!ctx->hwframes) + return AVERROR(EINVAL); + + for (i = 0; i < avbuf->num_planes; i++) { + if (avbuf->plane_info[i].dmafd < 0) + return AVERROR(EINVAL); + } + + desc = av_mallocz(sizeof(AVDRMFrameDescriptor)); + if (!desc) + return AVERROR(ENOMEM); + + for (i = 0; i < avbuf->num_planes; i++) { + desc->objects[i].fd = avbuf->plane_info[i].dmafd; + desc->objects[i].size = avbuf->plane_info[i].length; + } + desc->nb_objects = avbuf->num_planes; + + desc->nb_layers = 1; + layer = &desc->layers[0]; + layer->format = ff_v4l2_format_avfmt_to_drm(avbuf->context->av_pix_fmt); + layer->nb_planes = avbuf->num_planes; + for (i = 0; i < avbuf->num_planes; i++) { + layer->planes[i].object_index = i; + layer->planes[i].offset = 0; + layer->planes[i].pitch = avbuf->plane_info[i].bytesperline; + } + + /* fixup special cases */ + switch (avbuf->context->av_pix_fmt) { + case AV_PIX_FMT_NV12: + case AV_PIX_FMT_NV21: + if (avbuf->num_planes > 1) + break; + layer->nb_planes = 2; + layer->planes[1].object_index = 0; + layer->planes[1].offset = avbuf->plane_info[0].bytesperline * avbuf->context->format.fmt.pix_mp.height; + layer->planes[1].pitch = avbuf->plane_info[0].bytesperline; + break; + + case AV_PIX_FMT_YUV420P: + if (avbuf->num_planes > 1) + break; + layer->nb_planes = 3; + layer->planes[1].object_index = 0; + layer->planes[2].object_index = 0; + layer->planes[1].offset = avbuf->plane_info[0].bytesperline * avbuf->context->format.fmt.pix_mp.height; + layer->planes[2].offset = layer->planes[1].offset + ((avbuf->plane_info[0].bytesperline * avbuf->context->format.fmt.pix_mp.height) >> 2); + layer->planes[1].pitch = avbuf->plane_info[0].bytesperline >> 1; + layer->planes[2].pitch = avbuf->plane_info[0].bytesperline >> 1; + break; + + default: + break; + } + + avbuf->hwctx_ref = av_buffer_ref(ctx->hwdevice); + frame->buf[0] = av_buffer_create((uint8_t *)desc, sizeof(*desc), v4l2_free_buffer, avbuf, 0); + if (!frame->buf[0]) { + av_free(desc); + av_buffer_unref(&avbuf->hwctx_ref); + return AVERROR(ENOMEM); + } + frame->data[0] = (uint8_t *)desc; + frame->format = AV_PIX_FMT_DRM_PRIME; + frame->hw_frames_ctx = av_buffer_ref(ctx->hwframes); + + ret = v4l2_buf_increase_ref(avbuf); + if (ret) { + av_buffer_unref(&frame->buf[0]); + return ret; + } + + return 0; +} + static int v4l2_buffer_swframe_to_buf(const AVFrame *frame, V4L2Buffer *out) { int i, ret; @@ -418,12 +514,17 @@ int ff_v4l2_buffer_avframe_to_buf(const AVFrame *frame, V4L2Buffer *out) int ff_v4l2_buffer_buf_to_avframe(AVFrame *frame, V4L2Buffer *avbuf) { + V4L2m2mContext *s = buf_to_m2mctx(avbuf); int ret; av_frame_unref(frame); /* 1. get references to the actual data */ - ret = v4l2_buffer_buf_to_swframe(frame, avbuf); + if (s->avctx->pix_fmt == AV_PIX_FMT_DRM_PRIME) { + ret = v4l2_buffer_buf_to_hwframe(frame, avbuf); + } else { + ret = v4l2_buffer_buf_to_swframe(frame, avbuf); + } if (ret) return ret; @@ -494,6 +595,7 @@ int ff_v4l2_buffer_avpkt_to_buf(const AVPacket *pkt, V4L2Buffer *out) int ff_v4l2_buffer_initialize(V4L2Buffer* avbuf, int index) { V4L2Context *ctx = avbuf->context; + struct v4l2_exportbuffer exp; int ret, i; avbuf->buf.memory = V4L2_MEMORY_MMAP; @@ -539,6 +641,15 @@ int ff_v4l2_buffer_initialize(V4L2Buffer* avbuf, int index) if (avbuf->plane_info[i].mm_addr == MAP_FAILED) return AVERROR(ENOMEM); + + exp.type = ctx->type; + exp.index = avbuf->buf.index; + exp.plane = i; + exp.fd = -1; + exp.flags = O_CLOEXEC | O_RDWR; + if (ioctl(buf_to_m2mctx(avbuf)->fd, VIDIOC_EXPBUF, &exp)) + avbuf->context->support_dma_buf = 0; + avbuf->plane_info[i].dmafd = exp.fd; } avbuf->status = V4L2BUF_AVAILABLE; diff --git a/libavcodec/v4l2_buffers.h b/libavcodec/v4l2_buffers.h index 3d2ff1b9a5d7..04250cda175e 100644 --- a/libavcodec/v4l2_buffers.h +++ b/libavcodec/v4l2_buffers.h @@ -55,6 +55,7 @@ typedef struct V4L2Buffer { int bytesperline; void * mm_addr; size_t length; + int dmafd; } plane_info[VIDEO_MAX_PLANES]; int num_planes; @@ -66,6 +67,7 @@ typedef struct V4L2Buffer { int flags; enum V4L2Buffer_status status; + AVBufferRef *hwctx_ref; } V4L2Buffer; /** diff --git a/libavcodec/v4l2_context.c b/libavcodec/v4l2_context.c index b08f0015c2e5..9b578fdfe120 100644 --- a/libavcodec/v4l2_context.c +++ b/libavcodec/v4l2_context.c @@ -453,6 +453,10 @@ static int v4l2_release_buffers(V4L2Context* ctx) if (p->mm_addr && p->length) if (munmap(p->mm_addr, p->length) < 0) av_log(logger(ctx), AV_LOG_ERROR, "%s unmap plane (%s))\n", ctx->name, av_err2str(AVERROR(errno))); + if (p->dmafd >= 0) { + close(p->dmafd); + p->dmafd = -1; + } } } @@ -694,6 +698,53 @@ int ff_v4l2_context_set_format(V4L2Context* ctx) return ioctl(ctx_to_m2mctx(ctx)->fd, VIDIOC_S_FMT, &ctx->format); } + +int ff_v4l2_context_init_hw_ctx(V4L2Context *ctx) +{ + AVHWFramesContext *hwframes; + int ret; + + if (!ctx->support_dma_buf) + return AVERROR(EINVAL); + + ctx->hwdevice = av_hwdevice_ctx_alloc(AV_HWDEVICE_TYPE_DRM); + if (!ctx->hwdevice) { + ret = AVERROR(ENOMEM); + goto fail; + } + + ret = av_hwdevice_ctx_init(ctx->hwdevice); + if (ret < 0) + goto fail; + + ctx->hwframes = av_hwframe_ctx_alloc(ctx->hwdevice); + if (!ctx->hwframes) { + ret = AVERROR(ENOMEM); + goto fail; + } + + hwframes = (AVHWFramesContext*)ctx->hwframes->data; + hwframes->format = AV_PIX_FMT_DRM_PRIME; + hwframes->sw_format = ctx->av_pix_fmt; + hwframes->width = ctx->width; + hwframes->height = ctx->height; + ret = av_hwframe_ctx_init(ctx->hwframes); + if (ret < 0) + goto fail; + + return 0; +fail: + ff_v4l2_context_uninit_hw_ctx(ctx); + ctx->support_dma_buf = 0; + return ret; +} + +void ff_v4l2_context_uninit_hw_ctx(V4L2Context *ctx) +{ + av_buffer_unref(&ctx->hwframes); + av_buffer_unref(&ctx->hwdevice); +} + void ff_v4l2_context_release(V4L2Context* ctx) { int ret; @@ -706,6 +757,7 @@ void ff_v4l2_context_release(V4L2Context* ctx) av_log(logger(ctx), AV_LOG_WARNING, "V4L2 failed to unmap the %s buffers\n", ctx->name); av_freep(&ctx->buffers); + ff_v4l2_context_uninit_hw_ctx(ctx); } int ff_v4l2_context_init(V4L2Context* ctx) @@ -740,6 +792,7 @@ int ff_v4l2_context_init(V4L2Context* ctx) return AVERROR(ENOMEM); } + ctx->support_dma_buf = 1; for (i = 0; i < req.count; i++) { ctx->buffers[i].context = ctx; ret = ff_v4l2_buffer_initialize(&ctx->buffers[i], i); diff --git a/libavcodec/v4l2_context.h b/libavcodec/v4l2_context.h index 6f7460c89a9d..723d622e38c3 100644 --- a/libavcodec/v4l2_context.h +++ b/libavcodec/v4l2_context.h @@ -93,6 +93,9 @@ typedef struct V4L2Context { */ int done; + int support_dma_buf; + AVBufferRef *hwdevice; + AVBufferRef *hwframes; } V4L2Context; /** @@ -184,4 +187,18 @@ int ff_v4l2_context_enqueue_packet(V4L2Context* ctx, const AVPacket* pkt); */ int ff_v4l2_context_enqueue_frame(V4L2Context* ctx, const AVFrame* f); +/** + * Initializes the hw context of V4L2Context. + * + * @param[in] ctx A pointer to a V4L2Context. See V4L2Context description for required variables. + * @return 0 in case of success, a negative value representing the error otherwise. + */ +int ff_v4l2_context_init_hw_ctx(V4L2Context *ctx); + +/** + * Releases the hw context of V4L2Context. + * + * @param[in] ctx A pointer to a V4L2Context. + */ +void ff_v4l2_context_uninit_hw_ctx(V4L2Context *ctx); #endif // AVCODEC_V4L2_CONTEXT_H diff --git a/libavcodec/v4l2_fmt.c b/libavcodec/v4l2_fmt.c index 6df47e3f5a3c..a64b6d530283 100644 --- a/libavcodec/v4l2_fmt.c +++ b/libavcodec/v4l2_fmt.c @@ -29,83 +29,91 @@ #define AV_CODEC(x) AV_CODEC_ID_##x #define AV_FMT(x) AV_PIX_FMT_##x +#if CONFIG_LIBDRM +#include +#define DRM_FMT(x) DRM_FORMAT_##x +#else +#define DRM_FMT(x) 0 +#endif + static const struct fmt_conversion { enum AVPixelFormat avfmt; enum AVCodecID avcodec; uint32_t v4l2_fmt; + uint32_t drm_fmt; } fmt_map[] = { - { AV_FMT(RGB555LE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB555) }, - { AV_FMT(RGB555BE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB555X) }, - { AV_FMT(RGB565LE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB565) }, - { AV_FMT(RGB565BE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB565X) }, - { AV_FMT(BGR24), AV_CODEC(RAWVIDEO), V4L2_FMT(BGR24) }, - { AV_FMT(RGB24), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB24) }, - { AV_FMT(BGR0), AV_CODEC(RAWVIDEO), V4L2_FMT(BGR32) }, - { AV_FMT(0RGB), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB32) }, - { AV_FMT(GRAY8), AV_CODEC(RAWVIDEO), V4L2_FMT(GREY) }, - { AV_FMT(YUV420P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV420) }, - { AV_FMT(YUYV422), AV_CODEC(RAWVIDEO), V4L2_FMT(YUYV) }, - { AV_FMT(UYVY422), AV_CODEC(RAWVIDEO), V4L2_FMT(UYVY) }, - { AV_FMT(YUV422P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV422P) }, - { AV_FMT(YUV411P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV411P) }, - { AV_FMT(YUV410P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV410) }, - { AV_FMT(YUV410P), AV_CODEC(RAWVIDEO), V4L2_FMT(YVU410) }, - { AV_FMT(NV12), AV_CODEC(RAWVIDEO), V4L2_FMT(NV12) }, - { AV_FMT(NONE), AV_CODEC(MJPEG), V4L2_FMT(MJPEG) }, - { AV_FMT(NONE), AV_CODEC(MJPEG), V4L2_FMT(JPEG) }, + { AV_FMT(RGB555LE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB555), DRM_FMT(XRGB1555) }, + { AV_FMT(RGB555BE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB555X), DRM_FMT(XRGB1555) | DRM_FMT(BIG_ENDIAN) }, + { AV_FMT(RGB565LE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB565), DRM_FMT(RGB565) }, + { AV_FMT(RGB565BE), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB565X), DRM_FMT(RGB565) | DRM_FMT(BIG_ENDIAN) }, + { AV_FMT(BGR24), AV_CODEC(RAWVIDEO), V4L2_FMT(BGR24), DRM_FMT(BGR888) }, + { AV_FMT(RGB24), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB24), DRM_FMT(RGB888) }, + { AV_FMT(BGR0), AV_CODEC(RAWVIDEO), V4L2_FMT(BGR32), DRM_FMT(XRGB8888) }, + { AV_FMT(0RGB), AV_CODEC(RAWVIDEO), V4L2_FMT(RGB32), DRM_FMT(BGRX8888) }, + { AV_FMT(GRAY8), AV_CODEC(RAWVIDEO), V4L2_FMT(GREY), DRM_FMT(R8) }, + { AV_FMT(YUV420P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV420), DRM_FMT(YUV420) }, + { AV_FMT(YUYV422), AV_CODEC(RAWVIDEO), V4L2_FMT(YUYV), DRM_FMT(YUYV) }, + { AV_FMT(UYVY422), AV_CODEC(RAWVIDEO), V4L2_FMT(UYVY), DRM_FMT(UYVY) }, + { AV_FMT(YUV422P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV422P), DRM_FMT(YUV422) }, + { AV_FMT(YUV411P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV411P), DRM_FMT(YUV411) }, + { AV_FMT(YUV410P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV410), DRM_FMT(YUV410) }, + { AV_FMT(YUV410P), AV_CODEC(RAWVIDEO), V4L2_FMT(YVU410), DRM_FMT(YVU410) }, + { AV_FMT(NV12), AV_CODEC(RAWVIDEO), V4L2_FMT(NV12), DRM_FMT(NV12) }, + { AV_FMT(NONE), AV_CODEC(MJPEG), V4L2_FMT(MJPEG), DRM_FMT(INVALID) }, + { AV_FMT(NONE), AV_CODEC(MJPEG), V4L2_FMT(JPEG), DRM_FMT(INVALID) }, #ifdef V4L2_PIX_FMT_SRGGB8 - { AV_FMT(BAYER_BGGR8), AV_CODEC(RAWVIDEO), V4L2_FMT(SBGGR8) }, - { AV_FMT(BAYER_GBRG8), AV_CODEC(RAWVIDEO), V4L2_FMT(SGBRG8) }, - { AV_FMT(BAYER_GRBG8), AV_CODEC(RAWVIDEO), V4L2_FMT(SGRBG8) }, - { AV_FMT(BAYER_RGGB8), AV_CODEC(RAWVIDEO), V4L2_FMT(SRGGB8) }, + { AV_FMT(BAYER_BGGR8), AV_CODEC(RAWVIDEO), V4L2_FMT(SBGGR8), DRM_FMT(INVALID) }, + { AV_FMT(BAYER_GBRG8), AV_CODEC(RAWVIDEO), V4L2_FMT(SGBRG8), DRM_FMT(INVALID) }, + { AV_FMT(BAYER_GRBG8), AV_CODEC(RAWVIDEO), V4L2_FMT(SGRBG8), DRM_FMT(INVALID) }, + { AV_FMT(BAYER_RGGB8), AV_CODEC(RAWVIDEO), V4L2_FMT(SRGGB8), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_Y16 - { AV_FMT(GRAY16LE), AV_CODEC(RAWVIDEO), V4L2_FMT(Y16) }, + { AV_FMT(GRAY16LE), AV_CODEC(RAWVIDEO), V4L2_FMT(Y16), DRM_FMT(R16) }, #endif #ifdef V4L2_PIX_FMT_NV12M - { AV_FMT(NV12), AV_CODEC(RAWVIDEO), V4L2_FMT(NV12M) }, + { AV_FMT(NV12), AV_CODEC(RAWVIDEO), V4L2_FMT(NV12M), DRM_FMT(NV12) }, #endif #ifdef V4L2_PIX_FMT_NV21M - { AV_FMT(NV21), AV_CODEC(RAWVIDEO), V4L2_FMT(NV21M) }, + { AV_FMT(NV21), AV_CODEC(RAWVIDEO), V4L2_FMT(NV21M), DRM_FMT(NV21) }, #endif #ifdef V4L2_PIX_FMT_YUV420M - { AV_FMT(YUV420P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV420M) }, + { AV_FMT(YUV420P), AV_CODEC(RAWVIDEO), V4L2_FMT(YUV420M), DRM_FMT(YUV420) }, #endif #ifdef V4L2_PIX_FMT_NV16M - { AV_FMT(NV16), AV_CODEC(RAWVIDEO), V4L2_FMT(NV16M) }, + { AV_FMT(NV16), AV_CODEC(RAWVIDEO), V4L2_FMT(NV16M), DRM_FMT(NV16) }, #endif #ifdef V4L2_PIX_FMT_H263 - { AV_FMT(NONE), AV_CODEC(H263), V4L2_FMT(H263) }, + { AV_FMT(NONE), AV_CODEC(H263), V4L2_FMT(H263), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_H264 - { AV_FMT(NONE), AV_CODEC(H264), V4L2_FMT(H264) }, + { AV_FMT(NONE), AV_CODEC(H264), V4L2_FMT(H264), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_MPEG4 - { AV_FMT(NONE), AV_CODEC(MPEG4), V4L2_FMT(MPEG4) }, + { AV_FMT(NONE), AV_CODEC(MPEG4), V4L2_FMT(MPEG4), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_CPIA1 - { AV_FMT(NONE), AV_CODEC(CPIA), V4L2_FMT(CPIA1) }, + { AV_FMT(NONE), AV_CODEC(CPIA), V4L2_FMT(CPIA1), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_DV - { AV_FMT(NONE), AV_CODEC(DVVIDEO), V4L2_FMT(DV) }, + { AV_FMT(NONE), AV_CODEC(DVVIDEO), V4L2_FMT(DV), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_MPEG1 - { AV_FMT(NONE), AV_CODEC(MPEG1VIDEO), V4L2_FMT(MPEG1) }, + { AV_FMT(NONE), AV_CODEC(MPEG1VIDEO), V4L2_FMT(MPEG1), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_MPEG2 - { AV_FMT(NONE), AV_CODEC(MPEG2VIDEO), V4L2_FMT(MPEG2) }, + { AV_FMT(NONE), AV_CODEC(MPEG2VIDEO), V4L2_FMT(MPEG2), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_VP8 - { AV_FMT(NONE), AV_CODEC(VP8), V4L2_FMT(VP8) }, + { AV_FMT(NONE), AV_CODEC(VP8), V4L2_FMT(VP8), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_VP9 - { AV_FMT(NONE), AV_CODEC(VP9), V4L2_FMT(VP9) }, + { AV_FMT(NONE), AV_CODEC(VP9), V4L2_FMT(VP9), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_HEVC - { AV_FMT(NONE), AV_CODEC(HEVC), V4L2_FMT(HEVC) }, + { AV_FMT(NONE), AV_CODEC(HEVC), V4L2_FMT(HEVC), DRM_FMT(INVALID) }, #endif #ifdef V4L2_PIX_FMT_VC1_ANNEX_G - { AV_FMT(NONE), AV_CODEC(VC1), V4L2_FMT(VC1_ANNEX_G) }, + { AV_FMT(NONE), AV_CODEC(VC1), V4L2_FMT(VC1_ANNEX_G), DRM_FMT(INVALID) }, #endif }; @@ -139,3 +147,13 @@ enum AVPixelFormat ff_v4l2_format_v4l2_to_avfmt(uint32_t v4l2_fmt, enum AVCodecI } return AV_PIX_FMT_NONE; } + +uint32_t ff_v4l2_format_avfmt_to_drm(enum AVPixelFormat avfmt) +{ + int i; + for (i = 0; i < FF_ARRAY_ELEMS(fmt_map); i++) { + if (fmt_map[i].avfmt == avfmt) + return fmt_map[i].drm_fmt; + } + return DRM_FMT(INVALID); +} diff --git a/libavcodec/v4l2_fmt.h b/libavcodec/v4l2_fmt.h index 577e03a7a76c..f705fe9c0b30 100644 --- a/libavcodec/v4l2_fmt.h +++ b/libavcodec/v4l2_fmt.h @@ -31,5 +31,6 @@ enum AVPixelFormat ff_v4l2_format_v4l2_to_avfmt(uint32_t v4l2_fmt, enum AVCodecID avcodec); uint32_t ff_v4l2_format_avcodec_to_v4l2(enum AVCodecID avcodec); uint32_t ff_v4l2_format_avfmt_to_v4l2(enum AVPixelFormat avfmt); +uint32_t ff_v4l2_format_avfmt_to_drm(enum AVPixelFormat avfmt); #endif /* AVCODEC_V4L2_FMT_H*/ diff --git a/libavcodec/v4l2_m2m_dec.c b/libavcodec/v4l2_m2m_dec.c index 6b936b6df2a9..fe362e4598f4 100644 --- a/libavcodec/v4l2_m2m_dec.c +++ b/libavcodec/v4l2_m2m_dec.c @@ -29,6 +29,9 @@ #include "libavcodec/avcodec.h" #include "libavcodec/decode.h" #include "libavcodec/internal.h" +#include "hwconfig.h" +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_drm.h" #include "v4l2_context.h" #include "v4l2_m2m.h" @@ -99,6 +102,17 @@ static int v4l2_try_start(AVCodecContext *avctx) return ret; } + if (capture->support_dma_buf) { + ff_v4l2_context_uninit_hw_ctx(capture); + ret = ff_v4l2_context_init_hw_ctx(capture); + if (!ret) { + enum AVPixelFormat pix_fmts[3] = { AV_PIX_FMT_DRM_PRIME, + capture->av_pix_fmt, + AV_PIX_FMT_NONE }; + avctx->pix_fmt = ff_get_format(s->avctx, pix_fmts); + } + } + return 0; } @@ -224,6 +238,11 @@ static const AVOption options[] = { { NULL}, }; +static const AVCodecHWConfigInternal *const v4l2_m2m_dec_hw_configs[] = { + HW_CONFIG_INTERNAL(DRM_PRIME), + NULL +}; + #define M2MDEC_CLASS(NAME) \ static const AVClass v4l2_m2m_ ## NAME ## _dec_class = { \ .class_name = #NAME "_v4l2m2m_decoder", \ @@ -248,6 +267,7 @@ static const AVOption options[] = { .capabilities = AV_CODEC_CAP_HARDWARE | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AVOID_PROBING, \ .caps_internal = FF_CODEC_CAP_SETS_PKT_DTS | FF_CODEC_CAP_INIT_CLEANUP, \ .wrapper_name = "v4l2m2m", \ + .hw_configs = v4l2_m2m_dec_hw_configs, \ } M2MDEC(h264, "H.264", AV_CODEC_ID_H264, "h264_mp4toannexb");