From patchwork Mon Jan 8 23:46:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Rusak X-Patchwork-Id: 7213 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.78.2 with SMTP id r2csp3247518jaa; Mon, 8 Jan 2018 15:47:04 -0800 (PST) X-Google-Smtp-Source: ACJfBos97A1vbU8jFsS4ZmPPtqQ+IYxFeGiec8K0+HtNfnlUY9KmO1r24t9zq0f42SrNxK9CXymb X-Received: by 10.223.198.20 with SMTP id n20mr11551843wrg.191.1515455224749; Mon, 08 Jan 2018 15:47:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515455224; cv=none; d=google.com; s=arc-20160816; b=cT2Y/bQy+385RQklswxdS8YhON0XezV3u7eYTkTZSYEFwyDkBcZJTmpG+Ig9W7Wvro db97WhlzxsQp3BCSdjUWJfygb9kZTLZ7X9TZTaG3n9Gt8opqjWpdORqyqowy/cDprxrq 2a2zHlCJH3VnR7ITofcauEhEryJhyg33oUz3hSjfZMvy/LTbhg/ik9hn2K3q1snOOBx4 OADip1JABvionJyUSOgjcGzPEddAlXkPKYhBgjQQZ6vRjpyFxOghU+I+ysc22omFEMn4 rO/56ZsdIQrrVEXCnvXdCq9yQkelfu197iSI6L/F2tWc20Lto7++GWiysS2FKr60VXbK PEyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=HqHCce8yFW3zcXVxaKZ/m8rKlGJF1aMvYC/51yofD1M=; b=jCVJa5iBVEsomORXaHY54UDpZwJRFMxwMTTutB0qQzSYK5T/xtdPcYUdaEBr+xNwYf FoWFcpVslkhD/kUf+Apgh493AZzKKkNnYZD8YoGH+nVnHL7II/Z0xU53RdASzHNYCfJ0 V1LnSjxXBOL7VIHX2R4SltUR0HFacTON90X1t0labrgT+CzD76r6xt13v1Kmd9lrKT1z bXBXSbpGhVAqZZDH1KCJVLRX9RzD3UcPpbn383rvcX37aEA2Jf8Gdxn1YniC9kvU8wo9 mfunr3RqHEKIXn+dMqTIFLEk/ITHZJmbtMwbrluxgYx5KFPOid3HnRAFEunnypfwpcRg x1yA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=lreyCy7I; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h12si8396323wmc.244.2018.01.08.15.47.04; Mon, 08 Jan 2018 15:47:04 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=lreyCy7I; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 62E38689B6A; Tue, 9 Jan 2018 01:47:02 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg0-f51.google.com (mail-pg0-f51.google.com [74.125.83.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 153C9689A21 for ; Tue, 9 Jan 2018 01:46:56 +0200 (EET) Received: by mail-pg0-f51.google.com with SMTP id d6so5623902pgv.2 for ; Mon, 08 Jan 2018 15:46:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=INO5+aUry7j/jujzFK4ZuDdX9V1sUcrfVwN7AttajLQ=; b=lreyCy7IPBm1SwENsBQQcj8Lq7xvRMgzPYpqARLWCbZK7bLybCdoSuky4P9WX7vZKw n+QChkDcKPblFVcqiB0E15D+Swg+0iZiPNQ+E5UhPhciBbNwO+L1p/u76z3GpDPWW2os W/4o+XUqXApd8iBKQFc8grg1tQYORaQutJ00GiLfr6DckdhWuojglGpfc5sZvPLBViLG 47E1LY3h1Po+RCVoMEaQ3UbLefgWLpK2p5Zi3WX2TjJO5jwctP/S8uc0/uCw/MS/a/9f d/d/0EXztimBE1HYNnMwNJyXk7DdtyizOZ+V6s7Ml+Ihiv4eHPrEPBGfN4KrIDCiJja8 CvLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=INO5+aUry7j/jujzFK4ZuDdX9V1sUcrfVwN7AttajLQ=; b=skHKng/Kh2YOgxbRwulm0SFqM3WdFTzuW7NH62ZIYqN6nQWjFY4AxmTxY2iY7xm/sp RaKUHwyLkwMEhov6CLLLNMX02QF0SCX3+cnJWzqFNPSr7rKkqQ8+L40tTbKuWQ9O6LzF j2Nkd5iPVT4oKQ+cKI+8RlexuQZXNKP9K+a1cSAGb2FE7me/U5i5bHOgJ4Lmvd2BYCG+ e5qROhRAgtCmGMfDzgwQmGM+j3tcsCH6xeARamiINo9PFrlcaCbAJ8JYGSlImXpRz6Yy 40oBTywdEwSTLg7kqz2lIamvqqPTedvio4bZaJW8nRT6K6amcQe/93BgmXYQIaOd8PkD Y2IQ== X-Gm-Message-State: AKGB3mLKLlKkEQEpo7I/dDatNqO1XR9z71IynE7IIKuxJAm53hOlZEj4 2KJ+l5X5QH8BWkgsVCyOBkAGKRI8 X-Received: by 10.84.240.71 with SMTP id h7mr13524162plt.418.1515455215066; Mon, 08 Jan 2018 15:46:55 -0800 (PST) Received: from lukas-macbook-pro.telus (d154-20-200-72.bchsia.telus.net. [154.20.200.72]) by smtp.gmail.com with ESMTPSA id l64sm4979098pfi.46.2018.01.08.15.46.53 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 08 Jan 2018 15:46:54 -0800 (PST) From: Lukas Rusak To: ffmpeg-devel@ffmpeg.org Date: Mon, 8 Jan 2018 15:46:52 -0800 Message-Id: <20180108234652.27678-1-lorusak@gmail.com> X-Mailer: git-send-email 2.14.3 Subject: [FFmpeg-devel] [PATCH][RFC] libavcodec/v4l2_m2m_dec: output AVDRMFrameDescriptor X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This is a patch I have been testing for a while in combination with kodi's drmprime renderer I have been testing this with i.MX6 and Dragonboard 410c. So it covers single and multiplanar formats. I'm looking to get some comments on how to integrate this properly so that if we request AV_PIX_FMT_DRM_PRIME we can use that otherwise just use the default buffers. I basically followed how it was done in the rockchip decoder. --- libavcodec/v4l2_buffers.c | 97 +++++++++++++++++++++++++++++++++++------------ libavcodec/v4l2_buffers.h | 1 + libavcodec/v4l2_m2m_dec.c | 45 ++++++++++++---------- 3 files changed, 97 insertions(+), 46 deletions(-) diff --git a/libavcodec/v4l2_buffers.c b/libavcodec/v4l2_buffers.c index ba70c5d14b..065074c944 100644 --- a/libavcodec/v4l2_buffers.c +++ b/libavcodec/v4l2_buffers.c @@ -21,6 +21,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include #include #include #include @@ -29,6 +30,8 @@ #include #include "libavcodec/avcodec.h" #include "libavcodec/internal.h" +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_drm.h" #include "v4l2_context.h" #include "v4l2_buffers.h" #include "v4l2_m2m.h" @@ -202,11 +205,14 @@ static enum AVColorTransferCharacteristic v4l2_get_color_trc(V4L2Buffer *buf) return AVCOL_TRC_UNSPECIFIED; } -static void v4l2_free_buffer(void *opaque, uint8_t *unused) +static void v4l2_free_buffer(void *opaque, uint8_t *data) { + AVDRMFrameDescriptor *desc = (AVDRMFrameDescriptor *)data; V4L2Buffer* avbuf = opaque; V4L2m2mContext *s = buf_to_m2mctx(avbuf); + av_free(desc); + atomic_fetch_sub_explicit(&s->refcount, 1, memory_order_acq_rel); if (s->reinit) { if (!atomic_load(&s->refcount)) @@ -289,35 +295,15 @@ int ff_v4l2_buffer_avframe_to_buf(const AVFrame *frame, V4L2Buffer* out) int ff_v4l2_buffer_buf_to_avframe(AVFrame *frame, V4L2Buffer *avbuf) { V4L2m2mContext *s = buf_to_m2mctx(avbuf); - int i, ret; + AVDRMFrameDescriptor *desc = NULL; + AVDRMLayerDescriptor *layer = NULL; + int ret; av_frame_unref(frame); - /* 1. get references to the actual data */ - for (i = 0; i < avbuf->num_planes; i++) { - ret = v4l2_buf_to_bufref(avbuf, i, &frame->buf[i]); - if (ret) - return ret; - - frame->linesize[i] = avbuf->plane_info[i].bytesperline; - frame->data[i] = frame->buf[i]->data; - } - - /* 1.1 fixup special cases */ - switch (avbuf->context->av_pix_fmt) { - case AV_PIX_FMT_NV12: - if (avbuf->num_planes > 1) - break; - frame->linesize[1] = avbuf->plane_info[0].bytesperline; - frame->data[1] = frame->buf[0]->data + avbuf->plane_info[0].bytesperline * avbuf->context->format.fmt.pix_mp.height; - break; - default: - break; - } - /* 2. get frame information */ frame->key_frame = !!(avbuf->buf.flags & V4L2_BUF_FLAG_KEYFRAME); - frame->format = avbuf->context->av_pix_fmt; + frame->format = AV_PIX_FMT_DRM_PRIME; frame->color_primaries = v4l2_get_color_primaries(avbuf); frame->colorspace = v4l2_get_color_space(avbuf); frame->color_range = v4l2_get_color_range(avbuf); @@ -328,6 +314,42 @@ int ff_v4l2_buffer_buf_to_avframe(AVFrame *frame, V4L2Buffer *avbuf) frame->height = s->output.height; frame->width = s->output.width; + ret = ff_v4l2_buffer_export(avbuf); + if (ret < 0) { + return AVERROR(errno); + } + + desc = av_mallocz(sizeof(AVDRMFrameDescriptor)); + if (!desc) { + return AVERROR(ENOMEM); + } + + desc->nb_objects = 1; + + if (V4L2_TYPE_IS_MULTIPLANAR(avbuf->buf.type)) { + desc->objects[0].fd = avbuf->buf.m.planes[0].m.fd; + desc->objects[0].size = avbuf->buf.m.planes[0].length; + } else { + desc->objects[0].fd = avbuf->buf.m.fd; + desc->objects[0].size = avbuf->buf.length; + } + + desc->nb_layers = 1; + layer = &desc->layers[0]; + layer->format = DRM_FORMAT_NV12; + layer->nb_planes = 2; + + layer->planes[0].object_index = 0; + layer->planes[0].offset = 0; + layer->planes[0].pitch = avbuf->plane_info[0].bytesperline; + + layer->planes[1].object_index = 0; + layer->planes[1].offset = layer->planes[0].pitch * avbuf->context->format.fmt.pix_mp.height; + layer->planes[1].pitch = avbuf->plane_info[0].bytesperline; + + frame->data[0] = (uint8_t *)desc; + frame->buf[0] = av_buffer_create((uint8_t *)desc, sizeof(*desc), v4l2_free_buffer, avbuf, AV_BUFFER_FLAG_READONLY); + /* 3. report errors upstream */ if (avbuf->buf.flags & V4L2_BUF_FLAG_ERROR) { av_log(logger(avbuf), AV_LOG_ERROR, "%s: driver decode error\n", avbuf->context->name); @@ -461,3 +483,28 @@ int ff_v4l2_buffer_enqueue(V4L2Buffer* avbuf) return 0; } + +int ff_v4l2_buffer_export(V4L2Buffer* avbuf) +{ + int i, ret; + + for (i = 0; i < avbuf->num_planes; i++) { + + struct v4l2_exportbuffer expbuf; + memset(&expbuf, 0, sizeof(expbuf)); + expbuf.type = avbuf->buf.type; + expbuf.index = avbuf->buf.index; + expbuf.plane = i; + + ret = ioctl(buf_to_m2mctx(avbuf)->fd, VIDIOC_EXPBUF, &expbuf); + if (ret < 0) + return AVERROR(errno); + + if (V4L2_TYPE_IS_MULTIPLANAR(avbuf->buf.type)) + avbuf->buf.m.planes[i].m.fd = expbuf.fd; + else + avbuf->buf.m.fd = expbuf.fd; + } + + return 0; +} diff --git a/libavcodec/v4l2_buffers.h b/libavcodec/v4l2_buffers.h index e28a4a650d..317fdd1469 100644 --- a/libavcodec/v4l2_buffers.h +++ b/libavcodec/v4l2_buffers.h @@ -121,5 +121,6 @@ int ff_v4l2_buffer_initialize(V4L2Buffer* avbuf, int index); */ int ff_v4l2_buffer_enqueue(V4L2Buffer* avbuf); +int ff_v4l2_buffer_export(V4L2Buffer* avbuf); #endif // AVCODEC_V4L2_BUFFERS_H diff --git a/libavcodec/v4l2_m2m_dec.c b/libavcodec/v4l2_m2m_dec.c index 8308613978..4fd19ef115 100644 --- a/libavcodec/v4l2_m2m_dec.c +++ b/libavcodec/v4l2_m2m_dec.c @@ -23,6 +23,7 @@ #include #include +#include "libavutil/hwcontext_drm.h" #include "libavutil/pixfmt.h" #include "libavutil/pixdesc.h" #include "libavutil/opt.h" @@ -196,28 +197,30 @@ static const AVOption options[] = { { NULL}, }; +#define M2MDEC_CLASS(NAME) \ + static const AVClass v4l2_m2m_ ## NAME ## _dec_class = { \ + .class_name = #NAME "_v4l2_m2m_decoder", \ + .item_name = av_default_item_name, \ + .option = options, \ + .version = LIBAVUTIL_VERSION_INT, \ + }; + #define M2MDEC(NAME, LONGNAME, CODEC, bsf_name) \ -static const AVClass v4l2_m2m_ ## NAME ## _dec_class = {\ - .class_name = #NAME "_v4l2_m2m_decoder",\ - .item_name = av_default_item_name,\ - .option = options,\ - .version = LIBAVUTIL_VERSION_INT,\ -};\ -\ -AVCodec ff_ ## NAME ## _v4l2m2m_decoder = { \ - .name = #NAME "_v4l2m2m" ,\ - .long_name = NULL_IF_CONFIG_SMALL("V4L2 mem2mem " LONGNAME " decoder wrapper"),\ - .type = AVMEDIA_TYPE_VIDEO,\ - .id = CODEC ,\ - .priv_data_size = sizeof(V4L2m2mContext),\ - .priv_class = &v4l2_m2m_ ## NAME ## _dec_class,\ - .init = v4l2_decode_init,\ - .receive_frame = v4l2_receive_frame,\ - .close = ff_v4l2_m2m_codec_end,\ - .bsfs = bsf_name, \ - .capabilities = AV_CODEC_CAP_HARDWARE | AV_CODEC_CAP_DELAY, \ - .wrapper_name = "v4l2m2m", \ -}; + M2MDEC_CLASS(NAME) \ + AVCodec ff_ ## NAME ## _v4l2m2m_decoder = { \ + .name = #NAME "_v4l2m2m" , \ + .long_name = NULL_IF_CONFIG_SMALL("V4L2 mem2mem " LONGNAME " decoder wrapper"), \ + .type = AVMEDIA_TYPE_VIDEO, \ + .id = CODEC , \ + .priv_data_size = sizeof(V4L2m2mContext), \ + .priv_class = &v4l2_m2m_ ## NAME ## _dec_class, \ + .init = v4l2_decode_init, \ + .receive_frame = v4l2_receive_frame, \ + .close = ff_v4l2_m2m_codec_end, \ + .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_DRM_PRIME, \ + AV_PIX_FMT_NONE}, \ + .bsfs = bsf_name, \ + }; M2MDEC(h264, "H.264", AV_CODEC_ID_H264, "h264_mp4toannexb"); M2MDEC(hevc, "HEVC", AV_CODEC_ID_HEVC, "hevc_mp4toannexb");