From patchwork Tue May 8 18:24:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Rusak X-Patchwork-Id: 8875 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:155:0:0:0:0:0 with SMTP id c82-v6csp4324343jad; Tue, 8 May 2018 11:24:20 -0700 (PDT) X-Google-Smtp-Source: AB8JxZp8TfjA++c3BYBCA8PcAmx2WNkgPW02x49yIZ8q/J4TLVGSShiaWEyvJY24YmGEraqWl418 X-Received: by 10.28.23.201 with SMTP id 192mr3734005wmx.95.1525803860224; Tue, 08 May 2018 11:24:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525803860; cv=none; d=google.com; s=arc-20160816; b=Os3n7HfHhUDU8pFVfIB4u4dVsdWsyvf5VOQxx3+ygdCN9YNsCOdV62eYm8EIAs7Q9e 5rukrWU9KZMS+FTE/f9yLvU+p7Tq8+HUhl0mkmxpS1Yc4NuAaRDDFXtqIxITBDRSmyAu v31emAGhGe5xvk5hsmn1lmSUC0rxXjdzBCvLfRO5JzUDU7KQI6tJo6ocg9X/XB5Nrv7P ZDCgmJ8riUL7rHaqfqHXfM/hk9uvYLJwX+zVU5KmgIpYQS9UsbzEKkb4zdbVhBLgG+Bo BIMiAb3VzLcC+TfxvTXYYYcqBmTpuPbBa2Yn4A7XUnTNjSAtV1rNNnghoyJ5JKbmO+cd WO6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=Gw3donzyup2bwqoviUsWIkIEqhSn9xu7SqsVRmHalaU=; b=nojVAYG08o+02SgxylT01/bH843/NXKfWF82KKjBtSzIsyi3n1nUbXFkzx0yfxSBdv Bf2q8mrNJNCy8cdb4bwRppWSDV76kWM9O+WbVUIpXX6ApyzFnGxQxqLzP5tAfkRy7hL0 WgxBGQO5ZVV6r+UPco4Q6NNDv1GviXFS3LlZFapMk07wZGwr/Ur/gpVoOZD2Gwog182/ gYt01M/xlq2DRslzBBWTc4pIORtrxk11KdXvXrsIKdZSF5bfxAvcd1VKirgDZ6JkBNxP jvvpDa2MYe/CJoeXC2oypfD7yeHlP88xBiJg1qqbNIYxdODaSaLoDAW3peaID0NeO+hU JypA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=NXeehKCJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id z13-v6si1518080wmh.151.2018.05.08.11.24.19; Tue, 08 May 2018 11:24:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=NXeehKCJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 022B168A721; Tue, 8 May 2018 21:23:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl0-f42.google.com (mail-pl0-f42.google.com [209.85.160.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 046E268A3A5 for ; Tue, 8 May 2018 21:23:36 +0300 (EEST) Received: by mail-pl0-f42.google.com with SMTP id a39-v6so2709945pla.10 for ; Tue, 08 May 2018 11:24:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=Uy9Sw6L6Cu571zV52JpY+KpNWXPhRpGFh8DskqxiAqQ=; b=NXeehKCJdxU7YfCfS57mfr62adJeYLKUx0DWC5wSzj78WQDXbIjxO4zD02q5OtC7cV cJA7hIxPcWRoVoW1zBynun24V7RCRqctAa4w7aeLit1N8cd7p9ueBCQAi8ERAvSF2avT o4WHufNnv2pgcoDOAInMYDOSNIkm5LMAiqSxxa75+2dIxugFxW5c0WAfk0jJ+9PcQeqk SjQ8MnAtfFDBJi3OL6fiN5T14843Diwp59EmAhWn+s3iPtsPWYyJ801SnCMSluMFoiUn HWDXmzqXj2WCNPpvtfX+BKMFlABSwlqlAMfU3tKlZAOlYJXuVznvJXllk0qLTk8jDXjc mypg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=Uy9Sw6L6Cu571zV52JpY+KpNWXPhRpGFh8DskqxiAqQ=; b=dZCi86AH2vzjlctWDQ3uFPAmkK3crgNg/abyejbLQHRUqSyxK12ZRMfx/1qLEMtYfx hs/UkyfuXx4U7FgUq+5SB17X+GKrCxLrY8RB3DvId7hqsIdUSQnl0uUUtemES/86DuQ1 vl6+C6Ay06oQng9lwrQ7SPuwIlJ0DVgbX1EbN5O5zb0G6dpS/5rMAIwgE3V/mh5yt2fR RNO7Dise1bcym92V7iFLbX85/Gek6njnTLGLNQyL4Y3+073BZS7JoRosA6tSJxAQkNoS F24+lJoez7Ofrfu9QLL9tA7ejdQhHZAP27QtnQufwcAz8VAEQ57c9iufGeE3PZYuBeI9 5L4Q== X-Gm-Message-State: ALQs6tA3mvDakMzPZwanHzZE/ZG3B2F+NqT0hjz4fI7iR4VEjqqbWwEw xJnl3ceo94bKzErsBZc170UuV6Nc X-Received: by 2002:a17:902:a60d:: with SMTP id u13-v6mr23957542plq.40.1525803849214; Tue, 08 May 2018 11:24:09 -0700 (PDT) Received: from localhost.localdomain (d154-20-200-72.bchsia.telus.net. [154.20.200.72]) by smtp.gmail.com with ESMTPSA id w184-v6sm38311189pgb.20.2018.05.08.11.24.08 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 May 2018 11:24:08 -0700 (PDT) From: Lukas Rusak To: ffmpeg-devel@ffmpeg.org Date: Tue, 8 May 2018 11:24:05 -0700 Message-Id: <20180508182406.8846-2-lorusak@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180508182406.8846-1-lorusak@gmail.com> References: <20180508182406.8846-1-lorusak@gmail.com> Subject: [FFmpeg-devel] [PATCH 2/3] libavcodec: v4l2m2m: output AVDRMFrameDescriptor X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This is V2 of my previous patch. I have worked together with Jorge to get this working properly. We have made it so that AV_PIX_FMT_DRM_PRIME output can be selected by setting avctx->pix_fmt. This allows v4l2 to export the buffer so we can use it for zero-copy. If AV_PIX_FMT_DRM_PRIME is not selected then the standard pixel formats will be used and the buffers will not be exported. --- libavcodec/v4l2_buffers.c | 228 +++++++++++++++++++++++++++++++------- libavcodec/v4l2_buffers.h | 5 +- libavcodec/v4l2_context.c | 40 ++++++- libavcodec/v4l2_m2m.c | 4 +- libavcodec/v4l2_m2m.h | 3 + libavcodec/v4l2_m2m_dec.c | 23 ++++ 6 files changed, 257 insertions(+), 46 deletions(-) diff --git a/libavcodec/v4l2_buffers.c b/libavcodec/v4l2_buffers.c index aef911f3bb..d715bc6a7c 100644 --- a/libavcodec/v4l2_buffers.c +++ b/libavcodec/v4l2_buffers.c @@ -21,6 +21,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include #include #include #include @@ -29,6 +30,7 @@ #include #include "libavcodec/avcodec.h" #include "libavcodec/internal.h" +#include "libavutil/hwcontext.h" #include "v4l2_context.h" #include "v4l2_buffers.h" #include "v4l2_m2m.h" @@ -203,7 +205,65 @@ static enum AVColorTransferCharacteristic v4l2_get_color_trc(V4L2Buffer *buf) return AVCOL_TRC_UNSPECIFIED; } -static void v4l2_free_buffer(void *opaque, uint8_t *unused) +static uint8_t * v4l2_get_drm_frame(V4L2Buffer *avbuf) +{ + AVDRMFrameDescriptor *drm_desc = &avbuf->drm_frame; + AVDRMLayerDescriptor *layer; + + /* fill the DRM frame descriptor */ + drm_desc->nb_objects = 1; + drm_desc->nb_layers = 1; + + layer = &drm_desc->layers[0]; + layer->planes[0].object_index = 0; + layer->planes[0].offset = 0; + layer->planes[0].pitch = avbuf->plane_info[0].bytesperline; + + switch (avbuf->context->av_pix_fmt) { + case AV_PIX_FMT_NV12: + case AV_PIX_FMT_NV21: + + if (avbuf->num_planes > 1) + break; + + layer->format = avbuf->context->av_pix_fmt == AV_PIX_FMT_NV12 ? + DRM_FORMAT_NV12 : DRM_FORMAT_NV21; + layer->nb_planes = 2; + + layer->planes[1].object_index = 0; + layer->planes[1].offset = avbuf->plane_info[0].bytesperline * + avbuf->context->format.fmt.pix_mp.height; + layer->planes[1].pitch = avbuf->plane_info[0].bytesperline; + break; + + case AV_PIX_FMT_YUV420P: + + if (avbuf->num_planes > 1) + break; + + layer->format = DRM_FORMAT_YUV420; + layer->nb_planes = 3; + + layer->planes[1].object_index = 0; + layer->planes[1].offset = avbuf->plane_info[0].bytesperline * + avbuf->context->format.fmt.pix_mp.height; + layer->planes[1].pitch = avbuf->plane_info[0].bytesperline >> 1; + + layer->planes[2].object_index = 0; + layer->planes[2].offset = layer->planes[1].offset + + ((avbuf->plane_info[0].bytesperline * + avbuf->context->format.fmt.pix_mp.height) >> 2); + layer->planes[2].pitch = avbuf->plane_info[0].bytesperline >> 1; + break; + + default: + break; + } + + return (uint8_t *) drm_desc; +} + +static void v4l2_free_buffer(void *opaque, uint8_t *data) { V4L2Buffer* avbuf = opaque; V4L2m2mContext *s = buf_to_m2mctx(avbuf); @@ -227,27 +287,49 @@ static void v4l2_free_buffer(void *opaque, uint8_t *unused) } } -static int v4l2_buf_to_bufref(V4L2Buffer *in, int plane, AVBufferRef **buf) +static int v4l2_buffer_export_drm(V4L2Buffer* avbuf) { - V4L2m2mContext *s = buf_to_m2mctx(in); + struct v4l2_exportbuffer expbuf; + int i, ret; - if (plane >= in->num_planes) - return AVERROR(EINVAL); + for (i = 0; i < avbuf->num_planes; i++) { + memset(&expbuf, 0, sizeof(expbuf)); - /* even though most encoders return 0 in data_offset encoding vp8 does require this value */ - *buf = av_buffer_create((char *)in->plane_info[plane].mm_addr + in->planes[plane].data_offset, - in->plane_info[plane].length, v4l2_free_buffer, in, 0); - if (!*buf) - return AVERROR(ENOMEM); + expbuf.index = avbuf->buf.index; + expbuf.type = avbuf->buf.type; + expbuf.plane = i; + + ret = ioctl(buf_to_m2mctx(avbuf)->fd, VIDIOC_EXPBUF, &expbuf); + if (ret < 0) + return AVERROR(errno); + + if (V4L2_TYPE_IS_MULTIPLANAR(avbuf->buf.type)) { + avbuf->buf.m.planes[i].m.fd = expbuf.fd; + /* drm frame */ + avbuf->drm_frame.objects[i].size = avbuf->buf.m.planes[i].length; + avbuf->drm_frame.objects[i].fd = expbuf.fd; + } else { + avbuf->buf.m.fd = expbuf.fd; + /* drm frame */ + avbuf->drm_frame.objects[0].size = avbuf->buf.length; + avbuf->drm_frame.objects[0].fd = expbuf.fd; + } + } + + return 0; +} + +static int v4l2_buf_increase_ref(V4L2Buffer *in) +{ + V4L2m2mContext *s = buf_to_m2mctx(in); if (in->context_ref) atomic_fetch_add(&in->context_refcount, 1); else { in->context_ref = av_buffer_ref(s->self_ref); - if (!in->context_ref) { - av_buffer_unref(buf); + if (!in->context_ref) return AVERROR(ENOMEM); - } + in->context_refcount = 1; } @@ -257,6 +339,46 @@ static int v4l2_buf_to_bufref(V4L2Buffer *in, int plane, AVBufferRef **buf) return 0; } +static int v4l2_buf_to_bufref_drm(V4L2Buffer *in, AVBufferRef **buf) +{ + int ret; + + *buf = av_buffer_create((uint8_t *) &in->drm_frame, + sizeof(in->drm_frame), + v4l2_free_buffer, + in, AV_BUFFER_FLAG_READONLY); + if (!*buf) + return AVERROR(ENOMEM); + + ret = v4l2_buf_increase_ref(in); + if (ret) + av_buffer_unref(buf); + + return ret; +} + +static int v4l2_buf_to_bufref(V4L2Buffer *in, int plane, AVBufferRef **buf) +{ + int ret; + + if (plane >= in->num_planes) + return AVERROR(EINVAL); + + /* most encoders return 0 in data_offset but vp8 does require this value */ + *buf = av_buffer_create((char *)in->plane_info[plane].mm_addr + in->planes[plane].data_offset, + in->plane_info[plane].length, + v4l2_free_buffer, + in, 0); + if (!*buf) + return AVERROR(ENOMEM); + + ret = v4l2_buf_increase_ref(in); + if (ret) + av_buffer_unref(buf); + + return ret; +} + static int v4l2_bufref_to_buf(V4L2Buffer *out, int plane, const uint8_t* data, int size, AVBufferRef* bref) { unsigned int bytesused, length; @@ -267,7 +389,8 @@ static int v4l2_bufref_to_buf(V4L2Buffer *out, int plane, const uint8_t* data, i bytesused = FFMIN(size, out->plane_info[plane].length); length = out->plane_info[plane].length; - memcpy(out->plane_info[plane].mm_addr, data, FFMIN(size, out->plane_info[plane].length)); + memcpy(out->plane_info[plane].mm_addr, data, + FFMIN(size, out->plane_info[plane].length)); if (V4L2_TYPE_IS_MULTIPLANAR(out->buf.type)) { out->planes[plane].bytesused = bytesused; @@ -291,7 +414,10 @@ int ff_v4l2_buffer_avframe_to_buf(const AVFrame *frame, V4L2Buffer* out) int i, ret; for(i = 0; i < out->num_planes; i++) { - ret = v4l2_bufref_to_buf(out, i, frame->buf[i]->data, frame->buf[i]->size, frame->buf[i]); + ret = v4l2_bufref_to_buf(out, i, + frame->buf[i]->data, + frame->buf[i]->size, + frame->buf[i]); if (ret) return ret; } @@ -308,34 +434,46 @@ int ff_v4l2_buffer_buf_to_avframe(AVFrame *frame, V4L2Buffer *avbuf) av_frame_unref(frame); - /* 1. get references to the actual data */ - for (i = 0; i < avbuf->num_planes; i++) { - ret = v4l2_buf_to_bufref(avbuf, i, &frame->buf[i]); + if (buf_to_m2mctx(avbuf)->output_drm) { + /* 1. get references to the actual data */ + ret = v4l2_buf_to_bufref_drm(avbuf, &frame->buf[0]); if (ret) return ret; - frame->linesize[i] = avbuf->plane_info[i].bytesperline; - frame->data[i] = frame->buf[i]->data; - } + frame->data[0] = (uint8_t *) v4l2_get_drm_frame(avbuf); + frame->format = AV_PIX_FMT_DRM_PRIME; + } else { + /* 1. get references to the actual data */ + for (i = 0; i < avbuf->num_planes; i++) { + ret = v4l2_buf_to_bufref(avbuf, i, &frame->buf[i]); + if (ret) + return ret; + + frame->linesize[i] = avbuf->plane_info[i].bytesperline; + frame->data[i] = frame->buf[i]->data; + } - /* 1.1 fixup special cases */ - switch (avbuf->context->av_pix_fmt) { - case AV_PIX_FMT_NV12: - if (avbuf->num_planes > 1) + /* 1.1 fixup special cases */ + switch (avbuf->context->av_pix_fmt) { + case AV_PIX_FMT_NV12: + if (avbuf->num_planes > 1) + break; + frame->linesize[1] = avbuf->plane_info[0].bytesperline; + frame->data[1] = frame->buf[0]->data + + avbuf->plane_info[0].bytesperline * + avbuf->context->format.fmt.pix_mp.height; break; - frame->linesize[1] = avbuf->plane_info[0].bytesperline; - frame->data[1] = frame->buf[0]->data + avbuf->plane_info[0].bytesperline * avbuf->context->format.fmt.pix_mp.height; - break; - default: - break; + default: + break; + } + frame->format = avbuf->context->av_pix_fmt; } /* 2. get frame information */ frame->key_frame = !!(avbuf->buf.flags & V4L2_BUF_FLAG_KEYFRAME); - frame->format = avbuf->context->av_pix_fmt; frame->color_primaries = v4l2_get_color_primaries(avbuf); - frame->colorspace = v4l2_get_color_space(avbuf); frame->color_range = v4l2_get_color_range(avbuf); + frame->colorspace = v4l2_get_color_space(avbuf); frame->color_trc = v4l2_get_color_trc(avbuf); frame->pts = v4l2_get_pts(avbuf); @@ -361,7 +499,8 @@ int ff_v4l2_buffer_buf_to_avpkt(AVPacket *pkt, V4L2Buffer *avbuf) if (ret) return ret; - pkt->size = V4L2_TYPE_IS_MULTIPLANAR(avbuf->buf.type) ? avbuf->buf.m.planes[0].bytesused : avbuf->buf.bytesused; + pkt->size = V4L2_TYPE_IS_MULTIPLANAR(avbuf->buf.type) ? + avbuf->buf.m.planes[0].bytesused : avbuf->buf.bytesused; pkt->data = pkt->buf->data; if (avbuf->buf.flags & V4L2_BUF_FLAG_KEYFRAME) @@ -417,6 +556,7 @@ int ff_v4l2_buffer_initialize(V4L2Buffer* avbuf, int index) /* in MP, the V4L2 API states that buf.length means num_planes */ if (avbuf->num_planes >= avbuf->buf.length) break; + if (avbuf->buf.m.planes[avbuf->num_planes].length) avbuf->num_planes++; } @@ -433,12 +573,14 @@ int ff_v4l2_buffer_initialize(V4L2Buffer* avbuf, int index) avbuf->plane_info[i].length = avbuf->buf.m.planes[i].length; avbuf->plane_info[i].mm_addr = mmap(NULL, avbuf->buf.m.planes[i].length, PROT_READ | PROT_WRITE, MAP_SHARED, - buf_to_m2mctx(avbuf)->fd, avbuf->buf.m.planes[i].m.mem_offset); + buf_to_m2mctx(avbuf)->fd, + avbuf->buf.m.planes[i].m.mem_offset); } else { avbuf->plane_info[i].length = avbuf->buf.length; avbuf->plane_info[i].mm_addr = mmap(NULL, avbuf->buf.length, PROT_READ | PROT_WRITE, MAP_SHARED, - buf_to_m2mctx(avbuf)->fd, avbuf->buf.m.offset); + buf_to_m2mctx(avbuf)->fd, + avbuf->buf.m.offset); } if (avbuf->plane_info[i].mm_addr == MAP_FAILED) @@ -447,18 +589,23 @@ int ff_v4l2_buffer_initialize(V4L2Buffer* avbuf, int index) avbuf->status = V4L2BUF_AVAILABLE; - if (V4L2_TYPE_IS_OUTPUT(ctx->type)) - return 0; - if (V4L2_TYPE_IS_MULTIPLANAR(ctx->type)) { - avbuf->buf.m.planes = avbuf->planes; avbuf->buf.length = avbuf->num_planes; - + avbuf->buf.m.planes = avbuf->planes; } else { avbuf->buf.bytesused = avbuf->planes[0].bytesused; avbuf->buf.length = avbuf->planes[0].length; } + if (V4L2_TYPE_IS_OUTPUT(ctx->type)) + return 0; + + if (buf_to_m2mctx(avbuf)->output_drm) { + ret = v4l2_buffer_export_drm(avbuf); + if (ret) + return ret; + } + return ff_v4l2_buffer_enqueue(avbuf); } @@ -476,3 +623,4 @@ int ff_v4l2_buffer_enqueue(V4L2Buffer* avbuf) return 0; } + diff --git a/libavcodec/v4l2_buffers.h b/libavcodec/v4l2_buffers.h index dc5cc9e267..c609a6c676 100644 --- a/libavcodec/v4l2_buffers.h +++ b/libavcodec/v4l2_buffers.h @@ -27,6 +27,7 @@ #include #include +#include "libavutil/hwcontext_drm.h" #include "avcodec.h" enum V4L2Buffer_status { @@ -42,6 +43,9 @@ typedef struct V4L2Buffer { /* each buffer needs to have a reference to its context */ struct V4L2Context *context; + /* DRM descriptor */ + AVDRMFrameDescriptor drm_frame; + /* This object is refcounted per-plane, so we need to keep track * of how many context-refs we are holding. */ AVBufferRef *context_ref; @@ -127,5 +131,4 @@ int ff_v4l2_buffer_initialize(V4L2Buffer* avbuf, int index); */ int ff_v4l2_buffer_enqueue(V4L2Buffer* avbuf); - #endif // AVCODEC_V4L2_BUFFERS_H diff --git a/libavcodec/v4l2_context.c b/libavcodec/v4l2_context.c index efcb0426e4..9457fadb1e 100644 --- a/libavcodec/v4l2_context.c +++ b/libavcodec/v4l2_context.c @@ -393,22 +393,54 @@ static int v4l2_release_buffers(V4L2Context* ctx) struct v4l2_requestbuffers req = { .memory = V4L2_MEMORY_MMAP, .type = ctx->type, - .count = 0, /* 0 -> unmaps buffers from the driver */ + .count = 0, /* 0 -> unmap all buffers from the driver */ }; - int i, j; + int ret, i, j; for (i = 0; i < ctx->num_buffers; i++) { V4L2Buffer *buffer = &ctx->buffers[i]; for (j = 0; j < buffer->num_planes; j++) { struct V4L2Plane_info *p = &buffer->plane_info[j]; + + if (V4L2_TYPE_IS_OUTPUT(ctx->type)) { + /* output buffers are not EXPORTED */ + goto unmap; + } + + if (ctx_to_m2mctx(ctx)->output_drm) { + /* use the DRM frame to close */ + if (buffer->drm_frame.objects[j].fd >= 0) { + if (close(buffer->drm_frame.objects[j].fd) < 0) { + av_log(logger(ctx), AV_LOG_ERROR, "%s close drm fd " + "[buffer=%2d, plane=%d, fd=%2d] - %s \n", + ctx->name, i, j, buffer->drm_frame.objects[j].fd, + av_err2str(AVERROR(errno))); + } + } + } +unmap: if (p->mm_addr && p->length) if (munmap(p->mm_addr, p->length) < 0) - av_log(logger(ctx), AV_LOG_ERROR, "%s unmap plane (%s))\n", ctx->name, av_err2str(AVERROR(errno))); + av_log(logger(ctx), AV_LOG_ERROR, "%s unmap plane (%s))\n", + ctx->name, av_err2str(AVERROR(errno))); } } - return ioctl(ctx_to_m2mctx(ctx)->fd, VIDIOC_REQBUFS, &req); + ret = ioctl(ctx_to_m2mctx(ctx)->fd, VIDIOC_REQBUFS, &req); + if (ret < 0) { + av_log(logger(ctx), AV_LOG_ERROR, "release all %s buffers (%s)\n", + ctx->name, av_err2str(AVERROR(errno))); + + if (ctx_to_m2mctx(ctx)->output_drm) + av_log(logger(ctx), AV_LOG_ERROR, + "Make sure the DRM client releases all FB/GEM objects before closing the codec (ie):\n" + "for all buffers: \n" + " 1. drmModeRmFB(..)\n" + " 2. drmIoctl(.., DRM_IOCTL_GEM_CLOSE,... )\n"); + } + + return ret; } static inline int v4l2_try_raw_format(V4L2Context* ctx, enum AVPixelFormat pixfmt) diff --git a/libavcodec/v4l2_m2m.c b/libavcodec/v4l2_m2m.c index 427e165f58..7896326e80 100644 --- a/libavcodec/v4l2_m2m.c +++ b/libavcodec/v4l2_m2m.c @@ -159,7 +159,9 @@ static int v4l2_configure_contexts(V4L2m2mContext* s) goto error; } - /* decoder's buffers need to be updated at a later stage */ + /* decoder's capture buffers are updated during v4l2_try_start once we find + * the valid format. + */ if (!av_codec_is_decoder(s->avctx->codec)) { ret = ff_v4l2_context_init(&s->capture); if (ret) { diff --git a/libavcodec/v4l2_m2m.h b/libavcodec/v4l2_m2m.h index 452bf0d9bc..9ac5a2448d 100644 --- a/libavcodec/v4l2_m2m.h +++ b/libavcodec/v4l2_m2m.h @@ -59,6 +59,9 @@ typedef struct V4L2m2mContext { /* Reference to self; only valid while codec is active. */ AVBufferRef *self_ref; + + /* generate DRM frames */ + int output_drm; } V4L2m2mContext; typedef struct V4L2m2mPriv diff --git a/libavcodec/v4l2_m2m_dec.c b/libavcodec/v4l2_m2m_dec.c index ed5193ecc1..2b33badb08 100644 --- a/libavcodec/v4l2_m2m_dec.c +++ b/libavcodec/v4l2_m2m_dec.c @@ -23,12 +23,18 @@ #include #include + +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_drm.h" #include "libavutil/pixfmt.h" #include "libavutil/pixdesc.h" #include "libavutil/opt.h" #include "libavcodec/avcodec.h" #include "libavcodec/decode.h" +#include "libavcodec/hwaccel.h" +#include "libavcodec/internal.h" + #include "v4l2_context.h" #include "v4l2_m2m.h" #include "v4l2_fmt.h" @@ -183,6 +189,15 @@ static av_cold int v4l2_decode_init(AVCodecContext *avctx) capture->av_codec_id = AV_CODEC_ID_RAWVIDEO; capture->av_pix_fmt = avctx->pix_fmt; + /* the client requests the codec to generate DRM frames: + * - data[0] will therefore point to the returned AVDRMFrameDescriptor + * check the ff_v4l2_buffer_to_avframe conversion function. + * - the DRM frame format is passed in the DRM frame descriptor layer. + * check the v4l2_get_drm_frame function. + */ + if (avctx->pix_fmt == AV_PIX_FMT_DRM_PRIME) + s->output_drm = 1; + ret = ff_v4l2_m2m_codec_init(avctx); if (ret) { av_log(avctx, AV_LOG_ERROR, "can't configure decoder\n"); @@ -202,6 +217,11 @@ static const AVOption options[] = { { NULL}, }; +static const AVCodecHWConfigInternal *v4l2_m2m_hw_configs[] = { + HW_CONFIG_INTERNAL(DRM_PRIME), + NULL +}; + #define M2MDEC_CLASS(NAME) \ static const AVClass v4l2_m2m_ ## NAME ## _dec_class = { \ .class_name = #NAME "_v4l2_m2m_decoder", \ @@ -222,7 +242,10 @@ static const AVOption options[] = { .init = v4l2_decode_init, \ .receive_frame = v4l2_receive_frame, \ .close = ff_v4l2_m2m_codec_end, \ + .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_DRM_PRIME, \ + AV_PIX_FMT_NONE}, \ .bsfs = bsf_name, \ + .hw_configs = v4l2_m2m_hw_configs, \ .capabilities = AV_CODEC_CAP_HARDWARE | AV_CODEC_CAP_DELAY, \ .wrapper_name = "v4l2m2m", \ };