From patchwork Thu Jun 20 15:40:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ramiro Polla X-Patchwork-Id: 50021 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:8084:0:b0:482:c625:d099 with SMTP id q4csp462754vqg; Thu, 20 Jun 2024 08:47:57 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX2KhkQOFf9j7kOcCh3MCAsOYXL4G5RGWMa84ADhjCjMBGdLJhBggbzYHtWtI7fYVW7qlcHV46wBZtld/Z/+ntplbvtbZc6Y6bu+Q== X-Google-Smtp-Source: AGHT+IHBljXOwhOp56pnpQ4ldamqmW6E0vw93sY937HeNn5vPcdoBgsYdCN/8B4IjIOegf696vZC X-Received: by 2002:aa7:c392:0:b0:57d:107e:dd79 with SMTP id 4fb4d7f45d1cf-57d107edd96mr2467217a12.26.1718898476764; Thu, 20 Jun 2024 08:47:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718898476; cv=none; d=google.com; s=arc-20160816; b=DNOLg6kqL/szQzLweNuqR6rg+CwAyX0jQS8iD9cDyZvoe0DB/Oftl91fcBKnigtoeg KAcuzDiZdyFrJ80UVaf6IIyzdFcqgqEqqZBrZaQa5nO5cDOlVxFQBEEcJ802+iZgpioK Ua6rGnD9oWzVvmA6QkGCCwjdOqiEZB6XsDAhlqV73RDl/25VDsKiTqQ4c4Q6SQPPOPGz ilwphSDk6zwDwwwe14Y885DTcpLHiQEXyAhjpfINKfQ8JNsklSsoUapfElxkAgcNWi+u DZXJQJ8pF97uDkCaVEOtPmB/UOvhiLxPCqV+HP2oFzdzu2F+0soxdFuAHWCviLhdDj+f ghXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=hwuOLmC5krm5DRawCTYRMbjGhRnoxe4J0/e1thrkexc=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=nwvYbq52sRFi8LfJyUjNPGm1P3r5ACgIejzieT9/Pn5I60fAbp4KZTJ9kF2355auXr E0FJvg8pK5EM0QJlb+qAjXBeWmXcTQ9cVWQqFU6uEdZTdaNIgDrNMbXDcR2wXjgLKLm1 Oo3Z5ujAS0HJwWRUHF7wl+ecSdr+Gf//zuphlL1aZY8xySJH3xf1i1x/eC76YbsOILpc mSAeRMdDkpMu2ow1iErXCYpjl4D7x4x+LjV/IToF5LVDZOL4g/4xNQXGeWPgUOfV3UN+ /uu0xEypygtlfyVks7EDaO+hHhgUrAylUIydbl/Ctji9TVdeOPzcdPSHdNXNhFWguDbe gPIw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=NCXRJ6ae; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-57cbad009a6si7156135a12.329.2024.06.20.08.47.46; Thu, 20 Jun 2024 08:47:56 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=NCXRJ6ae; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3649568D83C; Thu, 20 Jun 2024 18:40:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DFE4A68D713 for ; Thu, 20 Jun 2024 18:40:42 +0300 (EEST) Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-421b9068274so10499515e9.1 for ; Thu, 20 Jun 2024 08:40:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718898041; x=1719502841; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=5aeF1QZ0iKv/Am246vwbLOV0LH5cZ/BFiDp76SbHnFU=; b=NCXRJ6ae2uEipFH2hfBVK5Ieu5DgVZQ8/K+78fQOB93PAV/qDSX+wYpaf6W3t2znxR Km/Ivmb560+pdgS4NflCT6Amu4aKVSBFZ0vyiGG8s4l1UrB0cEm4QeSLJmSX1ZBT+Oko lyc9W2st4bhKn64korw3O1l0t2Jo+tm8Kit/gHRgPR5GT1ZhN25lOdn8IuBuMqsiRxBW U83hW5PdslwKqPFnnw2eFi6L9kGVgILsm7xXNpac/YQFF7BpJ/+yyvgJG8BjqGv/n1O7 TE9e5MbM5kFI+++uMBU1+eBDSaA1TbQH95YhZQWujTptBSWJYParoj+uEZI2NDvc/s+B vGNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718898041; x=1719502841; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5aeF1QZ0iKv/Am246vwbLOV0LH5cZ/BFiDp76SbHnFU=; b=JL/ankU0l2flwH81q3fm7CsKWuNOqI1dHxMBRPYxDF8ORG/0jUe8NuSV1qM5F9d/oe RVAhSmbzNY5nOOhC7O7L/xFUdFe9Xdidlbx0Ci3kH+wzuOwB5pxYmOMdsRErbHasqWpU HN0sE8148SG4XNc/1DfyY5tZuFDr/dQ+Zb5i5EXd+NkM8XOlRBoeaLsJXw2vy7CrddA2 jCrujf19gbRK61fWHApp5kXGlYDj8guSAZgIoYRimABQprJruZgbvHoYJEwepmlqLRPv Qu1W8LffJlzXabD7cP25UOs5JLXjW7Vl4hGhngOGVQvwhdwDQ8mWz6xHxV0MjS0w5zHp LkUA== X-Gm-Message-State: AOJu0Yw2WF2t87QHohSsNFdYhWeK2IDSwlNFzz4dshlPtiu051w0xP7X LDeBGMQ/Pi0ePczTuKF1+BP3NTebx4Yv1EqzYn2dERV/a1hLP/nvw3vhbw== X-Received: by 2002:a05:600c:3423:b0:421:79a1:bd16 with SMTP id 5b1f17b1804b1-4247517780bmr44799645e9.16.1718898041302; Thu, 20 Jun 2024 08:40:41 -0700 (PDT) Received: from localhost.localdomain (176.140-201-80.adsl-dyn.isp.belgacom.be. [80.201.140.176]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4247d0b636bsm30254235e9.6.2024.06.20.08.40.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Jun 2024 08:40:40 -0700 (PDT) From: Ramiro Polla To: ffmpeg-devel@ffmpeg.org Date: Thu, 20 Jun 2024 17:40:39 +0200 Message-Id: <20240620154039.91460-1-ramiro.polla@gmail.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avdevice/v4l2: add limited support for multiplanar API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +d2dxAFvl9P4 This commit adds support for V4L2's multiplanar API, but only when the number of planes is 1. Adding full support for the multiplanar API would require a device that actually uses more than 1 plane, which I have not found yet. --- libavdevice/v4l2.c | 93 +++++++++++++++++++++++++++++++++------------- 1 file changed, 68 insertions(+), 25 deletions(-) diff --git a/libavdevice/v4l2.c b/libavdevice/v4l2.c index 74f43ef6a9..305c1f25dd 100644 --- a/libavdevice/v4l2.c +++ b/libavdevice/v4l2.c @@ -92,6 +92,9 @@ struct video_data { TimeFilter *timefilter; int64_t last_time_m; + int multiplanar; + enum v4l2_buf_type buf_type; + int buffers; atomic_int buffers_queued; void **buf_start; @@ -182,7 +185,13 @@ static int device_open(AVFormatContext *ctx, const char* device_path) av_log(ctx, AV_LOG_VERBOSE, "fd:%d capabilities:%x\n", fd, cap.capabilities); - if (!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)) { + if (cap.capabilities & V4L2_CAP_VIDEO_CAPTURE) { + s->multiplanar = 0; + s->buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + } else if (cap.capabilities & V4L2_CAP_VIDEO_CAPTURE_MPLANE) { + s->multiplanar = 1; + s->buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + } else { av_log(ctx, AV_LOG_ERROR, "Not a video capture device.\n"); err = AVERROR(ENODEV); goto fail; @@ -206,7 +215,7 @@ static int device_init(AVFormatContext *ctx, int *width, int *height, uint32_t pixelformat) { struct video_data *s = ctx->priv_data; - struct v4l2_format fmt = { .type = V4L2_BUF_TYPE_VIDEO_CAPTURE }; + struct v4l2_format fmt = { .type = s->buf_type }; int res = 0; fmt.fmt.pix.width = *width; @@ -288,7 +297,7 @@ static void list_framesizes(AVFormatContext *ctx, uint32_t pixelformat) static void list_formats(AVFormatContext *ctx, int type) { const struct video_data *s = ctx->priv_data; - struct v4l2_fmtdesc vfd = { .type = V4L2_BUF_TYPE_VIDEO_CAPTURE }; + struct v4l2_fmtdesc vfd = { .type = s->buf_type }; while(!v4l2_ioctl(s->fd, VIDIOC_ENUM_FMT, &vfd)) { enum AVCodecID codec_id = ff_fmt_v4l2codec(vfd.pixelformat); @@ -352,7 +361,7 @@ static int mmap_init(AVFormatContext *ctx) int i, res; struct video_data *s = ctx->priv_data; struct v4l2_requestbuffers req = { - .type = V4L2_BUF_TYPE_VIDEO_CAPTURE, + .type = s->buf_type, .count = desired_video_buffers, .memory = V4L2_MEMORY_MMAP }; @@ -381,10 +390,14 @@ static int mmap_init(AVFormatContext *ctx) } for (i = 0; i < req.count; i++) { + unsigned int buf_length, buf_offset; + struct v4l2_plane planes[VIDEO_MAX_PLANES]; struct v4l2_buffer buf = { - .type = V4L2_BUF_TYPE_VIDEO_CAPTURE, + .type = s->buf_type, .index = i, - .memory = V4L2_MEMORY_MMAP + .memory = V4L2_MEMORY_MMAP, + .m.planes = s->multiplanar ? planes : NULL, + .length = s->multiplanar ? VIDEO_MAX_PLANES : 0, }; if (v4l2_ioctl(s->fd, VIDIOC_QUERYBUF, &buf) < 0) { res = AVERROR(errno); @@ -392,16 +405,28 @@ static int mmap_init(AVFormatContext *ctx) return res; } - s->buf_len[i] = buf.length; + if (s->multiplanar) { + if (buf.length != 1) { + av_log(ctx, AV_LOG_ERROR, "multiplanar only supported when buf.length == 1\n"); + return AVERROR_PATCHWELCOME; + } + buf_length = buf.m.planes[0].length; + buf_offset = buf.m.planes[0].m.mem_offset; + } else { + buf_length = buf.length; + buf_offset = buf.m.offset; + } + + s->buf_len[i] = buf_length; if (s->frame_size > 0 && s->buf_len[i] < s->frame_size) { av_log(ctx, AV_LOG_ERROR, "buf_len[%d] = %d < expected frame size %d\n", i, s->buf_len[i], s->frame_size); return AVERROR(ENOMEM); } - s->buf_start[i] = v4l2_mmap(NULL, buf.length, + s->buf_start[i] = v4l2_mmap(NULL, buf_length, PROT_READ | PROT_WRITE, MAP_SHARED, - s->fd, buf.m.offset); + s->fd, buf_offset); if (s->buf_start[i] == MAP_FAILED) { res = AVERROR(errno); @@ -429,13 +454,16 @@ static int enqueue_buffer(struct video_data *s, struct v4l2_buffer *buf) static void mmap_release_buffer(void *opaque, uint8_t *data) { + struct v4l2_plane planes[VIDEO_MAX_PLANES]; struct v4l2_buffer buf = { 0 }; struct buff_data *buf_descriptor = opaque; struct video_data *s = buf_descriptor->s; - buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + buf.type = s->buf_type; buf.memory = V4L2_MEMORY_MMAP; buf.index = buf_descriptor->index; + buf.m.planes = s->multiplanar ? planes : NULL; + buf.length = s->multiplanar ? VIDEO_MAX_PLANES : 0; av_free(buf_descriptor); enqueue_buffer(s, &buf); @@ -505,11 +533,15 @@ static int convert_timestamp(AVFormatContext *ctx, int64_t *ts) static int mmap_read_frame(AVFormatContext *ctx, AVPacket *pkt) { struct video_data *s = ctx->priv_data; + struct v4l2_plane planes[VIDEO_MAX_PLANES]; struct v4l2_buffer buf = { - .type = V4L2_BUF_TYPE_VIDEO_CAPTURE, - .memory = V4L2_MEMORY_MMAP + .type = s->buf_type, + .memory = V4L2_MEMORY_MMAP, + .m.planes = s->multiplanar ? planes : NULL, + .length = s->multiplanar ? VIDEO_MAX_PLANES : 0, }; struct timeval buf_ts; + unsigned int bytesused; int res; pkt->size = 0; @@ -536,11 +568,16 @@ static int mmap_read_frame(AVFormatContext *ctx, AVPacket *pkt) // always keep at least one buffer queued av_assert0(atomic_load(&s->buffers_queued) >= 1); + bytesused = s->multiplanar ? buf.m.planes[0].bytesused : buf.bytesused; + #ifdef V4L2_BUF_FLAG_ERROR if (buf.flags & V4L2_BUF_FLAG_ERROR) { av_log(ctx, AV_LOG_WARNING, "Dequeued v4l2 buffer contains corrupted data (%d bytes).\n", - buf.bytesused); + bytesused); + if (s->multiplanar) + buf.m.planes[0].bytesused = 0; + else buf.bytesused = 0; } else #endif @@ -548,12 +585,15 @@ static int mmap_read_frame(AVFormatContext *ctx, AVPacket *pkt) /* CPIA is a compressed format and we don't know the exact number of bytes * used by a frame, so set it here as the driver announces it. */ if (ctx->video_codec_id == AV_CODEC_ID_CPIA) - s->frame_size = buf.bytesused; + s->frame_size = bytesused; - if (s->frame_size > 0 && buf.bytesused != s->frame_size) { + if (s->frame_size > 0 && bytesused != s->frame_size) { av_log(ctx, AV_LOG_WARNING, "Dequeued v4l2 buffer contains %d bytes, but %d were expected. Flags: 0x%08X.\n", - buf.bytesused, s->frame_size, buf.flags); + bytesused, s->frame_size, buf.flags); + if (s->multiplanar) + buf.m.planes[0].bytesused = 0; + else buf.bytesused = 0; } } @@ -561,13 +601,13 @@ static int mmap_read_frame(AVFormatContext *ctx, AVPacket *pkt) /* Image is at s->buff_start[buf.index] */ if (atomic_load(&s->buffers_queued) == FFMAX(s->buffers / 8, 1)) { /* when we start getting low on queued buffers, fall back on copying data */ - res = av_new_packet(pkt, buf.bytesused); + res = av_new_packet(pkt, bytesused); if (res < 0) { av_log(ctx, AV_LOG_ERROR, "Error allocating a packet.\n"); enqueue_buffer(s, &buf); return res; } - memcpy(pkt->data, s->buf_start[buf.index], buf.bytesused); + memcpy(pkt->data, s->buf_start[buf.index], bytesused); res = enqueue_buffer(s, &buf); if (res) { @@ -578,7 +618,7 @@ static int mmap_read_frame(AVFormatContext *ctx, AVPacket *pkt) struct buff_data *buf_descriptor; pkt->data = s->buf_start[buf.index]; - pkt->size = buf.bytesused; + pkt->size = bytesused; buf_descriptor = av_malloc(sizeof(struct buff_data)); if (!buf_descriptor) { @@ -615,10 +655,13 @@ static int mmap_start(AVFormatContext *ctx) int i, res; for (i = 0; i < s->buffers; i++) { + struct v4l2_plane planes[VIDEO_MAX_PLANES]; struct v4l2_buffer buf = { - .type = V4L2_BUF_TYPE_VIDEO_CAPTURE, + .type = s->buf_type, .index = i, - .memory = V4L2_MEMORY_MMAP + .memory = V4L2_MEMORY_MMAP, + .m.planes = s->multiplanar ? planes : NULL, + .length = s->multiplanar ? VIDEO_MAX_PLANES : 0, }; if (v4l2_ioctl(s->fd, VIDIOC_QBUF, &buf) < 0) { @@ -630,7 +673,7 @@ static int mmap_start(AVFormatContext *ctx) } atomic_store(&s->buffers_queued, s->buffers); - type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + type = s->buf_type; if (v4l2_ioctl(s->fd, VIDIOC_STREAMON, &type) < 0) { res = AVERROR(errno); av_log(ctx, AV_LOG_ERROR, "ioctl(VIDIOC_STREAMON): %s\n", @@ -646,7 +689,7 @@ static void mmap_close(struct video_data *s) enum v4l2_buf_type type; int i; - type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + type = s->buf_type; /* We do not check for the result, because we could * not do anything about it anyway... */ @@ -733,7 +776,7 @@ static int v4l2_set_parameters(AVFormatContext *ctx) tpf = &streamparm.parm.capture.timeperframe; } - streamparm.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + streamparm.type = s->buf_type; if (v4l2_ioctl(s->fd, VIDIOC_G_PARM, &streamparm) < 0) { ret = AVERROR(errno); av_log(ctx, AV_LOG_WARNING, "ioctl(VIDIOC_G_PARM): %s\n", av_err2str(ret)); @@ -921,7 +964,7 @@ static int v4l2_read_header(AVFormatContext *ctx) } if (!s->width && !s->height) { - struct v4l2_format fmt = { .type = V4L2_BUF_TYPE_VIDEO_CAPTURE }; + struct v4l2_format fmt = { .type = s->buf_type }; av_log(ctx, AV_LOG_VERBOSE, "Querying the device for the current frame size\n");