From patchwork Wed Feb 22 20:18:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Damien Riegel X-Patchwork-Id: 2647 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.65.149 with SMTP id x21csp1050763vsf; Wed, 22 Feb 2017 12:19:15 -0800 (PST) X-Received: by 10.223.163.26 with SMTP id c26mr14063746wrb.56.1487794755842; Wed, 22 Feb 2017 12:19:15 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b7si3436763wmb.54.2017.02.22.12.19.15; Wed, 22 Feb 2017 12:19:15 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6E4E4688276; Wed, 22 Feb 2017 22:19:04 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.savoirfairelinux.com (mail.savoirfairelinux.com [208.88.110.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 58D05687ECC for ; Wed, 22 Feb 2017 22:18:58 +0200 (EET) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id CA33F9C1A2F for ; Wed, 22 Feb 2017 15:19:04 -0500 (EST) Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id nFUXZ9MrJbVn for ; Wed, 22 Feb 2017 15:19:03 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id 9A79D9C1C19 for ; Wed, 22 Feb 2017 15:19:03 -0500 (EST) X-Virus-Scanned: amavisd-new at mail.savoirfairelinux.com Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id iS4N-JaXLfyj for ; Wed, 22 Feb 2017 15:19:03 -0500 (EST) Received: from workotop.localdomain (unknown [192.168.49.100]) by mail.savoirfairelinux.com (Postfix) with ESMTPSA id 66BD59C1A2F for ; Wed, 22 Feb 2017 15:19:03 -0500 (EST) Date: Wed, 22 Feb 2017 15:18:45 -0500 From: Damien Riegel To: FFmpeg development discussions and patches Message-ID: <20170222201845.r4wc5etnefoxkwvb@workotop.localdomain> References: <20170216174337.19488-1-damien.riegel@savoirfairelinux.com> <20170216174337.19488-2-damien.riegel@savoirfairelinux.com> <20170217200105.3vkg7zrsnstwtfpe@workotop.localdomain> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20170217200105.3vkg7zrsnstwtfpe@workotop.localdomain> User-Agent: NeoMutt/20161126 (1.7.1) Subject: Re: [FFmpeg-devel] [PATCH v1 1/2] codec: vrawdepay: add decoder for RFC4175 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On Fri, Feb 17, 2017 at 03:01:05PM -0500, Damien Riegel wrote: > Hi, > > On Thu, Feb 16, 2017 at 06:19:00PM +0000, Rostislav Pehlivanov wrote: > > > > > > No, do this in libavfilter and do not introduce another useless pseudo > > > codec > > > > > > > *libavformat, sorry > > The advantage of using a pseudo codec just to depack the stream is that > the input and the codec are in separate threads in ffmpeg, so it can > handle a heavier workload. Please find attached a v2, with the implementation in libavformat. Note that I don't want to send it as a patch of its own because the performance issue is not addressed. Basically, our test case is a raw input stream YUV 4:2:2 10 bits 1080p at 60fps. With the pseudo-codec, we are able to transcode it to h264 and dump it to a file. With unpacking done in the libavformat, the input thread gets too busy and can't stand the load. In the implementation you made [1] unpacking was done in libavcodec, so why is it not an acceptable solution for mainline? [1] https://github.com/funman/ffmpeg-sdi Thanks, From c81e69979433ed519932354e848889a90d25f1c5 Mon Sep 17 00:00:00 2001 From: Damien Riegel Date: Wed, 16 Nov 2016 15:27:13 -0500 Subject: [PATCH v2] rtp: rawvideo: add handler for YCbCr-4:2:2 This adds partial support for the RFC 4175 (raw video over RTP). The only supported formats are the YCbCr-4:2:2 8 and 10 bit. For 8-bit depth, the format is similar to AV_PIX_FMT_UYVY422, and for 10-bit, the stream is repacked to the planar format AV_PIX_FMT_YUV422P10LE. Signed-off-by: Damien Riegel --- libavformat/Makefile | 1 + libavformat/rtpdec.c | 1 + libavformat/rtpdec_formats.h | 1 + libavformat/rtpdec_rawvideo.c | 259 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 262 insertions(+) create mode 100644 libavformat/rtpdec_rawvideo.c diff --git a/libavformat/Makefile b/libavformat/Makefile index fc2d76067b..e17ba95c23 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -55,6 +55,7 @@ OBJS-$(CONFIG_RTPDEC) += rdt.o \ rtpdec_qcelp.o \ rtpdec_qdm2.o \ rtpdec_qt.o \ + rtpdec_rawvideo.o \ rtpdec_svq3.o \ rtpdec_vc2hq.o \ rtpdec_vp8.o \ diff --git a/libavformat/rtpdec.c b/libavformat/rtpdec.c index 53cdad7396..345318c32c 100644 --- a/libavformat/rtpdec.c +++ b/libavformat/rtpdec.c @@ -114,6 +114,7 @@ void ff_register_rtp_dynamic_payload_handlers(void) ff_register_dynamic_payload_handler(&ff_qt_rtp_vid_handler); ff_register_dynamic_payload_handler(&ff_quicktime_rtp_aud_handler); ff_register_dynamic_payload_handler(&ff_quicktime_rtp_vid_handler); + ff_register_dynamic_payload_handler(&ff_rawvideo_rtp_handler); ff_register_dynamic_payload_handler(&ff_svq3_dynamic_handler); ff_register_dynamic_payload_handler(&ff_theora_dynamic_handler); ff_register_dynamic_payload_handler(&ff_vc2hq_dynamic_handler); diff --git a/libavformat/rtpdec_formats.h b/libavformat/rtpdec_formats.h index 3292a3d265..fab12fd139 100644 --- a/libavformat/rtpdec_formats.h +++ b/libavformat/rtpdec_formats.h @@ -82,6 +82,7 @@ extern RTPDynamicProtocolHandler ff_qt_rtp_aud_handler; extern RTPDynamicProtocolHandler ff_qt_rtp_vid_handler; extern RTPDynamicProtocolHandler ff_quicktime_rtp_aud_handler; extern RTPDynamicProtocolHandler ff_quicktime_rtp_vid_handler; +extern RTPDynamicProtocolHandler ff_rawvideo_rtp_handler; extern RTPDynamicProtocolHandler ff_svq3_dynamic_handler; extern RTPDynamicProtocolHandler ff_theora_dynamic_handler; extern RTPDynamicProtocolHandler ff_vc2hq_dynamic_handler; diff --git a/libavformat/rtpdec_rawvideo.c b/libavformat/rtpdec_rawvideo.c new file mode 100644 index 0000000000..7edde84306 --- /dev/null +++ b/libavformat/rtpdec_rawvideo.c @@ -0,0 +1,259 @@ +/* + * RTP Depacketization of RAW video (TR-03) + * Copyright (c) 2016 Savoir-faire Linux, Inc + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* Development sponsored by CBC/Radio-Canada */ + +#include +#include +#include +#include +#include "avio_internal.h" +#include "rtpdec_formats.h" + +struct PayloadContext { + char *sampling; + int depth; + int width; + int height; + enum AVPixelFormat pixfmt; + + void *frame; + unsigned int frame_size; + unsigned int pgroup; /* size of the pixel group in bytes */ + unsigned int xinc; + + uint32_t timestamp; +}; + +static int raw_parse_format(AVStream *stream, + PayloadContext *data) +{ + enum AVPixelFormat pixfmt = AV_PIX_FMT_NONE; + int tag = 0; + + if (!strcmp(data->sampling, "YCbCr-4:2:2")) { + data->xinc = 2; + + if (data->depth == 8) { + tag = MKTAG('U', 'Y', 'V', 'Y'); + data->pgroup = 4; + pixfmt = AV_PIX_FMT_UYVY422; + } else if (data->depth == 10) { + tag = MKTAG('I', '2', 'A', 'L'); + data->pgroup = 5; + pixfmt = AV_PIX_FMT_YUV422P10; + } else { + return -1; + } + + stream->codecpar->bits_per_coded_sample = data->depth * 2; + } else { + return -1; + } + + stream->codecpar->codec_tag = tag; + stream->codecpar->field_order = AV_FIELD_PROGRESSIVE; + data->frame_size = av_image_get_buffer_size(pixfmt, data->width, data->height, 1); + data->pixfmt = pixfmt; + + return 0; +} + +static int raw_parse_fmtp(AVFormatContext *s, AVStream *stream, + PayloadContext *data, const char *attr, + const char *value) +{ + if (!strcmp(attr, "width")) + data->width = atoi(value); + else if (!strcmp(attr, "height")) + data->height = atoi(value); + else if (!strcmp(attr, "sampling")) + data->sampling = av_strdup(value); + else if (!strcmp(attr, "depth")) + data->depth = atoi(value); + + return 0; +} + +static int raw_parse_sdp_line(AVFormatContext *s, int st_index, + PayloadContext *data, const char *line) +{ + const char *p; + + if (st_index < 0) + return 0; + + if (av_strstart(line, "fmtp:", &p)) { + AVStream *stream = s->streams[st_index]; + int ret = ff_parse_fmtp(s, stream, data, p, raw_parse_fmtp); + + if (ret < 0) + return ret; + + + if (!data->sampling || !data->depth || !data->width || !data->height) + return -1; + + stream->codecpar->width = data->width; + stream->codecpar->height = data->height; + + ret = raw_parse_format(stream, data); + av_freep(&data->sampling); + + return ret; + } + + return 0; +} + +static int raw_finalize_packet(PayloadContext *data, AVPacket *pkt, + int stream_index) +{ + int ret; + + pkt->stream_index = stream_index; + ret = av_packet_from_data(pkt, data->frame, data->frame_size); + if (ret < 0) { + av_freep(&data->frame); + } + + data->frame = NULL; + + return ret; +} + +static int raw_handle_packet(AVFormatContext *ctx, PayloadContext *data, + AVStream *st, AVPacket *pkt, uint32_t *timestamp, + const uint8_t * buf, int len, + uint16_t seq, int flags) +{ + int length, line, offset, cont; + const uint8_t *headers = buf + 2; /* skip extended seqnum */ + const uint8_t *payload = buf + 2; + int payload_len = len - 2; + int missed_last_packet = 0; + + if (*timestamp != data->timestamp) { + if (data->frame) { + /* + * if we're here, it means that two RTP packets didn't have the + * same timestamp, which is a sign that they were packets from two + * different frames, but we didn't get the flag RTP_FLAG_MARKER on + * the first one of these frames (last packet of a frame). + * Finalize the previous frame anyway by filling the AVPacket. + */ + av_log(ctx, AV_LOG_ERROR, "Missed previous RTP Marker\n"); + missed_last_packet = 1; + raw_finalize_packet(data, pkt, st->index); + } + + data->timestamp = *timestamp; + + data->frame = av_malloc(data->frame_size); + if (!data->frame) { + av_log(ctx, AV_LOG_ERROR, "Out of memory.\n"); + return AVERROR(ENOMEM); + } + } + + /* + * looks for the 'Continuation bit' in scan lines' headers + * to find where data start + */ + do { + if (payload_len < 6) + return AVERROR_INVALIDDATA; + + cont = payload[4] & 0x80; + payload += 6; + payload_len -= 6; + } while (cont); + + /* and now iterate over every scan lines */ + do { + if (payload_len < data->pgroup) + return AVERROR_INVALIDDATA; + + length = (headers[0] << 8) | headers[1]; + line = ((headers[2] & 0x7f) << 8) | headers[3]; + offset = ((headers[4] & 0x7f) << 8) | headers[5]; + cont = headers[4] & 0x80; + headers += 6; + + if (length % data->pgroup) + return AVERROR_INVALIDDATA; + + if (length > payload_len) + length = payload_len; + + if (data->depth == 8) { + uint8_t *dest = data->frame; + + dest += (line * data->width + offset) * data->pgroup / data->xinc; + memcpy(dest, payload, length); + } + else if (data->depth == 10) { + uint16_t *y, *u, *v; + GetBitContext bc; + int i, ret; + + uint8_t* plans[4]; + int linesizes[4]; + + av_image_fill_arrays(plans, linesizes, data->frame, data->pixfmt, + data->width, data->height, 2); + y = (uint16_t*)(plans[0] + (line * data->width + offset) * 2); + u = (uint16_t*)(plans[1] + line * data->width + offset); + v = (uint16_t*)(plans[2] + line * data->width + offset); + + ret = init_get_bits(&bc, payload, length * 8); + if (ret) + return ret; + + for (i = 0; i < length / data->pgroup; i++) { + *u++ = get_bits(&bc, 10); + *y++ = get_bits(&bc, 10); + *v++ = get_bits(&bc, 10); + *y++ = get_bits(&bc, 10); + } + } + + payload += length; + payload_len -= length; + } while (cont); + + if ((flags & RTP_FLAG_MARKER)) { + return raw_finalize_packet(data, pkt, st->index); + } else if (missed_last_packet) { + return 0; + } + + return AVERROR(EAGAIN); +} + +RTPDynamicProtocolHandler ff_rawvideo_rtp_handler = { + .enc_name = "raw", + .codec_type = AVMEDIA_TYPE_VIDEO, + .codec_id = AV_CODEC_ID_RAWVIDEO, + .priv_data_size = sizeof(PayloadContext), + .parse_sdp_a_line = raw_parse_sdp_line, + .parse_packet = raw_handle_packet, +}; -- 2.11.1