From patchwork Tue Oct 31 18:53:09 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Breeden, Joshua" X-Patchwork-Id: 5793 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.90 with SMTP id m26csp4352791jah; Tue, 31 Oct 2017 11:53:19 -0700 (PDT) X-Google-Smtp-Source: ABhQp+SFPtNGG/oPT5JjRcYAR6nqGwa4ew3yfT5/Ux9vaPFq5IH+5z2AuwSYcSYS2wH/Ej5kQoFI X-Received: by 10.28.193.136 with SMTP id r130mr1767783wmf.141.1509475999859; Tue, 31 Oct 2017 11:53:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1509475999; cv=none; d=google.com; s=arc-20160816; b=f2FnBu3P5o5Uzkohn0Ghf2nhsRxt3ed8zV4MNuEHCspm4k2cICHKaXg3tbWSwBk1hE sI44oIlZHm33ODT7hEcc07GCtnMZWY6LvQ/zMSGmPKMJh16tRen+QYjHt04Nxz6X1KyX pq6hEqkY1vy4npdwDHEqic8md0hWR+/+rEaw+j84wIbqrSkdQP+fJALU14AAfPQtspSa +tQT88rn5lLeY1woAGxlH1jpMciuUOJ5Qn3Fdh+XM6q0QyWjGXO7R0GjiWxBUVnTQHjE nW+n4FdGBXYe1+yfR1nyIqxAz967S1jIXE4Y/Fh8L3Tztw/uYkMwEhrWlQRudXgJIsKc TT0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:content-language:accept-language:message-id:date :thread-index:thread-topic:to:from:dkim-signature:delivered-to :arc-authentication-results; bh=zc+HBKf9apiDPPpm4+sVv+1PP7hIYt3jJwoPBNTigEk=; b=dxb6ED9GaFWBrD0YIHTAaaUYJ8U57MIUR35Rrm3g4ppNNYcxX+y1qp1yECs6ITMMP7 fln8DQp5ghlOkbykqlUlVvJXjSX3cZPTxZrITsZ0rEdmvHYof0m79VTTCBTFNhEYnBjy Ntp6NapWuuhI+rGLXeqanaKs8d8cMfOTHusk7wZE9NEorrSDX/7T5/N/JR/EMRqF3hHt 2mO96f3Px8ONNJz4FrkiEnTrUxqnmpbRI4kn+24AoRstnc/TCtbssNYClC2H0w3Rqg4m 6K/kyMc/+5Vv2Ivq4Zr7ObKpQrpJ76LL/OU8qDb8LZV4KIbSk6xS1ohY+APbphkQHK+e orlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@harris.com header.s=key1 header.b=Zx1tzOUC; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a77si2103677wma.225.2017.10.31.11.53.19; Tue, 31 Oct 2017 11:53:19 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@harris.com header.s=key1 header.b=Zx1tzOUC; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9BB61689B62; Tue, 31 Oct 2017 20:53:08 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mlbxsmtp02.harris.com (mlbxsmtpout02.harris.com [192.52.234.93]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D7615680780 for ; Tue, 31 Oct 2017 20:53:01 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; d=harris.com; s=key1; c=relaxed/simple; q=dns/txt; i=@harris.com; t=1509475990; h=From:Sender:Reply-To:Subject:Date:Message-ID:To:Cc:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=fo7vU4U3z6sj+yEF5A8DRIo9kKtXMMPD+q3Q03rPeWY=; b=Zx1tzOUC+qflttg80AGolRWNH7LAkP58kioEGaPonqNsxTLyJVY2LNzIfix+trfc kcR5+oJbkvxRCyxePmGB3+xK37J5OKR6lDl7UDAXPp7ZNvz2uLhEh4YEXHYy0QL5 VmJLeP9+Pe9EteVvtv3ij+rLUd5b0tMfubI0NcYnQZpmo8AYzIOdXLoJ0NCTbuIh aCxj2e68C5bqiy+G2XnWw0ViloKJozqcIa1A13dC5j2olIgEPoXG6ppALCPOtnRQ hJAOnT4cRvAfdmjMOm6kO9j9YS06sOktF4cM5eRESRDX3xmX8Fvqajt1Qhgk0Onv dnF/h0OI/uXVIzOboAE8rA==; X-AuditID: c034ea5c-6a7ff70000001488-ab-59f8c6967daa Received: from MLBXCH16.cs.myharris.net ( [10.64.224.134]) (using TLS with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client did not present a certificate) by mlbxsmtp02.harris.com (mail) with SMTP id 21.3F.05256.696C8F95; Tue, 31 Oct 2017 14:53:10 -0400 (EDT) Received: from MLBXCH15.cs.myharris.net (10.64.224.133) by MLBXCH16.cs.myharris.net (10.64.224.134) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Tue, 31 Oct 2017 14:53:09 -0400 Received: from MLBXCH15.cs.myharris.net ([10.64.224.133]) by MLBXCH15.cs.myharris.net ([10.64.224.133]) with mapi id 15.00.1347.000; Tue, 31 Oct 2017 14:53:09 -0400 From: "Breeden, Joshua" To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: Support for h264/SVC over RTP and SVC base layer decoding in h264 Thread-Index: AQHTUnkinI8mFmw0jE+8SLrq04m+0A== Date: Tue, 31 Oct 2017 18:53:09 +0000 Message-ID: <1509476048958.76834@harris.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.64.224.63] MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrBIsWRmVeSWpSXmKPExsXC5fCgTXfasR+RBrevG1l8+3SG2YHR48+i zSwBjFFcNimpOZllqUX6dglcGWsfb2Qu2FhXsX7FbOYGxutlXYycHBICJhKnP/xm7GLk4hAS 6GeSeHS3kRnC2cEo8fr7U1YIZwWjxNzNL9hAWtgEdCVWLznDCGKLCJhLrN33jRnEFhbwkth5 9xdUPFDi9bcOdghbT2Jn41WwXhYBVYm5j9aA2bwCOhJPttxgAbEZBcQkvp9awwRiMwuIS9x6 Mp8J4jwRiYcXT7NB2KISLx//Y4WwDSS2Lt3HAmErSJyY0AvV6y0xsfE1I8R8QYmTM5+wTGAU noVk7CwkZbOQlEHEgyWuzp4AVaMncWPqFDYIW1ti2cLXzBC2rsSMf4dYMMXjJLYcug/UywFk +0mc+50wCxh0zAI3GCV2Nt+DqleUmNL9kB2iJlFiT28URDhQYsLDNiaI+tuMEr0nzrMiq1/A KLCKUTQ3J6miOLekwMBILyOxqCizWC85P3cTIzANHDB5FbODceUr+0OMAhyMSjy8Czb8iBRi TSwrrsw9xKgCNOfRhtUXGKVY8vLzUpVEeHtXAKV5UxIrq1KL8uOLSnNSiw8xSnOwKInzOh4A SgmkJ5akZqemFqQWwWSZODilGhgPb7nwdcMunqjpqpUHuvrXJzw71XBrVl93yuP/Mv9vWj7w mfhZsDWTIXNjjO0dB3GO16u73Zr/23Z93yIxJb9vqZRzlN57v7SmxvK52btc/k6JuDyh2LPe 2ZX1f9WUVT9nvPmx7uBajmNPkk5dz1zN/eugA5f9T6WUsEttCsu/szwt/3t4lpejEktxRqKh FnNRcSIAP9+aqAsDAAA= X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: [FFmpeg-devel] Support for h264/SVC over RTP and SVC base layer decoding in h264 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hello, First patch submission... hopefully I do all of this correctly.? Added basic support for H264 Annex G (Scalable Video Coding) to RTP module. Implement splitting of PACSI NAL units (type 30). Used existing H264 SPS decoder to parse Subset SPS NAL units (type 15) in h264 extradata. As a result, FFmpeg can now parse SVC over RTP. The video? base layer, which is AVC compatible, can be decoded. Signed-off-by: Joshua Breeden Joshua Breeden Software Engineer-Developer SPACE AND INTELLIGENCE SYSTEMS / HARRIS CORPORATION Office: +1-757-483-0226 x1008 From a9b89e1568b164031506164c2efd5533be93a56e Mon Sep 17 00:00:00 2001 From: Joshua Breeden Date: Tue, 31 Oct 2017 13:26:02 -0400 Subject: [PATCH 2/2] Add support for non-interleaved h264/SVC over RTP --- libavformat/rtpdec.c | 1 + libavformat/rtpdec_formats.h | 1 + libavformat/rtpdec_h264.c | 80 +++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 81 insertions(+), 1 deletion(-) diff --git a/libavformat/rtpdec.c b/libavformat/rtpdec.c index 4acb1ca629..1c5814749a 100644 --- a/libavformat/rtpdec.c +++ b/libavformat/rtpdec.c @@ -96,6 +96,7 @@ void ff_register_rtp_dynamic_payload_handlers(void) ff_register_dynamic_payload_handler(&ff_h263_2000_dynamic_handler); ff_register_dynamic_payload_handler(&ff_h263_rfc2190_dynamic_handler); ff_register_dynamic_payload_handler(&ff_h264_dynamic_handler); + ff_register_dynamic_payload_handler(&ff_h264_svc_dynamic_handler); ff_register_dynamic_payload_handler(&ff_hevc_dynamic_handler); ff_register_dynamic_payload_handler(&ff_ilbc_dynamic_handler); ff_register_dynamic_payload_handler(&ff_jpeg_dynamic_handler); diff --git a/libavformat/rtpdec_formats.h b/libavformat/rtpdec_formats.h index a436c9d62c..b67d1e338f 100644 --- a/libavformat/rtpdec_formats.h +++ b/libavformat/rtpdec_formats.h @@ -64,6 +64,7 @@ extern RTPDynamicProtocolHandler ff_h263_1998_dynamic_handler; extern RTPDynamicProtocolHandler ff_h263_2000_dynamic_handler; extern RTPDynamicProtocolHandler ff_h263_rfc2190_dynamic_handler; extern RTPDynamicProtocolHandler ff_h264_dynamic_handler; +extern RTPDynamicProtocolHandler ff_h264_svc_dynamic_handler; extern RTPDynamicProtocolHandler ff_hevc_dynamic_handler; extern RTPDynamicProtocolHandler ff_ilbc_dynamic_handler; extern RTPDynamicProtocolHandler ff_jpeg_dynamic_handler; diff --git a/libavformat/rtpdec_h264.c b/libavformat/rtpdec_h264.c index 6f8148ab6d..4c3f52be05 100644 --- a/libavformat/rtpdec_h264.c +++ b/libavformat/rtpdec_h264.c @@ -308,6 +308,69 @@ static int h264_handle_packet_fu_a(AVFormatContext *ctx, PayloadContext *data, A return ff_h264_handle_frag_packet(pkt, buf, len, start_bit, &nal, 1); } +static int h264_handle_packet_pacsi(AVFormatContext *ctx, PayloadContext *data, AVPacket *pkt, + const uint8_t *buf, int len, + int *nal_counters, int nal_mask) +{ + uint8_t *dst = NULL; + const uint8_t *src = buf; + int src_len = len; + int total_length = 0; + int ret; + int y, t; + int opt_offset; + + if (len < 5) + return AVERROR_INVALIDDATA; + + // skip some NAL header info + src += 4; + src_len -= 4; + + // skip optional fields if present + y = (src[0] >> 6) & 1; + t = (src[0] >> 5) & 1; + // 3 bytes indicated by y, 2 bytes for t, and skip over current byte + opt_offset = y * 3 + t * 2 + 1; + src += opt_offset; + src_len -= opt_offset; + + // calculate size of NALs plus start codes + while (src_len > 2) { + uint16_t nal_size = AV_RB16(src); + + total_length += sizeof(start_sequence) + nal_size; + src += 2 + nal_size; + src_len -= 2 + nal_size; + } + + // create packet to hold NALs, reset buffer to first NAL + if ((ret = av_new_packet(pkt, total_length)) < 0) + return ret; + dst = pkt->data; + src = buf + 4 + opt_offset; + src_len = len - 4 - opt_offset; + + while (src_len > 2) { + uint16_t nal_size = AV_RB16(src); + src += 2; + src_len -= 2; + // copying + memcpy(dst, start_sequence, sizeof(start_sequence)); + dst += sizeof(start_sequence); + memcpy(dst, src, nal_size); + if (nal_counters) + nal_counters[(*src) & nal_mask]++; + dst += nal_size; + + // eat what we handled + src += nal_size; + src_len -= nal_size; + } + + return 0; +} + // return 0 on packet, no more left, 1 on packet, 1 on partial packet static int h264_handle_packet(AVFormatContext *ctx, PayloadContext *data, AVStream *st, AVPacket *pkt, uint32_t *timestamp, @@ -360,7 +423,11 @@ static int h264_handle_packet(AVFormatContext *ctx, PayloadContext *data, NAL_COUNTERS, NAL_MASK); break; - case 30: // undefined + case 30: // PACSI (Annex G - SVC) + result = h264_handle_packet_pacsi(ctx, data, pkt, buf, len, + NAL_COUNTERS, NAL_MASK); + break; + case 31: // undefined default: av_log(ctx, AV_LOG_ERROR, "Undefined type (%d)\n", type); @@ -418,3 +485,14 @@ RTPDynamicProtocolHandler ff_h264_dynamic_handler = { .close = h264_close_context, .parse_packet = h264_handle_packet, }; + +RTPDynamicProtocolHandler ff_h264_svc_dynamic_handler = { + .enc_name = "H264-SVC", + .codec_type = AVMEDIA_TYPE_VIDEO, + .codec_id = AV_CODEC_ID_H264, + .need_parsing = AVSTREAM_PARSE_FULL, + .priv_data_size = sizeof(PayloadContext), + .parse_sdp_a_line = parse_h264_sdp_line, + .close = h264_close_context, + .parse_packet = h264_handle_packet, +}; \ No newline at end of file -- 2.11.0