From patchwork Tue Sep 19 07:40:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Riedl X-Patchwork-Id: 43773 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a886:b0:149:dfde:5c0a with SMTP id ca6csp324668pzb; Tue, 19 Sep 2023 00:41:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG5IKaAv/xAoNgIdNJgMlcGW8sx3jJInroc5T/eIOgcrIndXnwwt0OzfNVi+4yTODH/JIV9 X-Received: by 2002:a05:6402:17d0:b0:531:9c1:8262 with SMTP id s16-20020a05640217d000b0053109c18262mr4750599edy.8.1695109282560; Tue, 19 Sep 2023 00:41:22 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f18-20020a056402151200b0052a47fcbbfcsi9466733edw.585.2023.09.19.00.41.20; Tue, 19 Sep 2023 00:41:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@nativewaves.onmicrosoft.com header.s=selector2-nativewaves-onmicrosoft-com header.b=dqb9dl4b; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 322B068C852; Tue, 19 Sep 2023 10:41:18 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on2053.outbound.protection.outlook.com [40.107.14.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7D6EE68C60A for ; Tue, 19 Sep 2023 10:41:11 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cPIg6C/M9CGHYKs5lcB83X0O+E9LGY/gcTgFPAAxnvc/QdYC2Bn52lvHnaLyBTYR8vSc1i6fBemJLclnUc2ygD74Sc0SJ/4kJywEdZ+3B6FARm99aUWgah7k4BeUgvo+bI7UvpjiWwsPmv6OhQUdhabWN3jBGnJqsw/7idEIzO4qqjCN9+poNrc+Oxi+OpM83J9UdSGKuNrei/s2N0KGKZHMySWTUUwH7HGgy+ugDEEuobM0QEHrJuiKRT3bTU9NNotu2ESCMadPBzaBauzfSeY/8PJJ2oURpxrIWQFREulN0a9BQ0j6ToxiSW3HtCcEOuOrhnowaEM+ZEPYNzGm1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=By25BGsFquVJ0j19fypJA2RAWqcmMJpC3rxQjuOMfLE=; b=KzT5IC8FCtPcIY6t2/oF5PZiFCVYyWvqIqGf/g2LUVdSp4Vr3lMsqqAWC8f26PlJZnhcv4iuQQqoMmzL89t7/kiTSfxJnreOJjshAdJEyqHYqlbEymu5w3+eRFkjcEIQdGQlW2tq8P+PZBjYHhA3rHfVI/7F+je59ZNTx1Q/1iyC6f+fJBGPqIePu6vg5vneREZJ+hdnMUZjVeaZEKlEOuhr2B0SsxuD/SrVeQJQLsh51YA+EIViWHx8I1eALwaad6bZ71mbFGkIMAYGT+rM0rIizRvavhi/NA4J4pR/D2nsbJZWA0rAUbft5cm9DQqPcZ9TEh6/fxlU4CYKX6qfUw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nativewaves.com; dmarc=pass action=none header.from=nativewaves.com; dkim=pass header.d=nativewaves.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nativewaves.onmicrosoft.com; s=selector2-nativewaves-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=By25BGsFquVJ0j19fypJA2RAWqcmMJpC3rxQjuOMfLE=; b=dqb9dl4bAu5BwyvihcYSGbUQ3avn99iw0htGCmVbgJQQgJAPCvjgdJz3xlviJRG9k4wu4wR51RN9JP+1auDiBIK0CL7oiJr2uLoU9xaoC506NxsW7qv0SbVjFS3i+VWysWegH8Muk2A/1xYtwsZzKyjZhUrQMBVQu4x+cWFCWzs= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nativewaves.com; Received: from PA4PR03MB7167.eurprd03.prod.outlook.com (2603:10a6:102:10a::8) by DBAPR03MB6629.eurprd03.prod.outlook.com (2603:10a6:10:199::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.26; Tue, 19 Sep 2023 07:40:53 +0000 Received: from PA4PR03MB7167.eurprd03.prod.outlook.com ([fe80::d525:3ec1:f93a:9876]) by PA4PR03MB7167.eurprd03.prod.outlook.com ([fe80::d525:3ec1:f93a:9876%3]) with mapi id 15.20.6792.026; Tue, 19 Sep 2023 07:40:53 +0000 Message-ID: <4f742614-2375-4b5f-8d03-5f86ccf9c4df@nativewaves.com> Date: Tue, 19 Sep 2023 09:40:53 +0200 User-Agent: Mozilla Thunderbird From: Michael Riedl To: ffmpeg-devel@ffmpeg.org Content-Language: en-US X-ClientProxiedBy: FR2P281CA0079.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:9a::19) To PA4PR03MB7167.eurprd03.prod.outlook.com (2603:10a6:102:10a::8) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PA4PR03MB7167:EE_|DBAPR03MB6629:EE_ X-MS-Office365-Filtering-Correlation-Id: 4095eb25-9301-4a4a-94f7-08dbb8e3c104 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VauE8fzLL11E5WnQv1Ulr5uquAMasdRVF3yue653hrRZfRkzuTFHhgqnIhkQq3dZivN53h6zkALclSpjCNeiYtKVzeKyL8QNKQEjLf7Ve2k5LyKn3SlNec14GqXd5xMFkaNtbVCJU2A7XQWsZSrvTbhk5XXXOoSOi0plD10vGj+AzQNUXcKNxdX92oz6zgJ1M8BHAy2b/wBvizq8b444mQcEYAQ37sUz0kbN1wYx0/eQlRnge8G7TdRivWtOLBqtjdEt53nMh2yqNltgNYb31MR59/YQJzkhNMqud9U1irJuhlXoPETBuT8GxdjGe8GCs7iqLVyaJOjVcu9J+umVJ0m+YSdDktnG1G2KwhbzATYrqKZRUnsAe/26BOCbU0Hp/TJHaij0sb/O6krRNbhgqRPHPLWkVBgSaHY+ZZJ7MNeJ3OfMkHwjCojVnqgr165SJScMpNwJujtVC1gY737hOqkSoEz0V4XiRkX2Eiwf2GhcIF7U/oPsl4CNAtwnD4iSs6gGUaT1E7i6DW6UgMOaA4+xbTkmD/v7bVgSh/uJXZv3g8KqltXvVYQAtGQxW2EDOQMthbXZWZ+7GXwJ9TVJ2Zwo9rNvZSerBNzMCfkLehOHGBbehz7gMGz5NMPEUhnEs1DWYq7Bc1tGHsk5yar/HO15lsuaofd+BFoW8Oru+pt53CDXUZ6nF+d5hPMHTzX2hE44JaA1mBl1Ff+aD5cieg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PA4PR03MB7167.eurprd03.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(136003)(366004)(376002)(396003)(39830400003)(346002)(451199024)(186009)(1800799009)(31686004)(66899024)(478600001)(8676002)(6506007)(6486002)(2906002)(6512007)(316002)(2616005)(36756003)(41300700001)(86362001)(38100700002)(30864003)(26005)(31696002)(8936002)(5660300002)(66946007)(83380400001)(44832011)(66476007)(6916009)(66556008)(2004002)(43740500002)(45980500001)(357404004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?x29XtRWWt3hHneQHHj83kXusyQ9t?= =?utf-8?q?WYYKzYcWa1N7SRKulIRUz6BJ1kQWp+n/0ZzJCHN0U1a1rxywMuGy1rOY9nDtXy6zU?= =?utf-8?q?5KUswAJzbqiKlh4Lz2uzZgjCo6gJWSmZO/cG3/7OSdF2VqOgB+z6I1vSgUSAJxBfW?= =?utf-8?q?R5bUYm94JtiGr0jUFsjK2ZCqeKR9HRAfvlWaEjHYZWgkVkEP04kEbw3wrES+X87gs?= =?utf-8?q?9Vm2hZshUup8meDxziBvvbxmgWVKgSIMhpZStSEA0JQ6ddxEJ9g//RXSsclo5H8z3?= =?utf-8?q?VongHuRLbwvUJ1nILziSDdx2P4c8O1p8XWaNqafCJcpZWsXSWEoasqW1uvx4S55VI?= =?utf-8?q?VfREtfUZcTunZ/9n4rYy3g5+tmAqGsoYxlM1s0vkgry/UH5QBRsgLsYRwpxG/O4j6?= =?utf-8?q?d1GZYbLLLwuVHgv1NVAoCHu/9chu18n2kjmmoFComXlMGGtjuyQcJrALnjeUHnljD?= =?utf-8?q?qIjDl7TeQtT9cPVyE1LriCmAAjGCMkllJDcpHAzyIlbFYgrR5Eh9nVJ5woeJ117q7?= =?utf-8?q?krcstwgX46Q2flLATZ0UKzbbfUJwuV1D1KDmYrbzYJqEX/ibCklf5mfqMUmxl44NI?= =?utf-8?q?+gstsMxGDFw2b3GIIIaPMgxt/UiW7c5c3QtciaQniOYN/+Zqmk1E4GwVG6FMmXoqi?= =?utf-8?q?M8ZiyEn/It/fM1Z4UFj61/PcJy3jh5LC/pZEvOPLqKyQOER3bm6cflxJAfzlesy8g?= =?utf-8?q?50uRDu1eXmlYBR3VajxzIaeim/FJoyM60RZs96twDBZeT/8DNsH3dlZAet8zFx7vq?= =?utf-8?q?1hdkMqtH1BIdaDZuN/w2kaq1okJ/tmy8g6lTdCwFZ2oaw/N6l1y61ZGakF1c9OeI+?= =?utf-8?q?nMnwe09Or8b1LkswLGoCDD+F+l+McAeFgDO97ROmEedO+odoDFvCglXquWsLWUUJS?= =?utf-8?q?VacAFr5l6ZZ7UCy2Qo0Tsy4BE/O33Q8/FNaTLpJWoyBz2/8mrWIO7SBbDVFyra0/g?= =?utf-8?q?qToeEAby97qx6WI0zm1vEH5biLGG5X79HRdthKKoIEfY6p3wBZFoswmTl3JvrmGft?= =?utf-8?q?PY4A1WGN4pLowdBJIIoiOYX+kRAtjbO7Ugw/vjpaK8Hws3KMBDJBAdQOQOsBzwtrD?= =?utf-8?q?0iQBw7VmuHSR84+0OMPMJB8J2UYgZJmZp43QbsElKK+OOTsdFneL1an3Uj1J18nK3?= =?utf-8?q?85zUYplXFaEod12h3knN6sMmhkYWClYVF6HcXXa7weVDIo7vr97Yzog2tELIUAiZj?= =?utf-8?q?miq4ug+pm6oMU25whZ7wWvhNIDN05GG0Qe0PSG0pLgqOk9ecAtrUhpqmKrZNYcbr8?= =?utf-8?q?nZ+VmBd9ANZOr1jIuqRe/e6UXFHoNxEK78B6SBCi12pHrmQ/+EKUhO/PmjUutfi9i?= =?utf-8?q?77DvF9ly2TcM3afOF3/E0Oo2oFcrX1eOMSqvUUEE6z3xlSFDnwGivn1js8K1fCU+t?= =?utf-8?q?e3CbRR4a+MCJ6r/k7qRsY8HmbHcpyp9if0yWqhTR3OzHWzml+BrCxvBNpgyFp8wf0?= =?utf-8?q?mJ1dpKwlxIUCxZDhiCexDzlaOI2mUh/wQ7TsGxMm/UwTTVjOE3Vdc1LFIvDbeqNAr?= =?utf-8?q?2b0WW07MDqc2/6GDkxRkdtYOF7wef/X8ERKF5h+yINjJsIoDrPUqfU0=3D?= X-OriginatorOrg: nativewaves.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4095eb25-9301-4a4a-94f7-08dbb8e3c104 X-MS-Exchange-CrossTenant-AuthSource: PA4PR03MB7167.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Sep 2023 07:40:53.9167 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4239718-b000-4513-8314-02ef46bd0276 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CwIX8oR1ZOPSBqX3/jCVX+BtVuL65DqI/43pGe/k72qyFqBGkPr9F1Oh7JWVmq2I3VDKeR8L8wUauY3Gg/R4wBfT0MY1kltBqlWrPAeNPic= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR03MB6629 Subject: [FFmpeg-devel] [PATCH 2/2] libavformat/whip: add WebRTC-HTTP ingestion protocol (WHIP) X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dCHjcr0omeEx This is based on the library libdatachannel, which is much more lightweight than other libraries like libwebrtc. At the same time, using this library avoids reimplementing parts of WebRTC in FFmpeg. Signed-off-by: Michael Riedl --- configure | 5 + libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/whip.c | 521 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 528 insertions(+) create mode 100644 libavformat/whip.c +{ + switch (codec_id) + { + case AV_CODEC_ID_OPUS: + case AV_CODEC_ID_AAC: + case AV_CODEC_ID_H264: + case AV_CODEC_ID_HEVC: + return 1; + default: + return 0; + } +} + +#define OFFSET(x) offsetof(WHIPContext, x) +#define ENC AV_OPT_FLAG_ENCODING_PARAM +static const AVOption options[] = { + { "bearer_token", "optional Bearer token for authentication and authorization", OFFSET(bearer_token), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC }, + { "max_stored_packets_count", "maximum number of stored packets for retransmission", OFFSET(max_stored_packets_count), AV_OPT_TYPE_INT, { .i64 = 100 }, 0, INT_MAX, ENC }, + { "connection_timeout", "timeout in seconds for establishing a connection", OFFSET(connection_timeout), AV_OPT_TYPE_DURATION, { .i64 = 10000000 }, 100000, INT_MAX, ENC }, + { NULL }, +}; + +static const AVClass whip_muxer_class = { + .class_name = "WHIP muxer", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFOutputFormat ff_whip_muxer = { + .p.name = "whip", + .p.long_name = NULL_IF_CONFIG_SMALL("WebRTC-HTTP ingestion protocol (WHIP) muxer"), + .p.audio_codec = AV_CODEC_ID_OPUS, // supported by major browsers + .p.video_codec = AV_CODEC_ID_H264, + .p.flags = AVFMT_GLOBALHEADER | AVFMT_NOFILE, + .p.priv_class = &whip_muxer_class, + .priv_data_size = sizeof(WHIPContext), + .write_packet = whip_write_packet, + .write_header = whip_write_header, + .write_trailer = whip_write_trailer, + .init = whip_init, + .deinit = whip_deinit, + .query_codec = whip_query_codec, +}; diff --git a/configure b/configure index 48fee07f817..3bb68d3f20c 100755 --- a/configure +++ b/configure @@ -227,6 +227,7 @@ External library support: --enable-libcelt enable CELT decoding via libcelt [no] --enable-libcdio enable audio CD grabbing with libcdio [no] --enable-libcodec2 enable codec2 en/decoding using libcodec2 [no] + --enable-libdatachannel enable WHIP muxing via libdatachannel [no] --enable-libdav1d enable AV1 decoding via libdav1d [no] --enable-libdavs2 enable AVS2 decoding via libdavs2 [no] --enable-libdc1394 enable IIDC-1394 grabbing using libdc1394 @@ -1853,6 +1854,7 @@ EXTERNAL_LIBRARY_LIST=" libcaca libcelt libcodec2 + libdatachannel libdav1d libdc1394 libdrm @@ -3566,6 +3568,8 @@ wav_demuxer_select="riffdec" wav_muxer_select="riffenc" webm_chunk_muxer_select="webm_muxer" webm_dash_manifest_demuxer_select="matroska_demuxer" +whip_muxer_deps="libdatachannel" +whip_muxer_select="http_protocol rtpenc_chain" wtv_demuxer_select="mpegts_demuxer riffdec" wtv_muxer_select="mpegts_muxer riffenc" xmv_demuxer_select="riffdec" @@ -6691,6 +6695,7 @@ enabled libcelt && require libcelt celt/celt.h celt_decode -lcelt0 && enabled libcaca && require_pkg_config libcaca caca caca.h caca_create_canvas enabled libcodec2 && require libcodec2 codec2/codec2.h codec2_create -lcodec2 enabled libdav1d && require_pkg_config libdav1d "dav1d >= 0.5.0" "dav1d/dav1d.h" dav1d_version +enabled libdatachannel && require libdatachannel rtc/rtc.h rtcPreload -ldatachannel enabled libdavs2 && require_pkg_config libdavs2 "davs2 >= 1.6.0" davs2.h davs2_decoder_open enabled libdc1394 && require_pkg_config libdc1394 libdc1394-2 dc1394/dc1394.h dc1394_new enabled libdrm && require_pkg_config libdrm libdrm xf86drm.h drmGetVersion diff --git a/libavformat/Makefile b/libavformat/Makefile index 329055ccfd9..db9d8ec5d47 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -621,6 +621,7 @@ OBJS-$(CONFIG_WEBM_CHUNK_MUXER) += webm_chunk.o OBJS-$(CONFIG_WEBP_MUXER) += webpenc.o OBJS-$(CONFIG_WEBVTT_DEMUXER) += webvttdec.o subtitles.o OBJS-$(CONFIG_WEBVTT_MUXER) += webvttenc.o +OBJS-$(CONFIG_WHIP_MUXER) += whip.o OBJS-$(CONFIG_WSAUD_DEMUXER) += westwood_aud.o OBJS-$(CONFIG_WSAUD_MUXER) += westwood_audenc.o OBJS-$(CONFIG_WSD_DEMUXER) += wsddec.o rawdec.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index d4b505a5a32..e8825a92b54 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -504,6 +504,7 @@ extern const FFOutputFormat ff_webm_chunk_muxer; extern const FFOutputFormat ff_webp_muxer; extern const AVInputFormat ff_webvtt_demuxer; extern const FFOutputFormat ff_webvtt_muxer; +extern const FFOutputFormat ff_whip_muxer; extern const AVInputFormat ff_wsaud_demuxer; extern const FFOutputFormat ff_wsaud_muxer; extern const AVInputFormat ff_wsd_demuxer; diff --git a/libavformat/whip.c b/libavformat/whip.c new file mode 100644 index 00000000000..0bf162c51a1 --- /dev/null +++ b/libavformat/whip.c @@ -0,0 +1,521 @@ +/* + * WebRTC-HTTP ingestion protocol (WHIP) muxer using libdatachannel + * + * Copyright (C) 2023 NativeWaves GmbH + * This work is supported by FFG project 47168763. + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software Foundation, + * Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "mux.h" +#include "version.h" +#include "url.h" +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "libavutil/log.h" +#include "libavutil/uuid.h" +#include "libavutil/random_seed.h" +#include "libavutil/time.h" +#include "rtp.h" +#include "rtc/rtc.h" + +typedef struct WHIPContext { + AVClass *av_class; + int peer_connection; + int* tracks; + rtcState state; + char* resource_location; + + char* bearer_token; + int max_stored_packets_count; + int64_t connection_timeout; +} WHIPContext; + +static inline const char* whip_get_state_name(rtcState state) +{ + switch (state) + { + case RTC_NEW: + return "RTC_NEW"; + case RTC_CONNECTING: + return "RTC_CONNECTING"; + case RTC_CONNECTED: + return "RTC_CONNECTED"; + case RTC_DISCONNECTED: + return "RTC_DISCONNECTED"; + case RTC_FAILED: + return "RTC_FAILED"; + case RTC_CLOSED: + return "RTC_CLOSED"; + default: + return "UNKNOWN"; + } +} + +static void whip_on_state_change(int pc, rtcState state, void* ptr) +{ + AVFormatContext* avctx = (AVFormatContext*)ptr; + WHIPContext* s = (WHIPContext*)avctx->priv_data; + + av_log(avctx, AV_LOG_VERBOSE, "Connection state changed from %s to %s\n", whip_get_state_name(s->state), whip_get_state_name(state)); + s->state = state; +} + +static void whip_rtc_log(rtcLogLevel rtcLevel, const char *message) +{ + int level = AV_LOG_INFO; + switch (rtcLevel) + { + case RTC_LOG_NONE: + level = AV_LOG_QUIET; + break; + case RTC_LOG_DEBUG: + level = AV_LOG_DEBUG; + break; + case RTC_LOG_VERBOSE: + case RTC_LOG_INFO: + level = AV_LOG_VERBOSE; + break; + case RTC_LOG_WARNING: + level = AV_LOG_WARNING; + break; + case RTC_LOG_ERROR: + level = AV_LOG_ERROR; + break; + case RTC_LOG_FATAL: + level = AV_LOG_FATAL; + break; + } + av_log(NULL, level, "[libdatachannel] %s\n", message); +} + +static void generate_random_uuid(char buffer[37]) +{ + AVUUID uuid; + av_random_bytes(uuid, sizeof(uuid)); + av_uuid_unparse(uuid, buffer); +} + +static void whip_deinit(AVFormatContext* avctx); +static int whip_init(AVFormatContext* avctx) +{ + WHIPContext* s = avctx->priv_data; + rtcConfiguration config; + const AVStream* stream; + const AVCodecParameters* codecpar; + int i, ret; + char media_stream_id[37]; + rtcTrackInit track_init; + rtcPacketizationHandlerInit packetizer_init; + uint32_t ssrc; + uint8_t payload_type; + const uint32_t clock_rate = AV_TIME_BASE; + + memset(&config, 0, sizeof(rtcConfiguration)); + + rtcInitLogger(RTC_LOG_DEBUG, whip_rtc_log); + + if (!(s->peer_connection = rtcCreatePeerConnection(&config))) { + av_log(avctx, AV_LOG_ERROR, "Failed to create PeerConnection\n"); + return AVERROR(EINVAL); + } + rtcSetUserPointer(s->peer_connection, avctx); + if (rtcSetStateChangeCallback(s->peer_connection, whip_on_state_change)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set state change callback\n"); + ret = AVERROR(EINVAL); + goto fail; + } + + /* configure tracks */ + generate_random_uuid(media_stream_id); + if (!(s->tracks = av_malloc(sizeof(int) * avctx->nb_streams))) { + av_log(avctx, AV_LOG_ERROR, "Failed to allocate tracks\n"); + ret = AVERROR(ENOMEM); + goto fail; + } + + for (i = 0; i < avctx->nb_streams; ++i) { + stream = avctx->streams[i]; + codecpar = stream->codecpar; + + ssrc = av_get_random_seed(); + payload_type = (uint8_t)ff_rtp_get_payload_type(NULL, codecpar, i); + av_log(avctx, AV_LOG_VERBOSE, "ssrc: %u, payload_type: %u, clock_rate: %u\n", ssrc, payload_type, clock_rate); + + if (codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { + rtcCodec codec; + switch (codecpar->codec_id) { + case AV_CODEC_ID_OPUS: + codec = RTC_CODEC_OPUS; + break; + case AV_CODEC_ID_AAC: + codec = RTC_CODEC_AAC; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unsupported audio codec\n"); + ret = AVERROR(EINVAL); + goto fail; + } + + memset(&track_init, 0, sizeof(rtcTrackInit)); + track_init.direction = RTC_DIRECTION_SENDONLY; + track_init.codec = codec; + track_init.payloadType = payload_type; + track_init.ssrc = ssrc; + track_init.mid = av_asprintf("%d", i); + track_init.name = LIBAVFORMAT_IDENT; + track_init.msid = media_stream_id; + track_init.trackId = av_asprintf("%s-audio-%d", media_stream_id, i); + + memset(&packetizer_init, 0, sizeof(rtcPacketizationHandlerInit)); + packetizer_init.ssrc = ssrc; + packetizer_init.cname = LIBAVFORMAT_IDENT; + packetizer_init.payloadType = payload_type; + packetizer_init.clockRate = clock_rate; + + s->tracks[i] = rtcAddTrackEx(s->peer_connection, &track_init); + if (!s->tracks[i]) { + av_log(avctx, AV_LOG_ERROR, "Failed to add track\n"); + ret = AVERROR(EINVAL); + goto fail; + } + if (codec == RTC_CODEC_OPUS) { + if (rtcSetOpusPacketizationHandler(s->tracks[i], &packetizer_init)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set Opus packetization handler\n"); + ret = AVERROR(EIO); + goto fail; + } + } + else if (codec == RTC_CODEC_AAC) { + if (rtcSetAACPacketizationHandler(s->tracks[i], &packetizer_init)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set AAC packetization handler\n"); + ret = AVERROR(EIO); + goto fail; + } + } + if (rtcChainRtcpSrReporter(s->tracks[i])) { + av_log(avctx, AV_LOG_ERROR, "Failed to chain RTCP SR reporter\n"); + ret = AVERROR(EIO); + goto fail; + } + if (rtcChainRtcpNackResponder(s->tracks[i], s->max_stored_packets_count)) { + av_log(avctx, AV_LOG_ERROR, "Failed to chain RTCP NACK responder\n"); + ret = AVERROR(EIO); + goto fail; + } + } + else if (codecpar->codec_type == AVMEDIA_TYPE_VIDEO) + { + rtcCodec codec; + switch (codecpar->codec_id) + { + case AV_CODEC_ID_H264: + codec = RTC_CODEC_H264; + break; + case AV_CODEC_ID_HEVC: + codec = RTC_CODEC_H265; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unsupported video codec\n"); + ret = AVERROR(EINVAL); + goto fail; + } + + memset(&track_init, 0, sizeof(rtcTrackInit)); + track_init.direction = RTC_DIRECTION_SENDONLY; + track_init.codec = codec; + track_init.payloadType = payload_type; + track_init.ssrc = ssrc; + track_init.mid = av_asprintf("%d", i); + track_init.name = LIBAVFORMAT_IDENT; + track_init.msid = media_stream_id; + track_init.trackId = av_asprintf("%s-video-%d", media_stream_id, i); + + memset(&packetizer_init, 0, sizeof(rtcPacketizationHandlerInit)); + packetizer_init.ssrc = ssrc; + packetizer_init.cname = LIBAVFORMAT_IDENT; + packetizer_init.payloadType = payload_type; + packetizer_init.clockRate = clock_rate; + packetizer_init.nalSeparator = RTC_NAL_SEPARATOR_START_SEQUENCE; + + if (!(s->tracks[i] = rtcAddTrackEx(s->peer_connection, &track_init))) { + av_log(avctx, AV_LOG_ERROR, "Failed to add track\n"); + ret = AVERROR(EIO); + goto fail; + } + if (codec == RTC_CODEC_H264) { + if (rtcSetH264PacketizationHandler(s->tracks[i], &packetizer_init)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set H264 packetization handler\n"); + ret = AVERROR(EIO); + goto fail; + } + } + else if (codec == RTC_CODEC_H265) { + if (rtcSetH265PacketizationHandler(s->tracks[i], &packetizer_init)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set H265 packetization handler\n"); + ret = AVERROR(EIO); + goto fail; + } + } + if (rtcChainRtcpSrReporter(s->tracks[i])) { + av_log(avctx, AV_LOG_ERROR, "Failed to chain RTCP SR reporter\n"); + ret = AVERROR(EIO); + goto fail; + } + if (rtcChainRtcpNackResponder(s->tracks[i], s->max_stored_packets_count)) { + av_log(avctx, AV_LOG_ERROR, "Failed to chain RTCP NACK responder\n"); + ret = AVERROR(EIO); + goto fail; + } + } + } + + if (rtcSetLocalDescription(s->peer_connection, "offer")) { + av_log(avctx, AV_LOG_ERROR, "Failed to set local description\n"); + ret = AVERROR(EIO); + goto fail; + } + + return 0; + +fail: + whip_deinit(avctx); + return ret; +} + +static int whip_write_header(AVFormatContext* avctx) +{ + WHIPContext* s = avctx->priv_data; + int ret; + URLContext* h = NULL; + int64_t timeout; + char* headers; + char offer_sdp[4096] = {0}; + char response[4096] = {0}; + + if (rtcGetLocalDescription(s->peer_connection, offer_sdp, sizeof(offer_sdp)) < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to get local description\n"); + ret = AVERROR(EIO); + goto fail; + } + av_log(avctx, AV_LOG_VERBOSE, "offer_sdp: %s\n", offer_sdp); + + /* alloc the http context */ + if ((ret = ffurl_alloc(&h, avctx->url, AVIO_FLAG_READ_WRITE, NULL)) < 0) { + av_log(avctx, AV_LOG_ERROR, "ffurl_alloc failed\n"); + goto fail; + } + /* set options */ + headers = av_asprintf("Content-type: application/sdp\r\n"); + if (s->bearer_token) { + headers = av_asprintf("%sAuthorization: Bearer %s\r\n", headers, s->bearer_token); + } + av_log(avctx, AV_LOG_VERBOSE, "headers: %s\n", headers); + av_opt_set(h->priv_data, "headers", headers, 0); + av_opt_set(h->priv_data, "method", "POST", 0); + av_opt_set_bin(h->priv_data, "post_data", (uint8_t*)offer_sdp, strlen(offer_sdp), 0); + + /* open the http context */ + if ((ret = ffurl_connect(h, NULL)) < 0) { + av_log(avctx, AV_LOG_ERROR, "ffurl_connect failed\n"); + goto fail; + } + + /* read the server reply which contains a unique ID */ + ret = ffurl_read_complete(h, (unsigned char*)response, sizeof(response)); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "ffurl_read_complete failed\n"); + goto fail; + } + + av_log(avctx, AV_LOG_VERBOSE, "response: %s\n", response); + if (rtcSetRemoteDescription(s->peer_connection, response, "answer")) { + av_log(avctx, AV_LOG_ERROR, "Failed to set remote description\n"); + ret = AVERROR(EIO); + goto fail; + } + + /* save resource location for later use */ + av_opt_get(h->priv_data, "new_location", AV_OPT_SEARCH_CHILDREN, (uint8_t**)&s->resource_location); + av_log(avctx, AV_LOG_VERBOSE, "resource_location: %s\n", s->resource_location); + + /* close the http context */ + if ((ret = ffurl_closep(&h)) < 0) { + av_log(avctx, AV_LOG_ERROR, "ffurl_closep &failed\n"); + goto fail; + } + + /* wait for connection to be established */ + timeout = av_gettime_relative() + s->connection_timeout; + while (s->state != RTC_CONNECTED) { + if (s->state == RTC_FAILED || s->state == RTC_CLOSED || av_gettime_relative() > timeout) { + av_log(avctx, AV_LOG_ERROR, "Failed to open connection\n"); + ret = AVERROR(EIO); + goto fail; + } + + av_log(avctx, AV_LOG_VERBOSE, "Waiting for PeerConnection to open\n"); + av_usleep(100000); + } + + return 0; + +fail: + if (h) { + ffurl_closep(&h); + } + whip_deinit(avctx); + return ret; +} + +static int whip_write_packet(AVFormatContext* avctx, AVPacket* pkt) +{ + const WHIPContext* s = avctx->priv_data; + int64_t timestamp; + + if (s->state == RTC_DISCONNECTED || s->state == RTC_FAILED || s->state == RTC_CLOSED) { + return AVERROR_EOF; + } + + if (pkt->pts < 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid packet PTS, dropping packet\n"); + return AVERROR(EINVAL); + } + + timestamp = av_rescale_q(pkt->pts, avctx->streams[pkt->stream_index]->time_base, AV_TIME_BASE_Q); + if (rtcSetTrackRtpTimestamp(s->tracks[pkt->stream_index], (uint32_t)timestamp)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set track RTP timestamp\n"); + return AVERROR(EINVAL); + } + + if (rtcSendMessage(s->tracks[pkt->stream_index], (const char*)pkt->data, pkt->size)) { + av_log(avctx, AV_LOG_ERROR, "Failed to send message\n"); + return AVERROR(EINVAL); + } + + return 0; +} + +static int whip_write_trailer(AVFormatContext* avctx) +{ + WHIPContext* s = avctx->priv_data; + URLContext* h = NULL; + int ret; + char* headers; + + if (s->resource_location) { + av_log(avctx, AV_LOG_VERBOSE, "Closing resource %s\n", s->resource_location); + + /* alloc the http context */ + if ((ret = ffurl_alloc(&h, s->resource_location, AVIO_FLAG_READ_WRITE, NULL)) < 0) { + av_log(avctx, AV_LOG_ERROR, "ffurl_alloc failed\n"); + goto fail; + } + + /* set options */ + if (s->bearer_token) { + headers = av_asprintf("Authorization: Bearer %s\r\n", s->bearer_token); + av_log(avctx, AV_LOG_VERBOSE, "headers: %s\n", headers); + } + av_opt_set(h->priv_data, "method", "DELETE", 0); + + /* open the http context */ + if ((ret = ffurl_connect(h, NULL)) < 0) { + av_log(avctx, AV_LOG_ERROR, "ffurl_connect failed\n"); + goto fail; + } + + /* close the http context */ + if ((ret = ffurl_closep(&h)) < 0) { + av_log(avctx, AV_LOG_ERROR, "ffurl_close failed\n"); + goto fail; + } + + av_freep(&s->resource_location); + } + + return 0; + +fail: + if (h) { + ffurl_closep(&h); + } + return ret; +} + +static void whip_deinit(AVFormatContext* avctx) +{ + WHIPContext* s = avctx->priv_data; + if (s->tracks) { + for (int i = 0; i < avctx->nb_streams; ++i) { + if (s->tracks[i]) { + rtcDeleteTrack(s->tracks[i]); + } + } + av_freep(&s->tracks); + } + if (s->peer_connection) { + rtcDeletePeerConnection(s->peer_connection); + s->peer_connection = 0; + } +} + +static int whip_query_codec(enum AVCodecID codec_id, int std_compliance)