From patchwork Tue Jun 29 15:10:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 28728 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp3047348ios; Tue, 29 Jun 2021 08:32:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxpZ8c78F9xrcoIqtaVoyQT6sajBpXzLyr3QNswkWg7TnZ/6scn88hVQ8/KHAvB03Occ3u/ X-Received: by 2002:a50:c907:: with SMTP id o7mr40463739edh.120.1624980777265; Tue, 29 Jun 2021 08:32:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1624980777; cv=none; d=google.com; s=arc-20160816; b=DbYQUXLKeXPySak6haToW3hSbfSbEQD55ifJHOpRnv15y5VDkPDsQEMQwsYxVLmDMO IAUHJc4UOb+PuKuYnvE8z4aqIbRKPI4qycGxy5kiYIRwo9FAzQzzg04McSlh6chfhSox UXGaNy0/hnUPDA6IujG/1rG5E2BqHQNd1mSk8yKfYZ+TXofU8kafuUKPhHyUHNP58ekd rasMIyGa2oDnpWpkEv6Dc8PTzg252ujBEZdRtXBLFhPCqXDwp85YvxfAeMvv3+iKKyCb hU5yFrjbw8U+kHwrbtj/tTUwLm+nMuan5GePcd05aCjl3H1OMSy87/WZ/SmCeW2jka8h m3bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=QmgXQKHvl7L4OnOCN5M53jQtRUPCQc6KAKVIO079ang=; b=eFURssEqG3EQykXCEqg73UJufQlEigPGqINPAO51xE0Rd2gROna5yOfiQ4wXXlrO8M 47fzHCLmNKCwOfE5HbDj77SlU7tHiQae7NRl3TGs+LS65EDm2Btd0hvgfDkg96Pzi+cX slJ60qsL1QbR6Pt6Ow+gNAnkcTqwfzAnM9S4CXBcTV6sU8UXnKDsxhj1tBQkV3eyATcs Wmoocb9mSFdEEI+O333dsWhmupCfYYTC48XdXOoRf+ZELA8ou1mgwSO+wL3nOivaV9Cc CJFZ9a2Y89uSUoTyGi9vZkJBdXoZJiXrN3XkFDu0xriTRVwA404T4jg6ED0xVo7Lee7R ZMJg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=hlK1vdjo; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n5si5774866edv.524.2021.06.29.08.32.56; Tue, 29 Jun 2021 08:32:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=hlK1vdjo; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EF34068A2C6; Tue, 29 Jun 2021 18:32:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f179.google.com (mail-oi1-f179.google.com [209.85.167.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C6084680839 for ; Tue, 29 Jun 2021 18:32:46 +0300 (EEST) Received: by mail-oi1-f179.google.com with SMTP id a133so24489395oib.13 for ; Tue, 29 Jun 2021 08:32:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+hAmsB2gWERRasG0cA5bEWRPN/K0nJ+UM3q62xfa/ls=; b=hlK1vdjoSm8qNJ0vOF1Df35HI5/hoKta9ugKLO4+66KBbJcfNLEkI7SwvtCJFN7w8Q 2i/C3RFpeHT7ND6cIDNB0RpXUxka/QcaIDppztWPB/g5KUTMXZGgqAyKErkvbblmZVXN WRoUq4mz2Rt3ZjbOJbt9Qzt97rBnByxzBwZRxAb8ZFtpUw0uH/VKqYztHqAiJUEEsRsw /YCJRvcZaohObqSJ65k+kvHYdPGITulTBDSZQYWzlBSX6GcZINjy0VfdFw0ub5RaC6Ke 6K6wOVALqG2mGRtUMqTZl9Qj2bRtGmhcGb78anToqckpyWNEWflMi8fkLLvwZ9+8nXRM Oa+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+hAmsB2gWERRasG0cA5bEWRPN/K0nJ+UM3q62xfa/ls=; b=gZH3O3r9FKx1JiZtb1IY9voqsvK4sRG3nSMGkuoNK1LIaimNG7Dt1Y5tuvhVLm/N7v PCO7NgYFusIsubfT8Gfhmi8rKuYBS3OM/hrEmZ9XUpx0j/UrNiB1GiiRyWA5i06GDi7w AykAZb0Ph8BmHhtNw1ofYXptkEgyN6dGMkhuhhXf+unmfsuv4Zw1EyzsTUZuzceRvwL2 uiYsoWkmEcR6ebobz6p3yYkgySy2qp1CjcmjWWCZJDMX2jpY58I0WdZcIacHt3cabqIl V7FmkUBxxQbWSR+knDTe5F4ZR4hnSnf0gyh6a+yrqXuWOm0VR9hI4ZrbnWdz10SnhbRe OVKw== X-Gm-Message-State: AOAM5320f/9AoOh/5Pam7FIvQr95mTK7iYt5atOFwI+R6mBeo1d14axF ULFYD5pCGrmVT9NlOcAq6BGxaA+b/Xg= X-Received: by 2002:a17:90a:4fe3:: with SMTP id q90mr41942697pjh.113.1624979496617; Tue, 29 Jun 2021 08:11:36 -0700 (PDT) Received: from localhost.localdomain ([191.84.255.70]) by smtp.gmail.com with ESMTPSA id cx9sm6040242pjb.24.2021.06.29.08.11.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Jun 2021 08:11:35 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 29 Jun 2021 12:10:48 -0300 Message-Id: <20210629151048.1334-1-jamrial@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210628124848.6m6iw2xz5gtx7y7a@phare.normalesup.org> References: <20210628124848.6m6iw2xz5gtx7y7a@phare.normalesup.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3] avformat: add a concat protocol that takes a line break delimited list of resources X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 5S6dQjjSUPRp Suggested-by: ffmpeg@fb.com Signed-off-by: James Almer --- Now using the standard quoting and escaping syntax. doc/protocols.texi | 32 ++++++++++++ libavformat/Makefile | 1 + libavformat/concat.c | 109 ++++++++++++++++++++++++++++++++++++++++ libavformat/protocols.c | 1 + 4 files changed, 143 insertions(+) diff --git a/doc/protocols.texi b/doc/protocols.texi index ccdfb6e439..726e5f1c44 100644 --- a/doc/protocols.texi +++ b/doc/protocols.texi @@ -215,6 +215,38 @@ ffplay concat:split1.mpeg\|split2.mpeg\|split3.mpeg Note that you may need to escape the character "|" which is special for many shells. +@section concatf + +Physical concatenation protocol using a line break delimited list of +resources. + +Read and seek from many resources in sequence as if they were +a unique resource. + +A URL accepted by this protocol has the syntax: +@example +concatf:@var{URL} +@end example + +where @var{URL} is the url containing a line break delimited list of +resources to be concatenated, each one possibly specifying a distinct +protocol. Special characters must be escaped with backslash or single +quotes. See @ref{quoting_and_escaping,,the "Quoting and escaping" +section in the ffmpeg-utils(1) manual,ffmpeg-utils}. + +For example to read a sequence of files @file{split1.mpeg}, +@file{split2.mpeg}, @file{split3.mpeg} listed in separate lines within +a file @file{split.txt} with @command{ffplay} use the command: +@example +ffplay concatf:split.txt +@end example +Where @file{split.txt} contains the lines: +@example +split1.mpeg +split2.mpeg +split3.mpeg +@end example + @section crypto AES-encrypted stream reading protocol. diff --git a/libavformat/Makefile b/libavformat/Makefile index c9ef564523..caca95802a 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -616,6 +616,7 @@ OBJS-$(CONFIG_APPLEHTTP_PROTOCOL) += hlsproto.o OBJS-$(CONFIG_BLURAY_PROTOCOL) += bluray.o OBJS-$(CONFIG_CACHE_PROTOCOL) += cache.o OBJS-$(CONFIG_CONCAT_PROTOCOL) += concat.o +OBJS-$(CONFIG_CONCATF_PROTOCOL) += concat.o OBJS-$(CONFIG_CRYPTO_PROTOCOL) += crypto.o OBJS-$(CONFIG_DATA_PROTOCOL) += data_uri.o OBJS-$(CONFIG_FFRTMPCRYPT_PROTOCOL) += rtmpcrypt.o rtmpdigest.o rtmpdh.o diff --git a/libavformat/concat.c b/libavformat/concat.c index 278afd997d..aec1f52d8e 100644 --- a/libavformat/concat.c +++ b/libavformat/concat.c @@ -22,9 +22,11 @@ */ #include "libavutil/avstring.h" +#include "libavutil/bprint.h" #include "libavutil/mem.h" #include "avformat.h" +#include "avio_internal.h" #include "url.h" #define AV_CAT_SEPARATOR "|" @@ -56,6 +58,7 @@ static av_cold int concat_close(URLContext *h) return err < 0 ? -1 : 0; } +#if CONFIG_CONCAT_PROTOCOL static av_cold int concat_open(URLContext *h, const char *uri, int flags) { char *node_uri = NULL; @@ -124,6 +127,7 @@ static av_cold int concat_open(URLContext *h, const char *uri, int flags) data->total_size = total_size; return err; } +#endif static int concat_read(URLContext *h, unsigned char *buf, int size) { @@ -188,6 +192,7 @@ static int64_t concat_seek(URLContext *h, int64_t pos, int whence) return result; } +#if CONFIG_CONCAT_PROTOCOL const URLProtocol ff_concat_protocol = { .name = "concat", .url_open = concat_open, @@ -197,3 +202,107 @@ const URLProtocol ff_concat_protocol = { .priv_data_size = sizeof(struct concat_data), .default_whitelist = "concat,file,subfile", }; +#endif + +#if CONFIG_CONCATF_PROTOCOL +static av_cold int concatf_open(URLContext *h, const char *uri, int flags) +{ + AVBPrint bp; + struct concat_data *data = h->priv_data; + AVIOContext *in = NULL; + const char *cursor; + int64_t total_size = 0; + unsigned int nodes_size = 0; + size_t i = 0; + int err; + + if (!av_strstart(uri, "concatf:", &uri)) { + av_log(h, AV_LOG_ERROR, "URL %s lacks prefix\n", uri); + return AVERROR(EINVAL); + } + + /* handle input */ + if (!*uri) + return AVERROR(ENOENT); + + err = ffio_open_whitelist(&in, uri, AVIO_FLAG_READ, &h->interrupt_callback, + NULL, h->protocol_whitelist, h->protocol_blacklist); + if (err < 0) + return err; + + av_bprint_init(&bp, 0, AV_BPRINT_SIZE_UNLIMITED); + err = avio_read_to_bprint(in, &bp, SIZE_MAX); + avio_closep(&in); + if (err < 0) { + av_bprint_finalize(&bp, NULL); + return err; + } + + cursor = bp.str; + while (*cursor) { + struct concat_nodes *nodes; + URLContext *uc; + char *node_uri; + int64_t size; + size_t len = i; + + node_uri = av_get_token(&cursor, "\r\n"); + if (!node_uri) { + err = AVERROR(ENOMEM); + break; + } + cursor++; + + if (++len == SIZE_MAX / sizeof(*nodes)) { + av_free(node_uri); + err = AVERROR(ENAMETOOLONG); + break; + } + + /* creating URLContext */ + err = ffurl_open_whitelist(&uc, node_uri, flags, + &h->interrupt_callback, NULL, h->protocol_whitelist, h->protocol_blacklist, h); + av_free(node_uri); + if (err < 0) + break; + + /* creating size */ + if ((size = ffurl_size(uc)) < 0) { + ffurl_close(uc); + err = AVERROR(ENOSYS); + break; + } + + nodes = av_fast_realloc(data->nodes, &nodes_size, sizeof(*nodes) * len); + if (!nodes) { + ffurl_close(uc); + err = AVERROR(ENOMEM); + break; + } + data->nodes = nodes; + + /* assembling */ + data->nodes[i].uc = uc; + data->nodes[i++].size = size; + total_size += size; + } + av_bprint_finalize(&bp, NULL); + data->length = i; + + if (err < 0) + concat_close(h); + + data->total_size = total_size; + return err; +} + +const URLProtocol ff_concatf_protocol = { + .name = "concatf", + .url_open = concatf_open, + .url_read = concat_read, + .url_seek = concat_seek, + .url_close = concat_close, + .priv_data_size = sizeof(struct concat_data), + .default_whitelist = "concatf,concat,file,subfile", +}; +#endif diff --git a/libavformat/protocols.c b/libavformat/protocols.c index 4b6b1c8e98..7f08f151b6 100644 --- a/libavformat/protocols.c +++ b/libavformat/protocols.c @@ -27,6 +27,7 @@ extern const URLProtocol ff_async_protocol; extern const URLProtocol ff_bluray_protocol; extern const URLProtocol ff_cache_protocol; extern const URLProtocol ff_concat_protocol; +extern const URLProtocol ff_concatf_protocol; extern const URLProtocol ff_crypto_protocol; extern const URLProtocol ff_data_protocol; extern const URLProtocol ff_ffrtmpcrypt_protocol;