From patchwork Wed Jan 31 17:26:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 45938 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:2c82:b0:199:de12:6fa6 with SMTP id g2csp3912pzj; Wed, 31 Jan 2024 09:27:11 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVFMMLTSxVBp5L5ob6IgEbnN+saBpLa/q4h2RtftNfU+J3yj1DFgFFLwUHdCetHbYpbsxTvvmX3/AylD/waPvOHvaBiFWSVrOFyhA== X-Google-Smtp-Source: AGHT+IGW7URpWQkF3HE5THNTljpNiimHY5b93ABFx30PDQ/dCVHtJfQjSVgv+RcerXl43p5pBWqJ X-Received: by 2002:a17:906:3b85:b0:a31:234e:6a9c with SMTP id u5-20020a1709063b8500b00a31234e6a9cmr1552745ejf.5.1706722030760; Wed, 31 Jan 2024 09:27:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1706722030; cv=none; d=google.com; s=arc-20160816; b=uj+KtSSi+8S5ev7FZ9/gnHip4DoNN7SEDomXj7m5ZswbbrYwDbV50eDZZy7vT4Mn9B dxijKTVHeHpWcdJQ77DYPNPm0ktDQGa+OQVTCS9ScEIxrhrm7Fe9vRoBsR0gOpfEMXIi sqdiCG25Nqbwq3/7LFcsNknRT/jGxxBmouaLT//VvLZzWybjoFNeU/n0Y0Hmf0XsPlve 25DQbJ/C3ZZKsQXg+zsT0/ZQ3N87DCye2eddoPWcsheLHm0dphYv3v+A+n20fslAwAPw vnmcE0mdeTXVa0KXFENvc5eL+0VkLoiqVU9lshm0W/9kWYzvy+Z3KxXFlfy/zfORQENw 4bdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=FLxR8zuyq4fk4q9jshWY7/ArTNfYP+WVRga5CUsaPDA=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=LNKe7g2SiVh68ucDr1PtL65iTSi+VGWq8hn0OG8FLhaX8kG1VbVD/PMyPm4zsODj/P qe3Bk4OklDNiSoyKLXr4jN5WSXWRKGR2buaophr2L7qsij3SklsUcRdpZ5ThgAfOeiHg Km+kywtgUPfDFRIgtRQ2pOPKMNa4W5JIVJZHol6Z6YQ3tukQ1GPv6W/YhYie808G0pwZ c25BhqtSkCRUiFCRO+KVefSWHCVvw+oz+bAF61wqBKvri/Y8DBPKKhCqmI9Lm++k5v7J 9NRAYLIv40yG5AUmJKSDc+lgXeBMjZLLp6KTmOvAV+yeKldk7fz7jrDNRkW+TuEBMTAp u1cw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=eYuCyRLo; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w26-20020a17090652da00b00a2b22f642fasi5728523ejn.236.2024.01.31.09.27.10; Wed, 31 Jan 2024 09:27:10 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=eYuCyRLo; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5667168D118; Wed, 31 Jan 2024 19:26:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 65BDE68CAFA for ; Wed, 31 Jan 2024 19:26:48 +0200 (EET) Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1d911c9240dso15535405ad.2 for ; Wed, 31 Jan 2024 09:26:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706722006; x=1707326806; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=iVDXbV0NS0UVJL/Qg/i4CDJ4grAgWUzvGClGoMKd4y0=; b=eYuCyRLoR8qM9+18dzQ9N8cAXPnSNWU4v3SbbHIKSNk8obL7gt6wnQThjgxDiRl2Mq oGyC+eY596tp4P20GOJft0dSD54M+nF1SVwHLLLimmVkizAlproJXuaUu2oCaXDYF5Er AkkFdmR/s8bDcNYYLAi5bys8NLFwMbsL2yOKYiUv/pnswkGInlTdhVNiOu1mqs/FQtPj np57SG5Om2xdvJfz1Spb1yOO1lnGsBvACQKRY4+vr65d4hlK9+7VJ2KFVw83Z79qOrc7 pVkWlPnJsGTV54Ihy7sadxfb+M11RO9NTBFtV5LGFsfS3KD/kHohzEqu+DNZ+pYRy9ht +2Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706722006; x=1707326806; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iVDXbV0NS0UVJL/Qg/i4CDJ4grAgWUzvGClGoMKd4y0=; b=EB5GR/ht5xteEIH4R1f5yRNquOTjAGCqTXf0ikLAZ6bberCpsDDT+7+WnT8M2iiwzP rPHNpKcm9acaJUdz5fKYyxxis0ByMPUw67jj2MYMvLSNiWzuon98zOxunFNBfn2sFFlF cG55+L942ypQYrCW4gsqC3ztEmDUk35trI36wufLLf4UUuFSW9a653ALFgPQ8i3fgSgI l+ofCw/4+dgPRzYZMRG+3FnLPIHM9Fao0+IoG7LZLWnZN8uRzuRKtVsFDDPlKe+ndnf9 dWhD6+iopCJH90Jxg0XkLh02mbOJd7Bte0AmRGH940IlEf2NskKa0w3WvKbC5r0146IA hM7w== X-Gm-Message-State: AOJu0Yxj0wkgrH+A0hX7z0aISW/6SOsUK43qaZrYkaXs2GsD9DdYab06 cAkZZtaxKgaj55iC4r72/Qx6Qn2P09K5+TLOmyzYICGEgig4j5FpOq62a5Cs X-Received: by 2002:a17:902:c781:b0:1d7:690f:a961 with SMTP id w1-20020a170902c78100b001d7690fa961mr2164232pla.44.1706722005530; Wed, 31 Jan 2024 09:26:45 -0800 (PST) Received: from localhost.localdomain (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id k11-20020a170902f28b00b001d8e4b85636sm5762235plc.138.2024.01.31.09.26.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 09:26:44 -0800 (PST) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Wed, 31 Jan 2024 14:26:50 -0300 Message-ID: <20240131172654.15869-2-jamrial@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131172654.15869-1-jamrial@gmail.com> References: <20240131172654.15869-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/6 v3] avformat/demux: support inserting bitstream filters in demuxing scenarios X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 1MLxkKXnsnuM Packets will be passed to the bsf immediately after being generated by a demuxer, and no further data will be read from the input until all packets have been returned by the bsf. Signed-off-by: James Almer --- libavformat/avformat.c | 47 ++++++++++++ libavformat/demux.c | 162 ++++++++++++++++++++++++++++++----------- libavformat/internal.h | 13 +++- libavformat/mux.c | 43 ----------- libavformat/mux.h | 11 --- libavformat/rawenc.c | 1 + 6 files changed, 181 insertions(+), 96 deletions(-) diff --git a/libavformat/avformat.c b/libavformat/avformat.c index 8e8c6fbe55..0e22d47c8b 100644 --- a/libavformat/avformat.c +++ b/libavformat/avformat.c @@ -931,3 +931,50 @@ FF_ENABLE_DEPRECATION_WARNINGS *pb = NULL; return ret; } + +int ff_stream_add_bitstream_filter(AVStream *st, const char *name, const char *args) +{ + int ret; + const AVBitStreamFilter *bsf; + FFStream *const sti = ffstream(st); + AVBSFContext *bsfc; + + av_assert0(!sti->bsfc); + + if (name) { + bsf = av_bsf_get_by_name(name); + if (!bsf) { + av_log(NULL, AV_LOG_ERROR, "Unknown bitstream filter '%s'\n", name); + return AVERROR_BSF_NOT_FOUND; + } + ret = av_bsf_alloc(bsf, &bsfc); + } else + ret = av_bsf_get_null_filter(&bsfc); + if (ret < 0) + return ret; + + bsfc->time_base_in = st->time_base; + if ((ret = avcodec_parameters_copy(bsfc->par_in, st->codecpar)) < 0) { + av_bsf_free(&bsfc); + return ret; + } + + if (args && bsfc->filter->priv_class) { + if ((ret = av_set_options_string(bsfc->priv_data, args, "=", ":")) < 0) { + av_bsf_free(&bsfc); + return ret; + } + } + + if ((ret = av_bsf_init(bsfc)) < 0) { + av_bsf_free(&bsfc); + return ret; + } + + sti->bsfc = bsfc; + + av_log(NULL, AV_LOG_VERBOSE, + "Automatically inserted bitstream filter '%s'; args='%s'\n", + name, args ? args : ""); + return 1; +} diff --git a/libavformat/demux.c b/libavformat/demux.c index 6f640b92b1..fb9bf9e4ac 100644 --- a/libavformat/demux.c +++ b/libavformat/demux.c @@ -540,6 +540,109 @@ static int update_wrap_reference(AVFormatContext *s, AVStream *st, int stream_in return 1; } +static void update_timestamps(AVFormatContext *s, AVStream *st, AVPacket *pkt) +{ + FFStream *const sti = ffstream(st); + + if (update_wrap_reference(s, st, pkt->stream_index, pkt) && sti->pts_wrap_behavior == AV_PTS_WRAP_SUB_OFFSET) { + // correct first time stamps to negative values + if (!is_relative(sti->first_dts)) + sti->first_dts = wrap_timestamp(st, sti->first_dts); + if (!is_relative(st->start_time)) + st->start_time = wrap_timestamp(st, st->start_time); + if (!is_relative(sti->cur_dts)) + sti->cur_dts = wrap_timestamp(st, sti->cur_dts); + } + + pkt->dts = wrap_timestamp(st, pkt->dts); + pkt->pts = wrap_timestamp(st, pkt->pts); + + force_codec_ids(s, st); + + /* TODO: audio: time filter; video: frame reordering (pts != dts) */ + if (s->use_wallclock_as_timestamps) + pkt->dts = pkt->pts = av_rescale_q(av_gettime(), AV_TIME_BASE_Q, st->time_base); +} + +static int filter_packet(AVFormatContext *s, AVStream *st, AVPacket *pkt) +{ + FFFormatContext *const si = ffformatcontext(s); + FFStream *const sti = ffstream(st); + const AVPacket *pkt1; + int err; + + if (!sti->bsfc) { + const PacketListEntry *pktl = si->raw_packet_buffer.head; + if (AVPACKET_IS_EMPTY(pkt)) + return 0; + + update_timestamps(s, st, pkt); + + if (!pktl && sti->request_probe <= 0) + return 0; + + err = avpriv_packet_list_put(&si->raw_packet_buffer, pkt, NULL, 0); + if (err < 0) { + av_packet_unref(pkt); + return err; + } + + pkt1 = &si->raw_packet_buffer.tail->pkt; + si->raw_packet_buffer_size += pkt1->size; + + if (sti->request_probe <= 0) + return 0; + + return probe_codec(s, s->streams[pkt1->stream_index], pkt1); + } + + err = av_bsf_send_packet(sti->bsfc, pkt); + if (err < 0) { + av_log(s, AV_LOG_ERROR, + "Failed to send packet to filter %s for stream %d\n", + sti->bsfc->filter->name, st->index); + return err; + } + + do { + AVStream *out_st; + FFStream *out_sti; + + err = av_bsf_receive_packet(sti->bsfc, pkt); + if (err < 0) { + if (err == AVERROR(EAGAIN) || err == AVERROR_EOF) + return 0; + av_log(s, AV_LOG_ERROR, "Error applying bitstream filters to an output " + "packet for stream #%d: %s\n", st->index, av_err2str(err)); + if (!(s->error_recognition & AV_EF_EXPLODE) && err != AVERROR(ENOMEM)) + continue; + return err; + } + out_st = s->streams[pkt->stream_index]; + out_sti = ffstream(out_st); + + update_timestamps(s, out_st, pkt); + + err = avpriv_packet_list_put(&si->raw_packet_buffer, pkt, NULL, 0); + if (err < 0) { + av_packet_unref(pkt); + return err; + } + + pkt1 = &si->raw_packet_buffer.tail->pkt; + si->raw_packet_buffer_size += pkt1->size; + + if (out_sti->request_probe <= 0) + continue; + + err = probe_codec(s, out_st, pkt1); + if (err < 0) + return err; + } while (1); + + return 0; +} + int ff_read_packet(AVFormatContext *s, AVPacket *pkt) { FFFormatContext *const si = ffformatcontext(s); @@ -557,9 +660,6 @@ FF_ENABLE_DEPRECATION_WARNINGS for (;;) { PacketListEntry *pktl = si->raw_packet_buffer.head; - AVStream *st; - FFStream *sti; - const AVPacket *pkt1; if (pktl) { AVStream *const st = s->streams[pktl->pkt.stream_index]; @@ -582,16 +682,27 @@ FF_ENABLE_DEPRECATION_WARNINGS We must re-call the demuxer to get the real packet. */ if (err == FFERROR_REDO) continue; - if (!pktl || err == AVERROR(EAGAIN)) + if (err == AVERROR(EAGAIN)) return err; for (unsigned i = 0; i < s->nb_streams; i++) { AVStream *const st = s->streams[i]; FFStream *const sti = ffstream(st); + int ret; + + // Drain buffered packets in the bsf context on eof + if (err == AVERROR_EOF) + if ((ret = filter_packet(s, st, pkt)) < 0) + return ret; + pktl = si->raw_packet_buffer.head; + if (!pktl) + continue; if (sti->probe_packets || sti->request_probe > 0) - if ((err = probe_codec(s, st, NULL)) < 0) - return err; + if ((ret = probe_codec(s, st, NULL)) < 0) + return ret; av_assert0(sti->request_probe <= 0); } + if (!pktl) + return err; continue; } @@ -616,42 +727,11 @@ FF_ENABLE_DEPRECATION_WARNINGS av_assert0(pkt->stream_index < (unsigned)s->nb_streams && "Invalid stream index.\n"); - st = s->streams[pkt->stream_index]; - sti = ffstream(st); - - if (update_wrap_reference(s, st, pkt->stream_index, pkt) && sti->pts_wrap_behavior == AV_PTS_WRAP_SUB_OFFSET) { - // correct first time stamps to negative values - if (!is_relative(sti->first_dts)) - sti->first_dts = wrap_timestamp(st, sti->first_dts); - if (!is_relative(st->start_time)) - st->start_time = wrap_timestamp(st, st->start_time); - if (!is_relative(sti->cur_dts)) - sti->cur_dts = wrap_timestamp(st, sti->cur_dts); - } - - pkt->dts = wrap_timestamp(st, pkt->dts); - pkt->pts = wrap_timestamp(st, pkt->pts); - - force_codec_ids(s, st); - - /* TODO: audio: time filter; video: frame reordering (pts != dts) */ - if (s->use_wallclock_as_timestamps) - pkt->dts = pkt->pts = av_rescale_q(av_gettime(), AV_TIME_BASE_Q, st->time_base); - - if (!pktl && sti->request_probe <= 0) - return 0; - - err = avpriv_packet_list_put(&si->raw_packet_buffer, - pkt, NULL, 0); - if (err < 0) { - av_packet_unref(pkt); - return err; - } - pkt1 = &si->raw_packet_buffer.tail->pkt; - si->raw_packet_buffer_size += pkt1->size; - - if ((err = probe_codec(s, st, pkt1)) < 0) + err = filter_packet(s, s->streams[pkt->stream_index], pkt); + if (err < 0) return err; + if (!AVPACKET_IS_EMPTY(pkt)) + return 0; } } diff --git a/libavformat/internal.h b/libavformat/internal.h index f93832b3c4..c2738a420f 100644 --- a/libavformat/internal.h +++ b/libavformat/internal.h @@ -212,7 +212,7 @@ typedef struct FFStream { /** * bitstream filter to run on stream * - encoding: Set by muxer using ff_stream_add_bitstream_filter - * - decoding: unused + * - decoding: Set by demuxer using ff_stream_add_bitstream_filter */ struct AVBSFContext *bsfc; @@ -752,4 +752,15 @@ int ff_match_url_ext(const char *url, const char *extensions); struct FFOutputFormat; void avpriv_register_devices(const struct FFOutputFormat * const o[], const AVInputFormat * const i[]); +/** + * Add a bitstream filter to a stream. + * + * @param st output stream to add a filter to + * @param name the name of the filter to add + * @param args filter-specific argument string + * @return >0 on success; + * AVERROR code on failure + */ +int ff_stream_add_bitstream_filter(AVStream *st, const char *name, const char *args); + #endif /* AVFORMAT_INTERNAL_H */ diff --git a/libavformat/mux.c b/libavformat/mux.c index de10d2c008..4bc8627617 100644 --- a/libavformat/mux.c +++ b/libavformat/mux.c @@ -1344,49 +1344,6 @@ int av_get_output_timestamp(struct AVFormatContext *s, int stream, return 0; } -int ff_stream_add_bitstream_filter(AVStream *st, const char *name, const char *args) -{ - int ret; - const AVBitStreamFilter *bsf; - FFStream *const sti = ffstream(st); - AVBSFContext *bsfc; - - av_assert0(!sti->bsfc); - - if (!(bsf = av_bsf_get_by_name(name))) { - av_log(NULL, AV_LOG_ERROR, "Unknown bitstream filter '%s'\n", name); - return AVERROR_BSF_NOT_FOUND; - } - - if ((ret = av_bsf_alloc(bsf, &bsfc)) < 0) - return ret; - - bsfc->time_base_in = st->time_base; - if ((ret = avcodec_parameters_copy(bsfc->par_in, st->codecpar)) < 0) { - av_bsf_free(&bsfc); - return ret; - } - - if (args && bsfc->filter->priv_class) { - if ((ret = av_set_options_string(bsfc->priv_data, args, "=", ":")) < 0) { - av_bsf_free(&bsfc); - return ret; - } - } - - if ((ret = av_bsf_init(bsfc)) < 0) { - av_bsf_free(&bsfc); - return ret; - } - - sti->bsfc = bsfc; - - av_log(NULL, AV_LOG_VERBOSE, - "Automatically inserted bitstream filter '%s'; args='%s'\n", - name, args ? args : ""); - return 1; -} - int ff_write_chained(AVFormatContext *dst, int dst_stream, AVPacket *pkt, AVFormatContext *src, int interleave) { diff --git a/libavformat/mux.h b/libavformat/mux.h index b9ec75641d..ab3e8edd60 100644 --- a/libavformat/mux.h +++ b/libavformat/mux.h @@ -171,17 +171,6 @@ const AVPacket *ff_interleaved_peek(AVFormatContext *s, int stream); int ff_get_muxer_ts_offset(AVFormatContext *s, int stream_index, int64_t *offset); -/** - * Add a bitstream filter to a stream. - * - * @param st output stream to add a filter to - * @param name the name of the filter to add - * @param args filter-specific argument string - * @return >0 on success; - * AVERROR code on failure - */ -int ff_stream_add_bitstream_filter(AVStream *st, const char *name, const char *args); - /** * Write a packet to another muxer than the one the user originally * intended. Useful when chaining muxers, where one muxer internally diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index f916db13a2..ec31d76d88 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -25,6 +25,7 @@ #include "libavutil/intreadwrite.h" #include "avformat.h" +#include "internal.h" #include "rawenc.h" #include "mux.h"