From patchwork Mon Nov 27 18:43:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 44833 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bca6:b0:181:818d:5e7f with SMTP id fx38csp3615351pzb; Mon, 27 Nov 2023 10:44:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IH8Krjzn38S1nHmU0eOPdqbO3I32+piz4a9FJB/MJYCDhGPnUVuN8E6oFQXIJ2z9BBJCmSp X-Received: by 2002:a17:906:116:b0:9c5:cfa3:d030 with SMTP id 22-20020a170906011600b009c5cfa3d030mr9876435eje.54.1701110663169; Mon, 27 Nov 2023 10:44:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701110663; cv=none; d=google.com; s=arc-20160816; b=Ud50QZ7/Kd5i4UUsRyd8o6nm4EqLo8DyjJQGjU4r3x5iajoAuIelNtl+sdnGdSV016 tVOaa2HREw/2o9KbzL5ge8bLMXlhcaz/RE8JR8dhdh/oDU+/cJJs7ClKS7pBIN/y8hPa 70Cy/bRe1Tzzr2ucZoFODokHaxH0s8rfGrLJrHnCYiuh58+0x/TD9TRjDpqxegDt3tFZ y9ZsGpkGBwSwcjiRl9/+dtq14JIk8Z+JUH9lkNI4a1KeD3aSo1CJVwbvEFSz3/OkJdTX cEcJxDuHiQM8F+DOm8vwOU5W7NoNHIDyeWCJnEhk6a5mfWUzpShuY+VE/MiZactjQtfV Z51A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=HRakvgye9sUxoCeE+DcpFquUOA3P2YqS0gkQ2JCjWRY=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=eK7SKdku1uLkZuASTcHyj+rjJWTF80okrG2FYFiClwhMlnroh+kmP7qC7FGyTp1+w9 RPc5iBS93cQQjZhGPrwcaOHSCoTCEDjnJkFDzle5LFIGMBeBCD4a+L27X9v2j851IQtU K0JKqRrqeEpUHRXvmurMNCk/VDojvqdpyUwguj82D1KXNj2qVkkJV+IUhVh7TUYoyQum g97torJZqMTq/cWaTd4dd5QCOoy4NwE7X26z32DFUAPmZHgYf+fLxbUBfxcFvQ4uVeur NbN5ry3imSJh3D+kcXe+HVYlX+jRQMj9/U6m3jhc7kV5ZJZTrX0XS3q9cTJzUkLprnjN BhoQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=nXCiBrLt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hg14-20020a170906f34e00b00a0d4994d3b2si2274865ejb.678.2023.11.27.10.44.22; Mon, 27 Nov 2023 10:44:23 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=nXCiBrLt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 97B8D68CF7F; Mon, 27 Nov 2023 20:44:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B89AB68CD16 for ; Mon, 27 Nov 2023 20:44:00 +0200 (EET) Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-6c3363a2b93so4302018b3a.3 for ; Mon, 27 Nov 2023 10:44:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701110638; x=1701715438; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=C9/ErNJ/fmhyjb+x9i4Ix8AIGUtv7hF2O0rcKaO17kw=; b=nXCiBrLtZdL4t476rDysUKlid5Ggg/AAc6H3d2t9CtMyj9gQ/29uAE/gtN5EgohB7I vfFWA4HCHtDBoZ+eoweazAoM5l6ZmllZtoupOJd86+2QmLcMRBWn/8sStPc9sex39ORz nomHyG2qL7Rg+M+MLTmzWACDJhpLHXOvo7ZofQaVsrtRJWujWeS6q4XmkLDbOsj3KjBo hSfZJcZBTQbqCA5FKSdZ0+txNS7Lk6+88dKvbnhBr8x7eLqam624icgkoZJ018MIgA8n jGyWbcXXxr5vVftuA1lnmr4cnqeYElZDO9KXCQAVXMKZpHqrYgCX/7nNe/gRi6PbAlCH 183Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701110638; x=1701715438; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C9/ErNJ/fmhyjb+x9i4Ix8AIGUtv7hF2O0rcKaO17kw=; b=MB7fcyXduMaPX5NbHWJByR77j0BpHtYOWRO4BsaSfF/dNl/KmoBzAh6dM7TpXxRPIH JfuvploxGTibML1EZ1TNzh5+DKafFQTZayAvcu3/4sh6PeGIyy9veOrEtbNa9FHtEKgv qWJAZfPQ9ZPUzpeOheZI+XQsl4F3jvKShSvcyN+WydTEWvjv72SuGSNDmPFEsorpr/bU LJhrO8quhrvTRsj+vRhU8eMTgqb2Yd9iGzvp7W10OeHpfJ4wbxGlz6h51XLvVsFJBLJW 5xyfoXVu6GqC6oR7YkaaIQuZZ6UYQ/fJMGyXJpHU9ez+7OWNRsSIcrG8HSorO3q4VbVI Uy1A== X-Gm-Message-State: AOJu0YwSoyJ1AhrFWTDzicM/zYfsIkPPxSN4hDAO3o7Jc5lP7ZANoQ7m bie4P4f/LDTZFxrjo0sXXjQbecMN7io= X-Received: by 2002:a05:6a00:21c7:b0:68a:3ba3:e249 with SMTP id t7-20020a056a0021c700b0068a3ba3e249mr16269292pfj.16.1701110638112; Mon, 27 Nov 2023 10:43:58 -0800 (PST) Received: from localhost.localdomain (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id gx10-20020a056a001e0a00b006c107a9e8f0sm7516720pfb.128.2023.11.27.10.43.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Nov 2023 10:43:57 -0800 (PST) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 Nov 2023 15:43:55 -0300 Message-ID: <20231127184357.3361-2-jamrial@gmail.com> X-Mailer: git-send-email 2.42.1 In-Reply-To: <20231126012858.40388-1-jamrial@gmail.com> References: <20231126012858.40388-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 11/13] avformat/demux: support inserting bitstream filters in demuxing scenarios X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: tNla/o+fa0yr Packets will be passed to the bsf immediately after being generated by a demuxer, and no further data will be read from the input until all packets have been returned by the bsf. Signed-off-by: James Almer --- libavformat/avformat.c | 47 ++++++++++++ libavformat/demux.c | 162 ++++++++++++++++++++++++++++++----------- libavformat/internal.h | 13 +++- libavformat/mux.c | 43 ----------- libavformat/mux.h | 11 --- libavformat/rawenc.c | 1 + 6 files changed, 181 insertions(+), 96 deletions(-) diff --git a/libavformat/avformat.c b/libavformat/avformat.c index a02ec965dd..a41c0b391c 100644 --- a/libavformat/avformat.c +++ b/libavformat/avformat.c @@ -1033,3 +1033,50 @@ FF_ENABLE_DEPRECATION_WARNINGS *pb = NULL; return ret; } + +int ff_stream_add_bitstream_filter(AVStream *st, const char *name, const char *args) +{ + int ret; + const AVBitStreamFilter *bsf; + FFStream *const sti = ffstream(st); + AVBSFContext *bsfc; + + av_assert0(!sti->bsfc); + + if (name) { + bsf = av_bsf_get_by_name(name); + if (!bsf) { + av_log(NULL, AV_LOG_ERROR, "Unknown bitstream filter '%s'\n", name); + return AVERROR_BSF_NOT_FOUND; + } + ret = av_bsf_alloc(bsf, &bsfc); + } else + ret = av_bsf_get_null_filter(&bsfc); + if (ret < 0) + return ret; + + bsfc->time_base_in = st->time_base; + if ((ret = avcodec_parameters_copy(bsfc->par_in, st->codecpar)) < 0) { + av_bsf_free(&bsfc); + return ret; + } + + if (args && bsfc->filter->priv_class) { + if ((ret = av_set_options_string(bsfc->priv_data, args, "=", ":")) < 0) { + av_bsf_free(&bsfc); + return ret; + } + } + + if ((ret = av_bsf_init(bsfc)) < 0) { + av_bsf_free(&bsfc); + return ret; + } + + sti->bsfc = bsfc; + + av_log(NULL, AV_LOG_VERBOSE, + "Automatically inserted bitstream filter '%s'; args='%s'\n", + name, args ? args : ""); + return 1; +} diff --git a/libavformat/demux.c b/libavformat/demux.c index 6f640b92b1..fb9bf9e4ac 100644 --- a/libavformat/demux.c +++ b/libavformat/demux.c @@ -540,6 +540,109 @@ static int update_wrap_reference(AVFormatContext *s, AVStream *st, int stream_in return 1; } +static void update_timestamps(AVFormatContext *s, AVStream *st, AVPacket *pkt) +{ + FFStream *const sti = ffstream(st); + + if (update_wrap_reference(s, st, pkt->stream_index, pkt) && sti->pts_wrap_behavior == AV_PTS_WRAP_SUB_OFFSET) { + // correct first time stamps to negative values + if (!is_relative(sti->first_dts)) + sti->first_dts = wrap_timestamp(st, sti->first_dts); + if (!is_relative(st->start_time)) + st->start_time = wrap_timestamp(st, st->start_time); + if (!is_relative(sti->cur_dts)) + sti->cur_dts = wrap_timestamp(st, sti->cur_dts); + } + + pkt->dts = wrap_timestamp(st, pkt->dts); + pkt->pts = wrap_timestamp(st, pkt->pts); + + force_codec_ids(s, st); + + /* TODO: audio: time filter; video: frame reordering (pts != dts) */ + if (s->use_wallclock_as_timestamps) + pkt->dts = pkt->pts = av_rescale_q(av_gettime(), AV_TIME_BASE_Q, st->time_base); +} + +static int filter_packet(AVFormatContext *s, AVStream *st, AVPacket *pkt) +{ + FFFormatContext *const si = ffformatcontext(s); + FFStream *const sti = ffstream(st); + const AVPacket *pkt1; + int err; + + if (!sti->bsfc) { + const PacketListEntry *pktl = si->raw_packet_buffer.head; + if (AVPACKET_IS_EMPTY(pkt)) + return 0; + + update_timestamps(s, st, pkt); + + if (!pktl && sti->request_probe <= 0) + return 0; + + err = avpriv_packet_list_put(&si->raw_packet_buffer, pkt, NULL, 0); + if (err < 0) { + av_packet_unref(pkt); + return err; + } + + pkt1 = &si->raw_packet_buffer.tail->pkt; + si->raw_packet_buffer_size += pkt1->size; + + if (sti->request_probe <= 0) + return 0; + + return probe_codec(s, s->streams[pkt1->stream_index], pkt1); + } + + err = av_bsf_send_packet(sti->bsfc, pkt); + if (err < 0) { + av_log(s, AV_LOG_ERROR, + "Failed to send packet to filter %s for stream %d\n", + sti->bsfc->filter->name, st->index); + return err; + } + + do { + AVStream *out_st; + FFStream *out_sti; + + err = av_bsf_receive_packet(sti->bsfc, pkt); + if (err < 0) { + if (err == AVERROR(EAGAIN) || err == AVERROR_EOF) + return 0; + av_log(s, AV_LOG_ERROR, "Error applying bitstream filters to an output " + "packet for stream #%d: %s\n", st->index, av_err2str(err)); + if (!(s->error_recognition & AV_EF_EXPLODE) && err != AVERROR(ENOMEM)) + continue; + return err; + } + out_st = s->streams[pkt->stream_index]; + out_sti = ffstream(out_st); + + update_timestamps(s, out_st, pkt); + + err = avpriv_packet_list_put(&si->raw_packet_buffer, pkt, NULL, 0); + if (err < 0) { + av_packet_unref(pkt); + return err; + } + + pkt1 = &si->raw_packet_buffer.tail->pkt; + si->raw_packet_buffer_size += pkt1->size; + + if (out_sti->request_probe <= 0) + continue; + + err = probe_codec(s, out_st, pkt1); + if (err < 0) + return err; + } while (1); + + return 0; +} + int ff_read_packet(AVFormatContext *s, AVPacket *pkt) { FFFormatContext *const si = ffformatcontext(s); @@ -557,9 +660,6 @@ FF_ENABLE_DEPRECATION_WARNINGS for (;;) { PacketListEntry *pktl = si->raw_packet_buffer.head; - AVStream *st; - FFStream *sti; - const AVPacket *pkt1; if (pktl) { AVStream *const st = s->streams[pktl->pkt.stream_index]; @@ -582,16 +682,27 @@ FF_ENABLE_DEPRECATION_WARNINGS We must re-call the demuxer to get the real packet. */ if (err == FFERROR_REDO) continue; - if (!pktl || err == AVERROR(EAGAIN)) + if (err == AVERROR(EAGAIN)) return err; for (unsigned i = 0; i < s->nb_streams; i++) { AVStream *const st = s->streams[i]; FFStream *const sti = ffstream(st); + int ret; + + // Drain buffered packets in the bsf context on eof + if (err == AVERROR_EOF) + if ((ret = filter_packet(s, st, pkt)) < 0) + return ret; + pktl = si->raw_packet_buffer.head; + if (!pktl) + continue; if (sti->probe_packets || sti->request_probe > 0) - if ((err = probe_codec(s, st, NULL)) < 0) - return err; + if ((ret = probe_codec(s, st, NULL)) < 0) + return ret; av_assert0(sti->request_probe <= 0); } + if (!pktl) + return err; continue; } @@ -616,42 +727,11 @@ FF_ENABLE_DEPRECATION_WARNINGS av_assert0(pkt->stream_index < (unsigned)s->nb_streams && "Invalid stream index.\n"); - st = s->streams[pkt->stream_index]; - sti = ffstream(st); - - if (update_wrap_reference(s, st, pkt->stream_index, pkt) && sti->pts_wrap_behavior == AV_PTS_WRAP_SUB_OFFSET) { - // correct first time stamps to negative values - if (!is_relative(sti->first_dts)) - sti->first_dts = wrap_timestamp(st, sti->first_dts); - if (!is_relative(st->start_time)) - st->start_time = wrap_timestamp(st, st->start_time); - if (!is_relative(sti->cur_dts)) - sti->cur_dts = wrap_timestamp(st, sti->cur_dts); - } - - pkt->dts = wrap_timestamp(st, pkt->dts); - pkt->pts = wrap_timestamp(st, pkt->pts); - - force_codec_ids(s, st); - - /* TODO: audio: time filter; video: frame reordering (pts != dts) */ - if (s->use_wallclock_as_timestamps) - pkt->dts = pkt->pts = av_rescale_q(av_gettime(), AV_TIME_BASE_Q, st->time_base); - - if (!pktl && sti->request_probe <= 0) - return 0; - - err = avpriv_packet_list_put(&si->raw_packet_buffer, - pkt, NULL, 0); - if (err < 0) { - av_packet_unref(pkt); - return err; - } - pkt1 = &si->raw_packet_buffer.tail->pkt; - si->raw_packet_buffer_size += pkt1->size; - - if ((err = probe_codec(s, st, pkt1)) < 0) + err = filter_packet(s, s->streams[pkt->stream_index], pkt); + if (err < 0) return err; + if (!AVPACKET_IS_EMPTY(pkt)) + return 0; } } diff --git a/libavformat/internal.h b/libavformat/internal.h index c6181683ef..0a5d512697 100644 --- a/libavformat/internal.h +++ b/libavformat/internal.h @@ -212,7 +212,7 @@ typedef struct FFStream { /** * bitstream filter to run on stream * - encoding: Set by muxer using ff_stream_add_bitstream_filter - * - decoding: unused + * - decoding: Set by demuxer using ff_stream_add_bitstream_filter */ struct AVBSFContext *bsfc; @@ -757,4 +757,15 @@ int ff_match_url_ext(const char *url, const char *extensions); struct FFOutputFormat; void avpriv_register_devices(const struct FFOutputFormat * const o[], const AVInputFormat * const i[]); +/** + * Add a bitstream filter to a stream. + * + * @param st output stream to add a filter to + * @param name the name of the filter to add + * @param args filter-specific argument string + * @return >0 on success; + * AVERROR code on failure + */ +int ff_stream_add_bitstream_filter(AVStream *st, const char *name, const char *args); + #endif /* AVFORMAT_INTERNAL_H */ diff --git a/libavformat/mux.c b/libavformat/mux.c index de10d2c008..4bc8627617 100644 --- a/libavformat/mux.c +++ b/libavformat/mux.c @@ -1344,49 +1344,6 @@ int av_get_output_timestamp(struct AVFormatContext *s, int stream, return 0; } -int ff_stream_add_bitstream_filter(AVStream *st, const char *name, const char *args) -{ - int ret; - const AVBitStreamFilter *bsf; - FFStream *const sti = ffstream(st); - AVBSFContext *bsfc; - - av_assert0(!sti->bsfc); - - if (!(bsf = av_bsf_get_by_name(name))) { - av_log(NULL, AV_LOG_ERROR, "Unknown bitstream filter '%s'\n", name); - return AVERROR_BSF_NOT_FOUND; - } - - if ((ret = av_bsf_alloc(bsf, &bsfc)) < 0) - return ret; - - bsfc->time_base_in = st->time_base; - if ((ret = avcodec_parameters_copy(bsfc->par_in, st->codecpar)) < 0) { - av_bsf_free(&bsfc); - return ret; - } - - if (args && bsfc->filter->priv_class) { - if ((ret = av_set_options_string(bsfc->priv_data, args, "=", ":")) < 0) { - av_bsf_free(&bsfc); - return ret; - } - } - - if ((ret = av_bsf_init(bsfc)) < 0) { - av_bsf_free(&bsfc); - return ret; - } - - sti->bsfc = bsfc; - - av_log(NULL, AV_LOG_VERBOSE, - "Automatically inserted bitstream filter '%s'; args='%s'\n", - name, args ? args : ""); - return 1; -} - int ff_write_chained(AVFormatContext *dst, int dst_stream, AVPacket *pkt, AVFormatContext *src, int interleave) { diff --git a/libavformat/mux.h b/libavformat/mux.h index b9ec75641d..ab3e8edd60 100644 --- a/libavformat/mux.h +++ b/libavformat/mux.h @@ -171,17 +171,6 @@ const AVPacket *ff_interleaved_peek(AVFormatContext *s, int stream); int ff_get_muxer_ts_offset(AVFormatContext *s, int stream_index, int64_t *offset); -/** - * Add a bitstream filter to a stream. - * - * @param st output stream to add a filter to - * @param name the name of the filter to add - * @param args filter-specific argument string - * @return >0 on success; - * AVERROR code on failure - */ -int ff_stream_add_bitstream_filter(AVStream *st, const char *name, const char *args); - /** * Write a packet to another muxer than the one the user originally * intended. Useful when chaining muxers, where one muxer internally diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index f916db13a2..ec31d76d88 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -25,6 +25,7 @@ #include "libavutil/intreadwrite.h" #include "avformat.h" +#include "internal.h" #include "rawenc.h" #include "mux.h"