From patchwork Wed Jul 28 04:05:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gyan Doshi X-Patchwork-Id: 29077 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp5125995ios; Tue, 27 Jul 2021 21:05:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyEpqfxtsaZFPaCgPkJ3l6yzuu2GDUC+akrWzTA1digYqy1wIB+YWSGOgt9S3PEwlsyotRg X-Received: by 2002:a05:6402:747:: with SMTP id p7mr11074291edy.114.1627445152667; Tue, 27 Jul 2021 21:05:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627445152; cv=none; d=google.com; s=arc-20160816; b=F/SPCxansbZmrMQQPQTLmAdXEaKOwiNzEI5pwdL8yp3umn87y4fItRE2UVYuGKk2rO J/WADBjkmoXpje6mQEvqMspJ8X4VaFmMnZWgWpaDvzanfJu1qOquqsQn5JVvCOexBr2f dAz/NcII/RVvXFlqby+wGS8OfH9GhL6l7w7LEBPqugLtymxw9fJZQDSl4ORGW/txNEHL jXVlhUVRLgQw0sg08iFxcdCVqJ8ztsLC+Ae06hLE7Cp2q0sksau9UrJxcRBCHEDuZY+j FKMzXvgdrKCCyUOSQUp3NwVvY8xcegDgTMiWWFdY3fmUuqriul4T5bNKQBLw5eWAz8UR HIRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=dvpFtZFdpaAEujQ5W+s5xiHulu9XnlxrnFob14H7tOc=; b=SlAGwSD2dh5HhzZrLBFgW96/kWxlCwmyp2v1uvQfXJlk4J+YxPu9MwIgT6E9cJkl93 KCofWuZDugAgmIEEixgyrb8aU9nNpYW6L3VxCSY3Mz23/ZxjA3Npbfu/wDCQhox8p6AW 93LvwCm1CduDTh65v+SCKK1/Qi/0g65jzdCTjAZKH6gLcs//9GeAnYWl+3M6Zs3s9GgE to00Yu58flvOX7QAWd6Y/jeFFb9Sm9TFtPtH0Yi7EuDr+vA4114BJehGe+75E13Lse1G dfXUM5BMj746zw4XjyplXDYzergBcutEiYZE8Rzu/UrcRWPg0eb6sQzsJAD4iI/IddRm zVsQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e6si4407289ejk.740.2021.07.27.21.05.51; Tue, 27 Jul 2021 21:05:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F310E68A03A; Wed, 28 Jul 2021 07:05:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mout-p-103.mailbox.org (mout-p-103.mailbox.org [80.241.56.161]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6BE4E689D48 for ; Wed, 28 Jul 2021 07:05:41 +0300 (EEST) Received: from smtp1.mailbox.org (smtp1.mailbox.org [80.241.60.240]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-103.mailbox.org (Postfix) with ESMTPS id 4GZKnw3S1szQk9Z for ; Wed, 28 Jul 2021 06:05:40 +0200 (CEST) X-Virus-Scanned: amavisd-new at heinlein-support.de Received: from smtp1.mailbox.org ([80.241.60.240]) by spamfilter02.heinlein-hosting.de (spamfilter02.heinlein-hosting.de [80.241.56.116]) (amavisd-new, port 10030) with ESMTP id algToUSATo-C for ; Wed, 28 Jul 2021 06:05:35 +0200 (CEST) From: Gyan Doshi To: ffmpeg-devel@ffmpeg.org Date: Wed, 28 Jul 2021 09:35:21 +0530 Message-Id: <20210728040521.1323-1-ffmpeg@gyani.pro> MIME-Version: 1.0 X-Rspamd-Queue-Id: 73E4018B7 X-Rspamd-UID: 1980a1 Subject: [FFmpeg-devel] [PATCH v2] avcodec/noise_bsf: add expr support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: K7x5171gfLQ2 --- doc/bitstream_filters.texi | 64 ++++++++++++--- libavcodec/noise_bsf.c | 161 +++++++++++++++++++++++++++++++++---- tests/fate/matroska.mak | 2 +- 3 files changed, 199 insertions(+), 28 deletions(-) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index d10842ae47..46e4869f80 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -534,20 +534,62 @@ container. Can be used for fuzzing or testing error resilience/concealment. Parameters: @table @option @item amount -A numeral string, whose value is related to how often output bytes will -be modified. Therefore, values below or equal to 0 are forbidden, and -the lower the more frequent bytes will be modified, with 1 meaning -every byte is modified. -@item dropamount -A numeral string, whose value is related to how often packets will be dropped. -Therefore, values below or equal to 0 are forbidden, and the lower the more -frequent packets will be dropped, with 1 meaning every packet is dropped. +Accepts an expression whose evaluation per-packet determines how often bytes in that +packet will be modified. A value below 0 will result in a variable frequency. +Default is 0 which results in no modification. However, if neither amount or drop is specified, +amount will be set to @var{-1}. See below for accepted variables. +@item drop, dropamount +Accepts an expression evaluated per-packet whose value determines whether that packet is dropped. +Evaluation to a positive value results in the packet being dropped. Evaluation to a negative +value results in a variable chance of it being dropped, roughly inverse in proportion to the magnitude +of the value. Default is 0 which results in no drops. See below for accepted variables. @end table -The following example applies the modification to every byte but does not drop -any packets. +Both @code{amount} and @code{drop} accept expressions containing the following variables: + +@table @samp +@item n +The index of the packet, starting from zero. +@item tb +The timebase for packet timestamps. +@item pts +Packet presentation timestamp. +@item dts +Packet decoding timestamp. +@item nopts +Constant representing AV_NOPTS_VALUE. +@item startpts +First non-AV_NOPTS_VALUE PTS seen in the stream. +@item startdts +First non-AV_NOPTS_VALUE DTS seen in the stream. +@item duration +@itemx d +Packet duration, in timebase units. +@item pos +Packet position in input; may be -1 when unknown or not set. +@item size +Packet size, in bytes. +@item key +Whether packet is marked as a keyframe. +@item state +A pseudo random integer, primarily derived from the content of packet payload. +@end table + +@subsection Examples +Apply modification to every byte but don't drop any packets. +@example +ffmpeg -i INPUT -c copy -bsf noise=1 output.mkv +@end example + +Drop every video packet not marked as a keyframe after timestamp 30s but do not +modify any of the remaining packets. +@example +ffmpeg -i INPUT -c copy -bsf:v noise=drop='gt(t\,30)*not(key)' output.mkv +@end example + +Drop one second of audio every 10 seconds and add some random noise to the rest. @example -ffmpeg -i INPUT -c copy -bsf noise[=1] output.mkv +ffmpeg -i INPUT -c copy -bsf:a noise=amount=-1:drop='between(mod(t\,10)\,9\,10)' output.mkv @end example @section null diff --git a/libavcodec/noise_bsf.c b/libavcodec/noise_bsf.c index 6ebd369633..ab2221c5d1 100644 --- a/libavcodec/noise_bsf.c +++ b/libavcodec/noise_bsf.c @@ -23,55 +23,182 @@ #include "bsf.h" #include "bsf_internal.h" +#include "libavutil/avstring.h" #include "libavutil/log.h" #include "libavutil/opt.h" +#include "libavutil/eval.h" + +static const char *const var_names[] = { + "n", /// packet index, starting from zero + "tb", /// timebase + "pts", /// packet presentation timestamp + "dts", /// packet decoding timestamp + "nopts", /// AV_NOPTS_VALUE + "startpts", /// first seen non-AV_NOPTS_VALUE packet timestamp + "startdts", /// first seen non-AV_NOPTS_VALUE packet timestamp + "duration", "d", /// packet duration + "pos", /// original position of packet in its source + "size", /// packet size + "key" , /// packet keyframe flag + "state", /// random-ish state + NULL +}; + +enum var_name { + VAR_N, + VAR_TB, + VAR_PTS, + VAR_DTS, + VAR_NOPTS, + VAR_STARTPTS, + VAR_STARTDTS, + VAR_DURATION, VAR_D, + VAR_POS, + VAR_SIZE, + VAR_KEY, + VAR_STATE, + VAR_VARS_NB +}; typedef struct NoiseContext { const AVClass *class; - int amount; - int dropamount; + + char *amount_str; + char *drop_str; + + AVExpr *amount_pexpr; + AVExpr *drop_pexpr; + + double var_values[VAR_VARS_NB]; + unsigned int state; + unsigned int pkt_idx; } NoiseContext; -static int noise(AVBSFContext *ctx, AVPacket *pkt) +static int noise_init(AVBSFContext *ctx) { NoiseContext *s = ctx->priv_data; - int amount = s->amount > 0 ? s->amount : (s->state % 10001 + 1); - int i, ret; + int ret; + + if (!s->amount_str) { + s->amount_str = !s->drop_str ? av_strdup("-1") : av_strdup("0"); + if (!s->amount_str) + return AVERROR(ENOMEM); + } + + ret = av_expr_parse(&s->amount_pexpr, s->amount_str, + var_names, NULL, NULL, NULL, NULL, 0, ctx); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Error in parsing expr for amount: %s\n", s->amount_str); + return ret; + } + + if (s->drop_str) { + ret = av_expr_parse(&s->drop_pexpr, s->drop_str, + var_names, NULL, NULL, NULL, NULL, 0, ctx); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Error in parsing expr for drop: %s\n", s->drop_str); + return ret; + } + } + + s->var_values[VAR_TB] = ctx->time_base_out.den ? av_q2d(ctx->time_base_out) : 0; + s->var_values[VAR_NOPTS] = AV_NOPTS_VALUE; + s->var_values[VAR_STARTPTS] = AV_NOPTS_VALUE; + s->var_values[VAR_STARTDTS] = AV_NOPTS_VALUE; + s->var_values[VAR_STATE] = 0; + + return 0; +} - if (amount <= 0) - return AVERROR(EINVAL); +static int noise(AVBSFContext *ctx, AVPacket *pkt) +{ + NoiseContext *s = ctx->priv_data; + int i, ret, amount, drop; + double res; ret = ff_bsf_get_packet_ref(ctx, pkt); if (ret < 0) return ret; - if (s->dropamount > 0 && s->state % s->dropamount == 0) { - s->state++; + s->var_values[VAR_N] = s->pkt_idx++; + s->var_values[VAR_PTS] = pkt->pts; + s->var_values[VAR_DTS] = pkt->dts; + s->var_values[VAR_DURATION] = + s->var_values[VAR_D] = pkt->duration; + s->var_values[VAR_SIZE] = pkt->size; + s->var_values[VAR_KEY] = !!(pkt->flags & AV_PKT_FLAG_KEY); + s->var_values[VAR_POS] = pkt->pos; + + if (s->var_values[VAR_STARTPTS] == AV_NOPTS_VALUE) + s->var_values[VAR_STARTPTS] = pkt->pts; + + if (s->var_values[VAR_STARTDTS] == AV_NOPTS_VALUE) + s->var_values[VAR_STARTDTS] = pkt->dts; + + res = av_expr_eval(s->amount_pexpr, s->var_values, NULL); + + if (isnan(res)) + amount = 0; + else if (res < 0) + amount = (s->state % 10001 + 1); + else + amount = (int)res; + + if (s->drop_str) { + res = av_expr_eval(s->drop_pexpr, s->var_values, NULL); + + if (isnan(res)) + drop = 0; + else if (res < 0) + drop = !(s->state % FFABS((int)res)); + else + drop = !!res; + } + + av_log(ctx, AV_LOG_VERBOSE, "Stream #%d packet %d pts %"PRId64" - amount %d drop %d\n", + pkt->stream_index, (unsigned int)s->var_values[VAR_N], pkt->pts, amount, drop); + + if (s->drop_str && drop) { + s->var_values[VAR_STATE] = ++s->state; av_packet_unref(pkt); return AVERROR(EAGAIN); } - ret = av_packet_make_writable(pkt); - if (ret < 0) { - av_packet_unref(pkt); - return ret; + if (amount) { + ret = av_packet_make_writable(pkt); + if (ret < 0) { + av_packet_unref(pkt); + return ret; + } } for (i = 0; i < pkt->size; i++) { s->state += pkt->data[i] + 1; - if (s->state % amount == 0) + if (amount && s->state % amount == 0) pkt->data[i] = s->state; } + s->var_values[VAR_STATE] = s->state; + return 0; } +static void noise_close(AVBSFContext *bsf) +{ + NoiseContext *s = ctx->priv_data; + + av_expr_free(s->amount_pexpr); + av_expr_free(s->drop_pexpr); + s->amount_pexpr = s->drop_pexpr = NULL; +} + #define OFFSET(x) offsetof(NoiseContext, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_BSF_PARAM) static const AVOption options[] = { - { "amount", NULL, OFFSET(amount), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, - { "dropamount", NULL, OFFSET(dropamount), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, + { "amount", NULL, OFFSET(amount_str), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, FLAGS }, + { "drop", NULL, OFFSET(drop_str), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, FLAGS }, + { "dropamount", NULL, OFFSET(drop_str), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, FLAGS }, { NULL }, }; @@ -86,5 +213,7 @@ const AVBitStreamFilter ff_noise_bsf = { .name = "noise", .priv_data_size = sizeof(NoiseContext), .priv_class = &noise_class, + .init = noise_init, + .close = noise_close, .filter = noise, }; diff --git a/tests/fate/matroska.mak b/tests/fate/matroska.mak index ca7193a055..b57765280a 100644 --- a/tests/fate/matroska.mak +++ b/tests/fate/matroska.mak @@ -88,7 +88,7 @@ FATE_MATROSKA_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL MXF_DEMUXER \ MATROSKA_MUXER MATROSKA_DEMUXER \ FRAMECRC_MUXER PIPE_PROTOCOL) \ += fate-matroska-mastering-display-metadata -fate-matroska-mastering-display-metadata: CMD = transcode mxf $(TARGET_SAMPLES)/mxf/Meridian-Apple_ProResProxy-HDR10.mxf matroska "-map 0 -map 0:0 -c:v:0 copy -c:v:1 ffv1 -c:a:0 copy -bsf:a:0 noise=amount=3 -filter:a:1 aresample -c:a:1 pcm_s16be -bsf:a:1 noise=dropamount=4" "-map 0 -c copy" "" "-show_entries stream_side_data_list:stream=index,codec_name" +fate-matroska-mastering-display-metadata: CMD = transcode mxf $(TARGET_SAMPLES)/mxf/Meridian-Apple_ProResProxy-HDR10.mxf matroska "-map 0 -map 0:0 -c:v:0 copy -c:v:1 ffv1 -c:a:0 copy -bsf:a:0 noise=amount=3 -filter:a:1 aresample -c:a:1 pcm_s16be -bsf:a:1 noise=amount=-1:dropamount=-4" "-map 0 -c copy" "" "-show_entries stream_side_data_list:stream=index,codec_name" # This test tests remuxing annex B H.264 into Matroska. It also tests writing # the correct interlaced flags and overriding the sample aspect ratio, leading