From patchwork Tue Jul 27 16:22:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gyan Doshi X-Patchwork-Id: 29074 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a11:4023:0:0:0:0 with SMTP id ky35csp5002057pxb; Tue, 27 Jul 2021 09:22:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyMfvmoMPgrjAj7A8FwEQ4gIGxqj1mMZS3R1AjCapTx1HXK9I4WLJRtbSS4Fmj/8dB/5wYq X-Received: by 2002:a05:6402:424e:: with SMTP id g14mr28909091edb.364.1627402978581; Tue, 27 Jul 2021 09:22:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627402978; cv=none; d=google.com; s=arc-20160816; b=dBKM8/Tf8G9SSMCPWyqOX4LBp+MEh9U+H68W9TmC5wp6A98DQvS5r5RUJxEAlrNKbP ihONEyyBXDKm9NzjcAB29YJ2zaiNQpu8hFUk3R8e7Vp95/8ahksqbitpTHKLuGorN2Ew hXIncMIVMTX5pO7hVs3xL4n8276me9AKbUG2WmAS7kNwK0aAE19sZjSSjnVGZriPqfDc YSevNdVb8fB+SucULZG6MWe0UyiMDsfWiCp6EkZwSrva5gqBB3DRPlqDqmU9/gH6j8I1 82x+dE3SGhpumVd3v75bgK3xumBTu1Ck2IprWsML3MBagIIyTuGGSyN/PlH0j5gWtIwA I2pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=pZs4KHp1MVYvgB3W+tdcA1Yzc9C2QbLH7MVLPvla9cc=; b=cA7M2V8XiUT4hkxaPU0NWgBwf2E9Caydf1Xau7NQLJ/jOVqL6rlQZ61MiEmjbTk4p3 y+xvPYWrEtSKakxQ6KMDBHKNmkATf0XHLATU+73KpRQ3qy3neEoenclw4wJG1vl8YsUr uRMRocvxqwvz4V/QOqBQIbGsvf0yzKVCTUmBChDBWJrwLDd1PnL4PuLgT4ZzN2bHPPS8 FdLY6JqrvWmdj7GtPB6gfAI8hM5p1Q7NNvpy3UE2wH52jtomdQz7HQy6fZ7W64kSW5GM fREY0G1r1R/Nc7kqewZ/M9AyPpCuBQiGfBa2wY81Tvt8tPk8FbMOxLcFcR92FA9omjrb KW2g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r6si1489846edp.96.2021.07.27.09.22.57; Tue, 27 Jul 2021 09:22:58 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 489ED689F02; Tue, 27 Jul 2021 19:22:53 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mout-p-103.mailbox.org (mout-p-103.mailbox.org [80.241.56.161]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E012968085C for ; Tue, 27 Jul 2021 19:22:45 +0300 (EEST) Received: from smtp2.mailbox.org (smtp2.mailbox.org [IPv6:2001:67c:2050:105:465:1:2:0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-103.mailbox.org (Postfix) with ESMTPS id 4GZ2Bs21XgzQk9Z for ; Tue, 27 Jul 2021 18:22:45 +0200 (CEST) X-Virus-Scanned: amavisd-new at heinlein-support.de Received: from smtp2.mailbox.org ([80.241.60.241]) by gerste.heinlein-support.de (gerste.heinlein-support.de [91.198.250.173]) (amavisd-new, port 10030) with ESMTP id VL2w59b1V0Ms for ; Tue, 27 Jul 2021 18:22:41 +0200 (CEST) From: Gyan Doshi To: ffmpeg-devel@ffmpeg.org Date: Tue, 27 Jul 2021 21:52:26 +0530 Message-Id: <20210727162226.12968-1-ffmpeg@gyani.pro> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5018E1844 X-Rspamd-UID: 847af8 Subject: [FFmpeg-devel] [PATCH] avcodec/noise_bsf: add expr support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: CvSxIHkwIWeM --- doc/bitstream_filters.texi | 64 +++++++++++++--- libavcodec/noise_bsf.c | 151 +++++++++++++++++++++++++++++++++---- tests/fate/matroska.mak | 2 +- 3 files changed, 189 insertions(+), 28 deletions(-) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index d10842ae47..46e4869f80 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -534,20 +534,62 @@ container. Can be used for fuzzing or testing error resilience/concealment. Parameters: @table @option @item amount -A numeral string, whose value is related to how often output bytes will -be modified. Therefore, values below or equal to 0 are forbidden, and -the lower the more frequent bytes will be modified, with 1 meaning -every byte is modified. -@item dropamount -A numeral string, whose value is related to how often packets will be dropped. -Therefore, values below or equal to 0 are forbidden, and the lower the more -frequent packets will be dropped, with 1 meaning every packet is dropped. +Accepts an expression whose evaluation per-packet determines how often bytes in that +packet will be modified. A value below 0 will result in a variable frequency. +Default is 0 which results in no modification. However, if neither amount or drop is specified, +amount will be set to @var{-1}. See below for accepted variables. +@item drop, dropamount +Accepts an expression evaluated per-packet whose value determines whether that packet is dropped. +Evaluation to a positive value results in the packet being dropped. Evaluation to a negative +value results in a variable chance of it being dropped, roughly inverse in proportion to the magnitude +of the value. Default is 0 which results in no drops. See below for accepted variables. @end table -The following example applies the modification to every byte but does not drop -any packets. +Both @code{amount} and @code{drop} accept expressions containing the following variables: + +@table @samp +@item n +The index of the packet, starting from zero. +@item tb +The timebase for packet timestamps. +@item pts +Packet presentation timestamp. +@item dts +Packet decoding timestamp. +@item nopts +Constant representing AV_NOPTS_VALUE. +@item startpts +First non-AV_NOPTS_VALUE PTS seen in the stream. +@item startdts +First non-AV_NOPTS_VALUE DTS seen in the stream. +@item duration +@itemx d +Packet duration, in timebase units. +@item pos +Packet position in input; may be -1 when unknown or not set. +@item size +Packet size, in bytes. +@item key +Whether packet is marked as a keyframe. +@item state +A pseudo random integer, primarily derived from the content of packet payload. +@end table + +@subsection Examples +Apply modification to every byte but don't drop any packets. +@example +ffmpeg -i INPUT -c copy -bsf noise=1 output.mkv +@end example + +Drop every video packet not marked as a keyframe after timestamp 30s but do not +modify any of the remaining packets. +@example +ffmpeg -i INPUT -c copy -bsf:v noise=drop='gt(t\,30)*not(key)' output.mkv +@end example + +Drop one second of audio every 10 seconds and add some random noise to the rest. @example -ffmpeg -i INPUT -c copy -bsf noise[=1] output.mkv +ffmpeg -i INPUT -c copy -bsf:a noise=amount=-1:drop='between(mod(t\,10)\,9\,10)' output.mkv @end example @section null diff --git a/libavcodec/noise_bsf.c b/libavcodec/noise_bsf.c index 6ebd369633..37f08a0e08 100644 --- a/libavcodec/noise_bsf.c +++ b/libavcodec/noise_bsf.c @@ -23,55 +23,173 @@ #include "bsf.h" #include "bsf_internal.h" +#include "libavutil/avstring.h" #include "libavutil/log.h" #include "libavutil/opt.h" +#include "libavutil/eval.h" + +static const char *const var_names[] = { + "n", /// packet index, starting from zero + "tb", /// timebase + "pts", /// packet presentation timestamp + "dts", /// packet decoding timestamp + "nopts", /// AV_NOPTS_VALUE + "startpts", /// first seen non-AV_NOPTS_VALUE packet timestamp + "startdts", /// first seen non-AV_NOPTS_VALUE packet timestamp + "duration", "d", /// packet duration + "pos", /// original position of packet in its source + "size", /// packet size + "key" , /// packet keyframe flag + "state", /// random-ish state + NULL +}; + +enum var_name { + VAR_N, + VAR_TB, + VAR_PTS, + VAR_DTS, + VAR_NOPTS, + VAR_STARTPTS, + VAR_STARTDTS, + VAR_DURATION, VAR_D, + VAR_POS, + VAR_SIZE, + VAR_KEY, + VAR_STATE, + VAR_VARS_NB +}; typedef struct NoiseContext { const AVClass *class; - int amount; - int dropamount; + + char *amount_str; + char *drop_str; + + AVExpr *amount_pexpr; + AVExpr *drop_pexpr; + + double var_values[VAR_VARS_NB]; + unsigned int state; + unsigned int pkt_idx; } NoiseContext; -static int noise(AVBSFContext *ctx, AVPacket *pkt) +static int noise_init(AVBSFContext *ctx) { NoiseContext *s = ctx->priv_data; - int amount = s->amount > 0 ? s->amount : (s->state % 10001 + 1); - int i, ret; + int ret; - if (amount <= 0) - return AVERROR(EINVAL); + if (!s->amount_str) { + s->amount_str = !s->drop_str ? av_strdup("-1") : av_strdup("0"); + if (!s->amount_str) + return AVERROR(ENOMEM); + } + + ret = av_expr_parse(&s->amount_pexpr, s->amount_str, + var_names, NULL, NULL, NULL, NULL, 0, ctx); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Error in parsing expr for amount: %s\n", s->amount_str); + return ret; + } + + if (s->drop_str) { + ret = av_expr_parse(&s->drop_pexpr, s->drop_str, + var_names, NULL, NULL, NULL, NULL, 0, ctx); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Error in parsing expr for drop: %s\n", s->drop_str); + return ret; + } + } + + s->var_values[VAR_TB] = ctx->time_base_out.den ? av_q2d(ctx->time_base_out) : 0; + s->var_values[VAR_NOPTS] = AV_NOPTS_VALUE; + s->var_values[VAR_STARTPTS] = AV_NOPTS_VALUE; + s->var_values[VAR_STARTDTS] = AV_NOPTS_VALUE; + s->var_values[VAR_STATE] = 0; + + return 0; +} + +static int noise(AVBSFContext *ctx, AVPacket *pkt) +{ + NoiseContext *s = ctx->priv_data; + int i, ret, amount, drop; + double res; ret = ff_bsf_get_packet_ref(ctx, pkt); if (ret < 0) return ret; - if (s->dropamount > 0 && s->state % s->dropamount == 0) { - s->state++; + s->var_values[VAR_N] = s->pkt_idx++; + s->var_values[VAR_PTS] = pkt->pts; + s->var_values[VAR_DTS] = pkt->dts; + s->var_values[VAR_DURATION] = + s->var_values[VAR_D] = pkt->duration; + s->var_values[VAR_SIZE] = pkt->size; + s->var_values[VAR_KEY] = !!(pkt->flags & AV_PKT_FLAG_KEY); + s->var_values[VAR_POS] = pkt->pos; + + if (s->var_values[VAR_STARTPTS] == AV_NOPTS_VALUE) + s->var_values[VAR_STARTPTS] = pkt->pts; + + if (s->var_values[VAR_STARTDTS] == AV_NOPTS_VALUE) + s->var_values[VAR_STARTDTS] = pkt->dts; + + res = av_expr_eval(s->amount_pexpr, s->var_values, NULL); + + if (isnan(res)) + amount = 0; + else if (res < 0) + amount = (s->state % 10001 + 1); + else + amount = (int)res; + + if (s->drop_str) { + res = av_expr_eval(s->drop_pexpr, s->var_values, NULL); + + if (isnan(res)) + drop = 0; + else if (res < 0) + drop = !(s->state % FFABS((int)res)); + else + drop = !!res; + } + + av_log(ctx, AV_LOG_VERBOSE, "Stream #%d packet %d pts %"PRId64" - amount %d drop %d\n", + pkt->stream_index, (unsigned int)s->var_values[VAR_N], pkt->pts, amount, drop); + + if (s->drop_str && drop) { + s->var_values[VAR_STATE] = ++s->state; av_packet_unref(pkt); return AVERROR(EAGAIN); } - ret = av_packet_make_writable(pkt); - if (ret < 0) { - av_packet_unref(pkt); - return ret; + if (amount) { + ret = av_packet_make_writable(pkt); + if (ret < 0) { + av_packet_unref(pkt); + return ret; + } } for (i = 0; i < pkt->size; i++) { s->state += pkt->data[i] + 1; - if (s->state % amount == 0) + if (amount && s->state % amount == 0) pkt->data[i] = s->state; } + s->var_values[VAR_STATE] = s->state; + return 0; } #define OFFSET(x) offsetof(NoiseContext, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_BSF_PARAM) static const AVOption options[] = { - { "amount", NULL, OFFSET(amount), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, - { "dropamount", NULL, OFFSET(dropamount), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, + { "amount", NULL, OFFSET(amount_str), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, FLAGS }, + { "drop", NULL, OFFSET(drop_str), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, FLAGS }, + { "dropamount", NULL, OFFSET(drop_str), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, FLAGS }, { NULL }, }; @@ -86,5 +204,6 @@ const AVBitStreamFilter ff_noise_bsf = { .name = "noise", .priv_data_size = sizeof(NoiseContext), .priv_class = &noise_class, + .init = noise_init, .filter = noise, }; diff --git a/tests/fate/matroska.mak b/tests/fate/matroska.mak index ca7193a055..b57765280a 100644 --- a/tests/fate/matroska.mak +++ b/tests/fate/matroska.mak @@ -88,7 +88,7 @@ FATE_MATROSKA_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL MXF_DEMUXER \ MATROSKA_MUXER MATROSKA_DEMUXER \ FRAMECRC_MUXER PIPE_PROTOCOL) \ += fate-matroska-mastering-display-metadata -fate-matroska-mastering-display-metadata: CMD = transcode mxf $(TARGET_SAMPLES)/mxf/Meridian-Apple_ProResProxy-HDR10.mxf matroska "-map 0 -map 0:0 -c:v:0 copy -c:v:1 ffv1 -c:a:0 copy -bsf:a:0 noise=amount=3 -filter:a:1 aresample -c:a:1 pcm_s16be -bsf:a:1 noise=dropamount=4" "-map 0 -c copy" "" "-show_entries stream_side_data_list:stream=index,codec_name" +fate-matroska-mastering-display-metadata: CMD = transcode mxf $(TARGET_SAMPLES)/mxf/Meridian-Apple_ProResProxy-HDR10.mxf matroska "-map 0 -map 0:0 -c:v:0 copy -c:v:1 ffv1 -c:a:0 copy -bsf:a:0 noise=amount=3 -filter:a:1 aresample -c:a:1 pcm_s16be -bsf:a:1 noise=amount=-1:dropamount=-4" "-map 0 -c copy" "" "-show_entries stream_side_data_list:stream=index,codec_name" # This test tests remuxing annex B H.264 into Matroska. It also tests writing # the correct interlaced flags and overriding the sample aspect ratio, leading