From patchwork Tue Oct 25 09:13:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38975 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp2685364pzb; Tue, 25 Oct 2022 02:15:46 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7lNFu6UOqf0k5kwxeb58Nb+gx53pqAWNcGprt0KU8Gf8iyJQwqHpDiCY+WVdUZWP4Z1ufn X-Received: by 2002:a17:906:6791:b0:78d:4051:969f with SMTP id q17-20020a170906679100b0078d4051969fmr31901134ejp.171.1666689346161; Tue, 25 Oct 2022 02:15:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666689346; cv=none; d=google.com; s=arc-20160816; b=C/L1+EJgeFUGy3nw2xw0DaymGL7aoBNsLV5BRo5jvrpeYqyUZs9Y6oXD6ArMjCyHM8 PtAMAho200T0kHBzNX4wbzhgyNG+X6L5oAzKJ/AzRShTnX4/0k+ldNeT5DEB7jOyETLh hufd+1wYhUHmnhfbi4Xn1boW3/iHW8gNviZAxVWHVUrnMsNRGAthaGwceZZDE52Rjnq0 W3RP5Mn0LBzj+TrEvZ0NI+vXMtTObDJ72t8l4Kd9iehDSIY8ckqapiYgLWXoskdkodT1 o3ZWWiRYkd+S2mG2VaCwr43JxhbJelBW/m2xY2lY1nWK60K57jRRheW24Nd9Wu6frkyc eEaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=5JxqJaygJDPS4tlv1E08jC10QBA8M+yL0IH/Y6a5MnY=; b=B+sql9KB+lFaoq4MnMk6FDbd2JINltUrwXqmVFyDqMZ3vOtd7dMgeZc3jJxe5jOkwq Eij13bC8ywP+lBbYDv0spwWQ7VvPm8OOW2UUht20OmpEFKLCZhUz2TDF8bLZ0sHUGDt2 tg8AP63+X2lZQlUjB7AGctc9ZRh0cFCH2EJ6efcMbyL7id2zZHKZ2IvyZnS0LzM8/yYh ZSMi/9D15YagM3+9nh7fpKqxQPSA5+OqYJrhoXJwXWvy2TSpkz1jN0QAK7W1sS8+301Z nzMOAHskXLOXYkHygzzUYAb0OtxLlyz8+X3VfQ639Pu1/LzUMomQyLKM73mUsXHsOQzt LfaQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=DWn8W2vJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b9-20020a170906708900b0078d8614e809si1941591ejk.504.2022.10.25.02.15.45; Tue, 25 Oct 2022 02:15:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=DWn8W2vJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B9BB468BD38; Tue, 25 Oct 2022 12:14:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3340468BCCE for ; Tue, 25 Oct 2022 12:14:05 +0300 (EEST) Received: by mail-pj1-f52.google.com with SMTP id q9-20020a17090a178900b00212fe7c6bbeso5344041pja.4 for ; Tue, 25 Oct 2022 02:14:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=8mBq9SZ4u+dJ7tGA2IMtMgEb3DDSKovIkqH4+sl0N8M=; b=DWn8W2vJjsboKH5dlnmbvIOxiA7yPhGDgmy9qHeNBAnp2x6g0ZNqgpT0PED/C2oEyC t5BfQKJf0SeuEwRskS39d3khaRMhS6UANjJE2DlEyDc8hFx7rf5QKo01svRheag9fjSi j94U1O+MDDn+mCKnI//H8qgdGPeWDdthZUQmMZqJ1Qz1b3YiyzB54oAWTdZtDDjaSjNm /cTln9DqqXgS6GuCUFPKMXgUqFbATO4dY7zUROVi9qkX2ndVkI1pl8cF1Dp81yCgg+nP znRPP8SP2eEYfMiPJtVxEFjH7lXV3H6a5rvE2e2UBsXP7eHW8jK3HnHEHSI6Ywqq2HOq xQcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=8mBq9SZ4u+dJ7tGA2IMtMgEb3DDSKovIkqH4+sl0N8M=; b=zTOMS6Vprop2PJNPC+zwXkkUsoQI99vj6716ugdp/zX17arorcQdVzBzpD3ehtsN3x qSUz4heF9LcqHsqUHZ9wG0O+59HyM0Ej8yx67Xx+gZxAjx0MheJWHDW8NTdsDaicELDY DjXAHdoAWLxoAsUujMqihvbySjEPe++BS3c+5Q7yUMWjT6Un1C4yFiLaAvE+N7CMpnw1 CPvjTdfUYomEgXskg2wmdlb9J3CfHC+U083g5FY6gttnFo0fQ5h5xVEa2WpSPuzp0xXp cMAG+1SbP96/0NKJu1l+GN6iMNgodNxBMPHJjT4SJe2ZMABC55ZfRcydDcTq6QqjvbtQ 0iog== X-Gm-Message-State: ACrzQf2N57RqWJ9CfH7r7nq1CRm/Y0cjiQbyYP15N3WgZs2iNrNgGVc3 A2WqSmTDyyxQnzZfXlnvYwOqiJcgqdA= X-Received: by 2002:a17:90b:2306:b0:212:533:3188 with SMTP id mt6-20020a17090b230600b0021205333188mr31320261pjb.210.1666689243511; Tue, 25 Oct 2022 02:14:03 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id pj14-20020a17090b4f4e00b00212cf2fe8c3sm1284090pjb.1.2022.10.25.02.14.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Oct 2022 02:14:02 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: <066b754236ca1cad56b9fabff408271a347036e6.1666689226.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Tue, 25 Oct 2022 09:13:33 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v9 12/25] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: kAjTCViabRPN From: softworkz Signed-off-by: softworkz --- configure | 2 +- libavfilter/allfilters.c | 2 ++ libavfilter/buffersink.c | 54 ++++++++++++++++++++++++++++++ libavfilter/buffersink.h | 7 ++++ libavfilter/buffersrc.c | 72 ++++++++++++++++++++++++++++++++++++++++ libavfilter/buffersrc.h | 1 + 6 files changed, 137 insertions(+), 1 deletion(-) diff --git a/configure b/configure index 431fa5bf7a..b3f242469b 100755 --- a/configure +++ b/configure @@ -7954,7 +7954,7 @@ print_enabled_components(){ fi done if [ "$name" = "filter_list" ]; then - for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do + for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer vsink_buffer ssink_sbuffer; do printf " &ff_%s,\n" $c >> $TMPH done fi diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 119de40b25..173d85b982 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -577,8 +577,10 @@ extern const AVFilter ff_avsrc_movie; * being the same while having different 'types'). */ extern const AVFilter ff_asrc_abuffer; extern const AVFilter ff_vsrc_buffer; +extern const AVFilter ff_ssrc_sbuffer; extern const AVFilter ff_asink_abuffer; extern const AVFilter ff_vsink_buffer; +extern const AVFilter ff_ssink_sbuffer; extern const AVFilter ff_af_afifo; extern const AVFilter ff_vf_fifo; diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c index e269cf72d1..204e9bad6c 100644 --- a/libavfilter/buffersink.c +++ b/libavfilter/buffersink.c @@ -30,6 +30,8 @@ #include "libavutil/internal.h" #include "libavutil/opt.h" +#include "libavcodec/avcodec.h" + #define FF_INTERNAL_FIELDS 1 #include "framequeue.h" @@ -61,6 +63,10 @@ typedef struct BufferSinkContext { int *sample_rates; ///< list of accepted sample rates int sample_rates_size; + /* only used for subtitles */ + enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle types, must be terminated with -1 + int subtitle_types_size; + AVFrame *peeked_frame; } BufferSinkContext; @@ -372,6 +378,28 @@ static int asink_query_formats(AVFilterContext *ctx) return 0; } +static int ssink_query_formats(AVFilterContext *ctx) +{ + BufferSinkContext *buf = ctx->priv; + AVFilterFormats *formats = NULL; + unsigned i; + int ret; + + CHECK_LIST_SIZE(subtitle_types) + if (buf->subtitle_types_size) { + for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++) + if ((ret = ff_add_format(&formats, buf->subtitle_types[i])) < 0) + return ret; + if ((ret = ff_set_common_formats(ctx, formats)) < 0) + return ret; + } else { + if ((ret = ff_default_query_formats(ctx)) < 0) + return ret; + } + + return 0; +} + #define OFFSET(x) offsetof(BufferSinkContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM static const AVOption buffersink_options[] = { @@ -395,9 +423,16 @@ static const AVOption abuffersink_options[] = { { NULL }, }; #undef FLAGS +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM +static const AVOption sbuffersink_options[] = { + { "subtitle_types", "set the supported subtitle formats", OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS }, + { NULL }, +}; +#undef FLAGS AVFILTER_DEFINE_CLASS(buffersink); AVFILTER_DEFINE_CLASS(abuffersink); +AVFILTER_DEFINE_CLASS(sbuffersink); static const AVFilterPad avfilter_vsink_buffer_inputs[] = { { @@ -436,3 +471,22 @@ const AVFilter ff_asink_abuffer = { .outputs = NULL, FILTER_QUERY_FUNC(asink_query_formats), }; + +static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + }, +}; + +const AVFilter ff_ssink_sbuffer = { + .name = "sbuffersink", + .description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make them available to the end of the filter graph."), + .priv_class = &sbuffersink_class, + .priv_size = sizeof(BufferSinkContext), + .init = common_init, + .activate = activate, + FILTER_INPUTS(avfilter_ssink_sbuffer_inputs), + .outputs = NULL, + FILTER_QUERY_FUNC(ssink_query_formats), +}; diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h index 01e7c747d8..42c164e670 100644 --- a/libavfilter/buffersink.h +++ b/libavfilter/buffersink.h @@ -128,6 +128,13 @@ typedef struct AVABufferSinkParams { */ attribute_deprecated AVABufferSinkParams *av_abuffersink_params_alloc(void); + +/** + * Deprecated and unused struct to use for initializing an sbuffersink context. + */ +typedef struct AVSBufferSinkParams { + const int *subtitle_type; +} AVSBufferSinkParams; #endif /** diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c index ae8bba19b0..21465fe5e3 100644 --- a/libavfilter/buffersrc.c +++ b/libavfilter/buffersrc.c @@ -39,6 +39,7 @@ #include "formats.h" #include "internal.h" #include "video.h" +#include "libavcodec/avcodec.h" typedef struct BufferSourceContext { const AVClass *class; @@ -63,6 +64,9 @@ typedef struct BufferSourceContext { char *channel_layout_str; AVChannelLayout ch_layout; + /* subtitle only */ + enum AVSubtitleType subtitle_type; + int eof; } BufferSourceContext; @@ -143,6 +147,13 @@ FF_ENABLE_DEPRECATION_WARNINGS return ret; } break; + case AVMEDIA_TYPE_SUBTITLE: + s->subtitle_type = param->format; + if (param->width > 0) + s->w = param->width; + if (param->height > 0) + s->h = param->height; + break; default: return AVERROR_BUG; } @@ -224,6 +235,8 @@ FF_ENABLE_DEPRECATION_WARNINGS CHECK_AUDIO_PARAM_CHANGE(ctx, s, frame->sample_rate, frame->ch_layout, frame->format, frame->pts); break; + case AVMEDIA_TYPE_SUBTITLE: + break; default: return AVERROR(EINVAL); } @@ -303,6 +316,7 @@ unsigned av_buffersrc_get_nb_failed_requests(AVFilterContext *buffer_src) #define OFFSET(x) offsetof(BufferSourceContext, x) #define A AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_AUDIO_PARAM #define V AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +#define S AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM static const AVOption buffer_options[] = { { "width", NULL, OFFSET(w), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, @@ -332,6 +346,16 @@ static const AVOption abuffer_options[] = { AVFILTER_DEFINE_CLASS(abuffer); +static const AVOption sbuffer_options[] = { + { "time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, 0, INT_MAX, S }, + { "subtitle_type", NULL, OFFSET(subtitle_type), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, S }, + { "width", NULL, OFFSET(w), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, + { "height", NULL, OFFSET(h), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, V }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(sbuffer); + static av_cold int init_audio(AVFilterContext *ctx) { BufferSourceContext *s = ctx->priv; @@ -400,6 +424,21 @@ FF_ENABLE_DEPRECATION_WARNINGS return ret; } +static av_cold int init_subtitle(AVFilterContext *ctx) +{ + BufferSourceContext *c = ctx->priv; + + if (c->subtitle_type == AV_SUBTITLE_FMT_BITMAP) + av_log(ctx, AV_LOG_VERBOSE, "graphical subtitles - w:%d h:%d tb:%d/%d\n", + c->w, c->h, c->time_base.num, c->time_base.den); + else + av_log(ctx, AV_LOG_VERBOSE, "text subtitles - w:%d h:%d tb:%d/%d\n", + c->w, c->h, c->time_base.num, c->time_base.den); + + return 0; +} + + static av_cold void uninit(AVFilterContext *ctx) { BufferSourceContext *s = ctx->priv; @@ -433,6 +472,11 @@ static int query_formats(AVFilterContext *ctx) if ((ret = ff_set_common_channel_layouts(ctx, channel_layouts)) < 0) return ret; break; + case AVMEDIA_TYPE_SUBTITLE: + if ((ret = ff_add_format (&formats, c->subtitle_type)) < 0 || + (ret = ff_set_common_formats (ctx , formats )) < 0) + return ret; + break; default: return AVERROR(EINVAL); } @@ -463,6 +507,11 @@ static int config_props(AVFilterLink *link) return ret; } break; + case AVMEDIA_TYPE_SUBTITLE: + link->format = c->subtitle_type; + link->w = c->w; + link->h = c->h; + break; default: return AVERROR(EINVAL); } @@ -527,3 +576,26 @@ const AVFilter ff_asrc_abuffer = { FILTER_QUERY_FUNC(query_formats), .priv_class = &abuffer_class, }; + +static const AVFilterPad avfilter_ssrc_sbuffer_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_SUBTITLE, + .request_frame = request_frame, + .config_props = config_props, + }, +}; + +const AVFilter ff_ssrc_sbuffer = { + .name = "sbuffer", + .description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make them accessible to the filterchain."), + .priv_size = sizeof(BufferSourceContext), + + .init = init_subtitle, + .uninit = uninit, + + .inputs = NULL, + FILTER_OUTPUTS(avfilter_ssrc_sbuffer_outputs), + FILTER_QUERY_FUNC(query_formats), + .priv_class = &sbuffer_class, +}; diff --git a/libavfilter/buffersrc.h b/libavfilter/buffersrc.h index 3b248b37cd..45a657bbb6 100644 --- a/libavfilter/buffersrc.h +++ b/libavfilter/buffersrc.h @@ -74,6 +74,7 @@ typedef struct AVBufferSrcParameters { /** * video: the pixel format, value corresponds to enum AVPixelFormat * audio: the sample format, value corresponds to enum AVSampleFormat + * subtitles: the subtitle format, value corresponds to enum AVSubtitleType */ int format; /**