From patchwork Wed Apr 18 09:15:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ANURAG SINGH IIT BHU X-Patchwork-Id: 8484 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.85 with SMTP id c82csp2877976jad; Wed, 18 Apr 2018 02:22:37 -0700 (PDT) X-Google-Smtp-Source: AIpwx4+52fdXDlOlO8cdpzBOJCnhguSVYXwa0OCt9QvZdmSe9vER8HveV3JQF3JAZfBhCsqGcg4z X-Received: by 10.28.40.4 with SMTP id o4mr1219057wmo.44.1524043356950; Wed, 18 Apr 2018 02:22:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1524043356; cv=none; d=google.com; s=arc-20160816; b=WJV3nVU8yZkfdyTcPAIunlqy1Of/9z3GzD7EMENtpAQPnP7gwrw5nsPUdYdp4R87hb Ze5tP3d3wr7mmZ1IKzsg72fHQenstLc4CWu5tzDjxXU/NpxAwbSX2eCjAUDRqkV3blcm Bg8C6qlg4uQaskc936np72ZOSz1elftFsBUe+7PTJ85p0hPW03oB29llm/bu7DynpMx6 rPqbTM57dVM0GVX764AZN0RdaqvoEMIg0ih+k+P3tRqIQBipSLox4PIGHXB4YlBORZzM QSoQf1vTcCyHpRxuQRzfQn8HnC8hTzJT9VCgqEQMvcMoe1UJripe8OTwynQDWcIyZRRa 9VOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject:to :message-id:date:from:references:in-reply-to:mime-version :dkim-signature:delivered-to:arc-authentication-results; bh=FDN9YOjKLIP2EUIwyn+hvc7G3Rx7y/KGEo96dBhrh+8=; b=kV641zkxQ9895VfcFEkgHtVMbjvtkMa0ueVHUSy6VoAIPSaKLIjzUjQSLH/wt3vW7s yNVOKg1WzAIHuL9ULCW3zMQZELuFv/lVxIJGnA5F2z2hjRFXe3YM6FEZm1hGtb6FVFNC dmByKnCp0WYza8OSDLJQYp4gG2J04Yu8cxwEcRwvoM3hKDBRYxQSe9B29U9dP2W2SPyy 1BhNSIljExzeKTzFqLgMEdMta0zXB/7uxmNHTA4CbwkkIA3B1loLAuV04G6YW3Cr1dKE VOIDZCWm96s9j5hcYNHn5vJUBzZxYzVPQGzGJ50FSXPteR9ishVqA6Q4ENpd9w6Tcfk5 hW1g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@iitbhu.ac.in header.s=google header.b=npZ7pYmY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=iitbhu.ac.in Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i84si798737wmc.198.2018.04.18.02.22.36; Wed, 18 Apr 2018 02:22:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@iitbhu.ac.in header.s=google header.b=npZ7pYmY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=iitbhu.ac.in Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B452C680193; Wed, 18 Apr 2018 12:22:07 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-vk0-f46.google.com (mail-vk0-f46.google.com [209.85.213.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E2C36680D4E for ; Wed, 18 Apr 2018 12:22:00 +0300 (EEST) Received: by mail-vk0-f46.google.com with SMTP id v205so606858vkv.13 for ; Wed, 18 Apr 2018 02:22:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbhu.ac.in; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=sDJ3eX1RVetOg6WpyAHmzY3wkiHzU8OgIXgZtaIskHc=; b=npZ7pYmYEi+C2yK3UR892w4/2sAVi6tYFS/5YnrhFevyHmsslVVL7IOYouG/nR2wC3 vCbuvh0mofYQqnvkO3Q6czRF17hpPkvM9vKh5hPg0FT/sgA3W3wTzQ2qo55xnW2t0WoZ UWGLPIiLlKbbxCVk53B7A/0BsNkpAlyCUQDJE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=sDJ3eX1RVetOg6WpyAHmzY3wkiHzU8OgIXgZtaIskHc=; b=QTgMoLlqaxxgOthksVOodnwxwDZUD/uqIJlOG0ehtgM1m0uu3JG3W/SnHjZi11CYbi cJWj73m9oxpAkH09OzV9KMgoePwwdmlLOvwRKk1EAXANlSwsnZWTCJZ0cjVlDQ1/rcxY BYJQSkNlySHFww5IgMYKohU2Cu0lbe9frGTmLE9nWNlwuz7BUxSdGRA19VJtze7+Zv2B aL3UBN1bVop0VMI0Z9C+cw26PVnnrMM5jcTUltBzlMnwI/YdOsSYr1P6VRkfgrmkPgZI LptM1zax/0NhVJGDQZam1u7qfay13Ol9HvTnCI9Xssw0Dw/bnvmA3HxIj3pTjlslMNGb LhqQ== X-Gm-Message-State: ALQs6tB3Wjto7UDXCEohrPHUSU+vx2BfW1d1+NoFoek/9D4+wxihG4DQ qU5gJ/aSnrdQ8vUbVmkIe13ybtwIWGlT/T4ZRt8VdA== X-Received: by 10.31.148.135 with SMTP id w129mr746769vkd.7.1524042977290; Wed, 18 Apr 2018 02:16:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.201.14 with HTTP; Wed, 18 Apr 2018 02:15:36 -0700 (PDT) In-Reply-To: References: <20180412223455.GN20131@michaelspb> <20180415191917.GG20131@michaelspb> From: ANURAG SINGH IIT BHU Date: Wed, 18 Apr 2018 14:45:36 +0530 Message-ID: To: FFmpeg development discussions and patches X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: Re: [FFmpeg-devel] GSOC 2018 qualification task. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hello Sir, I have implemented the suggested changes, now the filter does not break builds, also now it works for all inputs and ffplay -f lavfi -i testsrc -vf hellosubs works. Sir I think passing the text using metadata to drawtext filter would not be an efficient way in terms of time as drawtext filter calls a number of functions which are not needed by the hellosubs filter, functions like expand_text(), expand_func() etc, i.e drawtext filter is of 1525 lines where as hellosubs is more than 1/3 rd times less in lines. and sir for realtime subtitles I think we should emphasise more on reducing the time taken by the filter to remove lag. Thanks and regards, Anurag Singh On Mon, Apr 16, 2018 at 11:57 AM, ANURAG SINGH IIT BHU < anurag.singh.phy15@iitbhu.ac.in> wrote: > > > > Hello sir, > > Okay I'll implement the suggested changes and make sure that the filter > does not break build without libfreetype. > > Thanks and regards > Anurag Singh. > > > On Mon, Apr 16, 2018 at 12:49 AM, Michael Niedermayer < > michael@niedermayer.cc> wrote: > >> On Sun, Apr 15, 2018 at 07:36:09PM +0530, ANURAG SINGH IIT BHU wrote: >> > Hello Sir, >> > >> > I have implemented the adviced changes for the hellosubs filter for the >> > qualification task which writes Hello World time on frames, now the >> filter >> > does not uses libavformat, and it uses libfreetype to draw over video >> > frames. I have attached the complete patch. >> > >> > libfretype and libfontconfig should be enabled to run the filter. >> > (libfontconfig if no font file is provided.) >> > >> > Command to run the filter >> > ffmpeg -i -vf hellosubs >> > >> > Thanks and regards, >> > Anurag Singh. >> > >> > >> > >> > >> > >> > ‌ >> > >> > On Fri, Apr 13, 2018 at 9:39 AM, ANURAG SINGH IIT BHU < >> > anurag.singh.phy15@iitbhu.ac.in> wrote: >> > >> > > Thank you sir, I'll implement the suggested reviews as soon as >> possible. >> > > >> > > >> > > >> > > >> > > ‌ >> > > >> > > On Fri, Apr 13, 2018 at 4:04 AM, Michael Niedermayer < >> > > michael@niedermayer.cc> wrote: >> > > >> > >> On Fri, Apr 13, 2018 at 02:13:53AM +0530, ANURAG SINGH IIT BHU wrote: >> > >> > Hello, >> > >> > I have implemented the reviews mentioned on previous patch, now >> there >> > >> is no >> > >> > need to provide any subtitle file to the filter, I am attaching the >> > >> > complete patch of the hellosubs filter. >> > >> > >> > >> > Command to run the filter >> > >> > ffmpeg -i -vf hellosubs= helloout.mp4 >> > >> > >> > >> > >> > >> > Thanks and regards, >> > >> > Anurag Singh. >> > >> > >> > >> > >> > >> > ‌ >> > >> > >> > >> > On Tue, Apr 10, 2018 at 4:55 AM, Rostislav Pehlivanov < >> > >> atomnuker@gmail.com> >> > >> > wrote: >> > >> > >> > >> > > On 9 April 2018 at 19:10, Paul B Mahol wrote: >> > >> > > >> > >> > > > On 4/9/18, Rostislav Pehlivanov wrote: >> > >> > > > > On 9 April 2018 at 03:59, ANURAG SINGH IIT BHU < >> > >> > > > > anurag.singh.phy15@iitbhu.ac.in> wrote: >> > >> > > > > >> > >> > > > >> This mail is regarding the qualification task assigned to >> me for >> > >> the >> > >> > > > >> GSOC project >> > >> > > > >> in FFmpeg for automatic real-time subtitle generation using >> > >> speech to >> > >> > > > text >> > >> > > > >> translation ML model. >> > >> > > > >> >> > >> > > > > >> > >> > > > > i really don't think lavfi is the correct place for such >> code, >> > >> nor that >> > >> > > > the >> > >> > > > > project's repo should contain such code at all. >> > >> > > > > This would need to be in another repo and a separate library. >> > >> > > > >> > >> > > > Why? Are you against ocr filter too? >> > >> > > > >> > >> > > >> > >> > > The OCR filter uses libtessract so I'm fine with it. Like I >> said, as >> > >> long >> > >> > > as the actual code to do it is in an external library I don't >> mind. >> > >> > > Mozilla recently released Deep Speech ( >> https://github.com/mozilla/ >> > >> > > DeepSpeech) >> > >> > > which does pretty much exactly speech to text and is considered >> to >> > >> have the >> > >> > > most accurate one out there. Someone just needs to convert the >> > >> tensorflow >> > >> > > code to something more usable. >> > >> > > _______________________________________________ >> > >> > > ffmpeg-devel mailing list >> > >> > > ffmpeg-devel@ffmpeg.org >> > >> > > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> > >> > > >> > >> >> > >> > Makefile | 1 >> > >> > allfilters.c | 1 >> > >> > vf_hellosubs.c | 513 ++++++++++++++++++++++++++++++ >> > >> +++++++++++++++++++++++++++ >> > >> > 3 files changed, 515 insertions(+) >> > >> > 2432f100fddb7ec84e771be8282d4b66e3d1f50a >> > >> 0001-avfilter-add-hellosubs-filter.patch >> > >> > From ac0e09d431ea68aebfaef6e2ed0b450e76d473d9 Mon Sep 17 00:00:00 >> 2001 >> > >> > From: ddosvulnerability >> > >> > Date: Thu, 12 Apr 2018 22:06:43 +0530 >> > >> > Subject: [PATCH] avfilter: add hellosubs filter. >> > >> > >> > >> > --- >> > >> > libavfilter/Makefile | 1 + >> > >> > libavfilter/allfilters.c | 1 + >> > >> > libavfilter/vf_hellosubs.c | 513 ++++++++++++++++++++++++++++++ >> > >> +++++++++++++++ >> > >> > 3 files changed, 515 insertions(+) >> > >> > create mode 100644 libavfilter/vf_hellosubs.c >> > >> > >> > >> > diff --git a/libavfilter/Makefile b/libavfilter/Makefile >> > >> > index a90ca30..770b1b5 100644 >> > >> > --- a/libavfilter/Makefile >> > >> > +++ b/libavfilter/Makefile >> > >> > @@ -331,6 +331,7 @@ OBJS-$(CONFIG_SSIM_FILTER) += >> > >> vf_ssim.o framesync.o >> > >> > OBJS-$(CONFIG_STEREO3D_FILTER) += vf_stereo3d.o >> > >> > OBJS-$(CONFIG_STREAMSELECT_FILTER) += f_streamselect.o >> > >> framesync.o >> > >> > OBJS-$(CONFIG_SUBTITLES_FILTER) += vf_subtitles.o >> > >> > +OBJS-$(CONFIG_HELLOSUBS_FILTER) += vf_hellosubs.o >> > >> > OBJS-$(CONFIG_SUPER2XSAI_FILTER) += vf_super2xsai.o >> > >> > OBJS-$(CONFIG_SWAPRECT_FILTER) += vf_swaprect.o >> > >> > OBJS-$(CONFIG_SWAPUV_FILTER) += vf_swapuv.o >> > >> > diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c >> > >> > index 6eac828..a008908 100644 >> > >> > --- a/libavfilter/allfilters.c >> > >> > +++ b/libavfilter/allfilters.c >> > >> > @@ -322,6 +322,7 @@ extern AVFilter ff_vf_ssim; >> > >> > extern AVFilter ff_vf_stereo3d; >> > >> > extern AVFilter ff_vf_streamselect; >> > >> > extern AVFilter ff_vf_subtitles; >> > >> > +extern AVFilter ff_vf_hellosubs; >> > >> > extern AVFilter ff_vf_super2xsai; >> > >> > extern AVFilter ff_vf_swaprect; >> > >> > extern AVFilter ff_vf_swapuv; >> > >> > diff --git a/libavfilter/vf_hellosubs.c >> b/libavfilter/vf_hellosubs.c >> > >> > new file mode 100644 >> > >> > index 0000000..b994050 >> > >> > --- /dev/null >> > >> > +++ b/libavfilter/vf_hellosubs.c >> > >> > @@ -0,0 +1,513 @@ >> > >> > +/* >> > >> > + * Copyright (c) 2011 Baptiste Coudurier >> > >> > + * Copyright (c) 2011 Stefano Sabatini >> > >> > + * Copyright (c) 2012 Clément Bœsch >> > >> > + * >> > >> > + * This file is part of FFmpeg. >> > >> > + * >> > >> > + * FFmpeg is free software; you can redistribute it and/or >> > >> > + * modify it under the terms of the GNU Lesser General Public >> > >> > + * License as published by the Free Software Foundation; either >> > >> > + * version 2.1 of the License, or (at your option) any later >> version. >> > >> > + * >> > >> > + * FFmpeg is distributed in the hope that it will be useful, >> > >> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of >> > >> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the >> GNU >> > >> > + * Lesser General Public License for more details. >> > >> > + * >> > >> > + * You should have received a copy of the GNU Lesser General >> Public >> > >> > + * License along with FFmpeg; if not, write to the Free Software >> > >> > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA >> > >> 02110-1301 USA >> > >> > + */ >> > >> > + >> > >> > +/** >> > >> > + * @file >> > >> > + * Libass hellosubs burning filter. >> > >> > + * >> > >> > + >> > >> > + */ >> > >> > + >> > >> > +#include >> > >> > + >> > >> > +#include "config.h" >> > >> > +#if CONFIG_SUBTITLES_FILTER >> > >> > +# include "libavcodec/avcodec.h" >> > >> > +# include "libavformat/avformat.h" >> > >> > +#endif >> > >> > +#include "libavutil/avstring.h" >> > >> > +#include "libavutil/imgutils.h" >> > >> > +#include "libavutil/opt.h" >> > >> > +#include "libavutil/parseutils.h" >> > >> > +#include "drawutils.h" >> > >> > +#include "avfilter.h" >> > >> > +#include "internal.h" >> > >> > +#include "formats.h" >> > >> > +#include "video.h" >> > >> > +#include >> > >> > +#include >> > >> > +#include >> > >> > + >> > >> > +typedef struct AssContext { >> > >> > + const AVClass *class; >> > >> > + ASS_Library *library; >> > >> > + ASS_Renderer *renderer; >> > >> > + ASS_Track *track; >> > >> > + char *filename; >> > >> > + char *fontsdir; >> > >> > + char *charenc; >> > >> > + char *force_style; >> > >> > + int stream_index; >> > >> > + int alpha; >> > >> > + uint8_t rgba_map[4]; >> > >> > + int pix_step[4]; ///< steps per pixel for each >> plane of >> > >> the main output >> > >> > + int original_w, original_h; >> > >> > + int shaping; >> > >> > + FFDrawContext draw; >> > >> > +} AssContext; >> > >> > + >> > >> > +#define OFFSET(x) offsetof(AssContext, x) >> > >> > +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM >> > >> > + >> > >> > +#define COMMON_OPTIONS \ >> > >> > + {"filename", "set the filename of file to read", >> > >> OFFSET(filename), AV_OPT_TYPE_STRING, {.str = >> NULL}, >> > >> CHAR_MIN, CHAR_MAX, FLAGS }, \ >> > >> > + {"f", "set the filename of file to read", >> > >> OFFSET(filename), AV_OPT_TYPE_STRING, {.str = >> NULL}, >> > >> CHAR_MIN, CHAR_MAX, FLAGS }, \ >> > >> > + {"original_size", "set the size of the original video (used >> to >> > >> scale fonts)", OFFSET(original_w), AV_OPT_TYPE_IMAGE_SIZE, {.str = >> NULL}, >> > >> CHAR_MIN, CHAR_MAX, FLAGS }, \ >> > >> > + {"fontsdir", "set the directory containing the fonts to >> > >> read", OFFSET(fontsdir), AV_OPT_TYPE_STRING, {.str = >> > >> NULL}, CHAR_MIN, CHAR_MAX, FLAGS }, \ >> > >> > + {"alpha", "enable processing of alpha channel", >> > >> OFFSET(alpha), AV_OPT_TYPE_BOOL, {.i64 = 0 >> }, >> > >> 0, 1, FLAGS }, \ >> > >> > + >> > >> > +/* libass supports a log level ranging from 0 to 7 */ >> > >> > +static const int ass_libavfilter_log_level_map[] = { >> > >> > + [0] = AV_LOG_FATAL, /* MSGL_FATAL */ >> > >> > + [1] = AV_LOG_ERROR, /* MSGL_ERR */ >> > >> > + [2] = AV_LOG_WARNING, /* MSGL_WARN */ >> > >> > + [3] = AV_LOG_WARNING, /* */ >> > >> > + [4] = AV_LOG_INFO, /* MSGL_INFO */ >> > >> > + [5] = AV_LOG_INFO, /* */ >> > >> > + [6] = AV_LOG_VERBOSE, /* MSGL_V */ >> > >> > + [7] = AV_LOG_DEBUG, /* MSGL_DBG2 */ >> > >> > +}; >> > >> > + >> > >> > +static void ass_log(int ass_level, const char *fmt, va_list args, >> void >> > >> *ctx) >> > >> > +{ >> > >> > + const int ass_level_clip = av_clip(ass_level, 0, >> > >> > + FF_ARRAY_ELEMS(ass_libavfilter_log_level_map) - 1); >> > >> > + const int level = ass_libavfilter_log_level_map[ >> ass_level_clip]; >> > >> > + >> > >> > + av_vlog(ctx, level, fmt, args); >> > >> > + av_log(ctx, level, "\n"); >> > >> > +} >> > >> > + >> > >> > +static av_cold int init(AVFilterContext *ctx) >> > >> > +{ >> > >> > + AssContext *ass = ctx->priv; >> > >> > + >> > >> > + if (!ass->filename) { >> > >> > + av_log(ctx, AV_LOG_ERROR, "No filename provided!\n"); >> > >> > + return AVERROR(EINVAL); >> > >> > + } >> > >> > + >> > >> > + ass->library = ass_library_init(); >> > >> > + if (!ass->library) { >> > >> > + av_log(ctx, AV_LOG_ERROR, "Could not initialize >> libass.\n"); >> > >> > + return AVERROR(EINVAL); >> > >> > + } >> > >> > + ass_set_message_cb(ass->library, ass_log, ctx); >> > >> > + >> > >> > + ass_set_fonts_dir(ass->library, ass->fontsdir); >> > >> > + >> > >> > + ass->renderer = ass_renderer_init(ass->library); >> > >> > + if (!ass->renderer) { >> > >> > + av_log(ctx, AV_LOG_ERROR, "Could not initialize libass >> > >> renderer.\n"); >> > >> > + return AVERROR(EINVAL); >> > >> > + } >> > >> > + >> > >> > + return 0; >> > >> > +} >> > >> > + >> > >> > +static av_cold void uninit(AVFilterContext *ctx) >> > >> > +{ >> > >> > + AssContext *ass = ctx->priv; >> > >> > + >> > >> > + if (ass->track) >> > >> > + ass_free_track(ass->track); >> > >> > + if (ass->renderer) >> > >> > + ass_renderer_done(ass->renderer); >> > >> > + if (ass->library) >> > >> > + ass_library_done(ass->library); >> > >> > +} >> > >> > + >> > >> > +static int query_formats(AVFilterContext *ctx) >> > >> > +{ >> > >> > + return ff_set_common_formats(ctx, >> ff_draw_supported_pixel_format >> > >> s(0)); >> > >> > +} >> > >> > + >> > >> > +static int config_input(AVFilterLink *inlink) >> > >> > +{ >> > >> > + AssContext *ass = inlink->dst->priv; >> > >> > + >> > >> > + ff_draw_init(&ass->draw, inlink->format, ass->alpha ? >> > >> FF_DRAW_PROCESS_ALPHA : 0); >> > >> > + >> > >> > + ass_set_frame_size (ass->renderer, inlink->w, inlink->h); >> > >> > + if (ass->original_w && ass->original_h) >> > >> > + ass_set_aspect_ratio(ass->renderer, (double)inlink->w / >> > >> inlink->h, >> > >> > + (double)ass->original_w / >> > >> ass->original_h); >> > >> > + if (ass->shaping != -1) >> > >> > + ass_set_shaper(ass->renderer, ass->shaping); >> > >> > + >> > >> > + return 0; >> > >> > +} >> > >> > + >> > >> > +/* libass stores an RGBA color in the format RRGGBBTT, where TT >> is the >> > >> transparency level */ >> > >> > +#define AR(c) ( (c)>>24) >> > >> > +#define AG(c) (((c)>>16)&0xFF) >> > >> > +#define AB(c) (((c)>>8) &0xFF) >> > >> > +#define AA(c) ((0xFF-(c)) &0xFF) >> > >> > + >> > >> > +static void overlay_ass_image(AssContext *ass, AVFrame *picref, >> > >> > + const ASS_Image *image) >> > >> > +{ >> > >> > + for (; image; image = image->next) { >> > >> > + uint8_t rgba_color[] = {AR(image->color), >> AG(image->color), >> > >> AB(image->color), AA(image->color)}; >> > >> > + FFDrawColor color; >> > >> > + ff_draw_color(&ass->draw, &color, rgba_color); >> > >> > + ff_blend_mask(&ass->draw, &color, >> > >> > + picref->data, picref->linesize, >> > >> > + picref->width, picref->height, >> > >> > + image->bitmap, image->stride, image->w, >> image->h, >> > >> > + 3, 0, image->dst_x, image->dst_y); >> > >> > + } >> > >> > +} >> > >> > + >> > >> > +static int filter_frame(AVFilterLink *inlink, AVFrame *picref) >> > >> > +{ >> > >> > + AVFilterContext *ctx = inlink->dst; >> > >> > + AVFilterLink *outlink = ctx->outputs[0]; >> > >> > + AssContext *ass = ctx->priv; >> > >> > + int detect_change = 0; >> > >> > + double time_ms = picref->pts * av_q2d(inlink->time_base) * >> 1000; >> > >> > + ASS_Image *image = ass_render_frame(ass->renderer, >> ass->track, >> > >> > + time_ms, &detect_change); >> > >> > + >> > >> > + if (detect_change) >> > >> > + av_log(ctx, AV_LOG_DEBUG, "Change happened at time >> ms:%f\n", >> > >> time_ms); >> > >> > + >> > >> > + overlay_ass_image(ass, picref, image); >> > >> > + >> > >> > + return ff_filter_frame(outlink, picref); >> > >> > +} >> > >> > + >> > >> > +static const AVFilterPad ass_inputs[] = { >> > >> > + { >> > >> > + .name = "default", >> > >> > + .type = AVMEDIA_TYPE_VIDEO, >> > >> > + .filter_frame = filter_frame, >> > >> > + .config_props = config_input, >> > >> > + .needs_writable = 1, >> > >> > + }, >> > >> > + { NULL } >> > >> > +}; >> > >> > + >> > >> > +static const AVFilterPad ass_outputs[] = { >> > >> > + { >> > >> > + .name = "default", >> > >> > + .type = AVMEDIA_TYPE_VIDEO, >> > >> > + }, >> > >> > + { NULL } >> > >> > +}; >> > >> > + >> > >> > + >> > >> > + >> > >> > + >> > >> > + >> > >> > +static const AVOption hellosubs_options[] = { >> > >> > + COMMON_OPTIONS >> > >> > + {"charenc", "set input character encoding", >> OFFSET(charenc), >> > >> AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, >> > >> > + {"stream_index", "set stream index", >> > >> OFFSET(stream_index), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, >> > >> INT_MAX, FLAGS}, >> > >> > + {"si", "set stream index", >> > >> OFFSET(stream_index), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, >> > >> INT_MAX, FLAGS}, >> > >> > + {"force_style", "force subtitle style", >> > >> OFFSET(force_style), AV_OPT_TYPE_STRING, {.str = NULL}, CHAR_MIN, >> > >> CHAR_MAX, FLAGS}, >> > >> > + {NULL}, >> > >> > +}; >> > >> > + >> > >> > +static const char * const font_mimetypes[] = { >> > >> > + "application/x-truetype-font", >> > >> > + "application/vnd.ms-opentype", >> > >> > + "application/x-font-ttf", >> > >> > + NULL >> > >> > +}; >> > >> > + >> > >> > +static int attachment_is_font(AVStream * st) >> > >> > +{ >> > >> > + const AVDictionaryEntry *tag = NULL; >> > >> > + int n; >> > >> > + >> > >> > + tag = av_dict_get(st->metadata, "mimetype", NULL, >> > >> AV_DICT_MATCH_CASE); >> > >> > + >> > >> > + if (tag) { >> > >> > + for (n = 0; font_mimetypes[n]; n++) { >> > >> > + if (av_strcasecmp(font_mimetypes[n], tag->value) == >> 0) >> > >> > + return 1; >> > >> > + } >> > >> > + } >> > >> > + return 0; >> > >> > +} >> > >> > + >> > >> > +AVFILTER_DEFINE_CLASS(hellosubs); >> > >> > + >> > >> > +static av_cold int init_hellosubs(AVFilterContext *ctx) >> > >> > +{ >> > >> > + int j, ret, sid;long int z=0;int t1=0; >> > >> > + int k = 0; >> > >> > + AVDictionary *codec_opts = NULL; >> > >> > + AVFormatContext *fmt = NULL; >> > >> > + AVCodecContext *dec_ctx = NULL; >> > >> > + AVCodec *dec = NULL; >> > >> > + const AVCodecDescriptor *dec_desc; >> > >> > + AVStream *st; >> > >> > + AVPacket pkt; >> > >> > + AssContext *ass = ctx->priv; >> > >> >> > >> > + FILE *file; >> > >> > + if ((file = fopen("hello.srt", "r"))) >> > >> >> > >> there is no need for accessing an external file for the task of >> > >> drawing a line of text. >> > >> >> > >> >> > >> > + { >> > >> > + fclose(file); >> > >> > + >> > >> > + } >> > >> > + else >> > >> > + { >> > >> > + FILE * fp; >> > >> > + fp = fopen ("hello.srt","w"); >> > >> >> > >> thats even more true for writing such file. >> > >> It also would not work predictable with multiple filters >> > >> >> > >> >> > >> > + fprintf (fp, "1\n"); >> > >> > + fprintf (fp, "00:00:05,615 --> 00:00:08,083\n"); >> > >> > + fprintf (fp, "%s",ass->filename); >> > >> > + fclose (fp); >> > >> > + >> > >> > + char cmd[300]; >> > >> > + strcpy(cmd,"ffmpeg -i "); >> > >> > + strcat(cmd,ass->filename); >> > >> > + char fn[200]; >> > >> > + strcpy(fn,ass->filename); >> > >> > + strcat(cmd," -vf hellosubs=hello.srt helloout"); >> > >> > + int m=0; >> > >> > + for(int w=(strlen(fn)-1);w>=0;w--) >> > >> > + {if (fn[w]=='.') >> > >> > + {m=w; >> > >> > + break;}} >> > >> > + char join[5]; >> > >> > + for(int loc=m;loc> > >> > + join[loc-m]=fn[loc]; >> > >> > + char rem[100]; >> > >> > + char join1[100]; >> > >> > + strcpy(join1,join); >> > >> > + strcpy(rem,"helloout"); >> > >> > + strcat(rem,join1); >> > >> > + remove(rem); >> > >> > + >> > >> > + strcat(cmd,join); >> > >> > + system(cmd); >> > >> > + remove("hello.srt"); >> > >> > + >> > >> > +exit(0); >> > >> >> > >> also a filter cannot call exit(), in fact a library like libavfilter >> must >> > >> not >> > >> call exit() >> > >> >> > >> >> > >> > +} >> > >> > + >> > >> > + /* Init libass */ >> > >> > + ret = init(ctx); >> > >> > + if (ret < 0) >> > >> > + return ret; >> > >> > + ass->track = ass_new_track(ass->library); >> > >> > + if (!ass->track) { >> > >> > + av_log(ctx, AV_LOG_ERROR, "Could not create a libass >> track\n"); >> > >> > + return AVERROR(EINVAL); >> > >> > + } >> > >> > + >> > >> > + >> > >> >> > >> > + ret = avformat_open_input(&fmt, ass->filename, NULL, NULL); >> > >> > + if (ret < 0) { >> > >> > + av_log(ctx, AV_LOG_ERROR, "Unable to open %s\n", >> > >> ass->filename); >> > >> > + >> > >> > + } >> > >> >> > >> also no function from libavformat is needed, this filter should draw >> a >> > >> line of >> > >> text, not demux a file. >> > >> You maybe misinterpredted my previous review. All unneeded code like >> > >> every bit of >> > >> libavformat use must be removed. >> > >> >> > >> You seem to be trying to workaround what i suggest not actually >> solve the >> > >> issues >> > >> raised. >> > >> Like writing a file to replace the impossibility of accessing some >> input >> > >> file >> > >> directly. There really is no file and none can be written. >> > >> >> > >> The goal of this filter was to create subtitle packets/frames and >> pass >> > >> them on. >> > >> As this turned out too hard in the time available. The simpler goal >> now >> > >> is to >> > >> draw that text on a video frame. >> > >> >> > >> The filter gets video frames on its input and it passes them on to >> the >> > >> output. >> > >> In there it should write that Hello world text with the advancing >> number >> > >> onto >> > >> it >> > >> For this there is no need to access any files, or use any demuxers. >> > >> you can use the libass code from the subtitle filter as you do but >> that >> > >> code >> > >> uses a external subtitle file. You have to change this so it no >> longer >> > >> uses a >> > >> external file or demuxes this with libavformat. These steps are not >> needed >> > >> and are incorrect for this task >> > >> >> > >> i suggest you remove "include "libavformat *" that way you will see >> > >> exactly what must be removed >> > >> and this should make the code simpler, it just isnt needed to have >> this >> > >> baggage between the avcodec/libass and what you want to draw >> > >> >> > >> the libavformat code is there to read a subtitle file. >> > >> There is no subtitle file. The filter should just draw a line saying >> > >> hello world with a number. >> > >> >> > >> >> > >> [...] >> > >> >> > >> -- >> > >> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC7 >> 87040B0FAB >> > >> >> > >> Dictatorship: All citizens are under surveillance, all their steps >> and >> > >> actions recorded, for the politicians to enforce control. >> > >> Democracy: All politicians are under surveillance, all their steps >> and >> > >> actions recorded, for the citizens to enforce control. >> > >> >> > >> _______________________________________________ >> > >> ffmpeg-devel mailing list >> > >> ffmpeg-devel@ffmpeg.org >> > >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> > >> >> > >> >> > > >> >> > Makefile | 1 >> > allfilters.c | 1 >> > vf_hellosubs.c | 960 ++++++++++++++++++++++++++++++ >> +++++++++++++++++++++++++++ >> > 3 files changed, 962 insertions(+) >> > 4232d543120e2ef383c33a584ddb06fe5f24ac11 >> 0001-avfilter-add-hellosubs-filter.patch >> > From 7004d913495a0274bc13d6e79aea18249e057698 Mon Sep 17 00:00:00 2001 >> > From: ddosvulnerability >> > Date: Sun, 15 Apr 2018 19:15:35 +0530 >> > Subject: [PATCH] avfilter: add hellosubs filter. >> > >> > --- >> > libavfilter/Makefile | 1 + >> > libavfilter/allfilters.c | 1 + >> > libavfilter/vf_hellosubs.c | 960 ++++++++++++++++++++++++++++++ >> +++++++++++++++ >> > 3 files changed, 962 insertions(+) >> > create mode 100644 libavfilter/vf_hellosubs.c >> > >> > diff --git a/libavfilter/Makefile b/libavfilter/Makefile >> > index 3a9fb02..531faad 100644 >> > --- a/libavfilter/Makefile >> > +++ b/libavfilter/Makefile >> > @@ -216,6 +216,7 @@ OBJS-$(CONFIG_GBLUR_FILTER) += >> vf_gblur.o >> > OBJS-$(CONFIG_GEQ_FILTER) += vf_geq.o >> > OBJS-$(CONFIG_GRADFUN_FILTER) += vf_gradfun.o >> > OBJS-$(CONFIG_HALDCLUT_FILTER) += vf_lut3d.o framesync.o >> > +OBJS-$(CONFIG_HELLOSUBS_FILTER) += vf_hellosubs.o >> > OBJS-$(CONFIG_HFLIP_FILTER) += vf_hflip.o >> > OBJS-$(CONFIG_HISTEQ_FILTER) += vf_histeq.o >> > OBJS-$(CONFIG_HISTOGRAM_FILTER) += vf_histogram.o >> > diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c >> > index 68b2992..bde8a2e 100644 >> > --- a/libavfilter/allfilters.c >> > +++ b/libavfilter/allfilters.c >> > @@ -208,6 +208,7 @@ extern AVFilter ff_vf_gblur; >> > extern AVFilter ff_vf_geq; >> > extern AVFilter ff_vf_gradfun; >> > extern AVFilter ff_vf_haldclut; >> > +extern AVFilter ff_vf_hellosubs; >> > extern AVFilter ff_vf_hflip; >> > extern AVFilter ff_vf_histeq; >> > extern AVFilter ff_vf_histogram; >> > diff --git a/libavfilter/vf_hellosubs.c b/libavfilter/vf_hellosubs.c >> > new file mode 100644 >> > index 0000000..65fa18b >> > --- /dev/null >> > +++ b/libavfilter/vf_hellosubs.c >> > @@ -0,0 +1,960 @@ >> > +/* >> > + * This file is part of FFmpeg. >> > + * >> > + * FFmpeg is free software; you can redistribute it and/or >> > + * modify it under the terms of the GNU Lesser General Public >> > + * License as published by the Free Software Foundation; either >> > + * version 2.1 of the License, or (at your option) any later version. >> > + * >> > + * FFmpeg is distributed in the hope that it will be useful, >> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of >> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >> > + * Lesser General Public License for more details. >> > + * >> > + * You should have received a copy of the GNU Lesser General Public >> > + * License along with FFmpeg; if not, write to the Free Software >> > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA >> 02110-1301 USA >> > + */ >> > + >> > +/** >> > + * @file >> > + * Libfreetype subtitles burning filter. >> >> > + * @see{http://www.matroska.org/technical/specs/subtitles/ssa.html} >> >> This is about SSA, the filter now is about libfreetype, this doesnt fit >> >> >> > + */ >> > + >> > +#include "config.h" >> > +#include >> > +#if HAVE_SYS_TIME_H >> > +#include >> > +#endif >> > +#include >> > +#include >> > +#include >> > +#if HAVE_UNISTD_H >> > +#include >> > +#endif >> > +#include >> > +#if CONFIG_LIBFONTCONFIG >> > +#include >> > +#endif >> > +#include "libavutil/avstring.h" >> > +#include "libavutil/bprint.h" >> > +#include "libavutil/common.h" >> > +#include "libavutil/file.h" >> > +#include "libavutil/eval.h" >> > +#include "libavutil/opt.h" >> > +#include "libavutil/random_seed.h" >> > +#include "libavutil/parseutils.h" >> > +#include "libavutil/timecode.h" >> > +#include "libavutil/time_internal.h" >> > +#include "libavutil/tree.h" >> > +#include "libavutil/lfg.h" >> > +#include "avfilter.h" >> > +#include "drawutils.h" >> > +#include "formats.h" >> > +#include "internal.h" >> > +#include "video.h" >> > +#include >> > +#include FT_FREETYPE_H >> > +#include FT_GLYPH_H >> > +#include FT_STROKER_H >> >> as you are copying the code from the drawtext filter, you also have to >> include the copyright / author information from the drawtext filter. >> >> also it doesnt work with all inputs, for example this doesnt work: >> >> ./ffplay -f lavfi -i testsrc -vf hellosubs >> >> also build fails without libfreetype. >> The original code copied does not break build without libfreetype >> ive previously pointed to this already and that you are missing >> dependancy stuff in configure >> >> and duplicating all of the drawtext filter is suboptimal >> it would be much cleaner if the hellosubs filter would send >> the to be drawn text as metadata to a subsequent drawtext filter >> >> you can look at libavfilter/vf_find_rect.c and libavfilter/vf_cover_rect.c >> which also communicate via metadata >> >> Thanks >> >> [...] >> >> -- >> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB >> >> Its not that you shouldnt use gotos but rather that you should write >> readable code and code with gotos often but not always is less readable >> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> > From 8a552f950aae0b71ad3fbf6be118ad166aab9b44 Mon Sep 17 00:00:00 2001 From: ddosvulnerability Date: Wed, 18 Apr 2018 14:21:30 +0530 Subject: [PATCH] avfilter: add hellosubs filter. --- configure | 4 + libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_hellosubs.c | 960 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 966 insertions(+) create mode 100644 libavfilter/vf_hellosubs.c diff --git a/configure b/configure index 535a59b..86ebe07 100755 --- a/configure +++ b/configure @@ -231,7 +231,9 @@ External library support: --enable-libfdk-aac enable AAC de/encoding via libfdk-aac [no] --enable-libflite enable flite (voice synthesis) support via libflite [no] --enable-libfontconfig enable libfontconfig, useful for drawtext filter [no] + --enable-libfontconfig enable libfontconfig, useful for hellosubs filter [no] --enable-libfreetype enable libfreetype, needed for drawtext filter [no] + --enable-libfreetype enable libfreetype, needed for hellosubs filter [no] --enable-libfribidi enable libfribidi, improves drawtext filter [no] --enable-libgme enable Game Music Emu via libgme [no] --enable-libgsm enable GSM de/encoding via libgsm [no] @@ -3315,7 +3317,9 @@ delogo_filter_deps="gpl" denoise_vaapi_filter_deps="vaapi VAProcPipelineParameterBuffer" deshake_filter_select="pixelutils" drawtext_filter_deps="libfreetype" +hellosubs_filter_deps="libfreetype" drawtext_filter_suggest="libfontconfig libfribidi" +hellosubs_filter_suggest="libfontconfig" elbg_filter_deps="avcodec" eq_filter_deps="gpl" fftfilt_filter_deps="avcodec" diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 7fc3de3..3903136 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -192,6 +192,7 @@ OBJS-$(CONFIG_DRAWBOX_FILTER) += vf_drawbox.o OBJS-$(CONFIG_DRAWGRAPH_FILTER) += f_drawgraph.o OBJS-$(CONFIG_DRAWGRID_FILTER) += vf_drawbox.o OBJS-$(CONFIG_DRAWTEXT_FILTER) += vf_drawtext.o +OBJS-$(CONFIG_HELLOSUBS_FILTER) += vf_hellosubs.o OBJS-$(CONFIG_EDGEDETECT_FILTER) += vf_edgedetect.o OBJS-$(CONFIG_ELBG_FILTER) += vf_elbg.o OBJS-$(CONFIG_ENTROPY_FILTER) += vf_entropy.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index bd55463..a3c6211 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -184,6 +184,7 @@ extern AVFilter ff_vf_drawbox; extern AVFilter ff_vf_drawgraph; extern AVFilter ff_vf_drawgrid; extern AVFilter ff_vf_drawtext; +extern AVFilter ff_vf_hellosubs; extern AVFilter ff_vf_edgedetect; extern AVFilter ff_vf_elbg; extern AVFilter ff_vf_entropy; diff --git a/libavfilter/vf_hellosubs.c b/libavfilter/vf_hellosubs.c new file mode 100644 index 0000000..a2d5074 --- /dev/null +++ b/libavfilter/vf_hellosubs.c @@ -0,0 +1,960 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Libfreetype subtitles burning filter. + * @see{http://www.matroska.org/technical/specs/subtitles/ssa.html} + */ + +#include "config.h" +#include +#if HAVE_SYS_TIME_H +#include +#endif +#include +#include +#include +#if HAVE_UNISTD_H +#include +#endif +#include +#if CONFIG_LIBFONTCONFIG +#include +#endif +#include "libavutil/avstring.h" +#include "libavutil/bprint.h" +#include "libavutil/common.h" +#include "libavutil/file.h" +#include "libavutil/eval.h" +#include "libavutil/opt.h" +#include "libavutil/random_seed.h" +#include "libavutil/parseutils.h" +#include "libavutil/timecode.h" +#include "libavutil/time_internal.h" +#include "libavutil/tree.h" +#include "libavutil/lfg.h" +#include "avfilter.h" +#include "drawutils.h" +#include "formats.h" +#include "internal.h" +#include "video.h" +#include +#include FT_FREETYPE_H +#include FT_GLYPH_H +#include FT_STROKER_H + +static const char *const var_names[] = { + "dar", + "hsub", "vsub", + "line_h", "lh", ///< line height, same as max_glyph_h + "main_h", "h", "H", ///< height of the input video + "main_w", "w", "W", ///< width of the input video + "max_glyph_a", "ascent", ///< max glyph ascent + "max_glyph_d", "descent", ///< min glyph descent + "max_glyph_h", ///< max glyph height + "max_glyph_w", ///< max glyph width + "n", ///< number of frame + "sar", + "t", ///< timestamp expressed in seconds + "text_h", "th", ///< height of the rendered text + "text_w", "tw", ///< width of the rendered text + "x", + "y", + "pict_type", + NULL +}; + +static const char *const fun2_names[] = { + "rand" +}; + +static double drand(void *opaque, double min, double max) +{ + return min + (max-min) / UINT_MAX * av_lfg_get(opaque); +} + +typedef double (*eval_func2)(void *, double a, double b); + +static const eval_func2 fun2[] = { + drand, + NULL +}; + +enum var_name { + VAR_DAR, + VAR_HSUB, VAR_VSUB, + VAR_LINE_H, VAR_LH, + VAR_MAIN_H, VAR_h, VAR_H, + VAR_MAIN_W, VAR_w, VAR_W, + VAR_MAX_GLYPH_A, VAR_ASCENT, + VAR_MAX_GLYPH_D, VAR_DESCENT, + VAR_MAX_GLYPH_H, + VAR_MAX_GLYPH_W, + VAR_N, + VAR_SAR, + VAR_T, + VAR_TEXT_H, VAR_TH, + VAR_TEXT_W, VAR_TW, + VAR_X, + VAR_Y, + VAR_PICT_TYPE, + VAR_VARS_NB +}; + + + +typedef struct hellosubsContext { + const AVClass *class; + + int reinit; ///< tells if the filter is being reinited +#if CONFIG_LIBFONTCONFIG + uint8_t *font; ///< font to be used +#endif + uint8_t *fontfile; ///< font to be used + uint8_t *text; ///< text to be drawn + AVBPrint expanded_text; ///< used to contain the expanded text + uint8_t *fontcolor_expr; ///< fontcolor expression to evaluate + AVBPrint expanded_fontcolor; ///< used to contain the expanded fontcolor spec + int ft_load_flags; ///< flags used for loading fonts, see FT_LOAD_* + FT_Vector *positions; ///< positions for each element in the text + size_t nb_positions; ///< number of elements of positions array + int x; ///< x position to start drawing text + int y; ///< y position to start drawing text + int max_glyph_w; ///< max glyph width + int max_glyph_h; ///< max glyph height + int borderw; ///< border width + char *fontsize_expr; ///< expression for fontsize + AVExpr *fontsize_pexpr; ///< parsed expressions for fontsize + unsigned int fontsize; ///< font size to use + unsigned int default_fontsize; ///< default font size to use + int line_spacing; ///< lines spacing in pixels + int use_kerning; ///< font kerning is used - true/false + int tabsize; ///< tab size + int fix_bounds; ///< do we let it go out of frame bounds - t/f + + FFDrawContext dc; + FFDrawColor fontcolor; ///< foreground color + FT_Library library; ///< freetype font library handle + FT_Face face; ///< freetype font face handle + FT_Stroker stroker; ///< freetype stroker handle + struct AVTreeNode *glyphs; ///< rendered glyphs, stored using the UTF-32 char code + char *x_expr; ///< expression for x position + char *y_expr; ///< expression for y position + AVExpr *x_pexpr, *y_pexpr; ///< parsed expressions for x and y + int64_t basetime; ///< base pts time in the real world for display + double var_values[VAR_VARS_NB]; + char *a_expr; + AVExpr *a_pexpr; + int alpha; + AVLFG prng; ///< random + char *tc_opt_string; ///< specified timecode option string + AVRational tc_rate; ///< frame rate for timecode + AVTimecode tc; ///< timecode context + int tc24hmax; ///< 1 if timecode is wrapped to 24 hours, 0 otherwise + int reload; ///< reload text file for each frame + int start_number; ///< starting frame number for n/frame_num var + + AVDictionary *metadata; +} hellosubsContext; + +#define OFFSET(x) offsetof(hellosubsContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM + +static const AVOption hellosubs_options[]= { + {"fontfile", "set font file", OFFSET(fontfile), AV_OPT_TYPE_STRING, {.str=NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"text", "set text", OFFSET(text), AV_OPT_TYPE_STRING, {.str="Hello world"}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fontcolor", "set foreground color", OFFSET(fontcolor.rgba), AV_OPT_TYPE_COLOR, {.str="white"}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"fontcolor_expr", "set foreground color expression", OFFSET(fontcolor_expr), AV_OPT_TYPE_STRING, {.str=""}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"line_spacing", "set line spacing in pixels", OFFSET(line_spacing), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX,FLAGS}, + {"fontsize", "set font size", OFFSET(fontsize_expr), AV_OPT_TYPE_STRING, {.str="h/20"}, CHAR_MIN, CHAR_MAX , FLAGS}, + {"x", "set x expression", OFFSET(x_expr), AV_OPT_TYPE_STRING, {.str="w/2.7"}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"y", "set y expression", OFFSET(y_expr), AV_OPT_TYPE_STRING, {.str="h/1.3"}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"borderw", "set border width", OFFSET(borderw), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX , FLAGS}, + {"tabsize", "set tab size", OFFSET(tabsize), AV_OPT_TYPE_INT, {.i64=4}, 0, INT_MAX , FLAGS}, + {"basetime", "set base time", OFFSET(basetime), AV_OPT_TYPE_INT64, {.i64=AV_NOPTS_VALUE}, INT64_MIN, INT64_MAX , FLAGS}, +#if CONFIG_LIBFONTCONFIG + { "font", "Font name", OFFSET(font), AV_OPT_TYPE_STRING, { .str = "Sans" }, .flags = FLAGS }, +#endif + {"timecode", "set initial timecode", OFFSET(tc_opt_string), AV_OPT_TYPE_STRING, {.str=NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"tc24hmax", "set 24 hours max (timecode only)", OFFSET(tc24hmax), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS}, + {"timecode_rate", "set rate (timecode only)", OFFSET(tc_rate), AV_OPT_TYPE_RATIONAL, {.dbl=0}, 0, INT_MAX, FLAGS}, + {"r", "set rate (timecode only)", OFFSET(tc_rate), AV_OPT_TYPE_RATIONAL, {.dbl=0}, 0, INT_MAX, FLAGS}, + {"rate", "set rate (timecode only)", OFFSET(tc_rate), AV_OPT_TYPE_RATIONAL, {.dbl=0}, 0, INT_MAX, FLAGS}, + {"reload", "reload text file for each frame", OFFSET(reload), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS}, + { "alpha", "apply alpha while rendering", OFFSET(a_expr), AV_OPT_TYPE_STRING, { .str = "1" }, .flags = FLAGS }, + {"fix_bounds", "check and fix text coords to avoid clipping", OFFSET(fix_bounds), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS}, + {"start_number", "start frame number for n/frame_num variable", OFFSET(start_number), AV_OPT_TYPE_INT, {.i64=0}, 0, INT_MAX, FLAGS}, + + + + /* FT_LOAD_* flags */ + { "ft_load_flags", "set font loading flags for libfreetype", OFFSET(ft_load_flags), AV_OPT_TYPE_FLAGS, { .i64 = FT_LOAD_DEFAULT }, 0, INT_MAX, FLAGS, "ft_load_flags" }, + { "default", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_DEFAULT }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "no_scale", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_NO_SCALE }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "no_hinting", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_NO_HINTING }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "render", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_RENDER }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "no_bitmap", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_NO_BITMAP }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "vertical_layout", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_VERTICAL_LAYOUT }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "force_autohint", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_FORCE_AUTOHINT }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "crop_bitmap", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_CROP_BITMAP }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "pedantic", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_PEDANTIC }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "ignore_global_advance_width", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_IGNORE_GLOBAL_ADVANCE_WIDTH }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "no_recurse", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_NO_RECURSE }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "ignore_transform", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_IGNORE_TRANSFORM }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "monochrome", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_MONOCHROME }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "linear_design", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_LINEAR_DESIGN }, .flags = FLAGS, .unit = "ft_load_flags" }, + { "no_autohint", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = FT_LOAD_NO_AUTOHINT }, .flags = FLAGS, .unit = "ft_load_flags" }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(hellosubs); + +#undef __FTERRORS_H__ +#define FT_ERROR_START_LIST { +#define FT_ERRORDEF(e, v, s) { (e), (s) }, +#define FT_ERROR_END_LIST { 0, NULL } }; + +static const struct ft_error { + int err; + const char *err_msg; +} ft_errors[] = +#include FT_ERRORS_H + +#define FT_ERRMSG(e) ft_errors[e].err_msg + +typedef struct Glyph { + FT_Glyph glyph; + FT_Glyph border_glyph; + uint32_t code; + unsigned int fontsize; + FT_Bitmap bitmap; ///< array holding bitmaps of font + FT_Bitmap border_bitmap; ///< array holding bitmaps of font border + FT_BBox bbox; + int advance; + int bitmap_left; + int bitmap_top; +} Glyph; + +static int glyph_cmp(const void *key, const void *b) +{ + const Glyph *a = key, *bb = b; + int64_t diff = (int64_t)a->code - (int64_t)bb->code; + + if (diff != 0) + return diff > 0 ? 1 : -1; + else + return FFDIFFSIGN((int64_t)a->fontsize, (int64_t)bb->fontsize); +} + +/** + * Load glyphs corresponding to the UTF-32 codepoint code. + */ +static int load_glyph(AVFilterContext *ctx, Glyph **glyph_ptr, uint32_t code) +{ + hellosubsContext *s = ctx->priv; + FT_BitmapGlyph bitmapglyph; + Glyph *glyph; + struct AVTreeNode *node = NULL; + int ret; + + /* load glyph into s->face->glyph */ + if (FT_Load_Char(s->face, code, s->ft_load_flags)) + return AVERROR(EINVAL); + + glyph = av_mallocz(sizeof(*glyph)); + if (!glyph) { + ret = AVERROR(ENOMEM); + goto error; + } + glyph->code = code; + glyph->fontsize = s->fontsize; + + if (FT_Get_Glyph(s->face->glyph, &glyph->glyph)) { + ret = AVERROR(EINVAL); + goto error; + } + if (s->borderw) { + glyph->border_glyph = glyph->glyph; + if (FT_Glyph_StrokeBorder(&glyph->border_glyph, s->stroker, 0, 0) || + FT_Glyph_To_Bitmap(&glyph->border_glyph, FT_RENDER_MODE_NORMAL, 0, 1)) { + ret = AVERROR_EXTERNAL; + goto error; + } + bitmapglyph = (FT_BitmapGlyph) glyph->border_glyph; + glyph->border_bitmap = bitmapglyph->bitmap; + } + if (FT_Glyph_To_Bitmap(&glyph->glyph, FT_RENDER_MODE_NORMAL, 0, 1)) { + ret = AVERROR_EXTERNAL; + goto error; + } + bitmapglyph = (FT_BitmapGlyph) glyph->glyph; + + glyph->bitmap = bitmapglyph->bitmap; + glyph->bitmap_left = bitmapglyph->left; + glyph->bitmap_top = bitmapglyph->top; + glyph->advance = s->face->glyph->advance.x >> 6; + + /* measure text height to calculate text_height (or the maximum text height) */ + FT_Glyph_Get_CBox(glyph->glyph, ft_glyph_bbox_pixels, &glyph->bbox); + + /* cache the newly created glyph */ + if (!(node = av_tree_node_alloc())) { + ret = AVERROR(ENOMEM); + goto error; + } + av_tree_insert(&s->glyphs, glyph, glyph_cmp, &node); + + if (glyph_ptr) + *glyph_ptr = glyph; + return 0; + +error: + if (glyph) + av_freep(&glyph->glyph); + + av_freep(&glyph); + av_freep(&node); + return ret; +} + +static av_cold int set_fontsize(AVFilterContext *ctx, unsigned int fontsize) +{ + int err; + hellosubsContext *s = ctx->priv; + + if ((err = FT_Set_Pixel_Sizes(s->face, 0, fontsize))) { + av_log(ctx, AV_LOG_ERROR, "Could not set font size to %d pixels: %s\n", + fontsize, FT_ERRMSG(err)); + return AVERROR(EINVAL); + } + + s->fontsize = fontsize; + + return 0; +} + +static av_cold int parse_fontsize(AVFilterContext *ctx) +{ + hellosubsContext *s = ctx->priv; + int err; + + if (s->fontsize_pexpr) + return 0; + + if (s->fontsize_expr == NULL) + return AVERROR(EINVAL); + + if ((err = av_expr_parse(&s->fontsize_pexpr, s->fontsize_expr, var_names, + NULL, NULL, fun2_names, fun2, 0, ctx)) < 0) + return err; + + return 0; +} + +static av_cold int update_fontsize(AVFilterContext *ctx) +{ + hellosubsContext *s = ctx->priv; + unsigned int fontsize = s->default_fontsize; + int err; + double size, roundedsize; + + // if no fontsize specified use the default + if (s->fontsize_expr != NULL) { + if ((err = parse_fontsize(ctx)) < 0) + return err; + + size = av_expr_eval(s->fontsize_pexpr, s->var_values, &s->prng); + + if (!isnan(size)) { + roundedsize = round(size); + // test for overflow before cast + if (!(roundedsize > INT_MIN && roundedsize < INT_MAX)) { + av_log(ctx, AV_LOG_ERROR, "fontsize overflow\n"); + return AVERROR(EINVAL); + } + + fontsize = roundedsize; + } + } + + if (fontsize == 0) + fontsize = 1; + + // no change + if (fontsize == s->fontsize) + return 0; + + return set_fontsize(ctx, fontsize); +} + +static int load_font_file(AVFilterContext *ctx, const char *path, int index) +{ + hellosubsContext *s = ctx->priv; + int err; + + err = FT_New_Face(s->library, path, index, &s->face); + if (err) { +#if !CONFIG_LIBFONTCONFIG + av_log(ctx, AV_LOG_ERROR, "Could not load font \"%s\": %s\n", + s->fontfile, FT_ERRMSG(err)); +#endif + return AVERROR(EINVAL); + } + return 0; +} + +#if CONFIG_LIBFONTCONFIG +static int load_font_fontconfig(AVFilterContext *ctx) +{ + hellosubsContext *s = ctx->priv; + FcConfig *fontconfig; + FcPattern *pat, *best; + FcResult result = FcResultMatch; + FcChar8 *filename; + int index; + double size; + int err = AVERROR(ENOENT); + int parse_err; + + fontconfig = FcInitLoadConfigAndFonts(); + if (!fontconfig) { + av_log(ctx, AV_LOG_ERROR, "impossible to init fontconfig\n"); + return AVERROR_UNKNOWN; + } + pat = FcNameParse(s->fontfile ? s->fontfile : + (uint8_t *)(intptr_t)"default"); + if (!pat) { + av_log(ctx, AV_LOG_ERROR, "could not parse fontconfig pat"); + return AVERROR(EINVAL); + } + + FcPatternAddString(pat, FC_FAMILY, s->font); + + parse_err = parse_fontsize(ctx); + if (!parse_err) { + double size = av_expr_eval(s->fontsize_pexpr, s->var_values, &s->prng); + + if (isnan(size)) { + av_log(ctx, AV_LOG_ERROR, "impossible to find font information"); + return AVERROR(EINVAL); + } + + FcPatternAddDouble(pat, FC_SIZE, size); + } + + FcDefaultSubstitute(pat); + + if (!FcConfigSubstitute(fontconfig, pat, FcMatchPattern)) { + av_log(ctx, AV_LOG_ERROR, "could not substitue fontconfig options"); /* very unlikely */ + FcPatternDestroy(pat); + return AVERROR(ENOMEM); + } + + best = FcFontMatch(fontconfig, pat, &result); + FcPatternDestroy(pat); + + if (!best || result != FcResultMatch) { + av_log(ctx, AV_LOG_ERROR, + "Cannot find a valid font for the family %s\n", + s->font); + goto fail; + } + + if ( + FcPatternGetInteger(best, FC_INDEX, 0, &index ) != FcResultMatch || + FcPatternGetDouble (best, FC_SIZE, 0, &size ) != FcResultMatch) { + av_log(ctx, AV_LOG_ERROR, "impossible to find font information"); + return AVERROR(EINVAL); + } + + if (FcPatternGetString(best, FC_FILE, 0, &filename) != FcResultMatch) { + av_log(ctx, AV_LOG_ERROR, "No file path for %s\n", + s->font); + goto fail; + } + + av_log(ctx, AV_LOG_INFO, "Using \"%s\"\n", filename); + if (parse_err) + s->default_fontsize = size + 0.5; + + err = load_font_file(ctx, filename, index); + if (err) + return err; + FcConfigDestroy(fontconfig); +fail: + FcPatternDestroy(best); + return err; +} +#endif + +static int load_font(AVFilterContext *ctx) +{ + hellosubsContext *s = ctx->priv; + int err; + + /* load the face, and set up the encoding, which is by default UTF-8 */ + err = load_font_file(ctx, s->fontfile, 0); + if (!err) + return 0; +#if CONFIG_LIBFONTCONFIG + err = load_font_fontconfig(ctx); + if (!err) + return 0; +#endif + return err; +} + + + +static inline int is_newline(uint32_t c) +{ + return c == '\n' || c == '\r' || c == '\f' || c == '\v'; +} + + + +static av_cold int init(AVFilterContext *ctx) +{ + int err; + hellosubsContext *s = ctx->priv; + Glyph *glyph; + + av_expr_free(s->fontsize_pexpr); + s->fontsize_pexpr = NULL; + + s->fontsize = 0; + s->default_fontsize = 16; + + if (!s->fontfile && !CONFIG_LIBFONTCONFIG) { + av_log(ctx, AV_LOG_ERROR, "No font filename provided\n"); + return AVERROR(EINVAL); + } + + + + + if (s->tc_opt_string) { + int ret = av_timecode_init_from_string(&s->tc, s->tc_rate, + s->tc_opt_string, ctx); + if (ret < 0) + return ret; + if (s->tc24hmax) + s->tc.flags |= AV_TIMECODE_FLAG_24HOURSMAX; + if (!s->text) + s->text = av_strdup(""); + } + + if ((err = FT_Init_FreeType(&(s->library)))) { + av_log(ctx, AV_LOG_ERROR, + "Could not load FreeType: %s\n", FT_ERRMSG(err)); + return AVERROR(EINVAL); + } + + if ((err = load_font(ctx)) < 0) + return err; + + if ((err = update_fontsize(ctx)) < 0) + return err; + + if (s->borderw) { + if (FT_Stroker_New(s->library, &s->stroker)) { + av_log(ctx, AV_LOG_ERROR, "Coult not init FT stroker\n"); + return AVERROR_EXTERNAL; + } + FT_Stroker_Set(s->stroker, s->borderw << 6, FT_STROKER_LINECAP_ROUND, + FT_STROKER_LINEJOIN_ROUND, 0); + } + + s->use_kerning = FT_HAS_KERNING(s->face); + + /* load the fallback glyph with code 0 */ + load_glyph(ctx, NULL, 0); + + /* set the tabsize in pixels */ + if ((err = load_glyph(ctx, &glyph, ' ')) < 0) { + av_log(ctx, AV_LOG_ERROR, "Could not set tabsize.\n"); + return err; + } + s->tabsize *= glyph->advance; + + + + av_bprint_init(&s->expanded_text, 0, AV_BPRINT_SIZE_UNLIMITED); + av_bprint_init(&s->expanded_fontcolor, 0, AV_BPRINT_SIZE_UNLIMITED); + + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + return ff_set_common_formats(ctx, ff_draw_supported_pixel_formats(0)); +} + +static int glyph_enu_free(void *opaque, void *elem) +{ + Glyph *glyph = elem; + + FT_Done_Glyph(glyph->glyph); + FT_Done_Glyph(glyph->border_glyph); + av_free(elem); + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + hellosubsContext *s = ctx->priv; + + av_expr_free(s->x_pexpr); + av_expr_free(s->y_pexpr); + av_expr_free(s->a_pexpr); + av_expr_free(s->fontsize_pexpr); + + s->x_pexpr = s->y_pexpr = s->a_pexpr = s->fontsize_pexpr = NULL; + + av_freep(&s->positions); + s->nb_positions = 0; + + av_tree_enumerate(s->glyphs, NULL, NULL, glyph_enu_free); + av_tree_destroy(s->glyphs); + s->glyphs = NULL; + + FT_Done_Face(s->face); + FT_Stroker_Done(s->stroker); + FT_Done_FreeType(s->library); + + av_bprint_finalize(&s->expanded_text, NULL); + av_bprint_finalize(&s->expanded_fontcolor, NULL); +} + +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + hellosubsContext *s = ctx->priv; + int ret; + + ff_draw_init(&s->dc, inlink->format, FF_DRAW_PROCESS_ALPHA); + ff_draw_color(&s->dc, &s->fontcolor, s->fontcolor.rgba); + s->var_values[VAR_w] = s->var_values[VAR_W] = s->var_values[VAR_MAIN_W] = inlink->w; + s->var_values[VAR_h] = s->var_values[VAR_H] = s->var_values[VAR_MAIN_H] = inlink->h; + s->var_values[VAR_SAR] = inlink->sample_aspect_ratio.num ? av_q2d(inlink->sample_aspect_ratio) : 1; + s->var_values[VAR_DAR] = (double)inlink->w / inlink->h * s->var_values[VAR_SAR]; + s->var_values[VAR_HSUB] = 1 << s->dc.hsub_max; + s->var_values[VAR_VSUB] = 1 << s->dc.vsub_max; + s->var_values[VAR_X] = NAN; + s->var_values[VAR_Y] = NAN; + s->var_values[VAR_T] = NAN; + + av_lfg_init(&s->prng, av_get_random_seed()); + + av_expr_free(s->x_pexpr); + av_expr_free(s->y_pexpr); + av_expr_free(s->a_pexpr); + s->x_pexpr = s->y_pexpr = s->a_pexpr = NULL; + + if ((ret = av_expr_parse(&s->x_pexpr, s->x_expr, var_names, + NULL, NULL, fun2_names, fun2, 0, ctx)) < 0 || + (ret = av_expr_parse(&s->y_pexpr, s->y_expr, var_names, + NULL, NULL, fun2_names, fun2, 0, ctx)) < 0 || + (ret = av_expr_parse(&s->a_pexpr, s->a_expr, var_names, + NULL, NULL, fun2_names, fun2, 0, ctx)) < 0) + + return AVERROR(EINVAL); + + return 0; +} + + +static int generatehellosub(AVFilterContext *ctx, AVBPrint *bp) + +{ + hellosubsContext *s = ctx->priv; + double pts = s->var_values[VAR_T]; + int64_t ms = llrint(pts * 1000); + + if (ms < 0) + ms = -ms; + av_bprintf(bp, "Hello world %d:%02d",(int)(ms / (60 * 1000)),(int)(ms / 1000) % 60); + return 0; +} + + +static int draw_glyphs(hellosubsContext *s, AVFrame *frame, + int width, int height, + FFDrawColor *color, + int x, int y, int borderw) +{ + char *text = s->expanded_text.str; + uint32_t code = 0; + int i, x1, y1; + uint8_t *p; + Glyph *glyph = NULL; + + for (i = 0, p = text; *p; i++) { + FT_Bitmap bitmap; + Glyph dummy = { 0 }; + GET_UTF8(code, *p++, continue;); + + /* skip new line chars, just go to new line */ + if (code == '\n' || code == '\r' || code == '\t') + continue; + + dummy.code = code; + dummy.fontsize = s->fontsize; + glyph = av_tree_find(s->glyphs, &dummy, glyph_cmp, NULL); + + bitmap = borderw ? glyph->border_bitmap : glyph->bitmap; + + if (glyph->bitmap.pixel_mode != FT_PIXEL_MODE_MONO && + glyph->bitmap.pixel_mode != FT_PIXEL_MODE_GRAY) + return AVERROR(EINVAL); + + x1 = s->positions[i].x+s->x+x - borderw; + y1 = s->positions[i].y+s->y+y - borderw; + + ff_blend_mask(&s->dc, color, + frame->data, frame->linesize, width, height, + bitmap.buffer, bitmap.pitch, + bitmap.width, bitmap.rows, + bitmap.pixel_mode == FT_PIXEL_MODE_MONO ? 0 : 3, + 0, x1, y1); + } + + return 0; +} + + +static void update_color_with_alpha(hellosubsContext *s, FFDrawColor *color, const FFDrawColor incolor) +{ + *color = incolor; + color->rgba[3] = (color->rgba[3] * s->alpha) / 255; + ff_draw_color(&s->dc, color, color->rgba); +} + +static void update_alpha(hellosubsContext *s) +{ + double alpha = av_expr_eval(s->a_pexpr, s->var_values, &s->prng); + + if (isnan(alpha)) + return; + + if (alpha >= 1.0) + s->alpha = 255; + else if (alpha <= 0) + s->alpha = 0; + else + s->alpha = 256 * alpha; +} + +static int draw_text(AVFilterContext *ctx, AVFrame *frame, + int width, int height) +{ + hellosubsContext *s = ctx->priv; + uint32_t code = 0, prev_code = 0; + int x = 0, y = 0, i = 0, ret; + int max_text_line_w = 0, len; + + char *text; + uint8_t *p; + int y_min = 32000, y_max = -32000; + int x_min = 32000, x_max = -32000; + FT_Vector delta; + Glyph *glyph = NULL, *prev_glyph = NULL; + Glyph dummy = { 0 }; + + AVBPrint *bp = &s->expanded_text; + + FFDrawColor fontcolor; + av_bprint_clear(bp); + + if ((ret = generatehellosub(ctx,&s->expanded_text)) < 0) + return ret; + text = s->expanded_text.str; + len=20; + if (len > s->nb_positions) { + if (!(s->positions = + av_realloc(s->positions, len*sizeof(*s->positions)))) + return AVERROR(ENOMEM); + s->nb_positions = len; + + } + + x = 0; + y = 0; + + if ((ret = update_fontsize(ctx)) < 0) + return ret; + + /* load and cache glyphs */ + for (i = 0, p = text; *p; i++) { + GET_UTF8(code, *p++, continue;); + + /* get glyph */ + dummy.code = code; + dummy.fontsize = s->fontsize; + glyph = av_tree_find(s->glyphs, &dummy, glyph_cmp, NULL); + if (!glyph) { + ret = load_glyph(ctx, &glyph, code); + if (ret < 0) + return ret; + } + + y_min = FFMIN(glyph->bbox.yMin, y_min); + y_max = FFMAX(glyph->bbox.yMax, y_max); + x_min = FFMIN(glyph->bbox.xMin, x_min); + x_max = FFMAX(glyph->bbox.xMax, x_max); + } + s->max_glyph_h = y_max - y_min; + s->max_glyph_w = x_max - x_min; + + /* compute and save position for each glyph */ + glyph = NULL; + for (i = 0, p = text; *p; i++) { + GET_UTF8(code, *p++, continue;); + + /* skip the \n in the sequence \r\n */ + if (prev_code == '\r' && code == '\n') + continue; + + prev_code = code; + if (is_newline(code)) { + + max_text_line_w = FFMAX(max_text_line_w, x); + y += s->max_glyph_h + s->line_spacing; + x = 0; + continue; + } + + /* get glyph */ + prev_glyph = glyph; + dummy.code = code; + dummy.fontsize = s->fontsize; + glyph = av_tree_find(s->glyphs, &dummy, glyph_cmp, NULL); + + /* kerning */ + if (s->use_kerning && prev_glyph && glyph->code) { + FT_Get_Kerning(s->face, prev_glyph->code, glyph->code, + ft_kerning_default, &delta); + x += delta.x >> 6; + } + + /* save position */ + s->positions[i].x = x + glyph->bitmap_left; + s->positions[i].y = y - glyph->bitmap_top + y_max; + if (code == '\t') x = (x / s->tabsize + 1)*s->tabsize; + else x += glyph->advance; + } + + max_text_line_w = FFMAX(x, max_text_line_w); + + s->var_values[VAR_TW] = s->var_values[VAR_TEXT_W] = max_text_line_w; + s->var_values[VAR_TH] = s->var_values[VAR_TEXT_H] = y + s->max_glyph_h; + + s->var_values[VAR_MAX_GLYPH_W] = s->max_glyph_w; + s->var_values[VAR_MAX_GLYPH_H] = s->max_glyph_h; + s->var_values[VAR_MAX_GLYPH_A] = s->var_values[VAR_ASCENT ] = y_max; + s->var_values[VAR_MAX_GLYPH_D] = s->var_values[VAR_DESCENT] = y_min; + + s->var_values[VAR_LINE_H] = s->var_values[VAR_LH] = s->max_glyph_h; + + s->x = s->var_values[VAR_X] = av_expr_eval(s->x_pexpr, s->var_values, &s->prng); + s->y = s->var_values[VAR_Y] = av_expr_eval(s->y_pexpr, s->var_values, &s->prng); + /* It is necessary if x is expressed from y */ + s->x = s->var_values[VAR_X] = av_expr_eval(s->x_pexpr, s->var_values, &s->prng); + + update_alpha(s); + update_color_with_alpha(s, &fontcolor , s->fontcolor ); + + if (s->fix_bounds) { + + /* calculate footprint of text effects */ + + + int offsetleft = FFMAX3(0,0,0); + + int offsettop = FFMAX3(0, 0,0); + + if (s->x - offsetleft < 0) s->x = offsetleft; + if (s->y - offsettop < 0) s->y = offsettop; + + + } + + + + if ((ret = draw_glyphs(s, frame, width, height, + &fontcolor, 0, 0, 0)) < 0) + return ret; + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + AVFilterLink *outlink = ctx->outputs[0]; + hellosubsContext *s = ctx->priv; + + s->var_values[VAR_N] = inlink->frame_count_out + s->start_number; + s->var_values[VAR_T] = frame->pts == AV_NOPTS_VALUE ? + NAN : frame->pts * av_q2d(inlink->time_base); + + s->var_values[VAR_PICT_TYPE] = frame->pict_type; + s->metadata = frame->metadata; + + draw_text(ctx, frame, frame->width, frame->height); + + av_log(ctx, AV_LOG_DEBUG, "n:%d t:%f text_w:%d text_h:%d x:%d y:%d\n", + (int)s->var_values[VAR_N], s->var_values[VAR_T], + (int)s->var_values[VAR_TEXT_W], (int)s->var_values[VAR_TEXT_H], + s->x, s->y); + + return ff_filter_frame(outlink, frame); +} + +static const AVFilterPad avfilter_vf_hellosubs_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + .needs_writable = 1, + }, + { NULL } +}; + +static const AVFilterPad avfilter_vf_hellosubs_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +AVFilter ff_vf_hellosubs = { + .name = "hellosubs", + .description = NULL_IF_CONFIG_SMALL("Writes hello world time on top of video frames using libfreetype library."), + .priv_size = sizeof(hellosubsContext), + .priv_class = &hellosubs_class, + .init = init, + .uninit = uninit, + .query_formats = query_formats, + .inputs = avfilter_vf_hellosubs_inputs, + .outputs = avfilter_vf_hellosubs_outputs, + .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_GENERIC, +}; -- 2.7.4