| Message ID | pull.18.v5.ffstaging.FFmpeg.1656151077.ffmpegagent@gmail.com |
|---|---|
| Headers | show |
| Series | Subtitle Filtering 2022 | expand |
On Sat, Jun 25, 2022 at 11:58 AM ffmpegagent <ffmpegagent@gmail.com> wrote: > > Subtitle Filtering 2022 > ======================= > > This is a substantial update to the earlier subtitle filtering patch > series. > A primary goal has been to address others' concerns as much as possible on > one side and to provide more clarity and control over the way things are > working. Clarity is is specifically important to allow for a better > understanding of the need for a subtitle start pts value that can be > different from the frame's pts value. This is done by refactoring the > subtitle timing fields in AVFrame, adding a frame field to indicate > repeated > subtitle frames, and finally the full removal of the heartbeat > functionality, replaced by a new 'subfeed' filter that provides different > modes for arbitrating subtitle frames in a filter graph. Finally, each > subtitle filter's documentation has been amended by a section describing > the > filter's timeline behavior (in v3 update). > > > Subtitle Filtering Demos > ======================== > > I published a demonstration of subtitle filtering capabilities with OCR, > text and bitmap subtitle manipulation involved: Demo 1: Text-Manipulation > with Bitmap Subtitles > [https://github.com/softworkz/SubtitleFilteringDemos/tree/master/Demo1] > > > v5 - Conversion to Graphic Subtitles, and other enhancements > ============================================================ > > * I'm glad to announce that Traian (@tcoza) has joined the project and > contributed a new 'text2graphicsub' filter to convert text subtitles to > graphic subtitles, which can in turn be encoded as dvd, dvb or x-subs > (and any other encoder for graphic subs that might be added in the > future). This filter closes the last open "gap" in subtitle processing. > * stripstyles filter: now allows very fine-grained control over which ASS > style codes should be preserved or stripped > * stripstyles: do not drop dialog margin values > * subfeed filter: eliminates duplicate frames with duplicate start times > when 'fix_overlap' is specified > * textmod: do not drop effect values > * graphicsub2text: reduce font size jitter > * ass_split: add function to selectively preserve elements when splitting > * add strim, snull and ssink and further unify subtitle frame handling > with > audio and video > * ffmpeg_filter: get simple filter notation working for subtitles > > > v4 - Quality Improvements > ========================= > > * finally an updated version > * includes many improvements from internal testing > * all FATE tests passed > * all example commands from the docs verified to work > * can't list all the detail changes.. > * I have left out the extra commits which can be handled separately, just > in case somebody wonders why these are missing: > * avcodec/webvttenc: Don't encode drawing codes and empty lines > * avcodec/webvttenc: convert hard-space tags to > * avutil/ass_split: Add parsing of hard-space tags (\h) > * avutil/ass_split: Treat all content in curly braces as hidden > * avutil/ass_split: Fix ass parsing of style codes with comments > > > v3 - Rebase > =========== > > due to merge conflicts - apologies. > > > Changes in v2 > ============= > > * added .gitattributes file to enforce binary diffs for the test refs that > cannot be applied when being sent via e-mail > * perform filter graph re-init due to subtitle "frame size" change only > when the size was unknown before and not set via -canvas_size > * overlaytextsubs: Make sure to request frames on the subtitle input > * avfilter/splitcc: Start parsing cc data on key frames only > * avcodec/webvttenc: Don't encode ass drawing codes and empty lines > * stripstyles: fix mem leak > * gs2t: improve color detection > * gs2t: empty frames must not be skipped > * subfeed: fix name > * textmod: preserve margins > * added .gitattributes file to enforce binary diffs for the test refs that > cannot be applied when being sent via e-mail > * perform filter graph re-init due to subtitle "frame size" change only > when the size was unknown before and not set via -canvas_size > * avcodec/dvbsubdec: Fix conditions for fallback to default resolution > * Made changes suggested by Andreas > * Fixed failing command line reported by Michael > > Changes from previous version v24: > > > AVFrame > ======= > > * Removed sub_start_time The start time is now added to the subtitle > start_pts during decoding The sub_end_time field is adjusted accordingly > * Renamed sub_end_time to duration which it is effectively after removing > the start_time > * Added a sub-struct 'subtitle_timing' to av frame Contains subtitle_pts > renamed to 'subtitle_timing.start_pts' and 'subtitle_timing.duration' > * Change both fields to (fixed) time_base AV_TIMEBASE > * add repeat_sub field provides a clear indication whether a subtitle > frame > is an actual subtitle event or a repeated subtitle frame in a filter > graph > > > Heartbeat Removal > ================= > > * completely removed the earlier heartbeat implementation > * filtering arbitration is now implemented in a new filter: 'subfeed' > * subfeed will be auto-inserted for compatiblity with sub2video command > lines > * the new behavior is not exactly identical to the earlier behavior, but > it > basically allows to achieve the same results > * there's a small remainder, now named subtitle kickoff which serves to > get > things (in the filter graph) going right from the start > > > New 'subfeed' Filter > ==================== > > * a versatile filter for solving all kinds of problems with subtile frame > flow in filter graphs > * Can be inserted at any position in a graph > * Auto-inserted for sub2video command lines (in repeat-mode) > * Allows duration fixup delay input frames with unknown duration and infer > duration from start of subsequent frame > * Provides multiple modes of operation: > * repeat mode (default) Queues input frames Outputs frames at a fixed > (configurable) rate Either sends a matching input frame (repeatedly) > or > empty frames otherwise > * scatter mode similar to repeat mode, but splits input frames by > duration into small segments with same content > * forward mode No fixed output rate Useful in combination with duration > fixup or overlap fixup > > > ffmpeg Tool Changes > =================== > > * delay subtitle output stream initialization (like for audio and video) > This is needed for example when a format header depends on having > received an initial frame to derive certain header values from > * decoding: set subtitle frame size from decoding context > * re-init graph when subtitle size changes > * always insert subscale filter for sub2video command lines (to ensure > correct scaling) > > > Subtitle Encoding > ================= > > * ignore repeated frames for encoding based on repeat_sub field in AVFrame > * support multi-area encoding for text subtitles Subtitle OCR can create > multiple areas at different positions. Previously, the texts were always > squashed into a single area ('subtitle rect'), which was not ideal. > Multiple text areas are now generally supported: > * ASS Encoder Changed to use the 'receive_packet' encoding API A single > frame with multiple text areas will create multiple packets now > * All other text subtitle encoders A newline is inserted between the > text > from multiple areas > > > graphicsub2text (OCR) > ===================== > > * enhanced preprocessing > * using elbg algorithm for color quantization > * detection and removal of text outlines > * map-based identification of colors per word (text, outline, > background) > * add option for duration fixup > * add option to dump preprocessing bitmaps > * Recognize formatting and apply as ASS inline styles > * per word(!) > * paragraph alignment > * positioning > * font names > * font size > * font style (italic, underline, bold) > * text color, outline color > > > Other Filter Changes > ==================== > > * all: Make sure to forward all link properties (time base, frame rate, w, > h) where appropriate > * overlaytextsubs: request frames on the subtitle input > * overlaytextsubs: disable read-order checking > * overlaytextsubs: improve implementation of render_latest_only > * overlaytextsubs: ensure equal in/out video formats > * splitcc: derive framerate from realtime_latency > * graphicsub2video: implement caching of converted frames > * graphicsub2video: use 1x1 output frame size as long as subtitle size is > unknown (0x0) > > Plus a dozen of things I forgot.. > > softworkz (24): > avcodec,avutil: Move enum AVSubtitleType to avutil, add new and > deprecate old values > avutil/frame: Prepare AVFrame for subtitle handling > avcodec/subtitles: Introduce new frame-based subtitle decoding API > avcodec/libzvbi: set subtitle type > avfilter/subtitles: Update vf_subtitles to use new decoding api > avcodec,avutil: Move ass helper functions to avutil as avpriv_ and > extend ass dialog parsing > avcodec/subtitles: Replace deprecated enum values > fftools/play,probe: Adjust for subtitle changes > avfilter/subtitles: Add subtitles.c for subtitle frame allocation > avfilter/avfilter: Handle subtitle frames > avfilter/avfilter: Fix hardcoded input index > avfilter/sbuffer: Add sbuffersrc and sbuffersink filters > avfilter/overlaygraphicsubs: Add overlaygraphicsubs and > graphicsub2video filters > avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video > filters > avfilter/textmod: Add textmod, censor and show_speaker filters > avfilter/stripstyles: Add stripstyles filter > avfilter/splitcc: Add splitcc filter for closed caption handling > avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) > avfilter/subscale: Add filter for scaling and/or re-arranging > graphical subtitles > avfilter/subfeed: add subtitle feed filter > avfilter/snull,strim: Add snull and strim filters > avcodec/subtitles: Migrate subtitle encoders to frame-based API > fftools/ffmpeg: Introduce subtitle filtering and new frame-based > subtitle encoding > avcodec/dvbsubdec: Fix conditions for fallback to default resolution > > Can this be properly finally be fully reviewed and accepted? Otherwise if its kept mainly ignored than it should be regarded as spam. Current status quo is bad. > tcoza (1): > avfilter/text2graphicsub: Added text2graphicsub subtitle filter > > configure | 10 +- > doc/filters.texi | 807 ++++++++++++++ > fftools/ffmpeg.c | 613 +++++----- > fftools/ffmpeg.h | 17 +- > fftools/ffmpeg_filter.c | 270 +++-- > fftools/ffmpeg_hw.c | 2 +- > fftools/ffmpeg_opt.c | 28 +- > fftools/ffplay.c | 102 +- > fftools/ffprobe.c | 47 +- > libavcodec/Makefile | 56 +- > libavcodec/ass.h | 151 +-- > libavcodec/ass_split.h | 191 ---- > libavcodec/assdec.c | 4 +- > libavcodec/assenc.c | 191 +++- > libavcodec/avcodec.c | 8 + > libavcodec/avcodec.h | 34 +- > libavcodec/ccaption_dec.c | 20 +- > libavcodec/codec_internal.h | 12 - > libavcodec/decode.c | 60 +- > libavcodec/dvbsubdec.c | 53 +- > libavcodec/dvbsubenc.c | 96 +- > libavcodec/dvdsubdec.c | 2 +- > libavcodec/dvdsubenc.c | 102 +- > libavcodec/encode.c | 61 +- > libavcodec/internal.h | 16 + > libavcodec/jacosubdec.c | 2 +- > libavcodec/libaribb24.c | 2 +- > libavcodec/libzvbi-teletextdec.c | 17 +- > libavcodec/microdvddec.c | 7 +- > libavcodec/movtextdec.c | 3 +- > libavcodec/movtextenc.c | 126 ++- > libavcodec/mpl2dec.c | 2 +- > libavcodec/pgssubdec.c | 2 +- > libavcodec/realtextdec.c | 2 +- > libavcodec/samidec.c | 2 +- > libavcodec/srtdec.c | 2 +- > libavcodec/srtenc.c | 116 +- > libavcodec/subviewerdec.c | 2 +- > libavcodec/tests/avcodec.c | 5 +- > libavcodec/textdec.c | 4 +- > libavcodec/ttmlenc.c | 114 +- > libavcodec/utils.c | 185 ++- > libavcodec/webvttdec.c | 2 +- > libavcodec/webvttenc.c | 94 +- > libavcodec/xsubdec.c | 2 +- > libavcodec/xsubenc.c | 88 +- > libavfilter/Makefile | 18 + > libavfilter/allfilters.c | 19 + > libavfilter/avfilter.c | 34 +- > libavfilter/avfilter.h | 11 + > libavfilter/avfiltergraph.c | 5 + > libavfilter/buffersink.c | 54 + > libavfilter/buffersink.h | 7 + > libavfilter/buffersrc.c | 72 ++ > libavfilter/buffersrc.h | 1 + > libavfilter/formats.c | 16 + > libavfilter/formats.h | 3 + > libavfilter/internal.h | 19 +- > libavfilter/sf_graphicsub2text.c | 1137 +++++++++++++++++++ > libavfilter/sf_snull.c | 50 + > libavfilter/sf_splitcc.c | 395 +++++++ > libavfilter/sf_stripstyles.c | 237 ++++ > libavfilter/sf_subfeed.c | 412 +++++++ > libavfilter/sf_subscale.c | 884 +++++++++++++++ > libavfilter/sf_text2graphicsub.c | 630 +++++++++++ > libavfilter/sf_textmod.c | 710 ++++++++++++ > libavfilter/subtitles.c | 63 ++ > libavfilter/subtitles.h | 44 + > libavfilter/trim.c | 46 +- > libavfilter/vf_overlaygraphicsubs.c | 765 +++++++++++++ > libavfilter/vf_overlaytextsubs.c | 680 +++++++++++ > libavfilter/vf_subtitles.c | 67 +- > libavutil/Makefile | 4 + > {libavcodec => libavutil}/ass.c | 115 +- > libavutil/ass_internal.h | 135 +++ > {libavcodec => libavutil}/ass_split.c | 179 ++- > libavutil/ass_split_internal.h | 254 +++++ > libavutil/frame.c | 206 +++- > libavutil/frame.h | 85 +- > libavutil/subfmt.c | 45 + > libavutil/subfmt.h | 115 ++ > libavutil/version.h | 1 + > tests/ref/fate/filter-overlay-dvdsub-2397 | 182 +-- > tests/ref/fate/sub-dvb | 162 +-- > tests/ref/fate/sub-scc | 1 - > tests/ref/fate/sub2video | 1091 +++++++++++++++++- > tests/ref/fate/sub2video_basic | 1238 +++++++++++++++++++-- > tests/ref/fate/sub2video_time_limited | 78 +- > 88 files changed, 12398 insertions(+), 1604 deletions(-) > delete mode 100644 libavcodec/ass_split.h > create mode 100644 libavfilter/sf_graphicsub2text.c > create mode 100644 libavfilter/sf_snull.c > create mode 100644 libavfilter/sf_splitcc.c > create mode 100644 libavfilter/sf_stripstyles.c > create mode 100644 libavfilter/sf_subfeed.c > create mode 100644 libavfilter/sf_subscale.c > create mode 100644 libavfilter/sf_text2graphicsub.c > create mode 100644 libavfilter/sf_textmod.c > create mode 100644 libavfilter/subtitles.c > create mode 100644 libavfilter/subtitles.h > create mode 100644 libavfilter/vf_overlaygraphicsubs.c > create mode 100644 libavfilter/vf_overlaytextsubs.c > rename {libavcodec => libavutil}/ass.c (59%) > create mode 100644 libavutil/ass_internal.h > rename {libavcodec => libavutil}/ass_split.c (71%) > create mode 100644 libavutil/ass_split_internal.h > create mode 100644 libavutil/subfmt.c > create mode 100644 libavutil/subfmt.h > > > base-commit: 6a82412bf33108111eb3f63076fd5a51349ae114 > Published-As: > https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-18%2Fsoftworkz%2Fsubmit_subfiltering-v5 > Fetch-It-Via > <https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-18%2Fsoftworkz%2Fsubmit_subfiltering-v5Fetch-It-Via>: > git fetch https://github.com/ffstaging/FFmpeg > pr-ffstaging-18/softworkz/submit_subfiltering-v5 > Pull-Request: https://github.com/ffstaging/FFmpeg/pull/18 > > Range-diff vs v4: > > 1: 2f3ba171f5 = 1: aa32b9048f avcodec,avutil: Move enum > AVSubtitleType to avutil, add new and deprecate old values > 2: ff101f8a76 ! 2: d5ab9d1919 avutil/frame: Prepare AVFrame for > subtitle handling > @@ libavutil/frame.c: FF_ENABLE_DEPRECATION_WARNINGS > + case AVMEDIA_TYPE_VIDEO: > return frame_copy_video(dst, src); > - else if (dst->nb_samples > 0 && > -- (av_channel_layout_check(&dst->ch_layout) > --#if FF_API_OLD_CHANNEL_LAYOUT > -- || dst->channel_layout || dst->channels > --#endif > -- )) > + case AVMEDIA_TYPE_AUDIO: > - return frame_copy_audio(dst, src); > ++ if (dst->nb_samples > 0 && > + (av_channel_layout_check(&dst->ch_layout) > + #if FF_API_OLD_CHANNEL_LAYOUT > + || dst->channels > 0 > + #endif > + )) > +- return frame_copy_audio(dst, src); > -FF_ENABLE_DEPRECATION_WARNINGS > -- > -- return AVERROR(EINVAL); > ++ return frame_copy_audio(dst, src); > ++ break; > + case AVMEDIA_TYPE_SUBTITLE: > + return frame_copy_subtitles(dst, src, 1); > -+ default: > -+ return AVERROR(EINVAL); > + } > - } > > - void av_frame_remove_side_data(AVFrame *frame, enum > AVFrameSideDataType type) > + return AVERROR(EINVAL); > + } > > ## libavutil/frame.h ## > @@ > 3: b8935d5e68 = 3: 0a685a6b19 avcodec/subtitles: Introduce new > frame-based subtitle decoding API > 4: 4b44732e07 = 4: 0b69b1ce19 avcodec/libzvbi: set subtitle type > 5: 8faa7a4043 = 5: 0c2091e57c avfilter/subtitles: Update vf_subtitles > to use new decoding api > 6: 1664026d7c ! 6: 4903cdd1cd avcodec,avutil: Move ass helper > functions to avutil as avpriv_ and extend ass dialog parsing > @@ Commit message > > - hard_space callback (for upcoming fix) > - extensible callback (for future extension) > + - new API which allows tag filtering > > Signed-off-by: softworkz <softworkz@hotmail.com> > > @@ libavcodec/ass.h > - const char *linebreaks, int > keep_ass_markup); > #endif /* AVCODEC_ASS_H */ > > + ## libavcodec/ass_split.h (deleted) ## > +@@ > +-/* > +- * SSA/ASS spliting functions > +- * Copyright (c) 2010 Aurelien Jacobs <aurel@gnuage.org> > +- * > +- * This file is part of FFmpeg. > +- * > +- * FFmpeg is free software; you can redistribute it and/or > +- * modify it under the terms of the GNU Lesser General Public > +- * License as published by the Free Software Foundation; either > +- * version 2.1 of the License, or (at your option) any later > version. > +- * > +- * FFmpeg is distributed in the hope that it will be useful, > +- * but WITHOUT ANY WARRANTY; without even the implied warranty of > +- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > +- * Lesser General Public License for more details. > +- * > +- * You should have received a copy of the GNU Lesser General Public > +- * License along with FFmpeg; if not, write to the Free Software > +- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA > +- */ > +- > +-#ifndef AVCODEC_ASS_SPLIT_H > +-#define AVCODEC_ASS_SPLIT_H > +- > +-/** > +- * fields extracted from the [Script Info] section > +- */ > +-typedef struct { > +- char *script_type; /**< SSA script format version (eg. > v4.00) */ > +- char *collisions; /**< how subtitles are moved to prevent > collisions */ > +- int play_res_x; /**< video width that ASS coords are > referring to */ > +- int play_res_y; /**< video height that ASS coords are > referring to */ > +- float timer; /**< time multiplier to apply to SSA > clock (in %) */ > +-} ASSScriptInfo; > +- > +-/** > +- * fields extracted from the [V4(+) Styles] section > +- */ > +-typedef struct { > +- char *name; /**< name of the tyle (case sensitive) */ > +- char *font_name; /**< font face (case sensitive) */ > +- int font_size; /**< font height */ > +- int primary_color; /**< color that a subtitle will normally > appear in */ > +- int secondary_color; > +- int outline_color; /**< color for outline in ASS, called > tertiary in SSA */ > +- int back_color; /**< color of the subtitle outline or > shadow */ > +- int bold; /**< whether text is bold (1) or not (0) > */ > +- int italic; /**< whether text is italic (1) or not > (0) */ > +- int underline; /**< whether text is underlined (1) or > not (0) */ > +- int strikeout; > +- float scalex; > +- float scaley; > +- float spacing; > +- float angle; > +- int border_style; > +- float outline; > +- float shadow; > +- int alignment; /**< position of the text (left, center, > top...), > +- defined after the layout of the > numpad > +- (1-3 sub, 4-6 mid, 7-9 top) */ > +- int margin_l; > +- int margin_r; > +- int margin_v; > +- int alpha_level; > +- int encoding; > +-} ASSStyle; > +- > +-/** > +- * fields extracted from the [Events] section > +- */ > +-typedef struct { > +- int readorder; > +- int layer; /**< higher numbered layers are drawn over > lower numbered */ > +- int start; /**< start time of the dialog in centiseconds */ > +- int end; /**< end time of the dialog in centiseconds */ > +- char *style; /**< name of the ASSStyle to use with this > dialog */ > +- char *name; > +- int margin_l; > +- int margin_r; > +- int margin_v; > +- char *effect; > +- char *text; /**< actual text which will be displayed as a > subtitle, > +- can include style override control codes > (see > +- ff_ass_split_override_codes()) */ > +-} ASSDialog; > +- > +-/** > +- * structure containing the whole split ASS data > +- */ > +-typedef struct { > +- ASSScriptInfo script_info; /**< general information about the > SSA script*/ > +- ASSStyle *styles; /**< array of split out styles */ > +- int styles_count; /**< number of ASSStyle in the > styles array */ > +- ASSDialog *dialogs; /**< array of split out dialogs */ > +- int dialogs_count; /**< number of ASSDialog in the > dialogs array*/ > +-} ASS; > +- > +-/** > +- * This struct can be casted to ASS to access to the split data. > +- */ > +-typedef struct ASSSplitContext ASSSplitContext; > +- > +-/** > +- * Split a full ASS file or a ASS header from a string buffer and > store > +- * the split structure in a newly allocated context. > +- * > +- * @param buf String containing the ASS formatted data. > +- * @return Newly allocated struct containing split data. > +- */ > +-ASSSplitContext *ff_ass_split(const char *buf); > +- > +-/** > +- * Free a dialogue obtained from ff_ass_split_dialog(). > +- */ > +-void ff_ass_free_dialog(ASSDialog **dialogp); > +- > +-/** > +- * Split one ASS Dialogue line from a string buffer. > +- * > +- * @param ctx Context previously initialized by ff_ass_split(). > +- * @param buf String containing the ASS "Dialogue" line. > +- * @return Pointer to the split ASSDialog. Must be freed with > ff_ass_free_dialog() > +- */ > +-ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char > *buf); > +- > +-/** > +- * Free all the memory allocated for an ASSSplitContext. > +- * > +- * @param ctx Context previously initialized by ff_ass_split(). > +- */ > +-void ff_ass_split_free(ASSSplitContext *ctx); > +- > +- > +-/** > +- * Set of callback functions corresponding to each override codes > that can > +- * be encountered in a "Dialogue" Text field. > +- */ > +-typedef struct { > +- /** > +- * @defgroup ass_styles ASS styles > +- * @{ > +- */ > +- void (*text)(void *priv, const char *text, int len); > +- void (*new_line)(void *priv, int forced); > +- void (*style)(void *priv, char style, int close); > +- void (*color)(void *priv, unsigned int /* color */, unsigned > int color_id); > +- void (*alpha)(void *priv, int alpha, int alpha_id); > +- void (*font_name)(void *priv, const char *name); > +- void (*font_size)(void *priv, int size); > +- void (*alignment)(void *priv, int alignment); > +- void (*cancel_overrides)(void *priv, const char *style); > +- /** @} */ > +- > +- /** > +- * @defgroup ass_functions ASS functions > +- * @{ > +- */ > +- void (*move)(void *priv, int x1, int y1, int x2, int y2, int > t1, int t2); > +- void (*origin)(void *priv, int x, int y); > +- /** @} */ > +- > +- /** > +- * @defgroup ass_end end of Dialogue Event > +- * @{ > +- */ > +- void (*end)(void *priv); > +- /** @} */ > +-} ASSCodesCallbacks; > +- > +-/** > +- * Split override codes out of a ASS "Dialogue" Text field. > +- * > +- * @param callbacks Set of callback functions called for each > override code > +- * encountered. > +- * @param priv Opaque pointer passed to the callback functions. > +- * @param buf The ASS "Dialogue" Text field to split. > +- * @return >= 0 on success otherwise an error code <0 > +- */ > +-int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, > void *priv, > +- const char *buf); > +- > +-/** > +- * Find an ASSStyle structure by its name. > +- * > +- * @param ctx Context previously initialized by ff_ass_split(). > +- * @param style name of the style to search for. > +- * @return the ASSStyle corresponding to style, or NULL if style > can't be found > +- */ > +-ASSStyle *ff_ass_style_get(ASSSplitContext *ctx, const char *style); > +- > +-#endif /* AVCODEC_ASS_SPLIT_H */ > + > ## libavcodec/assdec.c ## > @@ > #include <string.h> > @@ libavutil/ass.c: char *ff_ass_get_dialog(int readorder, int layer, > const char *s > - const char *speaker) > +char *avpriv_ass_get_dialog_ex(int readorder, int layer, const char > *style, > + const char *speaker, int margin_l, int > margin_r, > -+ int margin_v, const char *text) > ++ int margin_v, const char *effect, const > char *text) > { > - return ff_ass_add_rect2(sub, dialog, readorder, layer, style, > speaker, NULL); > -} > @@ libavutil/ass.c: char *ff_ass_get_dialog(int readorder, int layer, > const char *s > - FFASSDecoderContext *s = avctx->priv_data; > - if (!(avctx->flags2 & AV_CODEC_FLAG2_RO_FLUSH_NOOP)) > - s->readorder = 0; > -+ return av_asprintf("%d,%d,%s,%s,%d,%d,%d,,%s", > ++ return av_asprintf("%d,%d,%s,%s,%d,%d,%d,%s,%s", > + readorder, layer, style ? style : "Default", > + speaker ? speaker : "", margin_l, margin_r, > -+ margin_v, text); > ++ margin_v, effect ? effect : "", text); > } > > -void ff_ass_bprint_text_event(AVBPrint *buf, const char *p, int > size, > @@ libavutil/ass_internal.h (new) > + */ > +char *avpriv_ass_get_dialog_ex(int readorder, int layer, const char > *style, > + const char *speaker, int margin_l, int > margin_r, > -+ int margin_v, const char *text); > ++ int margin_v, const char *effect, const > char *text); > + > +/** > + * Escape a text subtitle using ASS syntax into an AVBPrint buffer. > @@ libavutil/ass_split.c: ASSDialog > *ff_ass_split_dialog(ASSSplitContext *ctx, cons > if (ctx) { > int i; > @@ libavutil/ass_split.c: void ff_ass_split_free(ASSSplitContext > *ctx) > + } > } > > ++static int ass_remove_empty_braces(AVBPrint* buffer) > ++{ > ++ char* tmp; > ++ int ret = 0, n = 0; > > -int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, > void *priv, > -+int avpriv_ass_split_override_codes(const ASSCodesCallbacks > *callbacks, void *priv, > - const char *buf) > +- const char *buf) > ++ if (buffer == NULL || buffer->len == 0 || > !av_bprint_is_complete(buffer)) > ++ return 0; > ++ > ++ ret = av_bprint_finalize(buffer, &tmp); > ++ if (ret) > ++ return ret; > ++ > ++ for (unsigned i = 0; i < buffer->len; i++) { > ++ if (tmp[i] == '{' && tmp[i+1] == '}') > ++ i++; > ++ else > ++ tmp[n++] = tmp[i]; > ++ } > ++ > ++ tmp[n++] = '\0'; > ++ > ++ av_bprint_init(buffer, n, n); > ++ av_bprint_append_data(buffer, tmp, n - 1); > ++ av_free(tmp); > ++ > ++ return ret; > ++} > ++ > ++static void ass_write_filtered_line(AVBPrint* buffer, const char > *buf, int len, enum ASSSplitComponents keep_flags, enum ASSSplitComponents > split_component) > ++{ > ++ if (buffer == NULL || buf == NULL || len == 0) > ++ return; > ++ > ++ if (split_component != ASS_SPLIT_ANY && !(keep_flags & > split_component)) > ++ return; > ++ > ++ > ++ av_bprint_append_data(buffer, buf, len - 1); > ++} > ++ > ++int avpriv_ass_filter_override_codes(const ASSCodesCallbacks > *callbacks, void *priv, const char *buf, AVBPrint* outbuffer, enum > ASSSplitComponents keep_flags) > { > const char *text = NULL; > -@@ libavutil/ass_split.c: int ff_ass_split_override_codes(const > ASSCodesCallbacks *callbacks, void *priv, > + char new_line[2]; > +- int text_len = 0; > ++ int text_len = 0, ret = 0; > + > + while (buf && *buf) { > +- if (text && callbacks->text && > +- (sscanf(buf, "\\%1[nN]", new_line) == 1 || > +- !strncmp(buf, "{\\", 2))) { > +- callbacks->text(priv, text, text_len); > ++ > ++ if (text && (sscanf(buf, "\\%1[nN]", new_line) == 1 || > !strncmp(buf, "{\\", 2))) { > ++ ass_write_filtered_line(outbuffer, text, text_len + 1, > keep_flags, ASS_SPLIT_TEXT | ASS_SPLIT_TEXT2); > ++ > ++ if (callbacks->text) > ++ callbacks->text(priv, text, text_len); > + text = NULL; > + } > ++ > + if (sscanf(buf, "\\%1[nN]", new_line) == 1) { > + if (callbacks->new_line) > + callbacks->new_line(priv, new_line[0] == 'N'); > ++ ass_write_filtered_line(outbuffer, buf, 3, keep_flags, > ASS_SPLIT_ANY); > + buf += 2; > + } else if (!strncmp(buf, "{\\", 2)) { > ++ ass_write_filtered_line(outbuffer, buf, 2, keep_flags, > ASS_SPLIT_ANY); > + buf++; > while (*buf == '\\') { > - char style[2], c[2], sep[2], c_num[2] = "0", > tmp[128] = {0}; > +- char style[2], c[2], sep[2], c_num[2] = "0", > tmp[128] = {0}; > ++ char style[4], c[2], axis[3], sep[3], c_num[2] = > "0", tmp[128] = {0}; > unsigned int color = 0xFFFFFFFF; > - int len, size = -1, an = -1, alpha = -1; > - int x1, y1, x2, y2, t1 = -1, t2 = -1; > + int len, size = -1, an = -1, alpha = -1, scale = 0; > -+ int x1, y1, x2, y2, t1 = -1, t2 = -1, accel = 1; > ++ float f1 = 1; > ++ int x1, y1, x2, y2, x3, t1 = -1, t2 = -1, t3 = -1, > t4 = -1, accel = 1; > if (sscanf(buf, "\\%1[bisu]%1[01\\}]%n", style, c, > &len) > 1) { > int close = c[0] == '0' ? 1 : c[0] == '1' ? 0 : > -1; > len += close != -1; > -@@ libavutil/ass_split.c: int ff_ass_split_override_codes(const > ASSCodesCallbacks *callbacks, void *priv, > ++ switch (c[0]) { > ++ case 'b': > ++ ass_write_filtered_line(outbuffer, buf, > len, keep_flags, ASS_SPLIT_FONT_BOLD); > ++ break; > ++ case 'u': > ++ ass_write_filtered_line(outbuffer, buf, > len, keep_flags, ASS_SPLIT_FONT_UNDERLINE); > ++ break; > ++ case 'i': > ++ ass_write_filtered_line(outbuffer, buf, > len, keep_flags, ASS_SPLIT_FONT_ITALIC); > ++ break; > ++ case 'a': > ++ ass_write_filtered_line(outbuffer, buf, > len, keep_flags, ASS_SPLIT_FONT_STRIKEOUT); > ++ break; > ++ } > + if (callbacks->style) > + callbacks->style(priv, style[0], close); > + } else if (sscanf(buf, "\\c%1[\\}]%n", sep, &len) > > 0 || > + sscanf(buf, "\\c&H%X&%1[\\}]%n", &color, > sep, &len) > 1 || > + sscanf(buf, "\\%1[1234]c%1[\\}]%n", > c_num, sep, &len) > 1 || > + sscanf(buf, "\\%1[1234]c&H%X&%1[\\}]%n", > c_num, &color, sep, &len) > 2) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_COLOR); > + if (callbacks->color) > + callbacks->color(priv, color, c_num[0] - > '0'); > + } else if (sscanf(buf, "\\alpha%1[\\}]%n", sep, > &len) > 0 || > + sscanf(buf, "\\alpha&H%2X&%1[\\}]%n", > &alpha, sep, &len) > 1 || > + sscanf(buf, "\\%1[1234]a%1[\\}]%n", > c_num, sep, &len) > 1 || > + sscanf(buf, > "\\%1[1234]a&H%2X&%1[\\}]%n", c_num, &alpha, sep, &len) > 2) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_ALPHA); > + if (callbacks->alpha) > + callbacks->alpha(priv, alpha, c_num[0] - > '0'); > + } else if (sscanf(buf, "\\fn%1[\\}]%n", sep, &len) > > 0 || > + sscanf(buf, "\\fn%127[^\\}]%1[\\}]%n", > tmp, sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_FONT_NAME); > + if (callbacks->font_name) > + callbacks->font_name(priv, tmp[0] ? tmp : > NULL); > + } else if (sscanf(buf, "\\fs%1[\\}]%n", sep, &len) > > 0 || > + sscanf(buf, "\\fs%u%1[\\}]%n", &size, > sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_FONT_SIZE); > + if (callbacks->font_size) > + callbacks->font_size(priv, size); > ++ } else if (sscanf(buf, "\\fscx%1[\\}]%n", sep, > &len) > 0 || > ++ sscanf(buf, "\\fscx%f%1[\\}]%n", &f1, > sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_FONT_SCALE); > ++ } else if (sscanf(buf, "\\fscy%1[\\}]%n", sep, > &len) > 0 || > ++ sscanf(buf, "\\fscy%f%1[\\}]%n", &f1, > sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_FONT_SCALE); > ++ } else if (sscanf(buf, "\\fsp%1[\\}]%n", sep, &len) > > 0 || > ++ sscanf(buf, "\\fsp%u%1[\\}]%n", &size, > sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_FONT_SPACING); > ++ } else if (sscanf(buf, "\\fe%1[\\}]%n", sep, &len) > > 0 || > ++ sscanf(buf, "\\fe%u%1[\\}]%n", &size, > sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_FONT_CHARSET); > ++ } else if (sscanf(buf, "\\bord%1[\\}]%n", sep, > &len) > 0 || > ++ sscanf(buf, "\\bord%u%1[\\}]%n", &size, > sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_TEXT_BORDER); > ++ } else if (sscanf(buf, "\\shad%1[\\}]%n", sep, > &len) > 0 || > ++ sscanf(buf, "\\shad%u%1[\\}]%n", &size, > sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_TEXT_SHADOW); > ++ } else if (sscanf(buf, "\\fr%1[\\}]%n", sep, &len) > > 0 || > ++ sscanf(buf, "\\fr%u%1[\\}]%n", &x1, sep, > &len) > 1 || > ++ sscanf(buf, "\\fr%1[xyz]%1[\\}]%n", > axis, sep, &len) > 1 || > ++ sscanf(buf, "\\fr%1[xyz]%u%1[\\}]%n", > axis, &size, sep, &len) > 2) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_TEXT_ROTATE); > ++ } else if (sscanf(buf, "\\blur%1[\\}]%n", sep, > &len) > 0 || > ++ sscanf(buf, "\\blur%u%1[\\}]%n", &size, > sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_TEXT_BLUR); > ++ } else if (sscanf(buf, "\\be%1[\\}]%n", sep, &len) > > 0 || > ++ sscanf(buf, "\\be%u%1[\\}]%n", &size, > sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_TEXT_BLUR); > ++ } else if (sscanf(buf, "\\q%1[\\}]%n", sep, &len) > > 0 || > ++ sscanf(buf, "\\q%u%1[\\}]%n", &size, > sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_TEXT_WRAP); > + } else if (sscanf(buf, "\\a%1[\\}]%n", sep, &len) > > 0 || > + sscanf(buf, "\\a%2u%1[\\}]%n", &an, sep, > &len) > 1 || > + sscanf(buf, "\\an%1[\\}]%n", sep, &len) > > 0 || > + sscanf(buf, "\\an%1u%1[\\}]%n", &an, > sep, &len) > 1) { > + if (an != -1 && buf[2] != 'n') > + an = (an&3) + (an&4 ? 6 : an&8 ? 3 : 0); > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_TEXT_ALIGNMENT); > + if (callbacks->alignment) > + callbacks->alignment(priv, an); > + } else if (sscanf(buf, "\\r%1[\\}]%n", sep, &len) > > 0 || > + sscanf(buf, "\\r%127[^\\}]%1[\\}]%n", > tmp, sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_CANCELLING); > + if (callbacks->cancel_overrides) > + callbacks->cancel_overrides(priv, tmp); > + } else if (sscanf(buf, > "\\move(%d,%d,%d,%d)%1[\\}]%n", &x1, &y1, &x2, &y2, sep, &len) > 4 || > + sscanf(buf, > "\\move(%d,%d,%d,%d,%d,%d)%1[\\}]%n", &x1, &y1, &x2, &y2, &t1, &t2, sep, > &len) > 6) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_MOVE); > + if (callbacks->move) > + callbacks->move(priv, x1, y1, x2, y2, t1, > t2); > + } else if (sscanf(buf, "\\pos(%d,%d)%1[\\}]%n", > &x1, &y1, sep, &len) > 2) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_POS); > + if (callbacks->move) > + callbacks->move(priv, x1, y1, x1, y1, -1, > -1); > } else if (sscanf(buf, "\\org(%d,%d)%1[\\}]%n", > &x1, &y1, sep, &len) > 2) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_ORIGIN); > if (callbacks->origin) > callbacks->origin(priv, x1, y1); > -+ } else if (sscanf(buf, "\\t(%d,%d,%1[\\}]%n", &t1, > &t2, sep, &len) > 2 || > ++ } else if (sscanf(buf, "\\t(%1[\\}]%n", sep, &len) > > 0 || > ++ sscanf(buf, "\\t(%d,%d,%1[\\}]%n", &t1, > &t2, sep, &len) > 2 || > + sscanf(buf, "\\t(%d,%d,%d,%1[\\}]%n", > &t1, &t2, &accel, sep, &len) > 3) { > ++ > ++ len = strcspn(buf, ")") + 2; > ++ > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_ANIMATE); > + if (callbacks->animate) > + callbacks->animate(priv, t1, t2, accel, > tmp); > ++ } else if (sscanf(buf, > "\\fade(%d,%d,%d,%d,%d,%d,%d)%1[\\}]%n", &x1, &x2, &x3, &t1, &t2, &t3, &t4, > sep, &len) > 7) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_FADE); > ++ } else if (sscanf(buf, "\\fad(%d,%d)%1[\\}]%n", > &t1, &t2, sep, &len) > 2) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_FADE); > ++ } else if (sscanf(buf, > "\\clip(%d,%d,%d,%d)%1[\\}]%n", &x1, &y1, &x2, &y2, sep, &len) > 4) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_CLIP); > + } else if (sscanf(buf, "\\p%1[\\}]%n", sep, &len) > > 0 || > + sscanf(buf, "\\p%u%1[\\}]%n", &scale, > sep, &len) > 1) { > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_DRAW); > + if (callbacks->drawing_mode) > + callbacks->drawing_mode(priv, scale); > } else { > - len = strcspn(buf+1, "\\}") + 2; /* skip > unknown code */ > - } > +- len = strcspn(buf+1, "\\}") + 2; /* skip > unknown code */ > +- } > ++ len = strcspn(buf+1, "\\}") + 2; /* unknown > code */ > ++ ass_write_filtered_line(outbuffer, buf, len, > keep_flags, ASS_SPLIT_UNKNOWN); > ++ } > + buf += len - 1; > + } > + if (*buf++ != '}') > + return AVERROR_INVALIDDATA; > +- } else { > ++ > ++ ass_write_filtered_line(outbuffer, "}", 2, keep_flags, > ASS_SPLIT_ANY); > ++ } else { > + if (!text) { > + text = buf; > + text_len = 1; > @@ libavutil/ass_split.c: int ff_ass_split_override_codes(const > ASSCodesCallbacks *callbacks, void *priv, > - return 0; > + buf++; > + } > + } > ++ if (text) > ++ ass_write_filtered_line(outbuffer, text, text_len + 1, > keep_flags, ASS_SPLIT_TEXT | ASS_SPLIT_TEXT2); > + if (text && callbacks->text) > + callbacks->text(priv, text, text_len); > + if (callbacks->end) > + callbacks->end(priv); > +- return 0; > ++ > ++ if (outbuffer) > ++ ret = ass_remove_empty_braces(outbuffer); > ++ > ++ return ret; > ++} > ++ > ++ > ++int avpriv_ass_split_override_codes(const ASSCodesCallbacks > *callbacks, void *priv, > ++ const char *buf) > ++{ > ++ return avpriv_ass_filter_override_codes(callbacks, priv, buf, > NULL, 0); > } > > -ASSStyle *ff_ass_style_get(ASSSplitContext *ctx, const char *style) > @@ libavutil/ass_split.c: int ff_ass_split_override_codes(const > ASSCodesCallbacks * > ASS *ass = &ctx->ass; > int i; > > - ## libavcodec/ass_split.h => libavutil/ass_split_internal.h ## > + ## libavutil/ass_split_internal.h (new) ## > @@ > - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA > - */ > - > --#ifndef AVCODEC_ASS_SPLIT_H > --#define AVCODEC_ASS_SPLIT_H > ++/* > ++ * SSA/ASS spliting functions > ++ * Copyright (c) 2010 Aurelien Jacobs <aurel@gnuage.org> > ++ * > ++ * This file is part of FFmpeg. > ++ * > ++ * FFmpeg is free software; you can redistribute it and/or > ++ * modify it under the terms of the GNU Lesser General Public > ++ * License as published by the Free Software Foundation; either > ++ * version 2.1 of the License, or (at your option) any later > version. > ++ * > ++ * FFmpeg is distributed in the hope that it will be useful, > ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of > ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > ++ * Lesser General Public License for more details. > ++ * > ++ * You should have received a copy of the GNU Lesser General Public > ++ * License along with FFmpeg; if not, write to the Free Software > ++ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA > ++ */ > ++ > +#ifndef AVUTIL_ASS_SPLIT_INTERNAL_H > +#define AVUTIL_ASS_SPLIT_INTERNAL_H > - > - /** > - * fields extracted from the [Script Info] section > -@@ libavutil/ass_split_internal.h: typedef struct { > - char *effect; > - char *text; /**< actual text which will be displayed as a > subtitle, > - can include style override control codes > (see > -- ff_ass_split_override_codes()) */ > -+ avpriv_ass_split_override_codes()) */ > - } ASSDialog; > - > - /** > -@@ libavutil/ass_split_internal.h: typedef struct ASSSplitContext > ASSSplitContext; > - * @param buf String containing the ASS formatted data. > - * @return Newly allocated struct containing split data. > - */ > --ASSSplitContext *ff_ass_split(const char *buf); > ++ > ++#include "bprint.h" > ++ > ++enum ASSSplitComponents > ++{ > ++ ASS_SPLIT_ANY = 0, > ++ ASS_SPLIT_TEXT = (1 << 0), > ++ ASS_SPLIT_TEXT2 = (1 << 1), // Same semantics as > ASS_SPLIT_TEXT. To work around help output default display. > ++ ASS_SPLIT_COLOR = (1 << 2), > ++ ASS_SPLIT_ALPHA = (1 << 3), > ++ ASS_SPLIT_FONT_NAME = (1 << 4), > ++ ASS_SPLIT_FONT_SIZE = (1 << 5), > ++ ASS_SPLIT_FONT_SCALE = (1 << 6), > ++ ASS_SPLIT_FONT_SPACING = (1 << 7), > ++ ASS_SPLIT_FONT_CHARSET = (1 << 8), > ++ ASS_SPLIT_FONT_BOLD = (1 << 9), > ++ ASS_SPLIT_FONT_ITALIC = (1 << 10), > ++ ASS_SPLIT_FONT_UNDERLINE = (1 << 11), > ++ ASS_SPLIT_FONT_STRIKEOUT = (1 << 12), > ++ ASS_SPLIT_TEXT_BORDER = (1 << 13), > ++ ASS_SPLIT_TEXT_SHADOW = (1 << 14), > ++ ASS_SPLIT_TEXT_ROTATE = (1 << 15), > ++ ASS_SPLIT_TEXT_BLUR = (1 << 16), > ++ ASS_SPLIT_TEXT_WRAP = (1 << 17), > ++ ASS_SPLIT_TEXT_ALIGNMENT = (1 << 18), > ++ ASS_SPLIT_CANCELLING = (1 << 19), > ++ ASS_SPLIT_MOVE = (1 << 20), > ++ ASS_SPLIT_POS = (1 << 21), > ++ ASS_SPLIT_ORIGIN = (1 << 22), > ++ ASS_SPLIT_DRAW = (1 << 23), > ++ ASS_SPLIT_ANIMATE = (1 << 24), > ++ ASS_SPLIT_FADE = (1 << 25), > ++ ASS_SPLIT_CLIP = (1 << 26), > ++ ASS_SPLIT_UNKNOWN = (1 << 27), > ++ > ++ ASS_SPLIT_BASIC = ASS_SPLIT_TEXT2 | ASS_SPLIT_COLOR | > ASS_SPLIT_ALPHA | ASS_SPLIT_FONT_NAME | ASS_SPLIT_FONT_SIZE | > ASS_SPLIT_FONT_SCALE | ASS_SPLIT_FONT_SPACING | ASS_SPLIT_FONT_CHARSET | > ASS_SPLIT_FONT_BOLD | ASS_SPLIT_FONT_ITALIC | ASS_SPLIT_FONT_UNDERLINE | > ASS_SPLIT_FONT_STRIKEOUT | ASS_SPLIT_TEXT_BORDER | ASS_SPLIT_TEXT_SHADOW | > ASS_SPLIT_TEXT_WRAP | ASS_SPLIT_TEXT_ALIGNMENT | ASS_SPLIT_POS | > ASS_SPLIT_CANCELLING, > ++ ASS_SPLIT_ALL_KNOWN = ASS_SPLIT_TEXT2 | ASS_SPLIT_COLOR | > ASS_SPLIT_ALPHA | ASS_SPLIT_FONT_NAME | ASS_SPLIT_FONT_SIZE | > ASS_SPLIT_FONT_SCALE | ASS_SPLIT_FONT_SPACING | ASS_SPLIT_FONT_CHARSET | > ASS_SPLIT_FONT_BOLD | ASS_SPLIT_FONT_ITALIC | ASS_SPLIT_FONT_UNDERLINE | > ASS_SPLIT_FONT_STRIKEOUT | ASS_SPLIT_TEXT_BORDER | ASS_SPLIT_TEXT_SHADOW | > ASS_SPLIT_TEXT_ROTATE | ASS_SPLIT_TEXT_BLUR | ASS_SPLIT_TEXT_WRAP | > ASS_SPLIT_TEXT_ALIGNMENT | ASS_SPLIT_CANCELLING | ASS_SPLIT_POS | > ASS_SPLIT_MOVE | ASS_SPLIT_ORIGIN | ASS_SPLIT_DRAW | ASS_SPLIT_ANIMATE | > ASS_SPLIT_FADE | ASS_SPLIT_CLIP, > ++}; > ++ > ++ /** > ++ * fields extracted from the [Script Info] section > ++ */ > ++ typedef struct { > ++ char *script_type; /**< SSA script format version (eg. v4.00) > */ > ++ char *collisions; /**< how subtitles are moved to prevent > collisions */ > ++ int play_res_x; /**< video width that ASS coords are referring > to */ > ++ int play_res_y; /**< video height that ASS coords are > referring to */ > ++ float timer; /**< time multiplier to apply to SSA clock (in > %) */ > ++} ASSScriptInfo; > ++ > ++/** > ++ * fields extracted from the [V4(+) Styles] section > ++ */ > ++typedef struct { > ++ char *name; /**< name of the tyle (case sensitive) */ > ++ char *font_name; /**< font face (case sensitive) */ > ++ int font_size; /**< font height */ > ++ int primary_color; /**< color that a subtitle will normally > appear in */ > ++ int secondary_color; > ++ int outline_color; /**< color for outline in ASS, called tertiary > in SSA */ > ++ int back_color; /**< color of the subtitle outline or shadow */ > ++ int bold; /**< whether text is bold (1) or not (0) */ > ++ int italic; /**< whether text is italic (1) or not (0) */ > ++ int underline; /**< whether text is underlined (1) or not (0) > */ > ++ int strikeout; > ++ float scalex; > ++ float scaley; > ++ float spacing; > ++ float angle; > ++ int border_style; > ++ float outline; > ++ float shadow; > ++ int alignment; /**< position of the text (left, center, top...), > ++ defined after the layout of the numpad > ++ (1-3 sub, 4-6 mid, 7-9 top) */ > ++ int margin_l; > ++ int margin_r; > ++ int margin_v; > ++ int alpha_level; > ++ int encoding; > ++} ASSStyle; > ++ > ++/** > ++ * fields extracted from the [Events] section > ++ */ > ++typedef struct { > ++ int readorder; > ++ int layer; /**< higher numbered layers are drawn over lower > numbered */ > ++ int start; /**< start time of the dialog in centiseconds */ > ++ int end; /**< end time of the dialog in centiseconds */ > ++ char *style; /**< name of the ASSStyle to use with this dialog */ > ++ char *name; > ++ int margin_l; > ++ int margin_r; > ++ int margin_v; > ++ char *effect; > ++ char *text; /**< actual text which will be displayed as a > subtitle, > ++ can include style override control codes (see > ++ avpriv_ass_split_override_codes()) */ > ++} ASSDialog; > ++ > ++/** > ++ * structure containing the whole split ASS data > ++ */ > ++typedef struct { > ++ ASSScriptInfo script_info; /**< general information about the SSA > script*/ > ++ ASSStyle *styles; /**< array of split out styles */ > ++ int styles_count; /**< number of ASSStyle in the styles > array */ > ++ ASSDialog *dialogs; /**< array of split out dialogs */ > ++ int dialogs_count; /**< number of ASSDialog in the > dialogs array*/ > ++} ASS; > ++ > ++/** > ++ * This struct can be casted to ASS to access to the split data. > ++ */ > ++typedef struct ASSSplitContext ASSSplitContext; > ++ > ++/** > ++ * Split a full ASS file or a ASS header from a string buffer and > store > ++ * the split structure in a newly allocated context. > ++ * > ++ * @param buf String containing the ASS formatted data. > ++ * @return Newly allocated struct containing split data. > ++ */ > +ASSSplitContext *avpriv_ass_split(const char *buf); > - > - /** > -- * Free a dialogue obtained from ff_ass_split_dialog(). > ++ > ++/** > + * Free a dialogue obtained from avpriv_ass_split_dialog(). > - */ > --void ff_ass_free_dialog(ASSDialog **dialogp); > ++ */ > +void avpriv_ass_free_dialog(ASSDialog **dialogp); > - > - /** > - * Split one ASS Dialogue line from a string buffer. > -@@ libavutil/ass_split_internal.h: void ff_ass_free_dialog(ASSDialog > **dialogp); > - * @param buf String containing the ASS "Dialogue" line. > - * @return Pointer to the split ASSDialog. Must be freed with > ff_ass_free_dialog() > - */ > --ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char > *buf); > ++ > ++/** > ++ * Split one ASS Dialogue line from a string buffer. > ++ * > ++ * @param ctx Context previously initialized by ff_ass_split(). > ++ * @param buf String containing the ASS "Dialogue" line. > ++ * @return Pointer to the split ASSDialog. Must be freed with > ++ * ff_ass_free_dialog() > ++ */ > +ASSDialog *avpriv_ass_split_dialog(ASSSplitContext *ctx, const char > *buf); > - > - /** > - * Free all the memory allocated for an ASSSplitContext. > - * > - * @param ctx Context previously initialized by ff_ass_split(). > - */ > --void ff_ass_split_free(ASSSplitContext *ctx); > ++ > ++/** > ++ * Free all the memory allocated for an ASSSplitContext. > ++ * > ++ * @param ctx Context previously initialized by ff_ass_split(). > ++ */ > +void avpriv_ass_split_free(ASSSplitContext *ctx); > - > - > - /** > -@@ libavutil/ass_split_internal.h: typedef struct { > - * @{ > - */ > - void (*text)(void *priv, const char *text, int len); > -+ void (*hard_space)(void *priv); > - void (*new_line)(void *priv, int forced); > - void (*style)(void *priv, char style, int close); > - void (*color)(void *priv, unsigned int /* color */, unsigned > int color_id); > -@@ libavutil/ass_split_internal.h: typedef struct { > - * @{ > - */ > - void (*move)(void *priv, int x1, int y1, int x2, int y2, int > t1, int t2); > -+ void (*animate)(void *priv, int t1, int t2, int accel, char > *style); > - void (*origin)(void *priv, int x, int y); > -+ void (*drawing_mode)(void *priv, int scale); > -+ /** @} */ > + > -+ /** > -+ * @defgroup ass_ext ASS extensible parsing callback > -+ * @{ > -+ */ > -+ void (*ext)(void *priv, int ext_id, const char *text, int p1, > int p2); > - /** @} */ > - > - /** > -@@ libavutil/ass_split_internal.h: typedef struct { > - * @param buf The ASS "Dialogue" Text field to split. > - * @return >= 0 on success otherwise an error code <0 > - */ > --int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, > void *priv, > -+int avpriv_ass_split_override_codes(const ASSCodesCallbacks > *callbacks, void *priv, > - const char *buf); > - > - /** > -@@ libavutil/ass_split_internal.h: int > ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, > - * @param style name of the style to search for. > - * @return the ASSStyle corresponding to style, or NULL if style > can't be found > - */ > --ASSStyle *ff_ass_style_get(ASSSplitContext *ctx, const char *style); > ++/** > ++ * Set of callback functions corresponding to each override codes > that can > ++ * be encountered in a "Dialogue" Text field. > ++ */ > ++typedef struct { > ++ /** > ++ * @defgroup ass_styles ASS styles > ++ * @{ > ++ */ > ++ void (*text)(void *priv, const char *text, int len); > ++ void (*hard_space)(void *priv); > ++ void (*new_line)(void *priv, int forced); > ++ void (*style)(void *priv, char style, int close); > ++ void (*color)(void *priv, unsigned int /* color */, unsigned int > color_id); > ++ void (*alpha)(void *priv, int alpha, int alpha_id); > ++ void (*font_name)(void *priv, const char *name); > ++ void (*font_size)(void *priv, int size); > ++ void (*alignment)(void *priv, int alignment); > ++ void (*cancel_overrides)(void *priv, const char *style); > ++ /** @} */ > ++ > ++ /** > ++ * @defgroup ass_functions ASS functions > ++ * @{ > ++ */ > ++ void (*move)(void *priv, int x1, int y1, int x2, int y2, int t1, > int t2); > ++ void (*animate)(void *priv, int t1, int t2, int accel, char > *style); > ++ void (*origin)(void *priv, int x, int y); > ++ void (*drawing_mode)(void *priv, int scale); > ++ /** @} */ > ++ > ++ /** > ++ * @defgroup ass_ext ASS extensible parsing callback > ++ * @{ > ++ */ > ++ void (*ext)(void *priv, int ext_id, const char *text, int p1, int > p2); > ++ /** @} */ > ++ > ++ /** > ++ * @defgroup ass_end end of Dialogue Event > ++ * @{ > ++ */ > ++ void (*end)(void *priv); > ++ /** @} */ > ++} ASSCodesCallbacks; > ++ > ++/** > ++ * Split override codes out of a ASS "Dialogue" Text field. > ++ * > ++ * @param callbacks Set of callback functions called for each > override code > ++ * encountered. > ++ * @param priv Opaque pointer passed to the callback functions. > ++ * @param buf The ASS "Dialogue" Text field to split. > ++ * @param outbuffer The output buffer. > ++ * @param keep_flags Flags for filtering ass codes. > ++ * @return >= 0 on success otherwise an error code <0 > ++ */ > ++int avpriv_ass_filter_override_codes(const ASSCodesCallbacks > *callbacks, > ++ void *priv, const char *buf, > ++ AVBPrint *outbuffer, enum > ASSSplitComponents keep_flags); > ++ > ++/** > ++ * Split override codes out of a ASS "Dialogue" Text field. > ++ * > ++ * @param callbacks Set of callback functions called for each > override code > ++ * encountered. > ++ * @param priv Opaque pointer passed to the callback functions. > ++ * @param buf The ASS "Dialogue" Text field to split. > ++ * @return >= 0 on success otherwise an error code <0 > ++ */ > ++int avpriv_ass_split_override_codes(const ASSCodesCallbacks > *callbacks, > ++ void *priv, const char *buf); > ++ > ++/** > ++ * Find an ASSStyle structure by its name. > ++ * > ++ * @param ctx Context previously initialized by ff_ass_split(). > ++ * @param style name of the style to search for. > ++ * @return the ASSStyle corresponding to style, or NULL if style > can't be found > ++ */ > +ASSStyle *avpriv_ass_style_get(ASSSplitContext *ctx, const char > *style); > - > --#endif /* AVCODEC_ASS_SPLIT_H */ > ++ > +#endif /* AVUTIL_ASS_SPLIT_INTERNAL_H */ > 7: 09d8cf7880 = 7: 98f12ad7e9 avcodec/subtitles: Replace deprecated > enum values > 8: 897299bf7f = 8: 12c8a308d3 fftools/play,probe: Adjust for subtitle > changes > 9: ca580c6d21 = 9: 2e55dbe180 avfilter/subtitles: Add subtitles.c for > subtitle frame allocation > 10: 0781e974a2 = 10: c931041103 avfilter/avfilter: Handle subtitle > frames > 11: d9d9f42558 = 11: 36cab55ff2 avfilter/avfilter: Fix hardcoded input > index > 12: af69a4b321 = 12: f41070479c avfilter/sbuffer: Add sbuffersrc and > sbuffersink filters > 13: f7e5b590a2 ! 13: 9bfaba4ace avfilter/overlaygraphicsubs: Add > overlaygraphicsubs and graphicsub2video filters > @@ libavfilter/allfilters.c: extern const AVFilter ff_vf_overlay_qsv; > extern const AVFilter ff_vf_owdenoise; > extern const AVFilter ff_vf_pad; > extern const AVFilter ff_vf_pad_opencl; > -@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showvolume; > +@@ libavfilter/allfilters.c: extern const AVFilter > ff_avf_showspectrumpic; > + extern const AVFilter ff_avf_showvolume; > extern const AVFilter ff_avf_showwaves; > extern const AVFilter ff_avf_showwavespic; > - extern const AVFilter ff_vaf_spectrumsynth; > +extern const AVFilter ff_svf_graphicsub2video; > + extern const AVFilter ff_vaf_spectrumsynth; > > /* multimedia sources */ > - extern const AVFilter ff_avsrc_avsynctest; > > ## libavfilter/vf_overlaygraphicsubs.c (new) ## > @@ > 14: 4c8092357f ! 14: 918fd9aaf5 avfilter/overlaytextsubs: Add > overlaytextsubs and textsubs2video filters > @@ libavfilter/allfilters.c: extern const AVFilter > ff_vf_overlay_vaapi; > extern const AVFilter ff_vf_owdenoise; > extern const AVFilter ff_vf_pad; > extern const AVFilter ff_vf_pad_opencl; > -@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showwaves; > +@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showvolume; > + extern const AVFilter ff_avf_showwaves; > extern const AVFilter ff_avf_showwavespic; > - extern const AVFilter ff_vaf_spectrumsynth; > extern const AVFilter ff_svf_graphicsub2video; > +extern const AVFilter ff_svf_textsub2video; > + extern const AVFilter ff_vaf_spectrumsynth; > > /* multimedia sources */ > - extern const AVFilter ff_avsrc_avsynctest; > > ## libavfilter/vf_overlaytextsubs.c (new) ## > @@ > 15: 8fdbdf7c5f ! 15: a361ad35c5 avfilter/textmod: Add textmod, censor > and show_speaker filters > @@ libavfilter/Makefile: OBJS-$(CONFIG_YUVTESTSRC_FILTER) > += vsrc_tests > OBJS-$(CONFIG_ADRAWGRAPH_FILTER) += f_drawgraph.o > > ## libavfilter/allfilters.c ## > -@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showvolume; > - extern const AVFilter ff_avf_showwaves; > - extern const AVFilter ff_avf_showwavespic; > - extern const AVFilter ff_vaf_spectrumsynth; > +@@ libavfilter/allfilters.c: extern const AVFilter > ff_avsrc_avsynctest; > + extern const AVFilter ff_avsrc_amovie; > + extern const AVFilter ff_avsrc_movie; > + > ++/* subtitle filters */ > +extern const AVFilter ff_sf_censor; > +extern const AVFilter ff_sf_showspeaker; > +extern const AVFilter ff_sf_textmod; > - extern const AVFilter ff_svf_graphicsub2video; > - extern const AVFilter ff_svf_textsub2video; > - > ++ > + /* those filters are part of public or internal API, > + * they are formatted to not be found by the grep > + * as they are manually added again (due to their 'names' > > ## libavfilter/sf_textmod.c (new) ## > @@ > @@ libavfilter/sf_textmod.c (new) > + > + av_bprint_finalize(&pbuf, &text); > + > -+ result = avpriv_ass_get_dialog_ex(dialog->readorder, > dialog->layer, dialog->style, dialog->name, dialog->margin_l, > dialog->margin_r, dialog->margin_v, text); > ++ result = avpriv_ass_get_dialog_ex(dialog->readorder, > dialog->layer, dialog->style, dialog->name, dialog->margin_l, > dialog->margin_r, dialog->margin_v, dialog->effect, text); > + > + av_free(text); > + avpriv_ass_free_dialog(&dialog); > @@ libavfilter/sf_textmod.c (new) > + if (!text) > + return NULL; > + > -+ result = avpriv_ass_get_dialog_ex(dialog->readorder, > dialog->layer, dialog->style, dialog->name, dialog->margin_l, > dialog->margin_r, dialog->margin_v, text); > ++ result = avpriv_ass_get_dialog_ex(dialog->readorder, > dialog->layer, dialog->style, dialog->name, dialog->margin_l, > dialog->margin_r, dialog->margin_v, dialog->effect, text); > + > + av_free(text); > + avpriv_ass_free_dialog(&dialog); > 16: d44b22f15b ! 16: bca90ebc3e avfilter/stripstyles: Add stripstyles > filter > @@ doc/filters.texi: ffmpeg -i " > http://streams.videolan.org/samples/sub/SSA/subtitl > > ## libavfilter/Makefile ## > @@ libavfilter/Makefile: OBJS-$(CONFIG_NULLSINK_FILTER) > += vsink_nullsink.o > + # subtitle filters > OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o > OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o > - OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o > +OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o > + OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o > > # multimedia filters > - OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o > > ## libavfilter/allfilters.c ## > -@@ libavfilter/allfilters.c: extern const AVFilter > ff_avf_showwavespic; > - extern const AVFilter ff_vaf_spectrumsynth; > +@@ libavfilter/allfilters.c: extern const AVFilter ff_avsrc_movie; > + /* subtitle filters */ > extern const AVFilter ff_sf_censor; > extern const AVFilter ff_sf_showspeaker; > +extern const AVFilter ff_sf_stripstyles; > extern const AVFilter ff_sf_textmod; > - extern const AVFilter ff_svf_graphicsub2video; > - extern const AVFilter ff_svf_textsub2video; > + > + /* those filters are part of public or internal API, > > ## libavfilter/sf_stripstyles.c (new) ## > @@ > @@ libavfilter/sf_stripstyles.c (new) > + > +#include "libavutil/opt.h" > +#include "internal.h" > ++#include "libavutil/ass_internal.h" > +#include "libavutil/ass_split_internal.h" > +#include "libavutil/bprint.h" > + > @@ libavfilter/sf_stripstyles.c (new) > + const AVClass *class; > + enum AVSubtitleType format; > + int remove_animated; > ++ enum ASSSplitComponents keep_flags; > + int select_layer; > +} StripStylesContext; > + > @@ libavfilter/sf_stripstyles.c (new) > + AVBPrint buffer; > + int drawing_scale; > + int is_animated; > ++ int plain_text_length; > +} DialogContext; > + > +static void dialog_text_cb(void *priv, const char *text, int len) > @@ libavfilter/sf_stripstyles.c (new) > + av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text); > + > + if (!s->drawing_scale && (!s->is_animated || > !s->ss_ctx->remove_animated)) > -+ av_bprint_append_data(&s->buffer, text, len); > ++ s->plain_text_length += len; > ++ ////av_bprint_append_data(&s->buffer, text, len); > +} > + > +static void dialog_new_line_cb(void *priv, int forced) > +{ > + DialogContext *s = priv; > + if (!s->drawing_scale && !s->is_animated) > -+ av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", > 2); > ++ s->plain_text_length += 2; > ++ ////av_bprint_append_data(&s->buffer, forced ? "\\N" : > "\\n", 2); > +} > + > +static void dialog_drawing_mode_cb(void *priv, int scale) > @@ libavfilter/sf_stripstyles.c (new) > + .move = dialog_move_cb, > +}; > + > -+static char *ass_get_line(int readorder, int layer, const char > *style, > -+ const char *speaker, const char *effect, > const char *text) > -+{ > -+ return av_asprintf("%d,%d,%s,%s,0,0,0,%s,%s", > -+ readorder, layer, style ? style : "Default", > -+ speaker ? speaker : "", effect, text); > -+} > -+ > +static char *process_dialog(StripStylesContext *s, const char > *ass_line) > +{ > + DialogContext dlg_ctx = { .ss_ctx = s }; > @@ libavfilter/sf_stripstyles.c (new) > + > + dlg_ctx.ss_ctx = s; > + > -+ av_bprint_init(&dlg_ctx.buffer, 0, AV_BPRINT_SIZE_UNLIMITED); > ++ av_bprint_init(&dlg_ctx.buffer, 512, AV_BPRINT_SIZE_UNLIMITED); > + > -+ avpriv_ass_split_override_codes(&dialog_callbacks, &dlg_ctx, > dialog->text); > ++ avpriv_ass_filter_override_codes(&dialog_callbacks, &dlg_ctx, > dialog->text, &dlg_ctx.buffer, s->keep_flags); > + > -+ if (av_bprint_is_complete(&dlg_ctx.buffer) > -+ && dlg_ctx.buffer.len > 0) > -+ result = ass_get_line(dialog->readorder, dialog->layer, > dialog->style, dialog->name, dialog->effect, dlg_ctx.buffer.str); > ++ if (av_bprint_is_complete(&dlg_ctx.buffer) && > dlg_ctx.buffer.len > 0 && dlg_ctx.plain_text_length > 0) > ++ result = avpriv_ass_get_dialog_ex(dialog->readorder, > dialog->layer, dialog->style, dialog->name, dialog->margin_l, > dialog->margin_r, dialog->margin_v, dialog->effect, dlg_ctx.buffer.str); > + > + av_bprint_finalize(&dlg_ctx.buffer, NULL); > + avpriv_ass_free_dialog(&dialog); > @@ libavfilter/sf_stripstyles.c (new) > + area->ass = process_dialog(s, area->ass); > + > + if (area->ass) { > -+ av_log(inlink->dst, AV_LOG_INFO, "original: %d > %s\n", i, tmp); > -+ av_log(inlink->dst, AV_LOG_INFO, "stripped: %d > %s\n", i, area->ass); > ++ av_log(inlink->dst, AV_LOG_DEBUG, "original: %d > %s\n", i, tmp); > ++ av_log(inlink->dst, AV_LOG_DEBUG, "stripped: %d > %s\n", i, area->ass); > + } > + else > + area->ass = NULL; > @@ libavfilter/sf_stripstyles.c (new) > +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | > AV_OPT_FLAG_FILTERING_PARAM) > + > +static const AVOption stripstyles_options[] = { > ++ { "keep_flags", "flags to control which override codes to > keep", OFFSET(keep_flags), AV_OPT_TYPE_FLAGS, { .i64 = ASS_SPLIT_TEXT }, > .flags = FLAGS, .unit = "keepflags" }, > ++ { "basic", "keep static style tags only", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_BASIC }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "all_known", "keep all known tags", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ALL_KNOWN }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "text", "keep text content", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "color", "keep color tags (\\c, \\<n>c)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_COLOR }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "alpha", "keep color alpha tags (\\alpha, > \\<n>a)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ALPHA }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "font_name", "keep font name tags (\\fn)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_NAME }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "font_size", "keep font size tags (\\fs)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_SIZE }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "font_scale", "keep font scale tags (\\fscx, > \\fscy)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_SCALE > }, .flags=FLAGS, .unit = "keepflags" }, > ++ { "font_spacing", "keep font spacing tags (\\fsp)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_SPACING }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "font_charset", "keep font charset tags (\\fe)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_CHARSET }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "font_bold", "keep font bold tags (\\b)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_BOLD }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "font_italic", "keep font italic tags (\\i)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_ITALIC }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "font_underline", "keep font underline tags (\\u)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_UNDERLINE }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "font_strikeout", "keep font strikeout tags (\\s)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_STRIKEOUT }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "text_border", "keep text border tags (\\bord)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_BORDER }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "text_shadow", "keep text shadow tags (\\shad)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_SHADOW }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "text_rotate", "keep text rotate tags (\\fr)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_ROTATE }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "text_blur", "keep text blur tags (\\blur, \\be)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_BLUR }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "text_wrap", "keep text wrap tags (\\q)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_WRAP }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "text_align", "keep text align tags (\\a, \\an)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_ALIGNMENT }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "reset_override", "keep override reset tags (\\r)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_CANCELLING }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "move", "keep move tags (\\move)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_MOVE }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "pos", "keep position tags (\\pos)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_POS }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "origin", "keep origin tags (\\org)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ORIGIN }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "draw", "keep drawing tags (\\p)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_DRAW }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "animate", "keep animation tags (\\t)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ANIMATE }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "fade", "keep fade tags (\\fad, \\fade)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FADE }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "clip", "keep clip tags (\\clip)", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_CLIP }, > .flags=FLAGS, .unit = "keepflags" }, > ++ { "unknown", "keep unknown tags", > .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_UNKNOWN }, > .flags=FLAGS, .unit = "keepflags" }, > + { "remove_animated", "remove animated text (default: yes)", > OFFSET(remove_animated), AV_OPT_TYPE_BOOL, {.i64 = 1 }, 0, 1, FLAGS, 0 }, > -+ { "select_layer", "process a specific ass layer only", > OFFSET(remove_animated), AV_OPT_TYPE_INT, {.i64 = -1 }, -1, INT_MAX, > FLAGS, 0 }, > ++ { "select_layer", "process a specific ass layer only", > OFFSET(select_layer), AV_OPT_TYPE_INT, {.i64 = -1 }, -1, INT_MAX, FLAGS, > 0 }, > + { NULL }, > +}; > + > 17: 28d75dc982 ! 17: 6e488e495f avfilter/splitcc: Add splitcc filter > for closed caption handling > @@ doc/filters.texi: ffmpeg -i INPUT -filter_complex > "showspeaker=format=colon:styl > > ## libavfilter/Makefile ## > @@ libavfilter/Makefile: OBJS-$(CONFIG_NULLSINK_FILTER) > += vsink_nullsink.o > + # subtitle filters > OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o > OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o > - OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o > +OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o > OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o > + OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o > > - # multimedia filters > > ## libavfilter/allfilters.c ## > -@@ libavfilter/allfilters.c: extern const AVFilter > ff_avf_showwavespic; > - extern const AVFilter ff_vaf_spectrumsynth; > +@@ libavfilter/allfilters.c: extern const AVFilter ff_avsrc_movie; > + /* subtitle filters */ > extern const AVFilter ff_sf_censor; > extern const AVFilter ff_sf_showspeaker; > +extern const AVFilter ff_sf_splitcc; > extern const AVFilter ff_sf_stripstyles; > extern const AVFilter ff_sf_textmod; > - extern const AVFilter ff_svf_graphicsub2video; > + > > ## libavfilter/sf_splitcc.c (new) ## > @@ > 18: 42d1d1c819 ! 18: 1057dff7da avfilter/graphicsub2text: Add new > graphicsub2text filter (OCR) > @@ libavfilter/Makefile: OBJS-$(CONFIG_GBLUR_FILTER) > += vf_gblur.o > OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o > > ## libavfilter/allfilters.c ## > -@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showwaves; > - extern const AVFilter ff_avf_showwavespic; > - extern const AVFilter ff_vaf_spectrumsynth; > +@@ libavfilter/allfilters.c: extern const AVFilter ff_avsrc_movie; > + > + /* subtitle filters */ > extern const AVFilter ff_sf_censor; > +extern const AVFilter ff_sf_graphicsub2text; > extern const AVFilter ff_sf_showspeaker; > @@ libavfilter/sf_graphicsub2text.c (new) > + } > + } > + > -+ if (pointsize != cur_pointsize && s->recognize & > RFLAGS_FONTSIZE) { > -+ av_log(s, AV_LOG_DEBUG, "pointsize - pointsize: > %d\n", pointsize); > -+ in_code = print_code(&s->buffer, in_code, "\\fs%d", > (int)(pointsize * font_factor)); > -+ cur_pointsize = pointsize; > ++ if (pointsize > 0 && pointsize != cur_pointsize && > s->recognize & RFLAGS_FONTSIZE) { > ++ float change_factor = (float)(FFABS(pointsize - > cur_pointsize)) / FFMAX(pointsize, cur_pointsize); > ++ > ++ // Avoid small changes due to recognition variance > ++ if (change_factor > 0.12f) { > ++ av_log(s, AV_LOG_DEBUG, "pointsize - pointsize: > %d\n", pointsize); > ++ in_code = print_code(&s->buffer, in_code, > "\\fs%d", (int)(pointsize * font_factor)); > ++ cur_pointsize = pointsize; > ++ } > + } > + > + if (is_italic && !cur_is_italic && s->recognize & > RFLAGS_FITALIC) > @@ libavfilter/sf_graphicsub2text.c (new) > + > + const int layer = s->recognize ? i : 0; > + char *tmp = area->ass; > -+ area->ass = > avpriv_ass_get_dialog_ex(s->readorder_counter++, layer, "Default", NULL, 0, > 0, margin_v, tmp); > ++ area->ass = > avpriv_ass_get_dialog_ex(s->readorder_counter++, layer, "Default", NULL, 0, > 0, margin_v, NULL, tmp); > + av_free(tmp); > + } > + } > 19: 7095e8aa26 ! 19: 4e85fb5d2f avfilter/subscale: Add filter for > scaling and/or re-arranging graphical subtitles > @@ doc/filters.texi: Set the rendering margin in pixels. > @chapter Multimedia Filters > > ## libavfilter/Makefile ## > -@@ libavfilter/Makefile: OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) > += sf_textmod.o > - OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o > +@@ libavfilter/Makefile: OBJS-$(CONFIG_CENSOR_FILTER) > += sf_textmod.o > + OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o > OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o > OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o > +OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o > + OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o > > # multimedia filters > - OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o > > ## libavfilter/allfilters.c ## > @@ libavfilter/allfilters.c: extern const AVFilter > ff_sf_graphicsub2text; > @@ libavfilter/allfilters.c: extern const AVFilter > ff_sf_graphicsub2text; > extern const AVFilter ff_sf_stripstyles; > +extern const AVFilter ff_sf_subscale; > extern const AVFilter ff_sf_textmod; > - extern const AVFilter ff_svf_graphicsub2video; > - extern const AVFilter ff_svf_textsub2video; > + > + /* those filters are part of public or internal API, > > ## libavfilter/sf_subscale.c (new) ## > @@ > 20: 697939451e ! 20: 88e8adb889 avfilter/subfeed: add subtitle feed > filter > @@ Commit message > Signed-off-by: softworkz <softworkz@hotmail.com> > > ## libavfilter/Makefile ## > -@@ libavfilter/Makefile: OBJS-$(CONFIG_TEXTMOD_FILTER) > += sf_textmod.o > +@@ libavfilter/Makefile: OBJS-$(CONFIG_CENSOR_FILTER) > += sf_textmod.o > + OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o > OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o > OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o > - OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o > +OBJS-$(CONFIG_SUBFEED_FILTER) += sf_subfeed.o > + OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o > + OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o > > - # multimedia filters > - OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o > > ## libavfilter/allfilters.c ## > -@@ libavfilter/allfilters.c: extern const AVFilter ff_sf_showspeaker; > +@@ libavfilter/allfilters.c: extern const AVFilter > ff_sf_graphicsub2text; > + extern const AVFilter ff_sf_showspeaker; > extern const AVFilter ff_sf_splitcc; > extern const AVFilter ff_sf_stripstyles; > - extern const AVFilter ff_sf_subscale; > +extern const AVFilter ff_sf_subfeed; > + extern const AVFilter ff_sf_subscale; > extern const AVFilter ff_sf_textmod; > - extern const AVFilter ff_svf_graphicsub2video; > - extern const AVFilter ff_svf_textsub2video; > + > > ## libavfilter/sf_subfeed.c (new) ## > @@ > @@ libavfilter/sf_subfeed.c (new) > + if (pts_diff <= 0) { > + av_log(ctx, AV_LOG_WARNING, "The pts_diff to the > previous frame (index #%"PRId64") is <= 0: %"PRId64" ms. The previous > frame duration is %"PRId64" ms.\n", > + index, avtb_to_ms(pts_diff), > avtb_to_ms(previous_frame->subtitle_timing.duration)); > ++ > ++ if (s->fix_overlap) { > ++ av_log(ctx, AV_LOG_VERBOSE, "Removing previous > frame\n"); > ++ previous_frame = ff_framequeue_take(&s->fifo); > ++ while (nb_queued_frames > 1) { > ++ ff_framequeue_add(&s->fifo, previous_frame); > ++ previous_frame = ff_framequeue_take(&s->fifo); > ++ nb_queued_frames--; > ++ } > ++ } > + } > + } > + > -: ---------- > 21: a96bb5c788 avfilter/text2graphicsub: Added > text2graphicsub subtitle filter > -: ---------- > 22: c4922f8466 avfilter/snull,strim: Add snull and > strim filters > 21: 32e9af0806 = 23: 848f84d5dc avcodec/subtitles: Migrate subtitle > encoders to frame-based API > 22: fa0b5c2077 ! 24: 2645a1a842 fftools/ffmpeg: Introduce subtitle > filtering and new frame-based subtitle encoding > @@ Commit message > Overlay results have slightly different CRCs due to different > blending implementation > > + - sub-scc > + The first entry is no longer in the output because it is before > + the actual start time and the strim filter removes such entries > + now (like for video and audio) > + > Signed-off-by: softworkz <softworkz@hotmail.com> > > ## fftools/ffmpeg.c ## > @@ fftools/ffmpeg.c: static int init_output_stream(OutputStream *ost, > AVFrame *fram > return AVERROR_INVALIDDATA; > } > } > +@@ fftools/ffmpeg.c: static int transcode_init(void) > + for (i = 0; i < nb_output_streams; i++) { > + if (!output_streams[i]->stream_copy && > + (output_streams[i]->enc_ctx->codec_type == > AVMEDIA_TYPE_VIDEO || > +- output_streams[i]->enc_ctx->codec_type == > AVMEDIA_TYPE_AUDIO)) > ++ output_streams[i]->enc_ctx->codec_type == > AVMEDIA_TYPE_AUDIO || > ++ output_streams[i]->enc_ctx->codec_type == > AVMEDIA_TYPE_SUBTITLE)) > + continue; > + > + ret = init_output_stream_wrapper(output_streams[i], NULL, > 0); > @@ fftools/ffmpeg.c: static OutputStream *choose_output(void) > av_rescale_q(ost->last_mux_dts, > ost->st->time_base, > AV_TIME_BASE_Q); > @@ fftools/ffmpeg_filter.c: static void init_input_filter(FilterGraph > *fg, AVFilter > continue; > if (check_stream_specifier(s, s->streams[i], *p == ':' > ? p + 1 : p) == 1) { > st = s->streams[i]; > +@@ fftools/ffmpeg_filter.c: static int insert_trim(int64_t > start_time, int64_t duration, > + const char *name = (type == AVMEDIA_TYPE_VIDEO) ? "trim" : > "atrim"; > + int ret = 0; > + > ++ switch (type) { > ++ case AVMEDIA_TYPE_VIDEO: > ++ name = "trim"; > ++ break; > ++ case AVMEDIA_TYPE_AUDIO: > ++ name = "atrim"; > ++ break; > ++ case AVMEDIA_TYPE_SUBTITLE: > ++ name = "strim"; > ++ break; > ++ default: > ++ av_log(NULL, AV_LOG_ERROR, "insert_trim: Invalid media > type: %d\n", type); > ++ return AVERROR_INVALIDDATA; > ++ } > ++ > + if (duration == INT64_MAX && start_time == AV_NOPTS_VALUE) > + return 0; > + > @@ fftools/ffmpeg_filter.c: static int insert_filter(AVFilterContext > **last_filter, int *pad_idx, > return 0; > } > @@ fftools/ffmpeg_filter.c: static int insert_filter(AVFilterContext > **last_filter, > +static int configure_output_subtitle_filter(FilterGraph *fg, > OutputFilter *ofilter, AVFilterInOut *out) > +{ > + OutputStream *ost = ofilter->ost; > ++ OutputFile *of = output_files[ost->file_index]; > + AVFilterContext *last_filter = out->filter_ctx; > + int pad_idx = out->pad_idx; > + int ret; > @@ fftools/ffmpeg_filter.c: static int insert_filter(AVFilterContext > **last_filter, > + return ret; > + } > + > -+ ////snprintf(name, sizeof(name), "trim_out_%d_%d", > -+ //// ost->file_index, ost->index); > -+ ////ret = insert_trim(of->start_time, of->recording_time, > -+ //// &last_filter, &pad_idx, name); > -+ ////if (ret < 0) > -+ //// return ret; > ++ snprintf(name, sizeof(name), "trim_out_%d_%d", > ++ ost->file_index, ost->index); > ++ ret = insert_trim(of->start_time, of->recording_time, > ++ &last_filter, &pad_idx, name); > ++ if (ret < 0) > ++ return ret; > + > + ////ost->st->codecpar->codec_tag = MKTAG('a', 's', 's', 's'); > + > @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) > + AVFilterContext *last_filter; > + const AVFilter *buffer_filt = avfilter_get_by_name("sbuffer"); > + InputStream *ist = ifilter->ist; > ++ InputFile *f = input_files[ist->file_index]; > + AVBPrint args; > + char name[255]; > + int ret, pad_idx = 0; > + int w, h; > ++ int64_t tsoffset = 0; > + AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); > + enum AVMediaType media_type; > + > @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) > - if (avf->streams[i]->codecpar->codec_type == > AVMEDIA_TYPE_VIDEO) { > - w = FFMAX(w, avf->streams[i]->codecpar->width); > - h = FFMAX(h, avf->streams[i]->codecpar->height); > -- } > -- } > -- if (!(w && h)) { > -- w = FFMAX(w, 720); > -- h = FFMAX(h, 576); > -- } > -- av_log(avf, AV_LOG_INFO, "sub2video: using %dx%d canvas\n", > w, h); > + w = ist->dec_ctx->width; > + h = ist->dec_ctx->height; > + } > @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) > + w = ass->script_info.play_res_x; > + h = ass->script_info.play_res_y; > + avpriv_ass_split_free(ass_ctx); > - } > -- ist->sub2video.w = ifilter->width = w; > -- ist->sub2video.h = ifilter->height = h; > - > -- ifilter->width = ist->dec_ctx->width ? ist->dec_ctx->width : > ist->sub2video.w; > -- ifilter->height = ist->dec_ctx->height ? ist->dec_ctx->height : > ist->sub2video.h; > ++ } > ++ > + ist->subtitle_kickoff.w = w; > + ist->subtitle_kickoff.h = h; > + av_log(ifilter, AV_LOG_INFO, "subtitle input filter: decoding > size %dx%d\n", ist->subtitle_kickoff.w, ist->subtitle_kickoff.h); > - > -- /* rectangles are AV_PIX_FMT_PAL8, but we have no guarantee > that the > -- palettes for all rectangles are identical or compatible */ > -- ifilter->format = AV_PIX_FMT_RGB32; > ++ > + ifilter->width = w; > + ifilter->height = h; > + ist->dec_ctx->width = w; > + ist->dec_ctx->height = h; > - > -- ist->sub2video.frame = av_frame_alloc(); > -- if (!ist->sub2video.frame) > -- return AVERROR(ENOMEM); > -- ist->sub2video.last_pts = INT64_MIN; > -- ist->sub2video.end_pts = INT64_MIN; > ++ > + ist->subtitle_kickoff.last_pts = INT64_MIN; > + > + snprintf(name, sizeof(name), "graph %d subtitle input from > stream %d:%d", fg->index, > @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) > + if ((ret = avfilter_graph_create_filter(&ifilter->filter, > buffer_filt, name, > + args.str, NULL, > fg->graph)) < 0) > + goto fail; > - > -- /* sub2video structure has been (re-)initialized. > -- Mark it as such so that the system will be > -- initialized with the first received heartbeat. */ > -- ist->sub2video.initialize = 1; > ++ > + par->hw_frames_ctx = ifilter->hw_frames_ctx; > + par->format = ifilter->format; > + par->width = ifilter->width; > @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) > + subscale_h = input->height; > + break; > + } > -+ } > -+ } > -+ > + } > + } > +- if (!(w && h)) { > +- w = FFMAX(w, 720); > +- h = FFMAX(h, 576); > +- } > +- av_log(avf, AV_LOG_INFO, "sub2video: using %dx%d canvas\n", > w, h); > +- } > +- ist->sub2video.w = ifilter->width = w; > +- ist->sub2video.h = ifilter->height = h; > + > +- ifilter->width = ist->dec_ctx->width ? ist->dec_ctx->width : > ist->sub2video.w; > +- ifilter->height = ist->dec_ctx->height ? ist->dec_ctx->height : > ist->sub2video.h; > + if (subscale_w && subscale_h) { > + char subscale_params[64]; > + snprintf(subscale_params, sizeof(subscale_params), > "w=%d:h=%d", subscale_w, subscale_h); > @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) > + if (ret < 0) > + return ret; > + } > -+ > + > +- /* rectangles are AV_PIX_FMT_PAL8, but we have no guarantee > that the > +- palettes for all rectangles are identical or compatible */ > +- ifilter->format = AV_PIX_FMT_RGB32; > + av_log(NULL, AV_LOG_INFO, "Auto-inserting graphicsub2video > filter\n"); > + ret = insert_filter(&last_filter, &pad_idx, > "graphicsub2video", NULL); > + if (ret < 0) > + return ret; > + } > -+ > + > +- ist->sub2video.frame = av_frame_alloc(); > +- if (!ist->sub2video.frame) > +- return AVERROR(ENOMEM); > +- ist->sub2video.last_pts = INT64_MIN; > +- ist->sub2video.end_pts = INT64_MIN; > ++ if (copy_ts) { > ++ tsoffset = f->start_time == AV_NOPTS_VALUE ? 0 : > f->start_time; > ++ if (!start_at_zero && f->ctx->start_time != AV_NOPTS_VALUE) > ++ tsoffset += f->ctx->start_time; > ++ } > ++ ret = insert_trim(((f->start_time == AV_NOPTS_VALUE) || > !f->accurate_seek) ? > ++ AV_NOPTS_VALUE : tsoffset, f->recording_time, > ++ &last_filter, &pad_idx, name); > ++ if (ret < 0) > ++ return ret; > + > +- /* sub2video structure has been (re-)initialized. > +- Mark it as such so that the system will be > +- initialized with the first received heartbeat. */ > +- ist->sub2video.initialize = 1; > + if ((ret = avfilter_link(last_filter, 0, in->filter_ctx, > in->pad_idx)) < 0) > + return ret; > > @@ fftools/ffmpeg_filter.c: static int > configure_input_video_filter(FilterGraph *fg > int64_t tsoffset = 0; > - AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); > + AVBufferSrcParameters *par; > - > ++ > + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { > + // Automatically insert conversion filter to retain > compatibility > + // with sub2video command lines > + return configure_input_subtitle_filter(fg, ifilter, in); > + } > -+ > + > + par = av_buffersrc_parameters_alloc(); > if (!par) > return AVERROR(ENOMEM); > @@ fftools/ffmpeg_opt.c: static void add_input_streams(OptionsContext > *o, AVFormatC > break; > } > case AVMEDIA_TYPE_ATTACHMENT: > +@@ fftools/ffmpeg_opt.c: static char *get_ost_filters(OptionsContext > *o, AVFormatContext *oc, > + > + if (ost->filters_script) > + return read_file(ost->filters_script); > +- else if (ost->filters) > ++ if (ost->filters) > + return av_strdup(ost->filters); > + > +- return av_strdup(st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO > ? > +- "null" : "anull"); > ++ switch (st->codecpar->codec_type) { > ++ case AVMEDIA_TYPE_VIDEO: return av_strdup("null"); > ++ case AVMEDIA_TYPE_AUDIO: return av_strdup("anull"); > ++ case AVMEDIA_TYPE_SUBTITLE: return av_strdup("snull"); > ++ default: av_assert0(0); return NULL; > ++ } > + } > + > + static void check_streamcopy_filters(OptionsContext *o, > AVFormatContext *oc, > +@@ fftools/ffmpeg_opt.c: static OutputStream > *new_subtitle_stream(OptionsContext *o, AVFormatContext *oc, > + > + subtitle_enc->codec_type = AVMEDIA_TYPE_SUBTITLE; > + > ++ MATCH_PER_STREAM_OPT(filter_scripts, str, ost->filters_script, > oc, st); > ++ MATCH_PER_STREAM_OPT(filters, str, ost->filters, > oc, st); > ++ > + if (!ost->stream_copy) { > + char *frame_size = NULL; > + > +@@ fftools/ffmpeg_opt.c: static OutputStream > *new_subtitle_stream(OptionsContext *o, AVFormatContext *oc, > + av_log(NULL, AV_LOG_FATAL, "Invalid frame size: %s.\n", > frame_size); > + exit_program(1); > + } > ++ > ++ ost->avfilter = get_ost_filters(o, oc, ost); > ++ if (!ost->avfilter) > ++ exit_program(1); > + } > + > + return ost; > @@ fftools/ffmpeg_opt.c: static void init_output_filter(OutputFilter > *ofilter, OptionsContext *o, > switch (ofilter->type) { > case AVMEDIA_TYPE_VIDEO: ost = new_video_stream(o, oc, -1); > break; > @@ fftools/ffmpeg_opt.c: static void init_output_filter(OutputFilter > *ofilter, Opti > "currently.\n"); > exit_program(1); > } > +@@ fftools/ffmpeg_opt.c: loop_end: > + ist->processing_needed = 1; > + > + if (ost->st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO > || > +- ost->st->codecpar->codec_type == > AVMEDIA_TYPE_AUDIO) { > ++ ost->st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO > || > ++ ost->st->codecpar->codec_type == > AVMEDIA_TYPE_SUBTITLE) { > + err = init_simple_filtergraph(ist, ost); > + if (err < 0) { > + av_log(NULL, AV_LOG_ERROR, > +@@ fftools/ffmpeg_opt.c: loop_end: > + } else if (ost->enc->ch_layouts) { > + f->ch_layouts = ost->enc->ch_layouts; > + } > ++ break; > ++ case AVMEDIA_TYPE_SUBTITLE: > ++ f->format = ost->enc_ctx->subtitle_type; > ++ > + break; > + } > + } > > ## tests/ref/fate/filter-overlay-dvdsub-2397 ## > @@ > @@ tests/ref/fate/sub-dvb > +0, 31400000, 31400000, 479000, 14, 0x0959015b > +0, 31879000, 31879000, 479000, 14, 0x09c9016b > > + ## tests/ref/fate/sub-scc ## > +@@ tests/ref/fate/sub-scc: Style: > Default,Monospace,16,&Hffffff,&Hffffff,&H0,&H0,0,0,0,0,100,100,0,0,3,1,0, > + > + [Events] > + Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, > Effect, Text > +-Dialogue: > 0,0:00:-2.-47,0:00:00.70,Default,,0,0,0,,{\an7}{\pos(76,228)}WE HAVE FOUND > A WITCH !\N{\an7}{\pos(76,243)}MAY WE BURN HER ? > + Dialogue: > 0,0:00:00.69,0:00:03.29,Default,,0,0,0,,{\an7}{\pos(115,228)}[ Crowd > ]\N{\an7}{\pos(115,243)}BURN HER ! BURN HER ! > + Dialogue: > 0,0:00:03.30,0:00:07.07,Default,,0,0,0,,{\an7}{\pos(38,197)}HOW DO YOU > KNOW\N{\an7}{\pos(38,213)}SHE IS A WITCH ?\N{\an7}{\pos(153,243)}SHE LOOKS > LIKE ONE ! > + Dialogue: > 0,0:00:07.07,0:00:09.27,Default,,0,0,0,,{\an7}{\pos(192,228)}[ > Shouting\N{\an7}{\pos(192,243)}\h\hAffirmations ] > + > ## tests/ref/fate/sub2video ## > @@ > 0, 47, 47, 1, 518400, 0xde69683f > 23: a66debd96e = 25: a90a6e1086 avcodec/dvbsubdec: Fix conditions for > fallback to default resolution > > -- > ffmpeg-codebot > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Paul B Mahol (12022-07-02):
> Can this be properly finally be fully reviewed and accepted?
As long as the patch does not have a solution to have all the utility
filters (setpts, trim, concat, etc.) working with subtitles too without
more code duplication, the review is just "NAK".
Paul B Mahol (12022-07-02): > Could you please mention in relevant thread of those subtitle work where > duplication in code happened? No. Currently, there is no solution, with or without code duplication.
On Sat, Jul 2, 2022 at 7:18 PM Nicolas George <george@nsup.org> wrote: > Paul B Mahol (12022-07-02): > > Can this be properly finally be fully reviewed and accepted? > > As long as the patch does not have a solution to have all the utility > filters (setpts, trim, concat, etc.) working with subtitles too without > more code duplication, the review is just "NAK". > Could you please mention in relevant thread of those subtitle work where duplication in code happened? > > -- > Nicolas George > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
From: Paul B Mahol <onemda@gmail.com>
Sent: Saturday, July 2, 2022 6:39 PM
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Cc: Michael Niedermayer <michael@niedermayer.cc>; softworkz <softworkz@hotmail.com>; Andriy Gelman <andriy.gelman@gmail.com>; Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Sat, Jun 25, 2022 at 11:58 AM ffmpegagent <ffmpegagent@gmail.com<mailto:ffmpegagent@gmail.com>> wrote:
Subtitle Filtering 2022
=======================
This is a substantial update to the earlier subtitle filtering patch series.
A primary goal has been to address others' concerns as much as possible on
one side and to provide more clarity and control over the way things are
working. Clarity is is specifically important to allow for a better
understanding of the need for a subtitle start pts value that can be
different from the frame's pts value. This is done by refactoring the
subtitle timing fields in AVFrame, adding a frame field to indicate repeated
subtitle frames, and finally the full removal of the heartbeat
functionality, replaced by a new 'subfeed' filter that provides different
modes for arbitrating subtitle frames in a filter graph. Finally, each
subtitle filter's documentation has been amended by a section describing the
filter's timeline behavior (in v3 update).
Subtitle Filtering Demos
========================
I published a demonstration of subtitle filtering capabilities with OCR,
text and bitmap subtitle manipulation involved: Demo 1: Text-Manipulation
with Bitmap Subtitles
[https://github.com/softworkz/SubtitleFilteringDemos/tree/master/Demo1]
v5 - Conversion to Graphic Subtitles, and other enhancements
============================================================
* I'm glad to announce that Traian (@tcoza) has joined the project and
contributed a new 'text2graphicsub' filter to convert text subtitles to
graphic subtitles, which can in turn be encoded as dvd, dvb or x-subs
(and any other encoder for graphic subs that might be added in the
future). This filter closes the last open "gap" in subtitle processing.
* stripstyles filter: now allows very fine-grained control over which ASS
style codes should be preserved or stripped
* stripstyles: do not drop dialog margin values
* subfeed filter: eliminates duplicate frames with duplicate start times
when 'fix_overlap' is specified
* textmod: do not drop effect values
* graphicsub2text: reduce font size jitter
* ass_split: add function to selectively preserve elements when splitting
* add strim, snull and ssink and further unify subtitle frame handling with
audio and video
* ffmpeg_filter: get simple filter notation working for subtitles
v4 - Quality Improvements
=========================
* finally an updated version
* includes many improvements from internal testing
* all FATE tests passed
* all example commands from the docs verified to work
* can't list all the detail changes..
* I have left out the extra commits which can be handled separately, just
in case somebody wonders why these are missing:
* avcodec/webvttenc: Don't encode drawing codes and empty lines
* avcodec/webvttenc: convert hard-space tags to
* avutil/ass_split: Add parsing of hard-space tags (\h)
* avutil/ass_split: Treat all content in curly braces as hidden
* avutil/ass_split: Fix ass parsing of style codes with comments
v3 - Rebase
===========
due to merge conflicts - apologies.
Changes in v2
=============
* added .gitattributes file to enforce binary diffs for the test refs that
cannot be applied when being sent via e-mail
* perform filter graph re-init due to subtitle "frame size" change only
when the size was unknown before and not set via -canvas_size
* overlaytextsubs: Make sure to request frames on the subtitle input
* avfilter/splitcc: Start parsing cc data on key frames only
* avcodec/webvttenc: Don't encode ass drawing codes and empty lines
* stripstyles: fix mem leak
* gs2t: improve color detection
* gs2t: empty frames must not be skipped
* subfeed: fix name
* textmod: preserve margins
* added .gitattributes file to enforce binary diffs for the test refs that
cannot be applied when being sent via e-mail
* perform filter graph re-init due to subtitle "frame size" change only
when the size was unknown before and not set via -canvas_size
* avcodec/dvbsubdec: Fix conditions for fallback to default resolution
* Made changes suggested by Andreas
* Fixed failing command line reported by Michael
Changes from previous version v24:
AVFrame
=======
* Removed sub_start_time The start time is now added to the subtitle
start_pts during decoding The sub_end_time field is adjusted accordingly
* Renamed sub_end_time to duration which it is effectively after removing
the start_time
* Added a sub-struct 'subtitle_timing' to av frame Contains subtitle_pts
renamed to 'subtitle_timing.start_pts' and 'subtitle_timing.duration'
* Change both fields to (fixed) time_base AV_TIMEBASE
* add repeat_sub field provides a clear indication whether a subtitle frame
is an actual subtitle event or a repeated subtitle frame in a filter
graph
Heartbeat Removal
=================
* completely removed the earlier heartbeat implementation
* filtering arbitration is now implemented in a new filter: 'subfeed'
* subfeed will be auto-inserted for compatiblity with sub2video command
lines
* the new behavior is not exactly identical to the earlier behavior, but it
basically allows to achieve the same results
* there's a small remainder, now named subtitle kickoff which serves to get
things (in the filter graph) going right from the start
New 'subfeed' Filter
====================
* a versatile filter for solving all kinds of problems with subtile frame
flow in filter graphs
* Can be inserted at any position in a graph
* Auto-inserted for sub2video command lines (in repeat-mode)
* Allows duration fixup delay input frames with unknown duration and infer
duration from start of subsequent frame
* Provides multiple modes of operation:
* repeat mode (default) Queues input frames Outputs frames at a fixed
(configurable) rate Either sends a matching input frame (repeatedly) or
empty frames otherwise
* scatter mode similar to repeat mode, but splits input frames by
duration into small segments with same content
* forward mode No fixed output rate Useful in combination with duration
fixup or overlap fixup
ffmpeg Tool Changes
===================
* delay subtitle output stream initialization (like for audio and video)
This is needed for example when a format header depends on having
received an initial frame to derive certain header values from
* decoding: set subtitle frame size from decoding context
* re-init graph when subtitle size changes
* always insert subscale filter for sub2video command lines (to ensure
correct scaling)
Subtitle Encoding
=================
* ignore repeated frames for encoding based on repeat_sub field in AVFrame
* support multi-area encoding for text subtitles Subtitle OCR can create
multiple areas at different positions. Previously, the texts were always
squashed into a single area ('subtitle rect'), which was not ideal.
Multiple text areas are now generally supported:
* ASS Encoder Changed to use the 'receive_packet' encoding API A single
frame with multiple text areas will create multiple packets now
* All other text subtitle encoders A newline is inserted between the text
from multiple areas
graphicsub2text (OCR)
=====================
* enhanced preprocessing
* using elbg algorithm for color quantization
* detection and removal of text outlines
* map-based identification of colors per word (text, outline, background)
* add option for duration fixup
* add option to dump preprocessing bitmaps
* Recognize formatting and apply as ASS inline styles
* per word(!)
* paragraph alignment
* positioning
* font names
* font size
* font style (italic, underline, bold)
* text color, outline color
Other Filter Changes
====================
* all: Make sure to forward all link properties (time base, frame rate, w,
h) where appropriate
* overlaytextsubs: request frames on the subtitle input
* overlaytextsubs: disable read-order checking
* overlaytextsubs: improve implementation of render_latest_only
* overlaytextsubs: ensure equal in/out video formats
* splitcc: derive framerate from realtime_latency
* graphicsub2video: implement caching of converted frames
* graphicsub2video: use 1x1 output frame size as long as subtitle size is
unknown (0x0)
Plus a dozen of things I forgot..
softworkz (24):
avcodec,avutil: Move enum AVSubtitleType to avutil, add new and
deprecate old values
avutil/frame: Prepare AVFrame for subtitle handling
avcodec/subtitles: Introduce new frame-based subtitle decoding API
avcodec/libzvbi: set subtitle type
avfilter/subtitles: Update vf_subtitles to use new decoding api
avcodec,avutil: Move ass helper functions to avutil as avpriv_ and
extend ass dialog parsing
avcodec/subtitles: Replace deprecated enum values
fftools/play,probe: Adjust for subtitle changes
avfilter/subtitles: Add subtitles.c for subtitle frame allocation
avfilter/avfilter: Handle subtitle frames
avfilter/avfilter: Fix hardcoded input index
avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
avfilter/overlaygraphicsubs: Add overlaygraphicsubs and
graphicsub2video filters
avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video
filters
avfilter/textmod: Add textmod, censor and show_speaker filters
avfilter/stripstyles: Add stripstyles filter
avfilter/splitcc: Add splitcc filter for closed caption handling
avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
avfilter/subscale: Add filter for scaling and/or re-arranging
graphical subtitles
avfilter/subfeed: add subtitle feed filter
avfilter/snull,strim: Add snull and strim filters
avcodec/subtitles: Migrate subtitle encoders to frame-based API
fftools/ffmpeg: Introduce subtitle filtering and new frame-based
subtitle encoding
avcodec/dvbsubdec: Fix conditions for fallback to default resolution
Can this be properly finally be fully reviewed and accepted?
Otherwise if its kept mainly ignored than it should be regarded as spam.
Current status quo is bad.
Do you think the patchset is bad or the situation is bad?
In the latter case I agree.
I would ask everybody to voice their concerns and objections so we
can get those addressed.
Thanks,
softworkz
> -----Original Message----- > From: Nicolas George <george@nsup.org> > Sent: Saturday, July 2, 2022 7:18 PM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Cc: Michael Niedermayer <michael@niedermayer.cc>; softworkz > <softworkz@hotmail.com>; Andriy Gelman <andriy.gelman@gmail.com>; > Andreas Rheinhardt <andreas.rheinhardt@outlook.com> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Paul B Mahol (12022-07-02): > > Can this be properly finally be fully reviewed and accepted? > > As long as the patch does not have a solution to have all the utility > filters (setpts, trim, concat, etc.) working with subtitles too > without > more code duplication, the review is just "NAK". The latest versions has added support for trim (strim). I strongly disagree on the "code duplication" argument. AFAIU, this is about having similar filters duplicated for each media type, e.g. trim, atrim and strim. The duplication has always existed already between audio and video filters. You could have unified and eliminated that duplication for years, but you didn't. Now that a third media type is being added, it is totally arbitrary to ask for unification and de-duplication of the code. You didn't do it for 2 media types, so why should 3 be the magic number where this is suddenly a requirement? This is an unjustified request. Even when I would follow that demand, it wouldn't be reasonable, because in that case, the patchset would dramatically widen its scope and start affecting audio and video as well - which is much too huge to get it tackled in a single patchset. It has always been possible to de-duplicate between audio and video, and in the future, it will always be possible to de-duplicate between audio, video and subtitles. It might make sense to do so at some point in time, but it is out of the scope of this patchset. And when this is the only objection, then I think that the patchset is actually in really good shape for getting merged. Thanks, softworkz
Soft Works (12022-07-02): > The duplication has always existed already between audio and video > filters. You could have unified and eliminated that duplication for > years, but you didn't. I have been preparing for it for years. You did not help. > Even when I would follow that demand, it wouldn't be reasonable, > because in that case, the patchset would dramatically widen its > scope and start affecting audio and video as well - which is much > too huge to get it tackled in a single patchset. We will not accept a bad patch just because you have bitten more than you could chew and reverted to a quick and dirty solution.
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Nicolas George > Sent: Saturday, July 2, 2022 9:17 PM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Cc: Michael Niedermayer <michael@niedermayer.cc>; Andriy Gelman > <andriy.gelman@gmail.com>; Andreas Rheinhardt > <andreas.rheinhardt@outlook.com> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Soft Works (12022-07-02): > > The duplication has always existed already between audio and video > > filters. You could have unified and eliminated that duplication for > > years, but you didn't. > > I have been preparing for it for years. You did not help. > > > Even when I would follow that demand, it wouldn't be reasonable, > > because in that case, the patchset would dramatically widen its > > scope and start affecting audio and video as well - which is much > > too huge to get it tackled in a single patchset. > > We will not accept a bad patch just because you have bitten more than > you could chew and reverted to a quick and dirty solution. All you do is keep spitting discrediting keywords, this is just disgusting. sw
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Nicolas George > Sent: Saturday, July 2, 2022 9:17 PM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Cc: Michael Niedermayer <michael@niedermayer.cc>; Andriy Gelman > <andriy.gelman@gmail.com>; Andreas Rheinhardt > <andreas.rheinhardt@outlook.com> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Soft Works (12022-07-02): > > The duplication has always existed already between audio and video > > filters. You could have unified and eliminated that duplication for > > years, but you didn't. > > I have been preparing for it for years. You did not help. This can easily be done AFTER my patchset has been merged. Despite all of your negative attitude, I will do my best to assist you in that process later. Thanks, sw
Soft Works (12022-07-02):
> This can easily be done AFTER my patchset has been merged.
With exponentially more work. Out of question.
This is all I have to say to you on the subject.
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Nicolas George > Sent: Saturday, July 2, 2022 9:36 PM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Cc: Michael Niedermayer <michael@niedermayer.cc>; Andriy Gelman > <andriy.gelman@gmail.com>; Andreas Rheinhardt > <andreas.rheinhardt@outlook.com> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Soft Works (12022-07-02): > > This can easily be done AFTER my patchset has been merged. > > With exponentially more work. Out of question. Previously it would have been about like: - Merging audio filter code with the video filter code (for the filters in question) Now it will be - Merging audio and subtitle filter code with the video filter code (for the same filters) TBH, I can't see any exponent here. I think "double work" would be closer to the truth and realistically it will be much less than that because the work for merging audio and subtitle code is very similar, so when you have merged audio code in a filter, merging the subtitle part will be very analogous, so in total it would be less than double. And when we look at the required amount of work in total, that calculation would only be valid when you would consider the value of MY work that I've already done as zero. I even think that it's a better approach overall to do the deduplication afterwards, because now - with the subtitle filtering patchset - the specific requirements for subtitle filtering are visible on the table and that way, the deduplication can already provision for those specific requirements whereas focusing on audio/video only, might have led to do changes that wouldn't accommodate for the needs of subtitle filtering. I am convinced that doing deduplication afterwards is a better order for getting this done. I'm also convinced that my patchset is pretty solid in the way it does handle subtitles, and I'm further convinced that you know that very well. During all the process I have watched very closely, and in several cases where others had objections about things I had done, you kept quiet, presumably because you were the only other one to know why it had to be done that way. Also, you never named any specific detail that would be wrong, and I'm sure you would have done if there had been any significant one. My impression is that your primary reason for objection is that my patchset interferes with your plans and visions you probably had in mind for quite a while and I'm very sorry about that. But in the end, my patchset doesn't stand in opposition to your plans, it just requires a bit of adaption regarding the order of doing the work. Neither do I stand in opposition to your plans. I respect the technical architecture of libavfilter, especially regarding its simplicity and effectivity compared to other filtering frameworks (like DirectShow) and my interest in Ffmpeg filtering is not limited to subtitles. We don't need to be friends, but when you would manage to act and communicate in a friendly way, you might gain somebody to help with and support your plans in the future and you would also do a favor to all readers of the ML by not having them read through despicable conversations. Best regards, softworkz
On Sat, Jul 2, 2022 at 10:32 PM Soft Works <softworkz@hotmail.com> wrote: > > > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > Nicolas George > > Sent: Saturday, July 2, 2022 9:36 PM > > To: FFmpeg development discussions and patches <ffmpeg- > > devel@ffmpeg.org> > > Cc: Michael Niedermayer <michael@niedermayer.cc>; Andriy Gelman > > <andriy.gelman@gmail.com>; Andreas Rheinhardt > > <andreas.rheinhardt@outlook.com> > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > > > Soft Works (12022-07-02): > > > This can easily be done AFTER my patchset has been merged. > > > > With exponentially more work. Out of question. > > Previously it would have been about like: > > - Merging audio filter code with the video filter code > (for the filters in question) > > Now it will be > > - Merging audio and subtitle filter code with the video filter code > (for the same filters) > > TBH, I can't see any exponent here. I think "double work" would be > closer to the truth and realistically it will be much less than that > because the work for merging audio and subtitle code is very similar, > so when you have merged audio code in a filter, merging the subtitle > part will be very analogous, so in total it would be less than > double. > > And when we look at the required amount of work in total, that > calculation would only be valid when you would consider the value > of MY work that I've already done as zero. > > I even think that it's a better approach overall to do the deduplication > afterwards, because now - with the subtitle filtering patchset - the > specific requirements for subtitle filtering are visible on the table > and that way, the deduplication can already provision for those > specific requirements whereas focusing on audio/video only, might have > led to do changes that wouldn't accommodate for the needs of subtitle > filtering. > > I am convinced that doing deduplication afterwards is a better order > for getting this done. I'm also convinced that my patchset is pretty > solid in the way it does handle subtitles, and I'm further convinced > that you know that very well. During all the process I have watched > very closely, and in several cases where others had objections about > things I had done, you kept quiet, presumably because you were the only > other one to know why it had to be done that way. Also, you never > named any specific detail that would be wrong, and I'm sure you would > have done if there had been any significant one. > My impression is that your primary reason for objection is that my > patchset interferes with your plans and visions you probably had in > mind for quite a while and I'm very sorry about that. > But in the end, my patchset doesn't stand in opposition to your plans, > it just requires a bit of adaption regarding the order of doing the work. > Neither do I stand in opposition to your plans. I respect the technical > architecture of libavfilter, especially regarding its simplicity and > effectivity compared to other filtering frameworks (like DirectShow) > and my interest in Ffmpeg filtering is not limited to subtitles. > We don't need to be friends, but when you would manage to act and > communicate in a friendly way, you might gain somebody to help with > and support your plans in the future and you would also do a favor > to all readers of the ML by not having them read through despicable > conversations. > AFAIK only NIcolas is for this merge of different types of filters into single filter, and filter type negotiation. Both ideas are very bad. And Nicolas is (still) not (yet) FFmpeg dictator. > Best regards, > softworkz > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Paul B Mahol > Sent: Saturday, July 2, 2022 10:40 PM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Cc: Michael Niedermayer <michael@niedermayer.cc>; Andriy Gelman > <andriy.gelman@gmail.com>; Andreas Rheinhardt > <andreas.rheinhardt@outlook.com> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > On Sat, Jul 2, 2022 at 10:32 PM Soft Works <softworkz@hotmail.com> > wrote: > > > > > > > > -----Original Message----- > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > Nicolas George > > > Sent: Saturday, July 2, 2022 9:36 PM > > > To: FFmpeg development discussions and patches <ffmpeg- > > > devel@ffmpeg.org> > > > Cc: Michael Niedermayer <michael@niedermayer.cc>; Andriy Gelman > > > <andriy.gelman@gmail.com>; Andreas Rheinhardt > > > <andreas.rheinhardt@outlook.com> > > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering > 2022 > > > > > > Soft Works (12022-07-02): > > > > This can easily be done AFTER my patchset has been merged. > > > > > > With exponentially more work. Out of question. > > > > Previously it would have been about like: > > > > - Merging audio filter code with the video filter code > > (for the filters in question) > > > > Now it will be > > > > - Merging audio and subtitle filter code with the video filter code > > (for the same filters) > > > > TBH, I can't see any exponent here. I think "double work" would be > > closer to the truth and realistically it will be much less than > that > > because the work for merging audio and subtitle code is very > similar, > > so when you have merged audio code in a filter, merging the > subtitle > > part will be very analogous, so in total it would be less than > > double. > > > > And when we look at the required amount of work in total, that > > calculation would only be valid when you would consider the value > > of MY work that I've already done as zero. > > > > I even think that it's a better approach overall to do the > deduplication > > afterwards, because now - with the subtitle filtering patchset - > the > > specific requirements for subtitle filtering are visible on the > table > > and that way, the deduplication can already provision for those > > specific requirements whereas focusing on audio/video only, might > have > > led to do changes that wouldn't accommodate for the needs of > subtitle > > filtering. > > > > I am convinced that doing deduplication afterwards is a better > order > > for getting this done. I'm also convinced that my patchset is > pretty > > solid in the way it does handle subtitles, and I'm further > convinced > > that you know that very well. During all the process I have watched > > very closely, and in several cases where others had objections > about > > things I had done, you kept quiet, presumably because you were the > only > > other one to know why it had to be done that way. Also, you never > > named any specific detail that would be wrong, and I'm sure you > would > > have done if there had been any significant one. > > My impression is that your primary reason for objection is that my > > patchset interferes with your plans and visions you probably had in > > mind for quite a while and I'm very sorry about that. > > But in the end, my patchset doesn't stand in opposition to your > plans, > > it just requires a bit of adaption regarding the order of doing the > work. > > Neither do I stand in opposition to your plans. I respect the > technical > > architecture of libavfilter, especially regarding its simplicity > and > > effectivity compared to other filtering frameworks (like > DirectShow) > > and my interest in Ffmpeg filtering is not limited to subtitles. > > We don't need to be friends, but when you would manage to act and > > communicate in a friendly way, you might gain somebody to help with > > and support your plans in the future and you would also do a favor > > to all readers of the ML by not having them read through despicable > > conversations. > > > > AFAIK only NIcolas is for this merge of different types of filters > into > single filter, > and filter type negotiation. > > Both ideas are very bad. And Nicolas is (still) not (yet) FFmpeg > dictator. I think it makes sense for certain filters like buffer source, buffer sink, null sink, trim, showinfo, copy, delay, repeat, etc. Yet there is no point in making this a pre-condition for subtitle filtering, it's out of scope. sw
Matt Zagrabelny (12022-07-02):
> Nicolas' arguments seem to be "The perfect is the enemy of the good."
No, my arguments are: you don't build a third floor when your
foundations are shaky.
I have posted on this very mailing list a long time ago simple tasks
that need to be done before considering adding more media types for
libavfilter. They have been rudely ignored.
Are you here to help, or are you here to pour oil on the flame?
On Sat, Jul 2, 2022 at 11:22 PM Nicolas George <george@nsup.org> wrote: > Matt Zagrabelny (12022-07-02): > > Nicolas' arguments seem to be "The perfect is the enemy of the good." > > No, my arguments are: you don't build a third floor when your > foundations are shaky. > > I have posted on this very mailing list a long time ago simple tasks > that need to be done before considering adding more media types for > libavfilter. They have been rudely ignored. > "Simple" tasks that are not necessary at all. > > Are you here to help, or are you here to pour oil on the flame? > > -- > Nicolas George > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Matt Zagrabelny (12022-07-02): > I guess your idea of "rudely" is different from mine. I find SW to be > polite. I guess you have not read everything since the beginning. > I am a simpleton that just reads what others write. I find subtitle support > to be desirable and SW's work to be a good step in that direction. It might seem so from the point of view of a user. From my point of view of the person who maintains the framework of libavfilter, support for subtitles is desirable, but SW's patches make my work later significantly harder.
Hello Paul, On Sat, 2 Jul 2022, at 22:40, Paul B Mahol wrote: >> I am convinced that doing deduplication afterwards is a better order >> for getting this done. I'm also convinced that my patchset is pretty >> [...] > > AFAIK only NIcolas is for this merge of different types of filters into > single filter, > and filter type negotiation. > > Both ideas are very bad. If you really feel so, you have a way to unlock this situation: call the TC, whose precise role is to do work on this. You can even request a GA vote on this topic, if you feel that more people should step in. But here, the discussion goes nowhere "I'm right", "No, I'm right", "No, I'm correct" and will only evolve in insults and slander. Best,
On Sun, Jul 3, 2022 at 9:59 AM Jean-Baptiste Kempf <jb@videolan.org> wrote: > Hello Paul, > > On Sat, 2 Jul 2022, at 22:40, Paul B Mahol wrote: > >> I am convinced that doing deduplication afterwards is a better order > >> for getting this done. I'm also convinced that my patchset is pretty > >> [...] > > > > AFAIK only NIcolas is for this merge of different types of filters into > > single filter, > > and filter type negotiation. > > > > Both ideas are very bad. > > If you really feel so, you have a way to unlock this situation: call the > TC, whose precise role is to do work on this. > You can even request a GA vote on this topic, if you feel that more people > should step in. > > But here, the discussion goes nowhere "I'm right", "No, I'm right", "No, > I'm correct" and will only evolve in insults and slander. > > Why should I do this work? I will just create my own fork. (Already did.) > Best, > > -- > Jean-Baptiste Kempf - President > +33 672 704 734 > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
On Sat, Jul 02, 2022 at 07:18:16PM +0200, Nicolas George wrote: > Paul B Mahol (12022-07-02): > > Can this be properly finally be fully reviewed and accepted? > > As long as the patch does not have a solution to have all the utility > filters (setpts, trim, concat, etc.) working with subtitles too without > more code duplication, the review is just "NAK". What is the timeline for the audio+video merge ? IIUC this would resolve this deadlock (with extra work adapting the patchset so it would be work for SW adapting it and it would be work for you finishing the merge) Also can others help nicolas moving his work forward What i suggest is to pick a time and then try to finish the merge before. If it succeeds this patchset needs updating and can move forward without this main objection OTOH if the time is not hit, we agree that the objection can no longer be used nothing above is meant as a objection to anything, just my oppinion. Some thought i had which i hope could move this forward. If people agree to anything else iam happy with that too, this mail is only meant to help! thx [...]
On Sun, Jul 3, 2022 at 7:07 PM Michael Niedermayer <michael@niedermayer.cc> wrote: > On Sat, Jul 02, 2022 at 07:18:16PM +0200, Nicolas George wrote: > > Paul B Mahol (12022-07-02): > > > Can this be properly finally be fully reviewed and accepted? > > > > As long as the patch does not have a solution to have all the utility > > filters (setpts, trim, concat, etc.) working with subtitles too without > > more code duplication, the review is just "NAK". > > What is the timeline for the audio+video merge ? > Eternity. > IIUC this would resolve this deadlock (with extra work adapting the > patchset > so it would be work for SW adapting it and it would be work for you > finishing > the merge) > Also can others help nicolas moving his work forward > > What i suggest is to pick a time and then try to finish the merge before. > If it succeeds this patchset needs updating and can move forward without > this main objection > OTOH if the time is not hit, we agree that the objection can no longer be > used > > nothing above is meant as a objection to anything, just my oppinion. Some > thought > i had which i hope could move this forward. > If people agree to anything else iam happy with that too, this mail is only > meant to help! > > thx > > [...] > > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > Observe your enemies, for they first find out your faults. -- Antisthenes > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
On Sun, 3 Jul 2022, at 12:42, Paul B Mahol wrote: >> But here, the discussion goes nowhere "I'm right", "No, I'm right", "No, >> I'm correct" and will only evolve in insults and slander. >> > Why should I do this work? > > I will just create my own fork. (Already did.) History shows that it's very hard to do a successful FFmpeg fork. Working together is more powerful, IMVHO.
I hesitated a long time before replying, but all considering, at this point expressing what is weighing on my heart cannot make things worse. Michael Niedermayer (12022-07-03): > What is the timeline for the audio+video merge ? I cannot give a timeline because I do not work on FFmpeg on a schedule, I work on FFmpeg in my free time, for fun. And lately, working on FFmpeg has been really un-fun. Recently, my contributions have been met with indifference at best (see when I proposed a XML parser six months ago: nobody cared), outright hostility at worst. Under these circumstances, whenever I am considering working on something FFmpeg-related, I usually find something else more fun to do. I do not recognize the project I started contributing to more than fifteen years ago. I do not even recognize the project that boasted the clever optimization framework that made FFVP9 possible, it has become increasingly hostile to trying new and more efficient ways of doing things in favor of a corporate never-take-risks style of coding. I am more and more often considering giving up and cutting my losses. > IIUC this would resolve this deadlock (with extra work adapting the patchset > so it would be work for SW adapting it and it would be work for you finishing > the merge) > Also can others help nicolas moving his work forward > > What i suggest is to pick a time and then try to finish the merge before. > If it succeeds this patchset needs updating and can move forward without > this main objection > OTOH if the time is not hit, we agree that the objection can no longer be > used My answer as maintainer of the framework of libavfilter is: no. Of course, maintainers are not dictators. The members of the project can collectively decide otherwise. But I have to warn you about the consequences. First, the issue about negotiation is not he only severe flaw in this patch series. I can immediately quote another one: for text subtitles, the approach of this proposal to synchronization is to feed everything to libass as it comes and see what comes out. It will work on easy cases, when the subtitles are interleaved with the video or come from a separate file. But as soon as a filter will, for example, adjust the timing to make the subtitles more early, it will just not work. Of course, it was not tested because this patch series does not even offer the feature to adjust the time of subtitles, which is frankly ridiculous, it is one of the most obvious thing people might want to do. Note that I did not have to perform a full review of the patch series to find this flaw. I have been preparing to implement subtitles filtering for years now, I know which aspects are tricky and hard to implement properly. I only had to check precisely how it was done. And it turns out it was not done at all. I suspect that if I were to do a full review, I would find a few other flaws. But the author has made painfully clear that they did not respect my expertise in this area, I have no reason to waste my time doing it. It illustrates what it means to be a maintainer. It does not only mean the task of reviewing and applying bug fixes. The maintainer holds in their head a knowledge of the code that cannot be fully shared in writing. The maintainer also holds in their head plans to evolve and extend the code. I have plans for libavfilter. Not just vague ideas, but precise plans on how to reach the goal. I have plans for subtitles filtering, of course. But not only. I have plans for filtering data packets, so that bistream filters do not need to have a separate and redundant API and ffmpeg does not need a separate code path for -c copy. I have plans for threading, or more precisely integrating filters in a global parallelized task and I/O scheduler. I have plans for seeking, with the seek target going back from the output to the input(s). I have plans for partial graph reconfiguration, to accommodate format changes mid-stream and let applications alter filtering in the middle. All of this is exciting. I am eager to work on any of this. Unfortunately, before it can happen, boring things need to be done. Parts of the framework of libavfilter are very fragile. Working on any of these is likely to break cases that were specifically fixed in the past. I can work on boring things if they are necessary to reach the exciting parts. What I cannot do is motivate myself to work on the boring things with the threat that the exciting things will be snatched under me by an inferior version from somebody who just refuses to engage with the boring necessary things. If this patch series gets applied, it will make the boring things a lot harder to do and it will ruin some of the plans I mentioned above. Under these circumstances, do not expect me to work on libavfilter again any time soon, even if it is to apply an obviously valid fix for an exploitable security issue. So, the choice is: - Apply this patch series, find a new maintainer for libavfilter and kiss goodbye to seeking, threading, reconfiguration (and subtitles filtering that work usefully). - Make it clear that this patch series is rejected until the framework is robust enough and has enough test coverage. - Let the situation continue to rot. Note: The most urgent and boring task is adding FATE tests to the heuristics of the negotiation process. It is actually rather easy. I would be happy to offer advice and even tutoring if somebody wants to contribute. Regards,
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Nicolas George > Sent: Sunday, July 24, 2022 5:10 PM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > I hesitated a long time before replying, but all considering, at this > point expressing what is weighing on my heart cannot make things > worse. > > > Michael Niedermayer (12022-07-03): > > What is the timeline for the audio+video merge ? > > I cannot give a timeline because I do not work on FFmpeg on a > schedule, > I work on FFmpeg in my free time, for fun. And lately, working on > FFmpeg > has been really un-fun. Recently, my contributions have been met with > indifference at best (see when I proposed a XML parser six months > ago: > nobody cared), outright hostility at worst. Under these > circumstances, > whenever I am considering working on something FFmpeg-related, I > usually > find something else more fun to do. > > I do not recognize the project I started contributing to more than > fifteen years ago. I do not even recognize the project that boasted > the > clever optimization framework that made FFVP9 possible, it has become > increasingly hostile to trying new and more efficient ways of doing > things in favor of a corporate never-take-risks style of coding. I am > more and more often considering giving up and cutting my losses. > > > IIUC this would resolve this deadlock (with extra work adapting the > patchset > > so it would be work for SW adapting it and it would be work for you > finishing > > the merge) > > Also can others help nicolas moving his work forward > > > > What i suggest is to pick a time and then try to finish the merge > before. > > If it succeeds this patchset needs updating and can move forward > without > > this main objection > > OTOH if the time is not hit, we agree that the objection can no > longer be > > used > > My answer as maintainer of the framework of libavfilter is: no. > > Of course, maintainers are not dictators. The members of the project > can > collectively decide otherwise. But I have to warn you about the > consequences. > > First, the issue about negotiation is not he only severe flaw in this > patch series. Negotiation hasn't been implemented for audio+video yet. Neither does that patchset do it for audio+video+subtitles. It is out of scope of this patchset. It can be done later or never, not everybody is a fan of doing so, as comments have shown. Clearly, this is in no way a showstopping reason as you had conceded yourself recently. > I can immediately quote another one: for text > subtitles, > the approach of this proposal to synchronization is to feed > everything > to libass as it comes and see what comes out. It will work on easy > cases, when the subtitles are interleaved with the video or come from > a separate file. - or come from decoded closed captions - or come from graphic subtitles converted with the graphicsub2text filter - or come from the subfeed filter after fixing durations - or come from the subfeed filter ensuring a regular repetition (heartbeat) Subtitle events don't need to come in linear order. Multiple events can have identical start times, subtitle events can overlap. The overlaytextsubs filter is meant to be a direct replacement for the existing subtitles filter, which performs additional opening, parsing and decoding of the source file in parallel, and avoiding that was one of the primary objectives I had for starting development. That's why it was very important for me to preserve the exact same behavior as the overlaytextsubs filter exposes. Other approaches for implementatino are surely possible as well. Traian, who did the text2graphicsub filter had initially an implementation that handled the timing manually instead of letting libass do it, but it turned out that this can quickly become a really complex task, especially when overlapping events or animations are part of the game, so it came down to feeding everything to libass in the end, like the overlaytextsubs filter and the subtitles filters do. The nice thing about having subtitle filtering is that there is no fixed functionality involved where you can argue about right or wrong: anyone is free to contribute another filter which is pursuing a different approach. I would welcome that and there may be cases where an alternative method could be advantageous, but it surely won't be superior in general. > But as soon as a filter will, for example, adjust the > timing to make the subtitles more early, it will just not work. Of > course, it was not tested because this patch series does not even > offer the feature to adjust the time of subtitles, which is frankly > ridiculous, it is one of the most obvious thing people might want to > do. The only reason why there is no timing adjustment filter is that I didn't need one. It is really easy to implement such filter. The abilities of such kind of filter are limited by the individual circumstances, though: You can always delay subtitle presentation, but the amount of time that you can move them ahead is limited by the situation. It depends on the amount of time they are muxed ahead in the source. This can range from zero (for example when originating from closed captions) to a few seconds (e.g. with ocr-ed DVB subs). In the latter case it's easy to subtract one or two seconds to show subtitles earlier - in the former case, there is no room for this - unless you would let the video frames queue up at the overlaytextsubs filter to synchronize with the subtitle frames. You are absolutely right here: the overlaytextsubs filter doesn't do that, it doesn't use framesync. Besides the reason to produce equal results to the existing subtitles filter (which framesync would interfere with), there's another consideration which kept me from using framesync: The benefit would be small and very limited. Let's look at an example: assuming we have a 4k 30fps video onto which we want to overlay text subtitles. The text subtitles are muxed "just-in-time" in the source and we want the subtitles to be shown 3 seconds earlier. In order to make this possible, it would be required to queue up 3s * 30 fps = 90 frames at the overlay filter. And this is not the muxing queue, it's inside a filtergraph where we have uncompressed frames. Assuming 4 bytes per pixel for simplicity * 3840 * 2160 = 32 MB per frame. For 90 frames, this makes 2.8 GB memory for 3 seconds delay. It is not unusual that subtitles are off by even larger time spans (e.g. video has cut off intro but sub timings assume intro to be present). Even with sufficient RAM - as soon as hardware acceleration is being used, you really wouldn't want to use previous GPU memory for subtitle timing adjustments. Eventually this brought me to the conclusion that this isn't a suitable approach for adjusting subtitle offsets except for small corrections. It's easy to add such filter to change timings and it's very well possible to implement an overlay filter with a different behavior - I would be sincerely interested to see which alternative solution you would come up for this. > this patch series does not even > offer the feature to adjust the time of subtitles, which is frankly > ridiculous, it is one of the most obvious thing people might want to > do. How did you come to assume that it wouldn't be possible to adjust the timing of subtitles at all? It's just that a much better and universal way than using a filter is the typical approach used for audio video sync correction, which can even be combined with the overlay filter: ffmpeg -i subtitlevideo.mkv -itsoffset -3 -i subtitlevideo.mkv -filter_complex "[1:s]subfeed[sub1];[0:v][sub1]overlaytextsubs[fout]" -map [fout] -map 0:a -sn -y out.mkv > Note that I did not have to perform a full review of the patch series > to > find this flaw. I have been preparing to implement subtitles > filtering > for years now, I know which aspects are tricky and hard to implement > properly. I only had to check precisely how it was done. And it turns > out it was not done at all. Then it's very weird that it's all working: dozens of examples for new functionality and all existing functionality is preserved. > I suspect that if I were to do a full review, I would find a few > other > flaws. But the author has made painfully clear that they did not > respect > my expertise in this area, You kept talking about your expertise and the lack of mine, without ever talking about any technical matters. This time, you mentioned technical issues and you see me answering in a very detailed way. We have seen how it cannot work and now you see how it could work. If you stick to technical matters without dropping the typical discrediting subtext, it could all go well. Thanks, softworkz
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Nicolas George > Sent: Sunday, July 24, 2022 5:10 PM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > I suspect that if I were to do a full review, I would find a few > other flaws. This is a very good example for the kind of behavior I'd kindly ask you to stop with. Do you think you have found a flaw? =================================== => please name it and explain it, ideally asking before calling it a "flaw" (like your one "flaw" wasn't really one as I had laid out) Do you think you COULD find a flaw? =================================== => Go ahead and find one - no need to tell Thanks, softworkz
Hi, On Sun, Jul 24, 2022 at 11:10 PM Nicolas George <george@nsup.org> wrote: > I do not recognize the project I started contributing to more than > fifteen years ago. I do not even recognize the project that boasted the > clever optimization framework that made FFVP9 possible, it has become > increasingly hostile to trying new and more efficient ways of doing > things in favor of a corporate never-take-risks style of coding. > We still do this, but currently outside FFmpeg (dav1d). So sadly, it's a bit more out of the sphere of this mailinglist. So we take risks, and if AV2-9 comes around, we might again. But we still consider ourselves part of the FFmpeg community, and I still try to review ffvp9 patches when they come up every once in a while. Don't lose hope, just find the positive things and work on them. Ignore the rest. I know I did. I'm having fun again. I don't know if I should offer advice, but my $.02: maybe adding tests (I don't mean ffmpeg invocations; some people would call this desired outcomes) might help here. You probably remember how Michael tested patches pre-FATE: for a particular patch, he'd send an FFmpeg commandline that either A) gives (undesirable) different output before vs. after patch, or B) should give some correct/desired output after the patch but (still) doesn't (and this second would be what I'm inviting you to do). Tests don't have to be scripted, they can simply be a list of features or behaviors desired in the new design. It's true that perfection is the enemy of progress, but I think you're right that we should try to strive for further improvement if it is within reach, especially for base design or things with API implications. (FFmpeg's API is a testament to poor design in some places.) You're likely better at making a comprehensive list than anyone else. Making the list public/explicit means other people can help accomplish the full list of features. <3 Ronald
On Sun, Jul 24, 2022 at 05:10:17PM +0200, Nicolas George wrote: > I hesitated a long time before replying, but all considering, at this > point expressing what is weighing on my heart cannot make things worse. > > > Michael Niedermayer (12022-07-03): > > What is the timeline for the audio+video merge ? > > I cannot give a timeline because I do not work on FFmpeg on a schedule, > I work on FFmpeg in my free time, for fun. And lately, working on FFmpeg > has been really un-fun. Recently, my contributions have been met with I think many if us, myself included have had un-fun periods in ffmpeg. This on its own is already bad and we should strive to make project contributions more fun to all. > indifference at best (see when I proposed a XML parser six months ago: > nobody cared), outright hostility at worst. Under these circumstances, > whenever I am considering working on something FFmpeg-related, I usually > find something else more fun to do. If we add a XML parser to FFmpeg. Such a thing would be used by several bits and we need to ensure it has redundant maintainership. So i think a xml parser needs broad support by developers otherwise we run the risk that if it has a single maintainer only and that maintainer stops maintaining it that could be annoying to more than just the parser itself. I cannot remember exactly without re-reading the old thread what the issues people had with the xml parser. If it was some component of it or in general. But i think its mostly a question if theres more than one person who wants to maintain it. [...] > It illustrates what it means to be a maintainer. It does not only mean > the task of reviewing and applying bug fixes. The maintainer holds in > their head a knowledge of the code that cannot be fully shared in > writing. The maintainer also holds in their head plans to evolve and > extend the code. > > I have plans for libavfilter. Not just vague ideas, but precise plans on > how to reach the goal. I have plans for subtitles filtering, of course. > But not only. > > I have plans for filtering data packets, so that bistream filters do not > need to have a separate and redundant API and ffmpeg does not need a > separate code path for -c copy. > > I have plans for threading, or more precisely integrating filters in a > global parallelized task and I/O scheduler. > > I have plans for seeking, with the seek target going back from the > output to the input(s). > > I have plans for partial graph reconfiguration, to accommodate format > changes mid-stream and let applications alter filtering in the middle. > > All of this is exciting. I am eager to work on any of this. > > Unfortunately, before it can happen, boring things need to be done. > Parts of the framework of libavfilter are very fragile. Working on any > of these is likely to break cases that were specifically fixed in the > past. > > I can work on boring things if they are necessary to reach the exciting > parts. > > What I cannot do is motivate myself to work on the boring things with > the threat that the exciting things will be snatched under me by an > inferior version from somebody who just refuses to engage with the > boring necessary things. If you work on libavfilter, noone will snatch it from you. If you dont work on it, eventually someone else will attempt to take over and push libavfilter in the direction he feels best which may or may not match what you want. If theres something i can do to make libavfilter work more fun for you please tell me. I think it would be best if people worked a bit more together on this. The current situation where work is split between your vission and this patchset is kind of a bad thing. Its a bit unfortunate noone involved seems to have the carismatic leader skills to pull everyone on his side and then get that side to the point that everyone is happy with the code and neither has a consensus emerged based on technical arguments. I think maybe more a "what is best for the project to move forward" and a less "how can i get my hard work in first" approuch might help but we can also try the technical commiitee of course but iam not sure how much that would really help, it would be better if some consensus would be reached and everyone would then work together on implementing that theres of course also the fork and merge approuch. Each side does whatever they like in their repository and then we at some point compare and merge one or both into ffmpeg. Iam not sure thats a good idea or not. thx [...]
Ronald S. Bultje (12022-07-25): > We still do this, but currently outside FFmpeg (dav1d). So sadly, it's a > bit more out of the sphere of this mailinglist. So we take risks, and if > AV2-9 comes around, we might again. But we still consider ourselves part of > the FFmpeg community, and I still try to review ffvp9 patches when they > come up every once in a while. Don't lose hope, just find the positive > things and work on them. Ignore the rest. I know I did. I'm having fun > again. > > I don't know if I should offer advice, but my $.02: maybe adding tests (I > don't mean ffmpeg invocations; some people would call this desired > outcomes) might help here. You probably remember how Michael tested patches > pre-FATE: for a particular patch, he'd send an FFmpeg commandline that > either A) gives (undesirable) different output before vs. after patch, or > B) should give some correct/desired output after the patch but (still) > doesn't (and this second would be what I'm inviting you to do). Tests don't > have to be scripted, they can simply be a list of features or behaviors > desired in the new design. It's true that perfection is the enemy of > progress, but I think you're right that we should try to strive for further > improvement if it is within reach, especially for base design or things > with API implications. (FFmpeg's API is a testament to poor design in some > places.) You're likely better at making a comprehensive list than anyone > else. Making the list public/explicit means other people can help > accomplish the full list of features. Thank you for these kind and encouraging words. The fact that the new developments happen in separate project kinds of confirms my impression that the project has become afraid of taking risks. It makes me think of these artists who found a rich patron and, dazzled by the money and luxury, become afraid of offending them and have their art lose its edge. FFmpeg is so eager to please our corporate users, to look “serious”, to look “professional” that sometimes it almost feels like grovelling, and it stifles novelty. Another symptom of this I see is that Michael spends most of his time managing releases and fuzzing and fixing trivial bugs, tasks that are way below his skills and talent, instead of writing new code. Or maybe I am just wrong and that is what he likes to do most. As for me… I am not good at shaving cycles from inner loops, which is the heart of FFmpeg's superiority, I cannot contribute to it. What I think I can contribute (and thank you for suggesting I am not wrong) is in areas of API and architecture. Unfortunately, while support for a new codec or optimization on an existing one will receive no objection on principle, work on API or architecture affect everybody and therefore is subject to much more discussion, possibly to the point of deadly bikeshedding or outright rejection. That makes it hard to invest time in it without some kind of a-priori assurance. Unfortunately my attempts to spark discussions have been met with indifference: https://ffmpeg.org/pipermail/ffmpeg-devel/2020-December/274167.html https://ffmpeg.org/pipermail/ffmpeg-devel/2020-December/thread.html#274167 (I realize I could probably work on internal error messages. As for event loop, the other one that has received positive feedback, I was trying to find a more efficient way out of a small quandary, I should probably go back to it.) Or even worse: https://ffmpeg.org/pipermail/ffmpeg-devel/2021-December/290226.html https://ffmpeg.org/pipermail/ffmpeg-devel/2021-December/thread.html#290226 (A good string API is a pre-requisite for several other projects.) What I think this project need is a leader, or a leading committee: somebody who can decide beforehand if a change in API or architecture is worth it, and therefore worth the time of the developer, with a decision that is binding to the project: when the code is written, other developers may discuss technical details but not reject the change itself. Maybe the technical committee could endorse this role, even though it was not exactly why it was elected. Thanks,
Michael Niedermayer (12022-07-25): > I think many if us, myself included have had un-fun periods in ffmpeg. > This on its own is already bad and we should strive to make project > contributions more fun to all. In my opinion, it would require at least two things: - That the members of the Community Committee step in more proactively in budding conflicts. And take sides: not “both be nice” but “you started it, shut up”. - A person or group of person capable of deciding ahead if a change that will affect every body is worth investing time implementing it. > If we add a XML parser to FFmpeg. Such a thing would be used by several > bits and we need to ensure it has redundant maintainership. > So i think a xml parser needs broad support by developers otherwise we > run the risk that if it has a single maintainer only and that maintainer > stops maintaining it that could be annoying to more than just the parser > itself. > I cannot remember exactly without re-reading the old thread what the issues > people had with the xml parser. If it was some component of it or in general. > But i think its mostly a question if theres more than one person who wants > to maintain it. There is nothing to re-read: https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/thread.html#296107 Nobody answered. Even to send files to make sure they would be supported. We would need reliable maintainers, you are right. But right now, we rely on libxml2, and that is not ideal: it is not a standard library on non-Linux systems, and even if it seems better nowadays, it used to be a “security issue of the week” kind of library. > If you work on libavfilter, noone will snatch it from you. If you > dont work on it, eventually someone else will attempt to take over and > push libavfilter in the direction he feels best which may or may not > match what you want. I will try to motivate myself to advance the FATE coverage, which is the blocking issue. Thanks,
On Mon, Jul 25, 2022 at 09:01:03PM +0200, Nicolas George wrote: [...] > > If we add a XML parser to FFmpeg. Such a thing would be used by several > > bits and we need to ensure it has redundant maintainership. > > So i think a xml parser needs broad support by developers otherwise we > > run the risk that if it has a single maintainer only and that maintainer > > stops maintaining it that could be annoying to more than just the parser > > itself. > > I cannot remember exactly without re-reading the old thread what the issues > > people had with the xml parser. If it was some component of it or in general. > > But i think its mostly a question if theres more than one person who wants > > to maintain it. > > There is nothing to re-read: > > https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/thread.html#296107 > > Nobody answered. Even to send files to make sure they would be > supported. that says "WIP" in the subject. I think some people when they see WIP they skip it and wait for a final version to review. thx [...]
> -----Original Message----- > From: ffmpegagent <ffmpegagent@gmail.com> > Sent: Saturday, June 25, 2022 11:58 AM > To: ffmpeg-devel@ffmpeg.org > Cc: Michael Niedermayer <michael@niedermayer.cc>; Andreas Rheinhardt > <andreas.rheinhardt@outlook.com>; Soft Works <softworkz@hotmail.com>; > Andriy Gelman <andriy.gelman@gmail.com>; softworkz > <softworkz@hotmail.com> > Subject: [PATCH v5 00/25] Subtitle Filtering 2022 > > > Subtitle Filtering 2022 > ======================= > > This is a substantial update to the earlier subtitle filtering patch > series. > A primary goal has been to address others' concerns as much as > possible on > one side and to provide more clarity and control over the way things > are > working. Clarity is is specifically important to allow for a better > understanding of the need for a subtitle start pts value that can be > different from the frame's pts value. This is done by refactoring the > subtitle timing fields in AVFrame, adding a frame field to indicate > repeated > subtitle frames, and finally the full removal of the heartbeat > functionality, replaced by a new 'subfeed' filter that provides > different > modes for arbitrating subtitle frames in a filter graph. Finally, > each > subtitle filter's documentation has been amended by a section > describing the > filter's timeline behavior (in v3 update). > > > Subtitle Filtering Demos > ======================== > > I published a demonstration of subtitle filtering capabilities with > OCR, > text and bitmap subtitle manipulation involved: Demo 1: Text- > Manipulation > with Bitmap Subtitles > [https://github.com/softworkz/SubtitleFilteringDemos/tree/master/Demo > 1] As there were no objections in any regard, I hope that we could make some progress regarding subtitle filtering soon. The only recent statement that has been made was something like "this may work in a limited number of cases but probably won't be capable to work for the whole range of real-life situations" I understand how easily such statements can induce doubt into the audience, even on those who tend and try to be neutral in their assessments. I also think that replying once another time with something like "no, this isn't true" won't be much impressive. Instead, I'm sharing a video with you, documenting my internal test runs for text subtitle overlay (with and without subtitle filtering, with and without hw overlay, with a range of hw accelerations for encoding and decoding, scaling or no scaling and two different kinds of source files). The diagrams at the right are a rather consumer focused, but still accurate. Just hwupload/download/hwmap are omitted and implied by the data changing the "swim lanes". https://github.com/softworkz/SubtitleFilteringDemos/tree/master/TestRun1 Commands and logs on request. Best wishes, softworkz
On Fri, Aug 12, 2022 at 12:51 AM Soft Works <softworkz@hotmail.com> wrote: > > > > -----Original Message----- > > From: ffmpegagent <ffmpegagent@gmail.com> > > Sent: Saturday, June 25, 2022 11:58 AM > > To: ffmpeg-devel@ffmpeg.org > > Cc: Michael Niedermayer <michael@niedermayer.cc>; Andreas Rheinhardt > > <andreas.rheinhardt@outlook.com>; Soft Works <softworkz@hotmail.com>; > > Andriy Gelman <andriy.gelman@gmail.com>; softworkz > > <softworkz@hotmail.com> > > Subject: [PATCH v5 00/25] Subtitle Filtering 2022 > > > > > > Subtitle Filtering 2022 > > ======================= > > > > This is a substantial update to the earlier subtitle filtering patch > > series. > > A primary goal has been to address others' concerns as much as > > possible on > > one side and to provide more clarity and control over the way things > > are > > working. Clarity is is specifically important to allow for a better > > understanding of the need for a subtitle start pts value that can be > > different from the frame's pts value. This is done by refactoring the > > subtitle timing fields in AVFrame, adding a frame field to indicate > > repeated > > subtitle frames, and finally the full removal of the heartbeat > > functionality, replaced by a new 'subfeed' filter that provides > > different > > modes for arbitrating subtitle frames in a filter graph. Finally, > > each > > subtitle filter's documentation has been amended by a section > > describing the > > filter's timeline behavior (in v3 update). > > > > > > Subtitle Filtering Demos > > ======================== > > > > I published a demonstration of subtitle filtering capabilities with > > OCR, > > text and bitmap subtitle manipulation involved: Demo 1: Text- > > Manipulation > > with Bitmap Subtitles > > [https://github.com/softworkz/SubtitleFilteringDemos/tree/master/Demo > > 1] > > > As there were no objections in any regard, I hope that we could make > some progress regarding subtitle filtering soon. > > The only recent statement that has been made was something like > > "this may work in a limited number of cases but probably won't be > capable to work for the whole range of real-life situations" > > I understand how easily such statements can induce doubt into the > audience, even on those who tend and try to be neutral in their > assessments. I also think that replying once another time with > something like "no, this isn't true" won't be much impressive. > > > Instead, I'm sharing a video with you, documenting my internal > test runs for text subtitle overlay (with and without subtitle > filtering, with and without hw overlay, with a range of hw accelerations > for encoding and decoding, scaling or no scaling and two different > kinds of source files). > > The diagrams at the right are a rather consumer focused, but still > accurate. Just hwupload/download/hwmap are omitted and implied > by the data changing the "swim lanes". > > https://github.com/softworkz/SubtitleFilteringDemos/tree/master/TestRun1 > > Commands and logs on request. > > We should more forward and merge this considerable subtitle work instead of waiting for broken promises from other inactive developers. > Best wishes, > softworkz > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Paul B Mahol (12022-08-21): > We should more forward and merge this considerable subtitle work instead of > waiting for broken promises from other inactive developers. > Then push to your fork.
On Sun, 21 Aug 2022, at 11:41, Paul B Mahol wrote:
> We should more forward and merge this considerable subtitle work
Are there parts of this work that have reach majority consensus?
Quoting Jean-Baptiste Kempf (2022-08-21 12:41:20) > > On Sun, 21 Aug 2022, at 11:41, Paul B Mahol wrote: > > We should more forward and merge this considerable subtitle work > > Are there parts of this work that have reach majority consensus? Almost exactly identical objections to the basic aspects of the API were raised independently by me, Lynne, and Hendrik. IIUC Soft Works still refuses to address them (though it's not so easy to tell in a 200-email thread).
On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > Almost exactly identical objections to the basic aspects of the API were > raised independently by me, Lynne, and Hendrik. > IIUC Soft Works still refuses to address them (though it's not so easy > to tell in a 200-email thread). OK. I lost the lists of objections, then.
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Jean-Baptiste Kempf > Sent: Monday, August 22, 2022 4:39 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > > Almost exactly identical objections to the basic aspects of the API > were > > raised independently by me, Lynne, and Hendrik. > > IIUC Soft Works still refuses to address them (though it's not so > easy > > to tell in a 200-email thread). > > OK. I lost the lists of objections, then. > > -- > Jean-Baptiste Kempf - President Could everybody who still has any objection PLEASE name it with reasoning and explain in which way it should be resolved? > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: ... > (though it's not so easy to tell in a 200-email thread) Yes that's true. For that reason it is not helpful to talk about unspecified objections from more than half a year ago. This is not further actionable without having a list of specific objections. When nobody responds, we need to assume that there aren't any left. Thanks, softworkz
Quoting Soft Works (2022-08-25 00:19:33) > > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > Jean-Baptiste Kempf > > Sent: Monday, August 22, 2022 4:39 PM > > To: ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > > > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > > > Almost exactly identical objections to the basic aspects of the API > > were > > > raised independently by me, Lynne, and Hendrik. > > > IIUC Soft Works still refuses to address them (though it's not so > > easy > > > to tell in a 200-email thread). > > > > OK. I lost the lists of objections, then. > > > > -- > > Jean-Baptiste Kempf - President > > > Could everybody who still has any objection PLEASE name it with reasoning > and explain in which way it should be resolved? Most of the main objections are mentioned in [1]. As far as I can tell, none of them were adequately addressed. > I wasn't refusing to make a change, but I have taken a lot of effort to > explain the reasons for that necessity. > I did that in several chats on IRC, on the ML, and recently, I have > written > an article especially to address that concern and better explain the > background: > > https://github.com/softworkz/SubtitleFilteringDemos/issues/1 > > It remained unresponded (but maybe unnoticed?). Sorry, but all explanations I've seen from you are walls of text from which I was never able to extract a solid argument that is not circular and does not leap to unsupported conclusions. And it is not just me, as far as I can tell, none of the others were convinced either. Frankly, you write too many words. A good argument about something like this should fit in a paragraph. Maybe followed by some extended explanations and clarifications, but the core of it should be quite short and right at the top, not buried under heaps of introductory remarks and examples. And if you cannot boil down your argument to a few words then maybe it's not a very strong one. > > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > ... > > (though it's not so easy to tell in a 200-email thread) > > Yes that's true. For that reason it is not helpful to talk about > unspecified objections from more than half a year ago. You are the author of this set, it is _your_ job to keep track of what has and has not been addressed. And if you want reviews, you should also try to make things easy to review. > This is not further actionable without having a list of specific objections. > When nobody responds, we need to assume that there aren't any left. Or maybe people just got tired of repeating the same objections to the same patches being submitted again and again. [1] http://lists.ffmpeg.org/pipermail/ffmpeg-devel/2021-December/288894.html Message-Id <MqASpFF--3-2@lynne.ee>
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Anton Khirnov > Sent: Friday, August 26, 2022 10:47 PM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Quoting Soft Works (2022-08-25 00:19:33) > > > > > -----Original Message----- > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > Jean-Baptiste Kempf > > > Sent: Monday, August 22, 2022 4:39 PM > > > To: ffmpeg-devel@ffmpeg.org > > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering > 2022 > > > > > > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > > > > Almost exactly identical objections to the basic aspects of the > API > > > were > > > > raised independently by me, Lynne, and Hendrik. > > > > IIUC Soft Works still refuses to address them (though it's not > so > > > easy > > > > to tell in a 200-email thread). > > > > > > OK. I lost the lists of objections, then. > > > > > > -- > > > Jean-Baptiste Kempf - President > > > > > > Could everybody who still has any objection PLEASE name it with > reasoning > > and explain in which way it should be resolved? > > Most of the main objections are mentioned in [1]. As far as I can > tell, none of them were adequately addressed. Hi Anton, thanks a lot for your reply. In fact, all of these were addressed long ago. http://lists.ffmpeg.org/pipermail/ffmpeg-devel/2021-December/288894.html That message you referenced is about 4 issues: 1. "Milliseconds?" I have changed fields to int64_t values in AV_TIME_BASE, please see below. 2. "There's no reason why this cannot be handled using the buffer and data fields" I had explained the reasons and in conversation on IRC, Lynne was no longer insisting on this AFAIR. 3. "I'm not going to accept a field which is instantly deprecated." I have reworked this and there is no longer a field that is instantly deprecated. 3. "As we've discussed multiple times, please merge this into the regular frame PTS field. We already have _2_ necessary stard/end fields." I have addressed this as well already. There are no longer 3 but only 2 fields remaining (int64_t in AV_TIME_BASE): AVFrame.subtitle_timing.start_pts AVFrame.subtitle_timing.duration > Frankly, you write too many words. A good argument about something > like > this should fit in a paragraph. Maybe followed by some extended > explanations and clarifications, but the core of it should be quite > short and right at the top, not buried under heaps of introductory > remarks and examples. And if you cannot boil down your argument to a > few words then maybe it's not a very strong one. It's a complicated matter and after I had seen that the situation wasn't understood, I started to get more into detail. I will try to write less text in the future. > Or maybe people just got tired of repeating the same objections to > the > same patches being submitted again and again. Or maybe people didn't realize that the objections were addressed already? > You are the author of this set, it is _your_ job to keep track of > what has and has not been addressed I do that, and the count of unaddressed objections that I'm aware of is zero. That's why I'm asking whether anybody would still have an objection or might not see her/his objection being addressed adequately. Thanks again, softworkz PS: the one thing that was under discussion on IRC was about why the subtitle start_pts cannot be the same as the AVFrame pts and that was the point I was explaining in so much detail.
Quoting Soft Works (2022-08-27 00:48:01) > 2. "There's no reason why this cannot be handled using the buffer > and data fields" > > I had explained the reasons and in conversation on IRC, Lynne was > no longer insisting on this AFAIR. I did not see this explanation, can you restate it here? If you claim the other points were addressed I will look at the patches again.
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Anton Khirnov > Sent: Wednesday, August 31, 2022 3:40 AM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Quoting Soft Works (2022-08-27 00:48:01) > > 2. "There's no reason why this cannot be handled using the buffer > > and data fields" > > > > I had explained the reasons and in conversation on IRC, Lynne was > > no longer insisting on this AFAIR. > > I did not see this explanation, can you restate it here? Sure. Let's look at the AVSubtitleArea struct first: typedef struct AVSubtitleArea { enum AVSubtitleType type; int flags; int x; ///< top left corner of area. int y; ///< top left corner of area. int w; ///< width of area. int h; ///< height of area. int nb_colors; ///< number of colors in bitmap palette (@ref pal). /** * Buffers and line sizes for the bitmap of this subtitle. * * @{ */ AVBufferRef *buf[AV_NUM_BUFFER_POINTERS]; int linesize[AV_NUM_BUFFER_POINTERS]; /** * @} */ uint32_t pal[256]; ///< RGBA palette for the bitmap. char *text; ///< 0-terminated plain UTF-8 text char *ass; ///< 0-terminated ASS/SSA compatible event line. } AVSubtitleArea; These structs are stored in AVFrame like this: /** * Number of items in the @ref subtitle_areas array. */ unsigned num_subtitle_areas; /** * Array of subtitle areas, may be empty. */ AVSubtitleArea **subtitle_areas; The question was "why this cannot be handled using the buffer and data fields?" - there are multiple reasons: 1. Area Count In ASS subtitles it's not uncommon that animations are defined through subtitle events (regular ass events). This can go as far as that dust particles are flying around on the screen and each particle has its own subtitle event/line which defines how it flies from x to y and how fast, etc. Not yet, but in a future update, the ass decoder should be improved in a way that it doesn't emit an individual AVFrame for each event line but bundles subsequent events together when these would have the same start time and duration. As a result, we can have AVFrames with dozens or even a hundred AVSubtitleArea items in extreme cases. The buffer and data fields are an array of fixed size (currently 8). Increasing to 16 or 32 would not help: there will still be cases where this does not suffice. 2. What exactly should be stored in frame->buf[]? Should we store the AVSubtitleArea structs as AVBuffer there? Or should we only store data in those buffers? If yes, which data? The subtitle bitmap probably. How about the subtitle text - maybe and area has even both. And should we also store the palette in some frame->buf[n]? If yes, how to keep track of which data is stored in which buffer? Further, there's a documented convention that requires that non-zero buffers are contiguous, i.e. there must not be an empty buffer pointer between two other a non-empty buffer pointers. This would require to re-arrange the buffers to close any gap when some subtitle area data is removed, zeroed out or has been converted to another format. Closing such gap also require to update any mapping information ("which buffer is related to which area?") That's a lot of tedious work and every API user (or codec/filter developer) would need to do it in the right way. 3. AVFrame Code Logic Mistmatch A subtitle frame can have 0 to N areas, which means that a frame without any area is still valid. Opposed to that, existing (code all over the place in ffmpeg) is treating an AVFrame as invalid when the first data buffer is empty, i.e. frame->buf[0] != NULL To accommodate to this situation, subtitle frames are always creating a 1-byte buffer for buf[0]. When we would want subtitle data to be stored in those buffers, every developer would need to be aware about the requirement to have that dummy buffer in the first array element. When something is to be stored, the dummy buffer would need to be freed first before storing the actual data. And when the last buffer gets deleted, API users would need to know to create a new dummy buffer. That fixed size array might be a good fit for image data, but it's not for subtitle data. The approach in my patchset is simple, intuitive and everybody can easily work with it without needing any special knowledge: unsigned num_subtitle_areas; AVSubtitleArea **subtitle_areas; IMHO, this is the most suitable pattern for this situation. > If you claim the other points were addressed I will look at the > patches > again. Oh cool, that would be great! Thanks, softworkz
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Soft Works > Sent: Wednesday, August 31, 2022 6:02 AM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > > > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > Anton Khirnov > > Sent: Wednesday, August 31, 2022 3:40 AM > > To: FFmpeg development discussions and patches <ffmpeg- > > devel@ffmpeg.org> > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering > 2022 > > > > Quoting Soft Works (2022-08-27 00:48:01) > > > 2. "There's no reason why this cannot be handled using the buffer > > > and data fields" > > > > > > I had explained the reasons and in conversation on IRC, Lynne was > > > no longer insisting on this AFAIR. > > > > I did not see this explanation, can you restate it here? > > Sure. Let's look at the AVSubtitleArea struct first: > > > typedef struct AVSubtitleArea { > > enum AVSubtitleType type; > int flags; > > int x; ///< top left corner of area. > int y; ///< top left corner of area. > int w; ///< width of area. > int h; ///< height of area. > int nb_colors; ///< number of colors in bitmap palette (@ref > pal). > > /** > * Buffers and line sizes for the bitmap of this subtitle. > * > * @{ > */ > AVBufferRef *buf[AV_NUM_BUFFER_POINTERS]; > int linesize[AV_NUM_BUFFER_POINTERS]; > /** > * @} > */ > > uint32_t pal[256]; ///< RGBA palette for the bitmap. > > char *text; ///< 0-terminated plain UTF-8 text > char *ass; ///< 0-terminated ASS/SSA compatible event > line. > > } AVSubtitleArea; > > > > These structs are stored in AVFrame like this: > > > /** > * Number of items in the @ref subtitle_areas array. > */ > unsigned num_subtitle_areas; > > /** > * Array of subtitle areas, may be empty. > */ > AVSubtitleArea **subtitle_areas; > > > > The question was "why this cannot be handled using the buffer > and data fields?" - there are multiple reasons: > > 1. Area Count > > In ASS subtitles it's not uncommon that animations are defined > through subtitle events (regular ass events). This can go as > far as that dust particles are flying around on the screen and > each particle has its own subtitle event/line which defines > how it flies from x to y and how fast, etc. > Not yet, but in a future update, the ass decoder should be > improved in a way that it doesn't emit an individual AVFrame > for each event line but bundles subsequent events together > when these would have the same start time and duration. > As a result, we can have AVFrames with dozens or even a hundred > AVSubtitleArea items in extreme cases. > > The buffer and data fields are an array of fixed size (currently > 8). Increasing to 16 or 32 would not help: there will still be > cases where this does not suffice. > > 2. What exactly should be stored in frame->buf[]? > > Should we store the AVSubtitleArea structs as AVBuffer there? > > Or should we only store data in those buffers? If yes, which > data? The subtitle bitmap probably. How about the subtitle > text - maybe and area has even both. And should we also > store the palette in some frame->buf[n]? > If yes, how to keep track of which data is stored in which > buffer? > Further, there's a documented convention that requires that > non-zero buffers are contiguous, i.e. there must not be > an empty buffer pointer between two other a non-empty buffer > pointers. This would require to re-arrange the buffers to > close any gap when some subtitle area data is removed, zeroed > out or has been converted to another format. > Closing such gap also require to update any mapping information > ("which buffer is related to which area?") > > That's a lot of tedious work and every API user (or codec/filter > developer) would need to do it in the right way. > > > > 3. AVFrame Code Logic Mistmatch > > A subtitle frame can have 0 to N areas, which means that a > frame without any area is still valid. Opposed to that, existing > (code all over the place in ffmpeg) is treating an AVFrame > as invalid when the first data buffer is empty, > i.e. frame->buf[0] != NULL > > To accommodate to this situation, subtitle frames are always > creating a 1-byte buffer for buf[0]. > > When we would want subtitle data to be stored in those buffers, > every developer would need to be aware about the requirement > to have that dummy buffer in the first array element. When something > is to be stored, the dummy buffer would need to be freed first > before storing the actual data. > And when the last buffer gets deleted, API users would need to > know to create a new dummy buffer. > > > That fixed size array might be a good fit for image data, but > it's not for subtitle data. > > > The approach in my patchset is simple, intuitive and everybody > can easily work with it without needing any special knowledge: > > > unsigned num_subtitle_areas; > AVSubtitleArea **subtitle_areas; > > IMHO, this is the most suitable pattern for this situation. > > > > > If you claim the other points were addressed I will look at the > > patches > > again. > > Oh cool, that would be great! Hi Anton, when do you think you could find some time to review the patch set? Just let me know, then I'd rebase and submit an updated version. Thanks, softworkz
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Anton Khirnov > Sent: Wednesday, August 31, 2022 3:40 AM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Quoting Soft Works (2022-08-27 00:48:01) > > 2. "There's no reason why this cannot be handled using the buffer > > and data fields" > > > > I had explained the reasons and in conversation on IRC, Lynne was > > no longer insisting on this AFAIR. > > I did not see this explanation, can you restate it here? > > If you claim the other points were addressed I will look at the > patches > again. > > -- > Anton Khirnov > _______________________________________________ Friendly Ping :-)
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Soft Works > Sent: Wednesday, August 31, 2022 6:02 AM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > > > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > Anton Khirnov > > Sent: Wednesday, August 31, 2022 3:40 AM > > To: FFmpeg development discussions and patches <ffmpeg- > > devel@ffmpeg.org> > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering > 2022 > > > > Quoting Soft Works (2022-08-27 00:48:01) > > > 2. "There's no reason why this cannot be handled using the buffer > > > and data fields" > > > > > > I had explained the reasons and in conversation on IRC, Lynne was > > > no longer insisting on this AFAIR. > > > > I did not see this explanation, can you restate it here? > You had asked me to restate the explanation. I did that (below) but you never responded. Thanks, softworkz > Sure. Let's look at the AVSubtitleArea struct first: > > > typedef struct AVSubtitleArea { > > enum AVSubtitleType type; > int flags; > > int x; ///< top left corner of area. > int y; ///< top left corner of area. > int w; ///< width of area. > int h; ///< height of area. > int nb_colors; ///< number of colors in bitmap palette (@ref > pal). > > /** > * Buffers and line sizes for the bitmap of this subtitle. > * > * @{ > */ > AVBufferRef *buf[AV_NUM_BUFFER_POINTERS]; > int linesize[AV_NUM_BUFFER_POINTERS]; > /** > * @} > */ > > uint32_t pal[256]; ///< RGBA palette for the bitmap. > > char *text; ///< 0-terminated plain UTF-8 text > char *ass; ///< 0-terminated ASS/SSA compatible event > line. > > } AVSubtitleArea; > > > > These structs are stored in AVFrame like this: > > > /** > * Number of items in the @ref subtitle_areas array. > */ > unsigned num_subtitle_areas; > > /** > * Array of subtitle areas, may be empty. > */ > AVSubtitleArea **subtitle_areas; > > > > The question was "why this cannot be handled using the buffer > and data fields?" - there are multiple reasons: > > 1. Area Count > > In ASS subtitles it's not uncommon that animations are defined > through subtitle events (regular ass events). This can go as > far as that dust particles are flying around on the screen and > each particle has its own subtitle event/line which defines > how it flies from x to y and how fast, etc. > Not yet, but in a future update, the ass decoder should be > improved in a way that it doesn't emit an individual AVFrame > for each event line but bundles subsequent events together > when these would have the same start time and duration. > As a result, we can have AVFrames with dozens or even a hundred > AVSubtitleArea items in extreme cases. > > The buffer and data fields are an array of fixed size (currently > 8). Increasing to 16 or 32 would not help: there will still be > cases where this does not suffice. > > 2. What exactly should be stored in frame->buf[]? > > Should we store the AVSubtitleArea structs as AVBuffer there? > > Or should we only store data in those buffers? If yes, which > data? The subtitle bitmap probably. How about the subtitle > text - maybe and area has even both. And should we also > store the palette in some frame->buf[n]? > If yes, how to keep track of which data is stored in which > buffer? > Further, there's a documented convention that requires that > non-zero buffers are contiguous, i.e. there must not be > an empty buffer pointer between two other a non-empty buffer > pointers. This would require to re-arrange the buffers to > close any gap when some subtitle area data is removed, zeroed > out or has been converted to another format. > Closing such gap also require to update any mapping information > ("which buffer is related to which area?") > > That's a lot of tedious work and every API user (or codec/filter > developer) would need to do it in the right way. > > > > 3. AVFrame Code Logic Mistmatch > > A subtitle frame can have 0 to N areas, which means that a > frame without any area is still valid. Opposed to that, existing > (code all over the place in ffmpeg) is treating an AVFrame > as invalid when the first data buffer is empty, > i.e. frame->buf[0] != NULL > > To accommodate to this situation, subtitle frames are always > creating a 1-byte buffer for buf[0]. > > When we would want subtitle data to be stored in those buffers, > every developer would need to be aware about the requirement > to have that dummy buffer in the first array element. When something > is to be stored, the dummy buffer would need to be freed first > before storing the actual data. > And when the last buffer gets deleted, API users would need to > know to create a new dummy buffer. > > > That fixed size array might be a good fit for image data, but > it's not for subtitle data. > > > The approach in my patchset is simple, intuitive and everybody > can easily work with it without needing any special knowledge: > > > unsigned num_subtitle_areas; > AVSubtitleArea **subtitle_areas; > > IMHO, this is the most suitable pattern for this situation. > > > > > If you claim the other points were addressed I will look at the > > patches > > again. > > Oh cool, that would be great! > > Thanks, > softworkz > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Subtitle Filtering 2022 ======================= This is a substantial update to the earlier subtitle filtering patch series. A primary goal has been to address others' concerns as much as possible on one side and to provide more clarity and control over the way things are working. Clarity is is specifically important to allow for a better understanding of the need for a subtitle start pts value that can be different from the frame's pts value. This is done by refactoring the subtitle timing fields in AVFrame, adding a frame field to indicate repeated subtitle frames, and finally the full removal of the heartbeat functionality, replaced by a new 'subfeed' filter that provides different modes for arbitrating subtitle frames in a filter graph. Finally, each subtitle filter's documentation has been amended by a section describing the filter's timeline behavior (in v3 update). Subtitle Filtering Demos ======================== I published a demonstration of subtitle filtering capabilities with OCR, text and bitmap subtitle manipulation involved: Demo 1: Text-Manipulation with Bitmap Subtitles [https://github.com/softworkz/SubtitleFilteringDemos/tree/master/Demo1] v5 - Conversion to Graphic Subtitles, and other enhancements ============================================================ * I'm glad to announce that Traian (@tcoza) has joined the project and contributed a new 'text2graphicsub' filter to convert text subtitles to graphic subtitles, which can in turn be encoded as dvd, dvb or x-subs (and any other encoder for graphic subs that might be added in the future). This filter closes the last open "gap" in subtitle processing. * stripstyles filter: now allows very fine-grained control over which ASS style codes should be preserved or stripped * stripstyles: do not drop dialog margin values * subfeed filter: eliminates duplicate frames with duplicate start times when 'fix_overlap' is specified * textmod: do not drop effect values * graphicsub2text: reduce font size jitter * ass_split: add function to selectively preserve elements when splitting * add strim, snull and ssink and further unify subtitle frame handling with audio and video * ffmpeg_filter: get simple filter notation working for subtitles v4 - Quality Improvements ========================= * finally an updated version * includes many improvements from internal testing * all FATE tests passed * all example commands from the docs verified to work * can't list all the detail changes.. * I have left out the extra commits which can be handled separately, just in case somebody wonders why these are missing: * avcodec/webvttenc: Don't encode drawing codes and empty lines * avcodec/webvttenc: convert hard-space tags to * avutil/ass_split: Add parsing of hard-space tags (\h) * avutil/ass_split: Treat all content in curly braces as hidden * avutil/ass_split: Fix ass parsing of style codes with comments v3 - Rebase =========== due to merge conflicts - apologies. Changes in v2 ============= * added .gitattributes file to enforce binary diffs for the test refs that cannot be applied when being sent via e-mail * perform filter graph re-init due to subtitle "frame size" change only when the size was unknown before and not set via -canvas_size * overlaytextsubs: Make sure to request frames on the subtitle input * avfilter/splitcc: Start parsing cc data on key frames only * avcodec/webvttenc: Don't encode ass drawing codes and empty lines * stripstyles: fix mem leak * gs2t: improve color detection * gs2t: empty frames must not be skipped * subfeed: fix name * textmod: preserve margins * added .gitattributes file to enforce binary diffs for the test refs that cannot be applied when being sent via e-mail * perform filter graph re-init due to subtitle "frame size" change only when the size was unknown before and not set via -canvas_size * avcodec/dvbsubdec: Fix conditions for fallback to default resolution * Made changes suggested by Andreas * Fixed failing command line reported by Michael Changes from previous version v24: AVFrame ======= * Removed sub_start_time The start time is now added to the subtitle start_pts during decoding The sub_end_time field is adjusted accordingly * Renamed sub_end_time to duration which it is effectively after removing the start_time * Added a sub-struct 'subtitle_timing' to av frame Contains subtitle_pts renamed to 'subtitle_timing.start_pts' and 'subtitle_timing.duration' * Change both fields to (fixed) time_base AV_TIMEBASE * add repeat_sub field provides a clear indication whether a subtitle frame is an actual subtitle event or a repeated subtitle frame in a filter graph Heartbeat Removal ================= * completely removed the earlier heartbeat implementation * filtering arbitration is now implemented in a new filter: 'subfeed' * subfeed will be auto-inserted for compatiblity with sub2video command lines * the new behavior is not exactly identical to the earlier behavior, but it basically allows to achieve the same results * there's a small remainder, now named subtitle kickoff which serves to get things (in the filter graph) going right from the start New 'subfeed' Filter ==================== * a versatile filter for solving all kinds of problems with subtile frame flow in filter graphs * Can be inserted at any position in a graph * Auto-inserted for sub2video command lines (in repeat-mode) * Allows duration fixup delay input frames with unknown duration and infer duration from start of subsequent frame * Provides multiple modes of operation: * repeat mode (default) Queues input frames Outputs frames at a fixed (configurable) rate Either sends a matching input frame (repeatedly) or empty frames otherwise * scatter mode similar to repeat mode, but splits input frames by duration into small segments with same content * forward mode No fixed output rate Useful in combination with duration fixup or overlap fixup ffmpeg Tool Changes =================== * delay subtitle output stream initialization (like for audio and video) This is needed for example when a format header depends on having received an initial frame to derive certain header values from * decoding: set subtitle frame size from decoding context * re-init graph when subtitle size changes * always insert subscale filter for sub2video command lines (to ensure correct scaling) Subtitle Encoding ================= * ignore repeated frames for encoding based on repeat_sub field in AVFrame * support multi-area encoding for text subtitles Subtitle OCR can create multiple areas at different positions. Previously, the texts were always squashed into a single area ('subtitle rect'), which was not ideal. Multiple text areas are now generally supported: * ASS Encoder Changed to use the 'receive_packet' encoding API A single frame with multiple text areas will create multiple packets now * All other text subtitle encoders A newline is inserted between the text from multiple areas graphicsub2text (OCR) ===================== * enhanced preprocessing * using elbg algorithm for color quantization * detection and removal of text outlines * map-based identification of colors per word (text, outline, background) * add option for duration fixup * add option to dump preprocessing bitmaps * Recognize formatting and apply as ASS inline styles * per word(!) * paragraph alignment * positioning * font names * font size * font style (italic, underline, bold) * text color, outline color Other Filter Changes ==================== * all: Make sure to forward all link properties (time base, frame rate, w, h) where appropriate * overlaytextsubs: request frames on the subtitle input * overlaytextsubs: disable read-order checking * overlaytextsubs: improve implementation of render_latest_only * overlaytextsubs: ensure equal in/out video formats * splitcc: derive framerate from realtime_latency * graphicsub2video: implement caching of converted frames * graphicsub2video: use 1x1 output frame size as long as subtitle size is unknown (0x0) Plus a dozen of things I forgot.. softworkz (24): avcodec,avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values avutil/frame: Prepare AVFrame for subtitle handling avcodec/subtitles: Introduce new frame-based subtitle decoding API avcodec/libzvbi: set subtitle type avfilter/subtitles: Update vf_subtitles to use new decoding api avcodec,avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing avcodec/subtitles: Replace deprecated enum values fftools/play,probe: Adjust for subtitle changes avfilter/subtitles: Add subtitles.c for subtitle frame allocation avfilter/avfilter: Handle subtitle frames avfilter/avfilter: Fix hardcoded input index avfilter/sbuffer: Add sbuffersrc and sbuffersink filters avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters avfilter/textmod: Add textmod, censor and show_speaker filters avfilter/stripstyles: Add stripstyles filter avfilter/splitcc: Add splitcc filter for closed caption handling avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles avfilter/subfeed: add subtitle feed filter avfilter/snull,strim: Add snull and strim filters avcodec/subtitles: Migrate subtitle encoders to frame-based API fftools/ffmpeg: Introduce subtitle filtering and new frame-based subtitle encoding avcodec/dvbsubdec: Fix conditions for fallback to default resolution tcoza (1): avfilter/text2graphicsub: Added text2graphicsub subtitle filter configure | 10 +- doc/filters.texi | 807 ++++++++++++++ fftools/ffmpeg.c | 613 +++++----- fftools/ffmpeg.h | 17 +- fftools/ffmpeg_filter.c | 270 +++-- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 28 +- fftools/ffplay.c | 102 +- fftools/ffprobe.c | 47 +- libavcodec/Makefile | 56 +- libavcodec/ass.h | 151 +-- libavcodec/ass_split.h | 191 ---- libavcodec/assdec.c | 4 +- libavcodec/assenc.c | 191 +++- libavcodec/avcodec.c | 8 + libavcodec/avcodec.h | 34 +- libavcodec/ccaption_dec.c | 20 +- libavcodec/codec_internal.h | 12 - libavcodec/decode.c | 60 +- libavcodec/dvbsubdec.c | 53 +- libavcodec/dvbsubenc.c | 96 +- libavcodec/dvdsubdec.c | 2 +- libavcodec/dvdsubenc.c | 102 +- libavcodec/encode.c | 61 +- libavcodec/internal.h | 16 + libavcodec/jacosubdec.c | 2 +- libavcodec/libaribb24.c | 2 +- libavcodec/libzvbi-teletextdec.c | 17 +- libavcodec/microdvddec.c | 7 +- libavcodec/movtextdec.c | 3 +- libavcodec/movtextenc.c | 126 ++- libavcodec/mpl2dec.c | 2 +- libavcodec/pgssubdec.c | 2 +- libavcodec/realtextdec.c | 2 +- libavcodec/samidec.c | 2 +- libavcodec/srtdec.c | 2 +- libavcodec/srtenc.c | 116 +- libavcodec/subviewerdec.c | 2 +- libavcodec/tests/avcodec.c | 5 +- libavcodec/textdec.c | 4 +- libavcodec/ttmlenc.c | 114 +- libavcodec/utils.c | 185 ++- libavcodec/webvttdec.c | 2 +- libavcodec/webvttenc.c | 94 +- libavcodec/xsubdec.c | 2 +- libavcodec/xsubenc.c | 88 +- libavfilter/Makefile | 18 + libavfilter/allfilters.c | 19 + libavfilter/avfilter.c | 34 +- libavfilter/avfilter.h | 11 + libavfilter/avfiltergraph.c | 5 + libavfilter/buffersink.c | 54 + libavfilter/buffersink.h | 7 + libavfilter/buffersrc.c | 72 ++ libavfilter/buffersrc.h | 1 + libavfilter/formats.c | 16 + libavfilter/formats.h | 3 + libavfilter/internal.h | 19 +- libavfilter/sf_graphicsub2text.c | 1137 +++++++++++++++++++ libavfilter/sf_snull.c | 50 + libavfilter/sf_splitcc.c | 395 +++++++ libavfilter/sf_stripstyles.c | 237 ++++ libavfilter/sf_subfeed.c | 412 +++++++ libavfilter/sf_subscale.c | 884 +++++++++++++++ libavfilter/sf_text2graphicsub.c | 630 +++++++++++ libavfilter/sf_textmod.c | 710 ++++++++++++ libavfilter/subtitles.c | 63 ++ libavfilter/subtitles.h | 44 + libavfilter/trim.c | 46 +- libavfilter/vf_overlaygraphicsubs.c | 765 +++++++++++++ libavfilter/vf_overlaytextsubs.c | 680 +++++++++++ libavfilter/vf_subtitles.c | 67 +- libavutil/Makefile | 4 + {libavcodec => libavutil}/ass.c | 115 +- libavutil/ass_internal.h | 135 +++ {libavcodec => libavutil}/ass_split.c | 179 ++- libavutil/ass_split_internal.h | 254 +++++ libavutil/frame.c | 206 +++- libavutil/frame.h | 85 +- libavutil/subfmt.c | 45 + libavutil/subfmt.h | 115 ++ libavutil/version.h | 1 + tests/ref/fate/filter-overlay-dvdsub-2397 | 182 +-- tests/ref/fate/sub-dvb | 162 +-- tests/ref/fate/sub-scc | 1 - tests/ref/fate/sub2video | 1091 +++++++++++++++++- tests/ref/fate/sub2video_basic | 1238 +++++++++++++++++++-- tests/ref/fate/sub2video_time_limited | 78 +- 88 files changed, 12398 insertions(+), 1604 deletions(-) delete mode 100644 libavcodec/ass_split.h create mode 100644 libavfilter/sf_graphicsub2text.c create mode 100644 libavfilter/sf_snull.c create mode 100644 libavfilter/sf_splitcc.c create mode 100644 libavfilter/sf_stripstyles.c create mode 100644 libavfilter/sf_subfeed.c create mode 100644 libavfilter/sf_subscale.c create mode 100644 libavfilter/sf_text2graphicsub.c create mode 100644 libavfilter/sf_textmod.c create mode 100644 libavfilter/subtitles.c create mode 100644 libavfilter/subtitles.h create mode 100644 libavfilter/vf_overlaygraphicsubs.c create mode 100644 libavfilter/vf_overlaytextsubs.c rename {libavcodec => libavutil}/ass.c (59%) create mode 100644 libavutil/ass_internal.h rename {libavcodec => libavutil}/ass_split.c (71%) create mode 100644 libavutil/ass_split_internal.h create mode 100644 libavutil/subfmt.c create mode 100644 libavutil/subfmt.h base-commit: 6a82412bf33108111eb3f63076fd5a51349ae114 Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-18%2Fsoftworkz%2Fsubmit_subfiltering-v5 Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-18/softworkz/submit_subfiltering-v5 Pull-Request: https://github.com/ffstaging/FFmpeg/pull/18 Range-diff vs v4: 1: 2f3ba171f5 = 1: aa32b9048f avcodec,avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values 2: ff101f8a76 ! 2: d5ab9d1919 avutil/frame: Prepare AVFrame for subtitle handling @@ libavutil/frame.c: FF_ENABLE_DEPRECATION_WARNINGS + case AVMEDIA_TYPE_VIDEO: return frame_copy_video(dst, src); - else if (dst->nb_samples > 0 && -- (av_channel_layout_check(&dst->ch_layout) --#if FF_API_OLD_CHANNEL_LAYOUT -- || dst->channel_layout || dst->channels --#endif -- )) + case AVMEDIA_TYPE_AUDIO: - return frame_copy_audio(dst, src); ++ if (dst->nb_samples > 0 && + (av_channel_layout_check(&dst->ch_layout) + #if FF_API_OLD_CHANNEL_LAYOUT + || dst->channels > 0 + #endif + )) +- return frame_copy_audio(dst, src); -FF_ENABLE_DEPRECATION_WARNINGS -- -- return AVERROR(EINVAL); ++ return frame_copy_audio(dst, src); ++ break; + case AVMEDIA_TYPE_SUBTITLE: + return frame_copy_subtitles(dst, src, 1); -+ default: -+ return AVERROR(EINVAL); + } - } - void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type) + return AVERROR(EINVAL); + } ## libavutil/frame.h ## @@ 3: b8935d5e68 = 3: 0a685a6b19 avcodec/subtitles: Introduce new frame-based subtitle decoding API 4: 4b44732e07 = 4: 0b69b1ce19 avcodec/libzvbi: set subtitle type 5: 8faa7a4043 = 5: 0c2091e57c avfilter/subtitles: Update vf_subtitles to use new decoding api 6: 1664026d7c ! 6: 4903cdd1cd avcodec,avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing @@ Commit message - hard_space callback (for upcoming fix) - extensible callback (for future extension) + - new API which allows tag filtering Signed-off-by: softworkz <softworkz@hotmail.com> @@ libavcodec/ass.h - const char *linebreaks, int keep_ass_markup); #endif /* AVCODEC_ASS_H */ + ## libavcodec/ass_split.h (deleted) ## +@@ +-/* +- * SSA/ASS spliting functions +- * Copyright (c) 2010 Aurelien Jacobs <aurel@gnuage.org> +- * +- * This file is part of FFmpeg. +- * +- * FFmpeg is free software; you can redistribute it and/or +- * modify it under the terms of the GNU Lesser General Public +- * License as published by the Free Software Foundation; either +- * version 2.1 of the License, or (at your option) any later version. +- * +- * FFmpeg is distributed in the hope that it will be useful, +- * but WITHOUT ANY WARRANTY; without even the implied warranty of +- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +- * Lesser General Public License for more details. +- * +- * You should have received a copy of the GNU Lesser General Public +- * License along with FFmpeg; if not, write to the Free Software +- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +- */ +- +-#ifndef AVCODEC_ASS_SPLIT_H +-#define AVCODEC_ASS_SPLIT_H +- +-/** +- * fields extracted from the [Script Info] section +- */ +-typedef struct { +- char *script_type; /**< SSA script format version (eg. v4.00) */ +- char *collisions; /**< how subtitles are moved to prevent collisions */ +- int play_res_x; /**< video width that ASS coords are referring to */ +- int play_res_y; /**< video height that ASS coords are referring to */ +- float timer; /**< time multiplier to apply to SSA clock (in %) */ +-} ASSScriptInfo; +- +-/** +- * fields extracted from the [V4(+) Styles] section +- */ +-typedef struct { +- char *name; /**< name of the tyle (case sensitive) */ +- char *font_name; /**< font face (case sensitive) */ +- int font_size; /**< font height */ +- int primary_color; /**< color that a subtitle will normally appear in */ +- int secondary_color; +- int outline_color; /**< color for outline in ASS, called tertiary in SSA */ +- int back_color; /**< color of the subtitle outline or shadow */ +- int bold; /**< whether text is bold (1) or not (0) */ +- int italic; /**< whether text is italic (1) or not (0) */ +- int underline; /**< whether text is underlined (1) or not (0) */ +- int strikeout; +- float scalex; +- float scaley; +- float spacing; +- float angle; +- int border_style; +- float outline; +- float shadow; +- int alignment; /**< position of the text (left, center, top...), +- defined after the layout of the numpad +- (1-3 sub, 4-6 mid, 7-9 top) */ +- int margin_l; +- int margin_r; +- int margin_v; +- int alpha_level; +- int encoding; +-} ASSStyle; +- +-/** +- * fields extracted from the [Events] section +- */ +-typedef struct { +- int readorder; +- int layer; /**< higher numbered layers are drawn over lower numbered */ +- int start; /**< start time of the dialog in centiseconds */ +- int end; /**< end time of the dialog in centiseconds */ +- char *style; /**< name of the ASSStyle to use with this dialog */ +- char *name; +- int margin_l; +- int margin_r; +- int margin_v; +- char *effect; +- char *text; /**< actual text which will be displayed as a subtitle, +- can include style override control codes (see +- ff_ass_split_override_codes()) */ +-} ASSDialog; +- +-/** +- * structure containing the whole split ASS data +- */ +-typedef struct { +- ASSScriptInfo script_info; /**< general information about the SSA script*/ +- ASSStyle *styles; /**< array of split out styles */ +- int styles_count; /**< number of ASSStyle in the styles array */ +- ASSDialog *dialogs; /**< array of split out dialogs */ +- int dialogs_count; /**< number of ASSDialog in the dialogs array*/ +-} ASS; +- +-/** +- * This struct can be casted to ASS to access to the split data. +- */ +-typedef struct ASSSplitContext ASSSplitContext; +- +-/** +- * Split a full ASS file or a ASS header from a string buffer and store +- * the split structure in a newly allocated context. +- * +- * @param buf String containing the ASS formatted data. +- * @return Newly allocated struct containing split data. +- */ +-ASSSplitContext *ff_ass_split(const char *buf); +- +-/** +- * Free a dialogue obtained from ff_ass_split_dialog(). +- */ +-void ff_ass_free_dialog(ASSDialog **dialogp); +- +-/** +- * Split one ASS Dialogue line from a string buffer. +- * +- * @param ctx Context previously initialized by ff_ass_split(). +- * @param buf String containing the ASS "Dialogue" line. +- * @return Pointer to the split ASSDialog. Must be freed with ff_ass_free_dialog() +- */ +-ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char *buf); +- +-/** +- * Free all the memory allocated for an ASSSplitContext. +- * +- * @param ctx Context previously initialized by ff_ass_split(). +- */ +-void ff_ass_split_free(ASSSplitContext *ctx); +- +- +-/** +- * Set of callback functions corresponding to each override codes that can +- * be encountered in a "Dialogue" Text field. +- */ +-typedef struct { +- /** +- * @defgroup ass_styles ASS styles +- * @{ +- */ +- void (*text)(void *priv, const char *text, int len); +- void (*new_line)(void *priv, int forced); +- void (*style)(void *priv, char style, int close); +- void (*color)(void *priv, unsigned int /* color */, unsigned int color_id); +- void (*alpha)(void *priv, int alpha, int alpha_id); +- void (*font_name)(void *priv, const char *name); +- void (*font_size)(void *priv, int size); +- void (*alignment)(void *priv, int alignment); +- void (*cancel_overrides)(void *priv, const char *style); +- /** @} */ +- +- /** +- * @defgroup ass_functions ASS functions +- * @{ +- */ +- void (*move)(void *priv, int x1, int y1, int x2, int y2, int t1, int t2); +- void (*origin)(void *priv, int x, int y); +- /** @} */ +- +- /** +- * @defgroup ass_end end of Dialogue Event +- * @{ +- */ +- void (*end)(void *priv); +- /** @} */ +-} ASSCodesCallbacks; +- +-/** +- * Split override codes out of a ASS "Dialogue" Text field. +- * +- * @param callbacks Set of callback functions called for each override code +- * encountered. +- * @param priv Opaque pointer passed to the callback functions. +- * @param buf The ASS "Dialogue" Text field to split. +- * @return >= 0 on success otherwise an error code <0 +- */ +-int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, +- const char *buf); +- +-/** +- * Find an ASSStyle structure by its name. +- * +- * @param ctx Context previously initialized by ff_ass_split(). +- * @param style name of the style to search for. +- * @return the ASSStyle corresponding to style, or NULL if style can't be found +- */ +-ASSStyle *ff_ass_style_get(ASSSplitContext *ctx, const char *style); +- +-#endif /* AVCODEC_ASS_SPLIT_H */ + ## libavcodec/assdec.c ## @@ #include <string.h> @@ libavutil/ass.c: char *ff_ass_get_dialog(int readorder, int layer, const char *s - const char *speaker) +char *avpriv_ass_get_dialog_ex(int readorder, int layer, const char *style, + const char *speaker, int margin_l, int margin_r, -+ int margin_v, const char *text) ++ int margin_v, const char *effect, const char *text) { - return ff_ass_add_rect2(sub, dialog, readorder, layer, style, speaker, NULL); -} @@ libavutil/ass.c: char *ff_ass_get_dialog(int readorder, int layer, const char *s - FFASSDecoderContext *s = avctx->priv_data; - if (!(avctx->flags2 & AV_CODEC_FLAG2_RO_FLUSH_NOOP)) - s->readorder = 0; -+ return av_asprintf("%d,%d,%s,%s,%d,%d,%d,,%s", ++ return av_asprintf("%d,%d,%s,%s,%d,%d,%d,%s,%s", + readorder, layer, style ? style : "Default", + speaker ? speaker : "", margin_l, margin_r, -+ margin_v, text); ++ margin_v, effect ? effect : "", text); } -void ff_ass_bprint_text_event(AVBPrint *buf, const char *p, int size, @@ libavutil/ass_internal.h (new) + */ +char *avpriv_ass_get_dialog_ex(int readorder, int layer, const char *style, + const char *speaker, int margin_l, int margin_r, -+ int margin_v, const char *text); ++ int margin_v, const char *effect, const char *text); + +/** + * Escape a text subtitle using ASS syntax into an AVBPrint buffer. @@ libavutil/ass_split.c: ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, cons if (ctx) { int i; @@ libavutil/ass_split.c: void ff_ass_split_free(ASSSplitContext *ctx) + } } ++static int ass_remove_empty_braces(AVBPrint* buffer) ++{ ++ char* tmp; ++ int ret = 0, n = 0; -int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, -+int avpriv_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, - const char *buf) +- const char *buf) ++ if (buffer == NULL || buffer->len == 0 || !av_bprint_is_complete(buffer)) ++ return 0; ++ ++ ret = av_bprint_finalize(buffer, &tmp); ++ if (ret) ++ return ret; ++ ++ for (unsigned i = 0; i < buffer->len; i++) { ++ if (tmp[i] == '{' && tmp[i+1] == '}') ++ i++; ++ else ++ tmp[n++] = tmp[i]; ++ } ++ ++ tmp[n++] = '\0'; ++ ++ av_bprint_init(buffer, n, n); ++ av_bprint_append_data(buffer, tmp, n - 1); ++ av_free(tmp); ++ ++ return ret; ++} ++ ++static void ass_write_filtered_line(AVBPrint* buffer, const char *buf, int len, enum ASSSplitComponents keep_flags, enum ASSSplitComponents split_component) ++{ ++ if (buffer == NULL || buf == NULL || len == 0) ++ return; ++ ++ if (split_component != ASS_SPLIT_ANY && !(keep_flags & split_component)) ++ return; ++ ++ ++ av_bprint_append_data(buffer, buf, len - 1); ++} ++ ++int avpriv_ass_filter_override_codes(const ASSCodesCallbacks *callbacks, void *priv, const char *buf, AVBPrint* outbuffer, enum ASSSplitComponents keep_flags) { const char *text = NULL; -@@ libavutil/ass_split.c: int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, + char new_line[2]; +- int text_len = 0; ++ int text_len = 0, ret = 0; + + while (buf && *buf) { +- if (text && callbacks->text && +- (sscanf(buf, "\\%1[nN]", new_line) == 1 || +- !strncmp(buf, "{\\", 2))) { +- callbacks->text(priv, text, text_len); ++ ++ if (text && (sscanf(buf, "\\%1[nN]", new_line) == 1 || !strncmp(buf, "{\\", 2))) { ++ ass_write_filtered_line(outbuffer, text, text_len + 1, keep_flags, ASS_SPLIT_TEXT | ASS_SPLIT_TEXT2); ++ ++ if (callbacks->text) ++ callbacks->text(priv, text, text_len); + text = NULL; + } ++ + if (sscanf(buf, "\\%1[nN]", new_line) == 1) { + if (callbacks->new_line) + callbacks->new_line(priv, new_line[0] == 'N'); ++ ass_write_filtered_line(outbuffer, buf, 3, keep_flags, ASS_SPLIT_ANY); + buf += 2; + } else if (!strncmp(buf, "{\\", 2)) { ++ ass_write_filtered_line(outbuffer, buf, 2, keep_flags, ASS_SPLIT_ANY); + buf++; while (*buf == '\\') { - char style[2], c[2], sep[2], c_num[2] = "0", tmp[128] = {0}; +- char style[2], c[2], sep[2], c_num[2] = "0", tmp[128] = {0}; ++ char style[4], c[2], axis[3], sep[3], c_num[2] = "0", tmp[128] = {0}; unsigned int color = 0xFFFFFFFF; - int len, size = -1, an = -1, alpha = -1; - int x1, y1, x2, y2, t1 = -1, t2 = -1; + int len, size = -1, an = -1, alpha = -1, scale = 0; -+ int x1, y1, x2, y2, t1 = -1, t2 = -1, accel = 1; ++ float f1 = 1; ++ int x1, y1, x2, y2, x3, t1 = -1, t2 = -1, t3 = -1, t4 = -1, accel = 1; if (sscanf(buf, "\\%1[bisu]%1[01\\}]%n", style, c, &len) > 1) { int close = c[0] == '0' ? 1 : c[0] == '1' ? 0 : -1; len += close != -1; -@@ libavutil/ass_split.c: int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, ++ switch (c[0]) { ++ case 'b': ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_BOLD); ++ break; ++ case 'u': ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_UNDERLINE); ++ break; ++ case 'i': ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_ITALIC); ++ break; ++ case 'a': ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_STRIKEOUT); ++ break; ++ } + if (callbacks->style) + callbacks->style(priv, style[0], close); + } else if (sscanf(buf, "\\c%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\c&H%X&%1[\\}]%n", &color, sep, &len) > 1 || + sscanf(buf, "\\%1[1234]c%1[\\}]%n", c_num, sep, &len) > 1 || + sscanf(buf, "\\%1[1234]c&H%X&%1[\\}]%n", c_num, &color, sep, &len) > 2) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_COLOR); + if (callbacks->color) + callbacks->color(priv, color, c_num[0] - '0'); + } else if (sscanf(buf, "\\alpha%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\alpha&H%2X&%1[\\}]%n", &alpha, sep, &len) > 1 || + sscanf(buf, "\\%1[1234]a%1[\\}]%n", c_num, sep, &len) > 1 || + sscanf(buf, "\\%1[1234]a&H%2X&%1[\\}]%n", c_num, &alpha, sep, &len) > 2) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_ALPHA); + if (callbacks->alpha) + callbacks->alpha(priv, alpha, c_num[0] - '0'); + } else if (sscanf(buf, "\\fn%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\fn%127[^\\}]%1[\\}]%n", tmp, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_NAME); + if (callbacks->font_name) + callbacks->font_name(priv, tmp[0] ? tmp : NULL); + } else if (sscanf(buf, "\\fs%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\fs%u%1[\\}]%n", &size, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_SIZE); + if (callbacks->font_size) + callbacks->font_size(priv, size); ++ } else if (sscanf(buf, "\\fscx%1[\\}]%n", sep, &len) > 0 || ++ sscanf(buf, "\\fscx%f%1[\\}]%n", &f1, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_SCALE); ++ } else if (sscanf(buf, "\\fscy%1[\\}]%n", sep, &len) > 0 || ++ sscanf(buf, "\\fscy%f%1[\\}]%n", &f1, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_SCALE); ++ } else if (sscanf(buf, "\\fsp%1[\\}]%n", sep, &len) > 0 || ++ sscanf(buf, "\\fsp%u%1[\\}]%n", &size, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_SPACING); ++ } else if (sscanf(buf, "\\fe%1[\\}]%n", sep, &len) > 0 || ++ sscanf(buf, "\\fe%u%1[\\}]%n", &size, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FONT_CHARSET); ++ } else if (sscanf(buf, "\\bord%1[\\}]%n", sep, &len) > 0 || ++ sscanf(buf, "\\bord%u%1[\\}]%n", &size, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_BORDER); ++ } else if (sscanf(buf, "\\shad%1[\\}]%n", sep, &len) > 0 || ++ sscanf(buf, "\\shad%u%1[\\}]%n", &size, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_SHADOW); ++ } else if (sscanf(buf, "\\fr%1[\\}]%n", sep, &len) > 0 || ++ sscanf(buf, "\\fr%u%1[\\}]%n", &x1, sep, &len) > 1 || ++ sscanf(buf, "\\fr%1[xyz]%1[\\}]%n", axis, sep, &len) > 1 || ++ sscanf(buf, "\\fr%1[xyz]%u%1[\\}]%n", axis, &size, sep, &len) > 2) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_ROTATE); ++ } else if (sscanf(buf, "\\blur%1[\\}]%n", sep, &len) > 0 || ++ sscanf(buf, "\\blur%u%1[\\}]%n", &size, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_BLUR); ++ } else if (sscanf(buf, "\\be%1[\\}]%n", sep, &len) > 0 || ++ sscanf(buf, "\\be%u%1[\\}]%n", &size, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_BLUR); ++ } else if (sscanf(buf, "\\q%1[\\}]%n", sep, &len) > 0 || ++ sscanf(buf, "\\q%u%1[\\}]%n", &size, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_WRAP); + } else if (sscanf(buf, "\\a%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\a%2u%1[\\}]%n", &an, sep, &len) > 1 || + sscanf(buf, "\\an%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\an%1u%1[\\}]%n", &an, sep, &len) > 1) { + if (an != -1 && buf[2] != 'n') + an = (an&3) + (an&4 ? 6 : an&8 ? 3 : 0); ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_TEXT_ALIGNMENT); + if (callbacks->alignment) + callbacks->alignment(priv, an); + } else if (sscanf(buf, "\\r%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\r%127[^\\}]%1[\\}]%n", tmp, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_CANCELLING); + if (callbacks->cancel_overrides) + callbacks->cancel_overrides(priv, tmp); + } else if (sscanf(buf, "\\move(%d,%d,%d,%d)%1[\\}]%n", &x1, &y1, &x2, &y2, sep, &len) > 4 || + sscanf(buf, "\\move(%d,%d,%d,%d,%d,%d)%1[\\}]%n", &x1, &y1, &x2, &y2, &t1, &t2, sep, &len) > 6) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_MOVE); + if (callbacks->move) + callbacks->move(priv, x1, y1, x2, y2, t1, t2); + } else if (sscanf(buf, "\\pos(%d,%d)%1[\\}]%n", &x1, &y1, sep, &len) > 2) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_POS); + if (callbacks->move) + callbacks->move(priv, x1, y1, x1, y1, -1, -1); } else if (sscanf(buf, "\\org(%d,%d)%1[\\}]%n", &x1, &y1, sep, &len) > 2) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_ORIGIN); if (callbacks->origin) callbacks->origin(priv, x1, y1); -+ } else if (sscanf(buf, "\\t(%d,%d,%1[\\}]%n", &t1, &t2, sep, &len) > 2 || ++ } else if (sscanf(buf, "\\t(%1[\\}]%n", sep, &len) > 0 || ++ sscanf(buf, "\\t(%d,%d,%1[\\}]%n", &t1, &t2, sep, &len) > 2 || + sscanf(buf, "\\t(%d,%d,%d,%1[\\}]%n", &t1, &t2, &accel, sep, &len) > 3) { ++ ++ len = strcspn(buf, ")") + 2; ++ ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_ANIMATE); + if (callbacks->animate) + callbacks->animate(priv, t1, t2, accel, tmp); ++ } else if (sscanf(buf, "\\fade(%d,%d,%d,%d,%d,%d,%d)%1[\\}]%n", &x1, &x2, &x3, &t1, &t2, &t3, &t4, sep, &len) > 7) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FADE); ++ } else if (sscanf(buf, "\\fad(%d,%d)%1[\\}]%n", &t1, &t2, sep, &len) > 2) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_FADE); ++ } else if (sscanf(buf, "\\clip(%d,%d,%d,%d)%1[\\}]%n", &x1, &y1, &x2, &y2, sep, &len) > 4) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_CLIP); + } else if (sscanf(buf, "\\p%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\p%u%1[\\}]%n", &scale, sep, &len) > 1) { ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_DRAW); + if (callbacks->drawing_mode) + callbacks->drawing_mode(priv, scale); } else { - len = strcspn(buf+1, "\\}") + 2; /* skip unknown code */ - } +- len = strcspn(buf+1, "\\}") + 2; /* skip unknown code */ +- } ++ len = strcspn(buf+1, "\\}") + 2; /* unknown code */ ++ ass_write_filtered_line(outbuffer, buf, len, keep_flags, ASS_SPLIT_UNKNOWN); ++ } + buf += len - 1; + } + if (*buf++ != '}') + return AVERROR_INVALIDDATA; +- } else { ++ ++ ass_write_filtered_line(outbuffer, "}", 2, keep_flags, ASS_SPLIT_ANY); ++ } else { + if (!text) { + text = buf; + text_len = 1; @@ libavutil/ass_split.c: int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, - return 0; + buf++; + } + } ++ if (text) ++ ass_write_filtered_line(outbuffer, text, text_len + 1, keep_flags, ASS_SPLIT_TEXT | ASS_SPLIT_TEXT2); + if (text && callbacks->text) + callbacks->text(priv, text, text_len); + if (callbacks->end) + callbacks->end(priv); +- return 0; ++ ++ if (outbuffer) ++ ret = ass_remove_empty_braces(outbuffer); ++ ++ return ret; ++} ++ ++ ++int avpriv_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, ++ const char *buf) ++{ ++ return avpriv_ass_filter_override_codes(callbacks, priv, buf, NULL, 0); } -ASSStyle *ff_ass_style_get(ASSSplitContext *ctx, const char *style) @@ libavutil/ass_split.c: int ff_ass_split_override_codes(const ASSCodesCallbacks * ASS *ass = &ctx->ass; int i; - ## libavcodec/ass_split.h => libavutil/ass_split_internal.h ## + ## libavutil/ass_split_internal.h (new) ## @@ - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - */ - --#ifndef AVCODEC_ASS_SPLIT_H --#define AVCODEC_ASS_SPLIT_H ++/* ++ * SSA/ASS spliting functions ++ * Copyright (c) 2010 Aurelien Jacobs <aurel@gnuage.org> ++ * ++ * This file is part of FFmpeg. ++ * ++ * FFmpeg is free software; you can redistribute it and/or ++ * modify it under the terms of the GNU Lesser General Public ++ * License as published by the Free Software Foundation; either ++ * version 2.1 of the License, or (at your option) any later version. ++ * ++ * FFmpeg is distributed in the hope that it will be useful, ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ++ * Lesser General Public License for more details. ++ * ++ * You should have received a copy of the GNU Lesser General Public ++ * License along with FFmpeg; if not, write to the Free Software ++ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ++ */ ++ +#ifndef AVUTIL_ASS_SPLIT_INTERNAL_H +#define AVUTIL_ASS_SPLIT_INTERNAL_H - - /** - * fields extracted from the [Script Info] section -@@ libavutil/ass_split_internal.h: typedef struct { - char *effect; - char *text; /**< actual text which will be displayed as a subtitle, - can include style override control codes (see -- ff_ass_split_override_codes()) */ -+ avpriv_ass_split_override_codes()) */ - } ASSDialog; - - /** -@@ libavutil/ass_split_internal.h: typedef struct ASSSplitContext ASSSplitContext; - * @param buf String containing the ASS formatted data. - * @return Newly allocated struct containing split data. - */ --ASSSplitContext *ff_ass_split(const char *buf); ++ ++#include "bprint.h" ++ ++enum ASSSplitComponents ++{ ++ ASS_SPLIT_ANY = 0, ++ ASS_SPLIT_TEXT = (1 << 0), ++ ASS_SPLIT_TEXT2 = (1 << 1), // Same semantics as ASS_SPLIT_TEXT. To work around help output default display. ++ ASS_SPLIT_COLOR = (1 << 2), ++ ASS_SPLIT_ALPHA = (1 << 3), ++ ASS_SPLIT_FONT_NAME = (1 << 4), ++ ASS_SPLIT_FONT_SIZE = (1 << 5), ++ ASS_SPLIT_FONT_SCALE = (1 << 6), ++ ASS_SPLIT_FONT_SPACING = (1 << 7), ++ ASS_SPLIT_FONT_CHARSET = (1 << 8), ++ ASS_SPLIT_FONT_BOLD = (1 << 9), ++ ASS_SPLIT_FONT_ITALIC = (1 << 10), ++ ASS_SPLIT_FONT_UNDERLINE = (1 << 11), ++ ASS_SPLIT_FONT_STRIKEOUT = (1 << 12), ++ ASS_SPLIT_TEXT_BORDER = (1 << 13), ++ ASS_SPLIT_TEXT_SHADOW = (1 << 14), ++ ASS_SPLIT_TEXT_ROTATE = (1 << 15), ++ ASS_SPLIT_TEXT_BLUR = (1 << 16), ++ ASS_SPLIT_TEXT_WRAP = (1 << 17), ++ ASS_SPLIT_TEXT_ALIGNMENT = (1 << 18), ++ ASS_SPLIT_CANCELLING = (1 << 19), ++ ASS_SPLIT_MOVE = (1 << 20), ++ ASS_SPLIT_POS = (1 << 21), ++ ASS_SPLIT_ORIGIN = (1 << 22), ++ ASS_SPLIT_DRAW = (1 << 23), ++ ASS_SPLIT_ANIMATE = (1 << 24), ++ ASS_SPLIT_FADE = (1 << 25), ++ ASS_SPLIT_CLIP = (1 << 26), ++ ASS_SPLIT_UNKNOWN = (1 << 27), ++ ++ ASS_SPLIT_BASIC = ASS_SPLIT_TEXT2 | ASS_SPLIT_COLOR | ASS_SPLIT_ALPHA | ASS_SPLIT_FONT_NAME | ASS_SPLIT_FONT_SIZE | ASS_SPLIT_FONT_SCALE | ASS_SPLIT_FONT_SPACING | ASS_SPLIT_FONT_CHARSET | ASS_SPLIT_FONT_BOLD | ASS_SPLIT_FONT_ITALIC | ASS_SPLIT_FONT_UNDERLINE | ASS_SPLIT_FONT_STRIKEOUT | ASS_SPLIT_TEXT_BORDER | ASS_SPLIT_TEXT_SHADOW | ASS_SPLIT_TEXT_WRAP | ASS_SPLIT_TEXT_ALIGNMENT | ASS_SPLIT_POS | ASS_SPLIT_CANCELLING, ++ ASS_SPLIT_ALL_KNOWN = ASS_SPLIT_TEXT2 | ASS_SPLIT_COLOR | ASS_SPLIT_ALPHA | ASS_SPLIT_FONT_NAME | ASS_SPLIT_FONT_SIZE | ASS_SPLIT_FONT_SCALE | ASS_SPLIT_FONT_SPACING | ASS_SPLIT_FONT_CHARSET | ASS_SPLIT_FONT_BOLD | ASS_SPLIT_FONT_ITALIC | ASS_SPLIT_FONT_UNDERLINE | ASS_SPLIT_FONT_STRIKEOUT | ASS_SPLIT_TEXT_BORDER | ASS_SPLIT_TEXT_SHADOW | ASS_SPLIT_TEXT_ROTATE | ASS_SPLIT_TEXT_BLUR | ASS_SPLIT_TEXT_WRAP | ASS_SPLIT_TEXT_ALIGNMENT | ASS_SPLIT_CANCELLING | ASS_SPLIT_POS | ASS_SPLIT_MOVE | ASS_SPLIT_ORIGIN | ASS_SPLIT_DRAW | ASS_SPLIT_ANIMATE | ASS_SPLIT_FADE | ASS_SPLIT_CLIP, ++}; ++ ++ /** ++ * fields extracted from the [Script Info] section ++ */ ++ typedef struct { ++ char *script_type; /**< SSA script format version (eg. v4.00) */ ++ char *collisions; /**< how subtitles are moved to prevent collisions */ ++ int play_res_x; /**< video width that ASS coords are referring to */ ++ int play_res_y; /**< video height that ASS coords are referring to */ ++ float timer; /**< time multiplier to apply to SSA clock (in %) */ ++} ASSScriptInfo; ++ ++/** ++ * fields extracted from the [V4(+) Styles] section ++ */ ++typedef struct { ++ char *name; /**< name of the tyle (case sensitive) */ ++ char *font_name; /**< font face (case sensitive) */ ++ int font_size; /**< font height */ ++ int primary_color; /**< color that a subtitle will normally appear in */ ++ int secondary_color; ++ int outline_color; /**< color for outline in ASS, called tertiary in SSA */ ++ int back_color; /**< color of the subtitle outline or shadow */ ++ int bold; /**< whether text is bold (1) or not (0) */ ++ int italic; /**< whether text is italic (1) or not (0) */ ++ int underline; /**< whether text is underlined (1) or not (0) */ ++ int strikeout; ++ float scalex; ++ float scaley; ++ float spacing; ++ float angle; ++ int border_style; ++ float outline; ++ float shadow; ++ int alignment; /**< position of the text (left, center, top...), ++ defined after the layout of the numpad ++ (1-3 sub, 4-6 mid, 7-9 top) */ ++ int margin_l; ++ int margin_r; ++ int margin_v; ++ int alpha_level; ++ int encoding; ++} ASSStyle; ++ ++/** ++ * fields extracted from the [Events] section ++ */ ++typedef struct { ++ int readorder; ++ int layer; /**< higher numbered layers are drawn over lower numbered */ ++ int start; /**< start time of the dialog in centiseconds */ ++ int end; /**< end time of the dialog in centiseconds */ ++ char *style; /**< name of the ASSStyle to use with this dialog */ ++ char *name; ++ int margin_l; ++ int margin_r; ++ int margin_v; ++ char *effect; ++ char *text; /**< actual text which will be displayed as a subtitle, ++ can include style override control codes (see ++ avpriv_ass_split_override_codes()) */ ++} ASSDialog; ++ ++/** ++ * structure containing the whole split ASS data ++ */ ++typedef struct { ++ ASSScriptInfo script_info; /**< general information about the SSA script*/ ++ ASSStyle *styles; /**< array of split out styles */ ++ int styles_count; /**< number of ASSStyle in the styles array */ ++ ASSDialog *dialogs; /**< array of split out dialogs */ ++ int dialogs_count; /**< number of ASSDialog in the dialogs array*/ ++} ASS; ++ ++/** ++ * This struct can be casted to ASS to access to the split data. ++ */ ++typedef struct ASSSplitContext ASSSplitContext; ++ ++/** ++ * Split a full ASS file or a ASS header from a string buffer and store ++ * the split structure in a newly allocated context. ++ * ++ * @param buf String containing the ASS formatted data. ++ * @return Newly allocated struct containing split data. ++ */ +ASSSplitContext *avpriv_ass_split(const char *buf); - - /** -- * Free a dialogue obtained from ff_ass_split_dialog(). ++ ++/** + * Free a dialogue obtained from avpriv_ass_split_dialog(). - */ --void ff_ass_free_dialog(ASSDialog **dialogp); ++ */ +void avpriv_ass_free_dialog(ASSDialog **dialogp); - - /** - * Split one ASS Dialogue line from a string buffer. -@@ libavutil/ass_split_internal.h: void ff_ass_free_dialog(ASSDialog **dialogp); - * @param buf String containing the ASS "Dialogue" line. - * @return Pointer to the split ASSDialog. Must be freed with ff_ass_free_dialog() - */ --ASSDialog *ff_ass_split_dialog(ASSSplitContext *ctx, const char *buf); ++ ++/** ++ * Split one ASS Dialogue line from a string buffer. ++ * ++ * @param ctx Context previously initialized by ff_ass_split(). ++ * @param buf String containing the ASS "Dialogue" line. ++ * @return Pointer to the split ASSDialog. Must be freed with ++ * ff_ass_free_dialog() ++ */ +ASSDialog *avpriv_ass_split_dialog(ASSSplitContext *ctx, const char *buf); - - /** - * Free all the memory allocated for an ASSSplitContext. - * - * @param ctx Context previously initialized by ff_ass_split(). - */ --void ff_ass_split_free(ASSSplitContext *ctx); ++ ++/** ++ * Free all the memory allocated for an ASSSplitContext. ++ * ++ * @param ctx Context previously initialized by ff_ass_split(). ++ */ +void avpriv_ass_split_free(ASSSplitContext *ctx); - - - /** -@@ libavutil/ass_split_internal.h: typedef struct { - * @{ - */ - void (*text)(void *priv, const char *text, int len); -+ void (*hard_space)(void *priv); - void (*new_line)(void *priv, int forced); - void (*style)(void *priv, char style, int close); - void (*color)(void *priv, unsigned int /* color */, unsigned int color_id); -@@ libavutil/ass_split_internal.h: typedef struct { - * @{ - */ - void (*move)(void *priv, int x1, int y1, int x2, int y2, int t1, int t2); -+ void (*animate)(void *priv, int t1, int t2, int accel, char *style); - void (*origin)(void *priv, int x, int y); -+ void (*drawing_mode)(void *priv, int scale); -+ /** @} */ + -+ /** -+ * @defgroup ass_ext ASS extensible parsing callback -+ * @{ -+ */ -+ void (*ext)(void *priv, int ext_id, const char *text, int p1, int p2); - /** @} */ - - /** -@@ libavutil/ass_split_internal.h: typedef struct { - * @param buf The ASS "Dialogue" Text field to split. - * @return >= 0 on success otherwise an error code <0 - */ --int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, -+int avpriv_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, - const char *buf); - - /** -@@ libavutil/ass_split_internal.h: int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, - * @param style name of the style to search for. - * @return the ASSStyle corresponding to style, or NULL if style can't be found - */ --ASSStyle *ff_ass_style_get(ASSSplitContext *ctx, const char *style); ++/** ++ * Set of callback functions corresponding to each override codes that can ++ * be encountered in a "Dialogue" Text field. ++ */ ++typedef struct { ++ /** ++ * @defgroup ass_styles ASS styles ++ * @{ ++ */ ++ void (*text)(void *priv, const char *text, int len); ++ void (*hard_space)(void *priv); ++ void (*new_line)(void *priv, int forced); ++ void (*style)(void *priv, char style, int close); ++ void (*color)(void *priv, unsigned int /* color */, unsigned int color_id); ++ void (*alpha)(void *priv, int alpha, int alpha_id); ++ void (*font_name)(void *priv, const char *name); ++ void (*font_size)(void *priv, int size); ++ void (*alignment)(void *priv, int alignment); ++ void (*cancel_overrides)(void *priv, const char *style); ++ /** @} */ ++ ++ /** ++ * @defgroup ass_functions ASS functions ++ * @{ ++ */ ++ void (*move)(void *priv, int x1, int y1, int x2, int y2, int t1, int t2); ++ void (*animate)(void *priv, int t1, int t2, int accel, char *style); ++ void (*origin)(void *priv, int x, int y); ++ void (*drawing_mode)(void *priv, int scale); ++ /** @} */ ++ ++ /** ++ * @defgroup ass_ext ASS extensible parsing callback ++ * @{ ++ */ ++ void (*ext)(void *priv, int ext_id, const char *text, int p1, int p2); ++ /** @} */ ++ ++ /** ++ * @defgroup ass_end end of Dialogue Event ++ * @{ ++ */ ++ void (*end)(void *priv); ++ /** @} */ ++} ASSCodesCallbacks; ++ ++/** ++ * Split override codes out of a ASS "Dialogue" Text field. ++ * ++ * @param callbacks Set of callback functions called for each override code ++ * encountered. ++ * @param priv Opaque pointer passed to the callback functions. ++ * @param buf The ASS "Dialogue" Text field to split. ++ * @param outbuffer The output buffer. ++ * @param keep_flags Flags for filtering ass codes. ++ * @return >= 0 on success otherwise an error code <0 ++ */ ++int avpriv_ass_filter_override_codes(const ASSCodesCallbacks *callbacks, ++ void *priv, const char *buf, ++ AVBPrint *outbuffer, enum ASSSplitComponents keep_flags); ++ ++/** ++ * Split override codes out of a ASS "Dialogue" Text field. ++ * ++ * @param callbacks Set of callback functions called for each override code ++ * encountered. ++ * @param priv Opaque pointer passed to the callback functions. ++ * @param buf The ASS "Dialogue" Text field to split. ++ * @return >= 0 on success otherwise an error code <0 ++ */ ++int avpriv_ass_split_override_codes(const ASSCodesCallbacks *callbacks, ++ void *priv, const char *buf); ++ ++/** ++ * Find an ASSStyle structure by its name. ++ * ++ * @param ctx Context previously initialized by ff_ass_split(). ++ * @param style name of the style to search for. ++ * @return the ASSStyle corresponding to style, or NULL if style can't be found ++ */ +ASSStyle *avpriv_ass_style_get(ASSSplitContext *ctx, const char *style); - --#endif /* AVCODEC_ASS_SPLIT_H */ ++ +#endif /* AVUTIL_ASS_SPLIT_INTERNAL_H */ 7: 09d8cf7880 = 7: 98f12ad7e9 avcodec/subtitles: Replace deprecated enum values 8: 897299bf7f = 8: 12c8a308d3 fftools/play,probe: Adjust for subtitle changes 9: ca580c6d21 = 9: 2e55dbe180 avfilter/subtitles: Add subtitles.c for subtitle frame allocation 10: 0781e974a2 = 10: c931041103 avfilter/avfilter: Handle subtitle frames 11: d9d9f42558 = 11: 36cab55ff2 avfilter/avfilter: Fix hardcoded input index 12: af69a4b321 = 12: f41070479c avfilter/sbuffer: Add sbuffersrc and sbuffersink filters 13: f7e5b590a2 ! 13: 9bfaba4ace avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters @@ libavfilter/allfilters.c: extern const AVFilter ff_vf_overlay_qsv; extern const AVFilter ff_vf_owdenoise; extern const AVFilter ff_vf_pad; extern const AVFilter ff_vf_pad_opencl; -@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showvolume; +@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showspectrumpic; + extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; - extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_svf_graphicsub2video; + extern const AVFilter ff_vaf_spectrumsynth; /* multimedia sources */ - extern const AVFilter ff_avsrc_avsynctest; ## libavfilter/vf_overlaygraphicsubs.c (new) ## @@ 14: 4c8092357f ! 14: 918fd9aaf5 avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters @@ libavfilter/allfilters.c: extern const AVFilter ff_vf_overlay_vaapi; extern const AVFilter ff_vf_owdenoise; extern const AVFilter ff_vf_pad; extern const AVFilter ff_vf_pad_opencl; -@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showwaves; +@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showvolume; + extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; - extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_svf_graphicsub2video; +extern const AVFilter ff_svf_textsub2video; + extern const AVFilter ff_vaf_spectrumsynth; /* multimedia sources */ - extern const AVFilter ff_avsrc_avsynctest; ## libavfilter/vf_overlaytextsubs.c (new) ## @@ 15: 8fdbdf7c5f ! 15: a361ad35c5 avfilter/textmod: Add textmod, censor and show_speaker filters @@ libavfilter/Makefile: OBJS-$(CONFIG_YUVTESTSRC_FILTER) += vsrc_tests OBJS-$(CONFIG_ADRAWGRAPH_FILTER) += f_drawgraph.o ## libavfilter/allfilters.c ## -@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showvolume; - extern const AVFilter ff_avf_showwaves; - extern const AVFilter ff_avf_showwavespic; - extern const AVFilter ff_vaf_spectrumsynth; +@@ libavfilter/allfilters.c: extern const AVFilter ff_avsrc_avsynctest; + extern const AVFilter ff_avsrc_amovie; + extern const AVFilter ff_avsrc_movie; + ++/* subtitle filters */ +extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_showspeaker; +extern const AVFilter ff_sf_textmod; - extern const AVFilter ff_svf_graphicsub2video; - extern const AVFilter ff_svf_textsub2video; - ++ + /* those filters are part of public or internal API, + * they are formatted to not be found by the grep + * as they are manually added again (due to their 'names' ## libavfilter/sf_textmod.c (new) ## @@ @@ libavfilter/sf_textmod.c (new) + + av_bprint_finalize(&pbuf, &text); + -+ result = avpriv_ass_get_dialog_ex(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->margin_l, dialog->margin_r, dialog->margin_v, text); ++ result = avpriv_ass_get_dialog_ex(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->margin_l, dialog->margin_r, dialog->margin_v, dialog->effect, text); + + av_free(text); + avpriv_ass_free_dialog(&dialog); @@ libavfilter/sf_textmod.c (new) + if (!text) + return NULL; + -+ result = avpriv_ass_get_dialog_ex(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->margin_l, dialog->margin_r, dialog->margin_v, text); ++ result = avpriv_ass_get_dialog_ex(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->margin_l, dialog->margin_r, dialog->margin_v, dialog->effect, text); + + av_free(text); + avpriv_ass_free_dialog(&dialog); 16: d44b22f15b ! 16: bca90ebc3e avfilter/stripstyles: Add stripstyles filter @@ doc/filters.texi: ffmpeg -i "http://streams.videolan.org/samples/sub/SSA/subtitl ## libavfilter/Makefile ## @@ libavfilter/Makefile: OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o + # subtitle filters OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o - OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o +OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o + OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o # multimedia filters - OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o ## libavfilter/allfilters.c ## -@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showwavespic; - extern const AVFilter ff_vaf_spectrumsynth; +@@ libavfilter/allfilters.c: extern const AVFilter ff_avsrc_movie; + /* subtitle filters */ extern const AVFilter ff_sf_censor; extern const AVFilter ff_sf_showspeaker; +extern const AVFilter ff_sf_stripstyles; extern const AVFilter ff_sf_textmod; - extern const AVFilter ff_svf_graphicsub2video; - extern const AVFilter ff_svf_textsub2video; + + /* those filters are part of public or internal API, ## libavfilter/sf_stripstyles.c (new) ## @@ @@ libavfilter/sf_stripstyles.c (new) + +#include "libavutil/opt.h" +#include "internal.h" ++#include "libavutil/ass_internal.h" +#include "libavutil/ass_split_internal.h" +#include "libavutil/bprint.h" + @@ libavfilter/sf_stripstyles.c (new) + const AVClass *class; + enum AVSubtitleType format; + int remove_animated; ++ enum ASSSplitComponents keep_flags; + int select_layer; +} StripStylesContext; + @@ libavfilter/sf_stripstyles.c (new) + AVBPrint buffer; + int drawing_scale; + int is_animated; ++ int plain_text_length; +} DialogContext; + +static void dialog_text_cb(void *priv, const char *text, int len) @@ libavfilter/sf_stripstyles.c (new) + av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text); + + if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated)) -+ av_bprint_append_data(&s->buffer, text, len); ++ s->plain_text_length += len; ++ ////av_bprint_append_data(&s->buffer, text, len); +} + +static void dialog_new_line_cb(void *priv, int forced) +{ + DialogContext *s = priv; + if (!s->drawing_scale && !s->is_animated) -+ av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2); ++ s->plain_text_length += 2; ++ ////av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2); +} + +static void dialog_drawing_mode_cb(void *priv, int scale) @@ libavfilter/sf_stripstyles.c (new) + .move = dialog_move_cb, +}; + -+static char *ass_get_line(int readorder, int layer, const char *style, -+ const char *speaker, const char *effect, const char *text) -+{ -+ return av_asprintf("%d,%d,%s,%s,0,0,0,%s,%s", -+ readorder, layer, style ? style : "Default", -+ speaker ? speaker : "", effect, text); -+} -+ +static char *process_dialog(StripStylesContext *s, const char *ass_line) +{ + DialogContext dlg_ctx = { .ss_ctx = s }; @@ libavfilter/sf_stripstyles.c (new) + + dlg_ctx.ss_ctx = s; + -+ av_bprint_init(&dlg_ctx.buffer, 0, AV_BPRINT_SIZE_UNLIMITED); ++ av_bprint_init(&dlg_ctx.buffer, 512, AV_BPRINT_SIZE_UNLIMITED); + -+ avpriv_ass_split_override_codes(&dialog_callbacks, &dlg_ctx, dialog->text); ++ avpriv_ass_filter_override_codes(&dialog_callbacks, &dlg_ctx, dialog->text, &dlg_ctx.buffer, s->keep_flags); + -+ if (av_bprint_is_complete(&dlg_ctx.buffer) -+ && dlg_ctx.buffer.len > 0) -+ result = ass_get_line(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->effect, dlg_ctx.buffer.str); ++ if (av_bprint_is_complete(&dlg_ctx.buffer) && dlg_ctx.buffer.len > 0 && dlg_ctx.plain_text_length > 0) ++ result = avpriv_ass_get_dialog_ex(dialog->readorder, dialog->layer, dialog->style, dialog->name, dialog->margin_l, dialog->margin_r, dialog->margin_v, dialog->effect, dlg_ctx.buffer.str); + + av_bprint_finalize(&dlg_ctx.buffer, NULL); + avpriv_ass_free_dialog(&dialog); @@ libavfilter/sf_stripstyles.c (new) + area->ass = process_dialog(s, area->ass); + + if (area->ass) { -+ av_log(inlink->dst, AV_LOG_INFO, "original: %d %s\n", i, tmp); -+ av_log(inlink->dst, AV_LOG_INFO, "stripped: %d %s\n", i, area->ass); ++ av_log(inlink->dst, AV_LOG_DEBUG, "original: %d %s\n", i, tmp); ++ av_log(inlink->dst, AV_LOG_DEBUG, "stripped: %d %s\n", i, area->ass); + } + else + area->ass = NULL; @@ libavfilter/sf_stripstyles.c (new) +#define FLAGS (AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_FILTERING_PARAM) + +static const AVOption stripstyles_options[] = { ++ { "keep_flags", "flags to control which override codes to keep", OFFSET(keep_flags), AV_OPT_TYPE_FLAGS, { .i64 = ASS_SPLIT_TEXT }, .flags = FLAGS, .unit = "keepflags" }, ++ { "basic", "keep static style tags only", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_BASIC }, .flags=FLAGS, .unit = "keepflags" }, ++ { "all_known", "keep all known tags", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ALL_KNOWN }, .flags=FLAGS, .unit = "keepflags" }, ++ { "text", "keep text content", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT }, .flags=FLAGS, .unit = "keepflags" }, ++ { "color", "keep color tags (\\c, \\<n>c)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_COLOR }, .flags=FLAGS, .unit = "keepflags" }, ++ { "alpha", "keep color alpha tags (\\alpha, \\<n>a)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ALPHA }, .flags=FLAGS, .unit = "keepflags" }, ++ { "font_name", "keep font name tags (\\fn)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_NAME }, .flags=FLAGS, .unit = "keepflags" }, ++ { "font_size", "keep font size tags (\\fs)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_SIZE }, .flags=FLAGS, .unit = "keepflags" }, ++ { "font_scale", "keep font scale tags (\\fscx, \\fscy)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_SCALE }, .flags=FLAGS, .unit = "keepflags" }, ++ { "font_spacing", "keep font spacing tags (\\fsp)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_SPACING }, .flags=FLAGS, .unit = "keepflags" }, ++ { "font_charset", "keep font charset tags (\\fe)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_CHARSET }, .flags=FLAGS, .unit = "keepflags" }, ++ { "font_bold", "keep font bold tags (\\b)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_BOLD }, .flags=FLAGS, .unit = "keepflags" }, ++ { "font_italic", "keep font italic tags (\\i)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_ITALIC }, .flags=FLAGS, .unit = "keepflags" }, ++ { "font_underline", "keep font underline tags (\\u)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_UNDERLINE }, .flags=FLAGS, .unit = "keepflags" }, ++ { "font_strikeout", "keep font strikeout tags (\\s)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FONT_STRIKEOUT }, .flags=FLAGS, .unit = "keepflags" }, ++ { "text_border", "keep text border tags (\\bord)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_BORDER }, .flags=FLAGS, .unit = "keepflags" }, ++ { "text_shadow", "keep text shadow tags (\\shad)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_SHADOW }, .flags=FLAGS, .unit = "keepflags" }, ++ { "text_rotate", "keep text rotate tags (\\fr)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_ROTATE }, .flags=FLAGS, .unit = "keepflags" }, ++ { "text_blur", "keep text blur tags (\\blur, \\be)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_BLUR }, .flags=FLAGS, .unit = "keepflags" }, ++ { "text_wrap", "keep text wrap tags (\\q)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_WRAP }, .flags=FLAGS, .unit = "keepflags" }, ++ { "text_align", "keep text align tags (\\a, \\an)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_TEXT_ALIGNMENT }, .flags=FLAGS, .unit = "keepflags" }, ++ { "reset_override", "keep override reset tags (\\r)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_CANCELLING }, .flags=FLAGS, .unit = "keepflags" }, ++ { "move", "keep move tags (\\move)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_MOVE }, .flags=FLAGS, .unit = "keepflags" }, ++ { "pos", "keep position tags (\\pos)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_POS }, .flags=FLAGS, .unit = "keepflags" }, ++ { "origin", "keep origin tags (\\org)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ORIGIN }, .flags=FLAGS, .unit = "keepflags" }, ++ { "draw", "keep drawing tags (\\p)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_DRAW }, .flags=FLAGS, .unit = "keepflags" }, ++ { "animate", "keep animation tags (\\t)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_ANIMATE }, .flags=FLAGS, .unit = "keepflags" }, ++ { "fade", "keep fade tags (\\fad, \\fade)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_FADE }, .flags=FLAGS, .unit = "keepflags" }, ++ { "clip", "keep clip tags (\\clip)", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_CLIP }, .flags=FLAGS, .unit = "keepflags" }, ++ { "unknown", "keep unknown tags", .type = AV_OPT_TYPE_CONST, { .i64 = ASS_SPLIT_UNKNOWN }, .flags=FLAGS, .unit = "keepflags" }, + { "remove_animated", "remove animated text (default: yes)", OFFSET(remove_animated), AV_OPT_TYPE_BOOL, {.i64 = 1 }, 0, 1, FLAGS, 0 }, -+ { "select_layer", "process a specific ass layer only", OFFSET(remove_animated), AV_OPT_TYPE_INT, {.i64 = -1 }, -1, INT_MAX, FLAGS, 0 }, ++ { "select_layer", "process a specific ass layer only", OFFSET(select_layer), AV_OPT_TYPE_INT, {.i64 = -1 }, -1, INT_MAX, FLAGS, 0 }, + { NULL }, +}; + 17: 28d75dc982 ! 17: 6e488e495f avfilter/splitcc: Add splitcc filter for closed caption handling @@ doc/filters.texi: ffmpeg -i INPUT -filter_complex "showspeaker=format=colon:styl ## libavfilter/Makefile ## @@ libavfilter/Makefile: OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o + # subtitle filters OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o - OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o +OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o + OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o - # multimedia filters ## libavfilter/allfilters.c ## -@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showwavespic; - extern const AVFilter ff_vaf_spectrumsynth; +@@ libavfilter/allfilters.c: extern const AVFilter ff_avsrc_movie; + /* subtitle filters */ extern const AVFilter ff_sf_censor; extern const AVFilter ff_sf_showspeaker; +extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; extern const AVFilter ff_sf_textmod; - extern const AVFilter ff_svf_graphicsub2video; + ## libavfilter/sf_splitcc.c (new) ## @@ 18: 42d1d1c819 ! 18: 1057dff7da avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) @@ libavfilter/Makefile: OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o ## libavfilter/allfilters.c ## -@@ libavfilter/allfilters.c: extern const AVFilter ff_avf_showwaves; - extern const AVFilter ff_avf_showwavespic; - extern const AVFilter ff_vaf_spectrumsynth; +@@ libavfilter/allfilters.c: extern const AVFilter ff_avsrc_movie; + + /* subtitle filters */ extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; @@ libavfilter/sf_graphicsub2text.c (new) + } + } + -+ if (pointsize != cur_pointsize && s->recognize & RFLAGS_FONTSIZE) { -+ av_log(s, AV_LOG_DEBUG, "pointsize - pointsize: %d\n", pointsize); -+ in_code = print_code(&s->buffer, in_code, "\\fs%d", (int)(pointsize * font_factor)); -+ cur_pointsize = pointsize; ++ if (pointsize > 0 && pointsize != cur_pointsize && s->recognize & RFLAGS_FONTSIZE) { ++ float change_factor = (float)(FFABS(pointsize - cur_pointsize)) / FFMAX(pointsize, cur_pointsize); ++ ++ // Avoid small changes due to recognition variance ++ if (change_factor > 0.12f) { ++ av_log(s, AV_LOG_DEBUG, "pointsize - pointsize: %d\n", pointsize); ++ in_code = print_code(&s->buffer, in_code, "\\fs%d", (int)(pointsize * font_factor)); ++ cur_pointsize = pointsize; ++ } + } + + if (is_italic && !cur_is_italic && s->recognize & RFLAGS_FITALIC) @@ libavfilter/sf_graphicsub2text.c (new) + + const int layer = s->recognize ? i : 0; + char *tmp = area->ass; -+ area->ass = avpriv_ass_get_dialog_ex(s->readorder_counter++, layer, "Default", NULL, 0, 0, margin_v, tmp); ++ area->ass = avpriv_ass_get_dialog_ex(s->readorder_counter++, layer, "Default", NULL, 0, 0, margin_v, NULL, tmp); + av_free(tmp); + } + } 19: 7095e8aa26 ! 19: 4e85fb5d2f avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles @@ doc/filters.texi: Set the rendering margin in pixels. @chapter Multimedia Filters ## libavfilter/Makefile ## -@@ libavfilter/Makefile: OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o - OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o +@@ libavfilter/Makefile: OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o + OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o +OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o + OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o # multimedia filters - OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o ## libavfilter/allfilters.c ## @@ libavfilter/allfilters.c: extern const AVFilter ff_sf_graphicsub2text; @@ libavfilter/allfilters.c: extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_stripstyles; +extern const AVFilter ff_sf_subscale; extern const AVFilter ff_sf_textmod; - extern const AVFilter ff_svf_graphicsub2video; - extern const AVFilter ff_svf_textsub2video; + + /* those filters are part of public or internal API, ## libavfilter/sf_subscale.c (new) ## @@ 20: 697939451e ! 20: 88e8adb889 avfilter/subfeed: add subtitle feed filter @@ Commit message Signed-off-by: softworkz <softworkz@hotmail.com> ## libavfilter/Makefile ## -@@ libavfilter/Makefile: OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o +@@ libavfilter/Makefile: OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o + OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o OBJS-$(CONFIG_SPLITCC_FILTER) += sf_splitcc.o OBJS-$(CONFIG_STRIPSTYLES_FILTER) += sf_stripstyles.o - OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o +OBJS-$(CONFIG_SUBFEED_FILTER) += sf_subfeed.o + OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o + OBJS-$(CONFIG_TEXTMOD_FILTER) += sf_textmod.o - # multimedia filters - OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o ## libavfilter/allfilters.c ## -@@ libavfilter/allfilters.c: extern const AVFilter ff_sf_showspeaker; +@@ libavfilter/allfilters.c: extern const AVFilter ff_sf_graphicsub2text; + extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; - extern const AVFilter ff_sf_subscale; +extern const AVFilter ff_sf_subfeed; + extern const AVFilter ff_sf_subscale; extern const AVFilter ff_sf_textmod; - extern const AVFilter ff_svf_graphicsub2video; - extern const AVFilter ff_svf_textsub2video; + ## libavfilter/sf_subfeed.c (new) ## @@ @@ libavfilter/sf_subfeed.c (new) + if (pts_diff <= 0) { + av_log(ctx, AV_LOG_WARNING, "The pts_diff to the previous frame (index #%"PRId64") is <= 0: %"PRId64" ms. The previous frame duration is %"PRId64" ms.\n", + index, avtb_to_ms(pts_diff), avtb_to_ms(previous_frame->subtitle_timing.duration)); ++ ++ if (s->fix_overlap) { ++ av_log(ctx, AV_LOG_VERBOSE, "Removing previous frame\n"); ++ previous_frame = ff_framequeue_take(&s->fifo); ++ while (nb_queued_frames > 1) { ++ ff_framequeue_add(&s->fifo, previous_frame); ++ previous_frame = ff_framequeue_take(&s->fifo); ++ nb_queued_frames--; ++ } ++ } + } + } + -: ---------- > 21: a96bb5c788 avfilter/text2graphicsub: Added text2graphicsub subtitle filter -: ---------- > 22: c4922f8466 avfilter/snull,strim: Add snull and strim filters 21: 32e9af0806 = 23: 848f84d5dc avcodec/subtitles: Migrate subtitle encoders to frame-based API 22: fa0b5c2077 ! 24: 2645a1a842 fftools/ffmpeg: Introduce subtitle filtering and new frame-based subtitle encoding @@ Commit message Overlay results have slightly different CRCs due to different blending implementation + - sub-scc + The first entry is no longer in the output because it is before + the actual start time and the strim filter removes such entries + now (like for video and audio) + Signed-off-by: softworkz <softworkz@hotmail.com> ## fftools/ffmpeg.c ## @@ fftools/ffmpeg.c: static int init_output_stream(OutputStream *ost, AVFrame *fram return AVERROR_INVALIDDATA; } } +@@ fftools/ffmpeg.c: static int transcode_init(void) + for (i = 0; i < nb_output_streams; i++) { + if (!output_streams[i]->stream_copy && + (output_streams[i]->enc_ctx->codec_type == AVMEDIA_TYPE_VIDEO || +- output_streams[i]->enc_ctx->codec_type == AVMEDIA_TYPE_AUDIO)) ++ output_streams[i]->enc_ctx->codec_type == AVMEDIA_TYPE_AUDIO || ++ output_streams[i]->enc_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE)) + continue; + + ret = init_output_stream_wrapper(output_streams[i], NULL, 0); @@ fftools/ffmpeg.c: static OutputStream *choose_output(void) av_rescale_q(ost->last_mux_dts, ost->st->time_base, AV_TIME_BASE_Q); @@ fftools/ffmpeg_filter.c: static void init_input_filter(FilterGraph *fg, AVFilter continue; if (check_stream_specifier(s, s->streams[i], *p == ':' ? p + 1 : p) == 1) { st = s->streams[i]; +@@ fftools/ffmpeg_filter.c: static int insert_trim(int64_t start_time, int64_t duration, + const char *name = (type == AVMEDIA_TYPE_VIDEO) ? "trim" : "atrim"; + int ret = 0; + ++ switch (type) { ++ case AVMEDIA_TYPE_VIDEO: ++ name = "trim"; ++ break; ++ case AVMEDIA_TYPE_AUDIO: ++ name = "atrim"; ++ break; ++ case AVMEDIA_TYPE_SUBTITLE: ++ name = "strim"; ++ break; ++ default: ++ av_log(NULL, AV_LOG_ERROR, "insert_trim: Invalid media type: %d\n", type); ++ return AVERROR_INVALIDDATA; ++ } ++ + if (duration == INT64_MAX && start_time == AV_NOPTS_VALUE) + return 0; + @@ fftools/ffmpeg_filter.c: static int insert_filter(AVFilterContext **last_filter, int *pad_idx, return 0; } @@ fftools/ffmpeg_filter.c: static int insert_filter(AVFilterContext **last_filter, +static int configure_output_subtitle_filter(FilterGraph *fg, OutputFilter *ofilter, AVFilterInOut *out) +{ + OutputStream *ost = ofilter->ost; ++ OutputFile *of = output_files[ost->file_index]; + AVFilterContext *last_filter = out->filter_ctx; + int pad_idx = out->pad_idx; + int ret; @@ fftools/ffmpeg_filter.c: static int insert_filter(AVFilterContext **last_filter, + return ret; + } + -+ ////snprintf(name, sizeof(name), "trim_out_%d_%d", -+ //// ost->file_index, ost->index); -+ ////ret = insert_trim(of->start_time, of->recording_time, -+ //// &last_filter, &pad_idx, name); -+ ////if (ret < 0) -+ //// return ret; ++ snprintf(name, sizeof(name), "trim_out_%d_%d", ++ ost->file_index, ost->index); ++ ret = insert_trim(of->start_time, of->recording_time, ++ &last_filter, &pad_idx, name); ++ if (ret < 0) ++ return ret; + + ////ost->st->codecpar->codec_tag = MKTAG('a', 's', 's', 's'); + @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) + AVFilterContext *last_filter; + const AVFilter *buffer_filt = avfilter_get_by_name("sbuffer"); + InputStream *ist = ifilter->ist; ++ InputFile *f = input_files[ist->file_index]; + AVBPrint args; + char name[255]; + int ret, pad_idx = 0; + int w, h; ++ int64_t tsoffset = 0; + AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); + enum AVMediaType media_type; + @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) - if (avf->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) { - w = FFMAX(w, avf->streams[i]->codecpar->width); - h = FFMAX(h, avf->streams[i]->codecpar->height); -- } -- } -- if (!(w && h)) { -- w = FFMAX(w, 720); -- h = FFMAX(h, 576); -- } -- av_log(avf, AV_LOG_INFO, "sub2video: using %dx%d canvas\n", w, h); + w = ist->dec_ctx->width; + h = ist->dec_ctx->height; + } @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) + w = ass->script_info.play_res_x; + h = ass->script_info.play_res_y; + avpriv_ass_split_free(ass_ctx); - } -- ist->sub2video.w = ifilter->width = w; -- ist->sub2video.h = ifilter->height = h; - -- ifilter->width = ist->dec_ctx->width ? ist->dec_ctx->width : ist->sub2video.w; -- ifilter->height = ist->dec_ctx->height ? ist->dec_ctx->height : ist->sub2video.h; ++ } ++ + ist->subtitle_kickoff.w = w; + ist->subtitle_kickoff.h = h; + av_log(ifilter, AV_LOG_INFO, "subtitle input filter: decoding size %dx%d\n", ist->subtitle_kickoff.w, ist->subtitle_kickoff.h); - -- /* rectangles are AV_PIX_FMT_PAL8, but we have no guarantee that the -- palettes for all rectangles are identical or compatible */ -- ifilter->format = AV_PIX_FMT_RGB32; ++ + ifilter->width = w; + ifilter->height = h; + ist->dec_ctx->width = w; + ist->dec_ctx->height = h; - -- ist->sub2video.frame = av_frame_alloc(); -- if (!ist->sub2video.frame) -- return AVERROR(ENOMEM); -- ist->sub2video.last_pts = INT64_MIN; -- ist->sub2video.end_pts = INT64_MIN; ++ + ist->subtitle_kickoff.last_pts = INT64_MIN; + + snprintf(name, sizeof(name), "graph %d subtitle input from stream %d:%d", fg->index, @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) + if ((ret = avfilter_graph_create_filter(&ifilter->filter, buffer_filt, name, + args.str, NULL, fg->graph)) < 0) + goto fail; - -- /* sub2video structure has been (re-)initialized. -- Mark it as such so that the system will be -- initialized with the first received heartbeat. */ -- ist->sub2video.initialize = 1; ++ + par->hw_frames_ctx = ifilter->hw_frames_ctx; + par->format = ifilter->format; + par->width = ifilter->width; @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) + subscale_h = input->height; + break; + } -+ } -+ } -+ + } + } +- if (!(w && h)) { +- w = FFMAX(w, 720); +- h = FFMAX(h, 576); +- } +- av_log(avf, AV_LOG_INFO, "sub2video: using %dx%d canvas\n", w, h); +- } +- ist->sub2video.w = ifilter->width = w; +- ist->sub2video.h = ifilter->height = h; + +- ifilter->width = ist->dec_ctx->width ? ist->dec_ctx->width : ist->sub2video.w; +- ifilter->height = ist->dec_ctx->height ? ist->dec_ctx->height : ist->sub2video.h; + if (subscale_w && subscale_h) { + char subscale_params[64]; + snprintf(subscale_params, sizeof(subscale_params), "w=%d:h=%d", subscale_w, subscale_h); @@ fftools/ffmpeg_filter.c: void check_filter_outputs(void) + if (ret < 0) + return ret; + } -+ + +- /* rectangles are AV_PIX_FMT_PAL8, but we have no guarantee that the +- palettes for all rectangles are identical or compatible */ +- ifilter->format = AV_PIX_FMT_RGB32; + av_log(NULL, AV_LOG_INFO, "Auto-inserting graphicsub2video filter\n"); + ret = insert_filter(&last_filter, &pad_idx, "graphicsub2video", NULL); + if (ret < 0) + return ret; + } -+ + +- ist->sub2video.frame = av_frame_alloc(); +- if (!ist->sub2video.frame) +- return AVERROR(ENOMEM); +- ist->sub2video.last_pts = INT64_MIN; +- ist->sub2video.end_pts = INT64_MIN; ++ if (copy_ts) { ++ tsoffset = f->start_time == AV_NOPTS_VALUE ? 0 : f->start_time; ++ if (!start_at_zero && f->ctx->start_time != AV_NOPTS_VALUE) ++ tsoffset += f->ctx->start_time; ++ } ++ ret = insert_trim(((f->start_time == AV_NOPTS_VALUE) || !f->accurate_seek) ? ++ AV_NOPTS_VALUE : tsoffset, f->recording_time, ++ &last_filter, &pad_idx, name); ++ if (ret < 0) ++ return ret; + +- /* sub2video structure has been (re-)initialized. +- Mark it as such so that the system will be +- initialized with the first received heartbeat. */ +- ist->sub2video.initialize = 1; + if ((ret = avfilter_link(last_filter, 0, in->filter_ctx, in->pad_idx)) < 0) + return ret; @@ fftools/ffmpeg_filter.c: static int configure_input_video_filter(FilterGraph *fg int64_t tsoffset = 0; - AVBufferSrcParameters *par = av_buffersrc_parameters_alloc(); + AVBufferSrcParameters *par; - ++ + if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { + // Automatically insert conversion filter to retain compatibility + // with sub2video command lines + return configure_input_subtitle_filter(fg, ifilter, in); + } -+ + + par = av_buffersrc_parameters_alloc(); if (!par) return AVERROR(ENOMEM); @@ fftools/ffmpeg_opt.c: static void add_input_streams(OptionsContext *o, AVFormatC break; } case AVMEDIA_TYPE_ATTACHMENT: +@@ fftools/ffmpeg_opt.c: static char *get_ost_filters(OptionsContext *o, AVFormatContext *oc, + + if (ost->filters_script) + return read_file(ost->filters_script); +- else if (ost->filters) ++ if (ost->filters) + return av_strdup(ost->filters); + +- return av_strdup(st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO ? +- "null" : "anull"); ++ switch (st->codecpar->codec_type) { ++ case AVMEDIA_TYPE_VIDEO: return av_strdup("null"); ++ case AVMEDIA_TYPE_AUDIO: return av_strdup("anull"); ++ case AVMEDIA_TYPE_SUBTITLE: return av_strdup("snull"); ++ default: av_assert0(0); return NULL; ++ } + } + + static void check_streamcopy_filters(OptionsContext *o, AVFormatContext *oc, +@@ fftools/ffmpeg_opt.c: static OutputStream *new_subtitle_stream(OptionsContext *o, AVFormatContext *oc, + + subtitle_enc->codec_type = AVMEDIA_TYPE_SUBTITLE; + ++ MATCH_PER_STREAM_OPT(filter_scripts, str, ost->filters_script, oc, st); ++ MATCH_PER_STREAM_OPT(filters, str, ost->filters, oc, st); ++ + if (!ost->stream_copy) { + char *frame_size = NULL; + +@@ fftools/ffmpeg_opt.c: static OutputStream *new_subtitle_stream(OptionsContext *o, AVFormatContext *oc, + av_log(NULL, AV_LOG_FATAL, "Invalid frame size: %s.\n", frame_size); + exit_program(1); + } ++ ++ ost->avfilter = get_ost_filters(o, oc, ost); ++ if (!ost->avfilter) ++ exit_program(1); + } + + return ost; @@ fftools/ffmpeg_opt.c: static void init_output_filter(OutputFilter *ofilter, OptionsContext *o, switch (ofilter->type) { case AVMEDIA_TYPE_VIDEO: ost = new_video_stream(o, oc, -1); break; @@ fftools/ffmpeg_opt.c: static void init_output_filter(OutputFilter *ofilter, Opti "currently.\n"); exit_program(1); } +@@ fftools/ffmpeg_opt.c: loop_end: + ist->processing_needed = 1; + + if (ost->st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO || +- ost->st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { ++ ost->st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO || ++ ost->st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) { + err = init_simple_filtergraph(ist, ost); + if (err < 0) { + av_log(NULL, AV_LOG_ERROR, +@@ fftools/ffmpeg_opt.c: loop_end: + } else if (ost->enc->ch_layouts) { + f->ch_layouts = ost->enc->ch_layouts; + } ++ break; ++ case AVMEDIA_TYPE_SUBTITLE: ++ f->format = ost->enc_ctx->subtitle_type; ++ + break; + } + } ## tests/ref/fate/filter-overlay-dvdsub-2397 ## @@ @@ tests/ref/fate/sub-dvb +0, 31400000, 31400000, 479000, 14, 0x0959015b +0, 31879000, 31879000, 479000, 14, 0x09c9016b + ## tests/ref/fate/sub-scc ## +@@ tests/ref/fate/sub-scc: Style: Default,Monospace,16,&Hffffff,&Hffffff,&H0,&H0,0,0,0,0,100,100,0,0,3,1,0, + + [Events] + Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text +-Dialogue: 0,0:00:-2.-47,0:00:00.70,Default,,0,0,0,,{\an7}{\pos(76,228)}WE HAVE FOUND A WITCH !\N{\an7}{\pos(76,243)}MAY WE BURN HER ? + Dialogue: 0,0:00:00.69,0:00:03.29,Default,,0,0,0,,{\an7}{\pos(115,228)}[ Crowd ]\N{\an7}{\pos(115,243)}BURN HER ! BURN HER ! + Dialogue: 0,0:00:03.30,0:00:07.07,Default,,0,0,0,,{\an7}{\pos(38,197)}HOW DO YOU KNOW\N{\an7}{\pos(38,213)}SHE IS A WITCH ?\N{\an7}{\pos(153,243)}SHE LOOKS LIKE ONE ! + Dialogue: 0,0:00:07.07,0:00:09.27,Default,,0,0,0,,{\an7}{\pos(192,228)}[ Shouting\N{\an7}{\pos(192,243)}\h\hAffirmations ] + ## tests/ref/fate/sub2video ## @@ 0, 47, 47, 1, 518400, 0xde69683f 23: a66debd96e = 25: a90a6e1086 avcodec/dvbsubdec: Fix conditions for fallback to default resolution