From patchwork Mon Sep 19 09:46:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thilo Borgmann X-Patchwork-Id: 38046 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp1113037pzh; Mon, 19 Sep 2022 02:47:16 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7J92/6Xmv9/4tq6R22RBnaVHJiIgG3GjIU6V/4Tx1YW4oTn3mXICZ25T/d1x4eP+8rP13w X-Received: by 2002:a05:6402:3549:b0:454:414e:a7fd with SMTP id f9-20020a056402354900b00454414ea7fdmr1100199edd.69.1663580836197; Mon, 19 Sep 2022 02:47:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1663580836; cv=none; d=google.com; s=arc-20160816; b=gZAVN30JsuTgVR2z+TNYrReSKph3fmq/Bo+1hklUC7g0KsNCPcubQ2E7VqcJ+mAZce VvUm3xm5yBNcm52NI+nlshBhC08NaXIBpR3G+28+nE3K8QQ97koDbOICAa0yIzhxa+82 nv1iu3Hs2CyEn4I/MJZ3uct2LNaFXE/CokIW1vkWf/kgO1OnrfmC89trACWqBZ0B38fm ljOTIt4v+mezF4Sdq+A62iv7rP1huaCCtUOGNuQL74gw71K5LW0IGvvmzIJascyUm0MK umhr+0sHakqZcOU2PfoKGbLSLyQTBQug7NYzytvjYA1+UEldXqgonqanc6UOpFdv65Ik fVdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=QZRtGWU52msurbaXDJy8srJUbgxRRcyUib1F8Hb8H18=; b=zuIvT5bwH4e5eelPgfqtzH3msEbKI824N5qdQ8+J9yBatfVlXID6ZfO7DZ3APGPr/A pTJ35PCdn/IoST7s+vntTf8P309cRQgKJcGyffEnfhm5bmCjHd4kmwQSyXQ3Hfw8+MqU qZ8vf3R8v4bdpYKAC2+Iat0l6WTQVHAV0IU9GKTZvg9AP08TP6SNA6EQxqOR25B7vS+b AZ9XJH1liGAFe9ndTTcDARgBQYiJ90Eq1j4gcFTFdGtaXxXDBVCSvPC8PiYs/vCypcZD ubAGzglcLQwUeE5+BjkZOGjxNiKvyEH107z5wzlsdL2kHWXwAz+9rhNZq0Mr1WeEAV10 ZRVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@mail.de header.s=mailde202009 header.b=5dkZo6w4; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=mail.de Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dn8-20020a17090794c800b0077b45792af2si3995934ejc.464.2022.09.19.02.47.15; Mon, 19 Sep 2022 02:47:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@mail.de header.s=mailde202009 header.b=5dkZo6w4; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=mail.de Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 795D568BC84; Mon, 19 Sep 2022 12:46:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from shout02.mail.de (shout02.mail.de [62.201.172.25]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 64DF268BC13 for ; Mon, 19 Sep 2022 12:46:12 +0300 (EEST) Received: from postfix01.mail.de (postfix01.bt.mail.de [10.0.121.125]) by shout02.mail.de (Postfix) with ESMTP id F2FA9A0C0F for ; Mon, 19 Sep 2022 11:46:06 +0200 (CEST) Received: from smtp01.mail.de (smtp01.bt.mail.de [10.0.121.211]) by postfix01.mail.de (Postfix) with ESMTP id DA04B802BB for ; Mon, 19 Sep 2022 11:46:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mail.de; s=mailde202009; t=1663580766; bh=bI0c07TkkuYO4msVJI/6v+38eNAjrmjY4Vobj79oIhU=; h=From:To:Subject:Date:Message-Id:From:To:CC:Subject:Reply-To; b=5dkZo6w45vjLzTdTUcpSrGdCxv6IyS2BuRhVzs1gYO9YCOUQi/iVloU6uI5z4z8rF bJE2O1VLs6w94Usca4fmxfoCxRO04eSo7st8tJvWV7r4mxCZshtxNazw7qYBKYtEGe w1wD7QpyCUT0VM318Zu7yH9XicAaQfZsqyoxz4h3b8Cn3F9zbsqkStCok10XCSepg/ Ppa+ZDsMO/QauoAEt/0n4fpkkGPtHQV4lI2BfSIRTTP3XtpOoqeEjv5iacNTPmKwN0 cFAhfCSn0QqcuGGXrwUPCbEZvVR5o3mReTZcak5oFD4qJFyxh1N8ul8dFAdnWdlyMa tnPrES99OiNCQ== Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp01.mail.de (Postfix) with ESMTPSA id 91ADA100107 for ; Mon, 19 Sep 2022 11:46:06 +0200 (CEST) From: Thilo Borgmann To: ffmpeg-devel@ffmpeg.org Date: Mon, 19 Sep 2022 11:46:03 +0200 Message-Id: <20220919094604.4645-6-thilo.borgmann@mail.de> In-Reply-To: <20220919094604.4645-1-thilo.borgmann@mail.de> References: <20220919094604.4645-1-thilo.borgmann@mail.de> MIME-Version: 1.0 X-purgate: clean X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate-type: clean X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate: clean X-purgate-size: 12663 X-purgate-ID: 154282::1663580766-CD7F95FC-9ED4071B/0/0 Subject: [FFmpeg-devel] [PATCH v4 5/6] ffmpeg: Add display_matrix option X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: bo79zX4PHKuz From: Jan Ekström This enables overriding the rotation as well as horizontal/vertical flip state of a specific video stream on the input side. Additionally, switch the singular test that was utilizing the rotation metadata to instead override the input display rotation, thus leading to the same result. --- doc/ffmpeg.texi | 18 ++++++ fftools/ffmpeg.h | 2 + fftools/ffmpeg_filter.c | 13 ++++ fftools/ffmpeg_opt.c | 123 ++++++++++++++++++++++++++++++++++++ tests/fate/filter-video.mak | 2 +- 5 files changed, 157 insertions(+), 1 deletion(-) diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index 42440d93b4..ee86faedc8 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -912,6 +912,24 @@ If used together with @option{-vcodec copy}, it will affect the aspect ratio stored at container level, but not the aspect ratio stored in encoded frames, if it exists. +@item -display_matrix[:@var{stream_specifier}] @var{opt1=val1[,opt2=val2]...} (@emph{input,per-stream}) +Set the video display matrix according to given options. +We support scaling, flipping and rotation. The order of application to an Image is scale, flip, rotate. + +@table @option +@item rotation=@var{number} +Set the rotation using a floating point number that describes a pure +counter-clockwise rotation in degrees. +The @code{-autorotate} logic will be affected. +@item hflip=@var{[0,1]} +@item vflip=@var{[0,1]} +Set a horizontal or vertical flip. +@item hscale=@var{number} +Set a horizontal scaling by factor of the given floating-point value. +@item vscale=@var{number} +Set a vertical scaling by factor of the given floating-point value. +@end table + @item -vn (@emph{input/output}) As an input option, blocks all video streams of a file from being filtered or being automatically selected or mapped for any output. See @code{-discard} diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index ede0b2bd96..524a1d415a 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -193,6 +193,8 @@ typedef struct OptionsContext { int nb_force_fps; SpecifierOpt *frame_aspect_ratios; int nb_frame_aspect_ratios; + SpecifierOpt *display_matrixes; + int nb_display_matrixes; SpecifierOpt *rc_overrides; int nb_rc_overrides; SpecifierOpt *intra_matrices; diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 7a5308425d..8bcba0d3a2 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -779,9 +779,22 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, if (ist->autorotate && !(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) { int32_t *displaymatrix = ifilter->displaymatrix; double theta; + double hscale = 1.0f; + double vscale = 1.0f; if (!displaymatrix) displaymatrix = (int32_t *)av_stream_get_side_data(ist->st, AV_PKT_DATA_DISPLAYMATRIX, NULL); + + if (displaymatrix) { + hscale = av_display_hscale_get(displaymatrix); + vscale = av_display_vscale_get(displaymatrix); + if (hscale != 1.0f || vscale != 1.0f) { + char scale_buf[128]; + snprintf(scale_buf, sizeof(scale_buf), "%f*iw:%f*ih", hscale, vscale); + ret = insert_filter(&last_filter, &pad_idx, "scale", scale_buf); + } + } + theta = get_rotation(displaymatrix); if (fabs(theta - 90) < 1.0) { diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index be1cd673f6..ba56267460 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -20,6 +20,7 @@ #include "config.h" +#include #include #if HAVE_SYS_RESOURCE_H @@ -45,6 +46,7 @@ #include "libavutil/avutil.h" #include "libavutil/bprint.h" #include "libavutil/channel_layout.h" +#include "libavutil/display.h" #include "libavutil/getenv_utf8.h" #include "libavutil/intreadwrite.h" #include "libavutil/fifo.h" @@ -101,6 +103,7 @@ static const char *const opt_name_forced_key_frames[] = {"forced_key_fra static const char *const opt_name_fps_mode[] = {"fps_mode", NULL}; static const char *const opt_name_force_fps[] = {"force_fps", NULL}; static const char *const opt_name_frame_aspect_ratios[] = {"aspect", NULL}; +static const char *const opt_name_display_matrixes[] = {"display_matrix", NULL}; static const char *const opt_name_rc_overrides[] = {"rc_override", NULL}; static const char *const opt_name_intra_matrices[] = {"intra_matrix", NULL}; static const char *const opt_name_inter_matrices[] = {"inter_matrix", NULL}; @@ -126,6 +129,38 @@ static const char *const opt_name_time_bases[] = {"time_base", NU static const char *const opt_name_enc_time_bases[] = {"enc_time_base", NULL}; static const char *const opt_name_bits_per_raw_sample[] = {"bits_per_raw_sample", NULL}; +// XXX this should probably go into a seperate file _args.c and #included here + struct DisplayMatrix { + const AVClass *class; + double rotation; + int hflip; + int vflip; + double hscale; + double vscale; + }; +#define OFFSET(x) offsetof(struct DisplayMatrix, x) + static const AVOption display_matrix_args[] = { + { "rotation", "set rotation", OFFSET(rotation), AV_OPT_TYPE_DOUBLE, + { .dbl = DBL_MAX }, -(DBL_MAX), DBL_MAX - 1.0f, AV_OPT_FLAG_ARGUMENT}, + { "hflip", "set hflip", OFFSET(hflip), AV_OPT_TYPE_BOOL, + { .i64 = -1 }, 0, 1, AV_OPT_FLAG_ARGUMENT}, + { "vflip", "set vflip", OFFSET(vflip), AV_OPT_TYPE_BOOL, + { .i64 = -1 }, 0, 1, AV_OPT_FLAG_ARGUMENT}, + { "hscale", "set horizontal scale factor", OFFSET(hscale), AV_OPT_TYPE_DOUBLE, + { .dbl = DBL_MAX }, 0.0f, DBL_MAX - 1.0f, AV_OPT_FLAG_ARGUMENT}, + { "vscale", "set vertical scale factor", OFFSET(vscale), AV_OPT_TYPE_DOUBLE, + { .dbl = DBL_MAX }, 0.0f, DBL_MAX - 1.0f, AV_OPT_FLAG_ARGUMENT}, + { NULL }, + }; + static const AVClass class_display_matrix_args = { + .class_name = "display_matrix_args", + .item_name = av_default_item_name, + .option = display_matrix_args, + .version = LIBAVUTIL_VERSION_INT, + }; +#undef OFFSET +// XXX + #define WARN_MULTIPLE_OPT_USAGE(name, type, so, st)\ {\ char namestr[128] = "";\ @@ -824,6 +859,85 @@ static int opt_recording_timestamp(void *optctx, const char *opt, const char *ar return 0; } +static void add_display_matrix_to_stream(OptionsContext *o, + AVFormatContext *ctx, AVStream *st) +{ + int hflip_set = 0, vflip_set = 0, display_rotation_set = 0, hscale_set = 0, vscale_set = 0; + uint8_t *buf = NULL; + + struct DisplayMatrix test_args = { + .class = &class_display_matrix_args, + .rotation = DBL_MAX, + .hflip = -1, + .vflip = -1, + .hscale = DBL_MAX, + .vscale = DBL_MAX, + }; + + AVDictionary *global_args = NULL; + AVDictionary *local_args = NULL; + AVDictionaryEntry *en = NULL; + + MATCH_PER_STREAM_OPT(display_matrixes, dict, global_args, ctx, st); + + if (!global_args) + return; + + // make a copy of the dict so it doesn't get freed from underneath us + if (av_dict_copy(&local_args, global_args, 0) < 0) { + av_log(NULL, AV_LOG_FATAL, + "Failed to copy argument dict for display matrix!\n"); + exit_program(1); + } + + if (av_opt_set_dict2(&test_args, &local_args, 0) < 0) { + av_log(NULL, AV_LOG_FATAL, + "Failed to set options for a display matrix!\n"); + exit_program(1); + } + + while ((en = av_dict_get(local_args, "", en, AV_DICT_IGNORE_SUFFIX))) { + av_log(NULL, AV_LOG_FATAL, + "Unknown option=value pair for display matrix: " + "key: '%s', value: '%s'!\n", + en->key, en->value); + } + + if (av_dict_count(local_args)) { + exit_program(1); + } + + av_dict_free(&local_args); + + display_rotation_set = test_args.rotation != DBL_MAX; + hflip_set = test_args.hflip != -1; + vflip_set = test_args.vflip != -1; + hscale_set = test_args.hscale != DBL_MAX; + vscale_set = test_args.vscale != DBL_MAX; + + if (!display_rotation_set && !hflip_set && !vflip_set && + !hscale_set && !vscale_set) + return; + + if (!(buf = av_stream_new_side_data(st, AV_PKT_DATA_DISPLAYMATRIX, + sizeof(int32_t) * 9))) { + av_log(NULL, AV_LOG_FATAL, "Failed to generate a display matrix!\n"); + exit_program(1); + } + + av_display_rotation_set((int32_t *)buf, + display_rotation_set ? -(test_args.rotation) : + -0.0f); + av_display_matrix_flip((int32_t *)buf, + hflip_set ? test_args.hflip : 0, + vflip_set ? test_args.vflip : 0); + + av_display_matrix_scale((int32_t *)buf, + hscale_set ? test_args.hscale : 1.0f, + vscale_set ? test_args.vscale : 1.0f); +} + + static const AVCodec *find_codec_or_die(const char *name, enum AVMediaType type, int encoder) { const AVCodecDescriptor *desc; @@ -958,6 +1072,8 @@ static void add_input_streams(OptionsContext *o, AVFormatContext *ic) } if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) { + add_display_matrix_to_stream(o, ic, st); + MATCH_PER_STREAM_OPT(hwaccels, str, hwaccel, ic, st); MATCH_PER_STREAM_OPT(hwaccel_output_formats, str, hwaccel_output_format, ic, st); @@ -1883,6 +1999,8 @@ static OutputStream *new_video_stream(OptionsContext *o, AVFormatContext *oc, in ost->frame_aspect_ratio = q; } + add_display_matrix_to_stream(o, oc, st); + MATCH_PER_STREAM_OPT(filter_scripts, str, ost->filters_script, oc, st); MATCH_PER_STREAM_OPT(filters, str, ost->filters, oc, st); @@ -4044,6 +4162,11 @@ const OptionDef options[] = { { "aspect", OPT_VIDEO | HAS_ARG | OPT_STRING | OPT_SPEC | OPT_OUTPUT, { .off = OFFSET(frame_aspect_ratios) }, "set aspect ratio (4:3, 16:9 or 1.3333, 1.7777)", "aspect" }, + { "display_matrix", OPT_VIDEO | HAS_ARG | OPT_DICT | OPT_SPEC | + OPT_INPUT, { .off = OFFSET(display_matrixes) }, + "define a display matrix with rotation, and/or horizontal/vertical " + "flip/scale for stream(s)", + "arguments", &class_display_matrix_args }, { "pix_fmt", OPT_VIDEO | HAS_ARG | OPT_EXPERT | OPT_STRING | OPT_SPEC | OPT_INPUT | OPT_OUTPUT, { .off = OFFSET(frame_pix_fmts) }, "set pixel format", "format" }, diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak index 372c70bba7..763390ea51 100644 --- a/tests/fate/filter-video.mak +++ b/tests/fate/filter-video.mak @@ -691,7 +691,7 @@ fate-filter-metadata-avf-aphase-meter-out-of-phase: SRC = $(TARGET_SAMPLES)/filt fate-filter-metadata-avf-aphase-meter-out-of-phase: CMD = run $(FILTER_METADATA_COMMAND) "amovie='$(SRC)',aphasemeter=video=0" FATE_FILTER_SAMPLES-$(call TRANSCODE, RAWVIDEO H264, MOV, ARESAMPLE_FILTER AAC_FIXED_DECODER) += fate-filter-meta-4560-rotate0 -fate-filter-meta-4560-rotate0: CMD = transcode mov $(TARGET_SAMPLES)/filter/sample-in-issue-505.mov mov "-c copy -metadata:s:v:0 rotate=0" "-af aresample" "" "" "-flags +bitexact -c:a aac_fixed" +fate-filter-meta-4560-rotate0: CMD = transcode "mov -display_matrix:v:0 rotation=0" $(TARGET_SAMPLES)/filter/sample-in-issue-505.mov mov "-c copy" "-af aresample" "" "" "-flags +bitexact -c:a aac_fixed" FATE_FILTER_CMP_METADATA-$(CONFIG_BLOCKDETECT_FILTER) += fate-filter-refcmp-blockdetect-yuv fate-filter-refcmp-blockdetect-yuv: CMD = cmp_metadata blockdetect yuv420p 0.015