From patchwork Thu Mar 9 20:27:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Calvin Walton X-Patchwork-Id: 2862 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.50.79 with SMTP id y76csp495217vsy; Thu, 9 Mar 2017 12:27:39 -0800 (PST) X-Received: by 10.28.109.147 with SMTP id b19mr30720502wmi.69.1489091259411; Thu, 09 Mar 2017 12:27:39 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d7si10000683wrc.336.2017.03.09.12.27.38; Thu, 09 Mar 2017 12:27:39 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@kepstin.ca; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=REJECT dis=NONE) header.from=kepstin.ca Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 881B568075E; Thu, 9 Mar 2017 22:27:21 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-it0-f65.google.com (mail-it0-f65.google.com [209.85.214.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 242C468046B for ; Thu, 9 Mar 2017 22:27:15 +0200 (EET) Received: by mail-it0-f65.google.com with SMTP id f203so11236242itf.2 for ; Thu, 09 Mar 2017 12:27:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kepstin.ca; s=google; h=from:to:cc:subject:date:message-id; bh=m8hRzziBTisXZvZGA9THoonXG6AXCld1i+u4r+strZ8=; b=tLvyPZXxBo4rVMLSUTWL+kgi9b8jwQTIq5zm1Yr9YkTsmRPE3J8oYbEZnGD7MPVErU FpGxdjuX4Q9a1WvN95jxEjerLwkWT1GYRUVIrEnHlIEjRzZCdVzSIGA+rHRCwd/SwCXc zAcBZaVuIWSZLmAeOTgqxTDYqd8RONeph5GfA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=m8hRzziBTisXZvZGA9THoonXG6AXCld1i+u4r+strZ8=; b=Uoeo/eET6xL+ATfTfJT0oH66VJLF2qTSWDbL0Tbw4W36xRFlrHB7YxOy0vXDeMxYg4 RYPauAqGeryrQSW3BP6m1VH1dVlOKighn5OV58/45IYnoBGvZ46DVFb8gkU5hAae81Ut pgERCVf01SheXQtNU2hySpv/XbBhF24OZQAPF9r2PD3odUBLJ2mcnKC/FErnkXZrEY+y k1jVmnw4RrwUT6GxGCLXiYXTY+9PgKbYF2ao+HQLYulk7mFALDQD+/8ru+YjZuDD14rx RTSs7mEfR39RrSttrwUWsYAeq+3PaEK4Rp8cafS4QuAN05fE5XDQrvdIO26D2GlJuW+p RRMA== X-Gm-Message-State: AMke39lXB4jAxetoL4VVr/ZSBnYGn2wEtvlAt4ov04mdNBo/eHcmzpOJ9pum6WwKR6Rk/Q== X-Received: by 10.36.10.13 with SMTP id 13mr13924002itw.110.1489091247529; Thu, 09 Mar 2017 12:27:27 -0800 (PST) Received: from rocky.ottawa.blindsidenetworks.com ([2001:470:8924:82da::f82b]) by smtp.gmail.com with ESMTPSA id f85sm3410079ioi.26.2017.03.09.12.27.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Mar 2017 12:27:26 -0800 (PST) From: Calvin Walton To: ffmpeg-devel@ffmpeg.org Date: Thu, 9 Mar 2017 15:27:16 -0500 Message-Id: <20170309202716.7757-1-calvin.walton@kepstin.ca> X-Mailer: git-send-email 2.9.3 Subject: [FFmpeg-devel] [PATCH v2] Allow 'concat' filter to support inputs with different frame rates X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Calvin Walton MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Right now, the concat filter does not set the frame_rate value on any of the out links. As a result, the default ffmpeg behaviour kicks in - to copy the framerate from the first input to the outputs. If a later input is higher framerate, this results in dropped frames; if a later input is lower framerate it might cause judder. This patch checks if all of the video inputs have the same framerate, and if not it sets the out link to use '1/0' as the frame rate, the value meaning "unknown/vfr". A test is added to verify the VFR behaviour. The existing test for CFR behaviour passes unchanged. --- libavfilter/avf_concat.c | 15 ++- tests/fate/filter-video.mak | 4 +- tests/ref/fate/filter-concat-vfr | 224 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 241 insertions(+), 2 deletions(-) create mode 100644 tests/ref/fate/filter-concat-vfr diff --git a/libavfilter/avf_concat.c b/libavfilter/avf_concat.c index 56e4179..2258663 100644 --- a/libavfilter/avf_concat.c +++ b/libavfilter/avf_concat.c @@ -133,8 +133,21 @@ static int config_output(AVFilterLink *outlink) outlink->h = inlink->h; outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; outlink->format = inlink->format; + outlink->frame_rate = inlink->frame_rate; + + for (seg = 1; seg < cat->nb_segments; seg++) { + inlink = ctx->inputs[in_no + seg * ctx->nb_outputs]; + if (outlink->frame_rate.num != inlink->frame_rate.num || + outlink->frame_rate.den != outlink->frame_rate.den) { + av_log(ctx, AV_LOG_VERBOSE, + "Video inputs have different frame rates, output will be VFR"); + outlink->frame_rate = av_make_q(1, 0); + break; + } + } + for (seg = 1; seg < cat->nb_segments; seg++) { - inlink = ctx->inputs[in_no += ctx->nb_outputs]; + inlink = ctx->inputs[in_no + seg * ctx->nb_outputs]; if (!outlink->sample_aspect_ratio.num) outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; /* possible enhancement: unsafe mode, do not check */ diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak index c57c9c7..5a814e2 100644 --- a/tests/fate/filter-video.mak +++ b/tests/fate/filter-video.mak @@ -358,9 +358,11 @@ FATE_FILTER_SAMPLES-$(call ALLYES, VMD_DEMUXER VMDVIDEO_DECODER FORMAT_FILTER PE fate-filter-gradfun-sample: tests/data/filtergraphs/gradfun fate-filter-gradfun-sample: CMD = framecrc -i $(TARGET_SAMPLES)/vmd/12.vmd -filter_script $(TARGET_PATH)/tests/data/filtergraphs/gradfun -an -frames:v 20 -FATE_FILTER-$(call ALLYES, TESTSRC_FILTER SINE_FILTER CONCAT_FILTER) += fate-filter-concat +FATE_FILTER-$(call ALLYES, TESTSRC_FILTER SINE_FILTER CONCAT_FILTER) += fate-filter-concat fate-filter-concat-vfr fate-filter-concat: tests/data/filtergraphs/concat fate-filter-concat: CMD = framecrc -filter_complex_script $(TARGET_PATH)/tests/data/filtergraphs/concat +fate-filter-concat-vfr: tests/data/filtergraphs/concat-vfr +fate-filter-concat-vfr: CMD = framecrc -filter_complex_script $(TARGET_PATH)/tests/data/filtergraphs/concat-vfr FATE_FILTER-$(call ALLYES, TESTSRC2_FILTER FPS_FILTER MPDECIMATE_FILTER) += fate-filter-mpdecimate fate-filter-mpdecimate: CMD = framecrc -lavfi testsrc2=r=2:d=10,fps=3,mpdecimate -r 3 -pix_fmt yuv420p diff --git a/tests/ref/fate/filter-concat-vfr b/tests/ref/fate/filter-concat-vfr new file mode 100644 index 0000000..0dece67 --- /dev/null +++ b/tests/ref/fate/filter-concat-vfr @@ -0,0 +1,224 @@ +#tb 0: 1/1000000 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 320x240 +#sar 0: 1/1 +#tb 1: 1/44100 +#media_type 1: audio +#codec_id 1: pcm_s16le +#sample_rate 1: 44100 +#channel_layout 1: 4 +#channel_layout_name 1: mono +0, 0, 0, 0, 230400, 0x88c4d19a +1, 0, 0, 1024, 2048, 0xb3f10192 +1, 1024, 1024, 1024, 2048, 0xb340fe4e +1, 2048, 2048, 1024, 2048, 0x0a5f0111 +1, 3072, 3072, 1024, 2048, 0x51be06b8 +1, 4096, 4096, 1024, 2048, 0x71a1ffcb +1, 5120, 5120, 1024, 2048, 0x7f64f50f +1, 6144, 6144, 1024, 2048, 0x70a8fa17 +1, 7168, 7168, 1024, 2048, 0x0dad072a +1, 8192, 8192, 1024, 2048, 0x5e810c51 +0, 200000, 200000, 0, 230400, 0x0d77c977 +1, 9216, 9216, 1024, 2048, 0xbe5bf462 +1, 10240, 10240, 1024, 2048, 0xbcd9faeb +1, 11264, 11264, 1024, 2048, 0x0d5bfe9c +1, 12288, 12288, 1024, 2048, 0x97d80297 +1, 13312, 13312, 1024, 2048, 0xba0f0894 +1, 14336, 14336, 1024, 2048, 0xcc22f291 +1, 15360, 15360, 1024, 2048, 0x11a9fa03 +1, 16384, 16384, 1024, 2048, 0x9a920378 +1, 17408, 17408, 1024, 2048, 0x901b0525 +0, 400000, 400000, 0, 230400, 0x242629d7 +1, 18432, 18432, 1024, 2048, 0x74b2003f +1, 19456, 19456, 1024, 2048, 0xa20ef3ed +1, 20480, 20480, 1024, 2048, 0x44cef9de +1, 21504, 21504, 1024, 2048, 0x4b2e039b +1, 22528, 22528, 1024, 2048, 0x198509a1 +1, 23552, 23552, 1024, 2048, 0xcab6f9e5 +1, 24576, 24576, 1024, 2048, 0x67f8f608 +1, 25600, 25600, 1024, 2048, 0x8d7f03fa +0, 600000, 600000, 0, 230400, 0x62cdc018 +1, 26624, 26624, 1024, 2048, 0x3e1e0566 +1, 27648, 27648, 1024, 2048, 0x2cfe0308 +1, 28672, 28672, 1024, 2048, 0x1ceaf702 +1, 29696, 29696, 1024, 2048, 0x38a9f3d1 +1, 30720, 30720, 1024, 2048, 0x6c3306b7 +1, 31744, 31744, 1024, 2048, 0x600f0579 +1, 32768, 32768, 1024, 2048, 0x3e5afa28 +1, 33792, 33792, 1024, 2048, 0x053ff47a +1, 34816, 34816, 1024, 2048, 0x0d28fed9 +0, 800000, 800000, 0, 230400, 0x248ad058 +1, 35840, 35840, 1024, 2048, 0x279805cc +1, 36864, 36864, 1024, 2048, 0xb16a0a12 +1, 37888, 37888, 1024, 2048, 0xb45af340 +1, 38912, 38912, 1024, 2048, 0x1834f972 +1, 39936, 39936, 1024, 2048, 0xb5d206ae +1, 40960, 40960, 1024, 2048, 0xc5760375 +1, 41984, 41984, 1024, 2048, 0x503800ce +1, 43008, 43008, 1024, 2048, 0xa3bbf4af +1, 44032, 44032, 68, 136, 0xc8d751c7 +0, 1000000, 1000000, 0, 230400, 0x223d134f +1, 44100, 44100, 9600, 19200, 0x00000000 +0, 1200000, 1200000, 0, 230400, 0xbf1c3d34 +1, 53700, 53700, 9600, 19200, 0x00000000 +0, 1400000, 1400000, 0, 230400, 0xae0efe96 +1, 63300, 63300, 9600, 19200, 0x00000000 +0, 1600000, 1600000, 0, 230400, 0x0cd624d1 +1, 72900, 72900, 9600, 19200, 0x00000000 +0, 1800000, 1800000, 0, 230400, 0x6dedf2c0 +1, 82500, 82500, 5700, 11400, 0x00000000 +0, 2000000, 2000000, 0, 230400, 0x88c4d19a +1, 88200, 88200, 1024, 2048, 0x283efb3a +1, 89224, 89224, 1024, 2048, 0x7692fb8f +1, 90248, 90248, 1024, 2048, 0xbaaafcc0 +0, 2066667, 2066667, 0, 230400, 0x5bbc2f63 +1, 91272, 91272, 1024, 2048, 0xadc8017e +1, 92296, 92296, 1024, 2048, 0x4f4dffdc +1, 93320, 93320, 1024, 2048, 0x7ffbff48 +0, 2133333, 2133333, 0, 230400, 0x3becbfad +1, 94344, 94344, 1024, 2048, 0x2f990719 +1, 95368, 95368, 1024, 2048, 0xe2caf65c +1, 96392, 96392, 1024, 2048, 0x825208e4 +0, 2200000, 2200000, 0, 230400, 0x0d77c977 +1, 97416, 97416, 1024, 2048, 0xf563f13b +1, 98440, 98440, 1024, 2048, 0x855d03e9 +1, 99464, 99464, 1024, 2048, 0x0ba9fa4b +0, 2266667, 2266667, 0, 230400, 0x436cf4b2 +1, 100488, 100488, 1024, 2048, 0x83e1fb92 +1, 101512, 101512, 1024, 2048, 0x1162f965 +1, 102536, 102536, 1024, 2048, 0x0cfef73d +0, 2333333, 2333333, 0, 230400, 0x39210f27 +1, 103560, 103560, 1024, 2048, 0x5688ff75 +1, 104584, 104584, 1024, 2048, 0xf6c0ede9 +1, 105608, 105608, 1024, 2048, 0xfdb20602 +0, 2400000, 2400000, 0, 230400, 0x242629d7 +1, 106632, 106632, 1024, 2048, 0x40c5f17b +1, 107656, 107656, 1024, 2048, 0x559600b1 +1, 108680, 108680, 1024, 2048, 0xccc3f930 +0, 2466667, 2466667, 0, 230400, 0x771c2293 +1, 109704, 109704, 1024, 2048, 0xdc800045 +1, 110728, 110728, 1024, 2048, 0xdce4fb3e +0, 2533333, 2533333, 0, 230400, 0xec2af9a9 +1, 111752, 111752, 1024, 2048, 0x1e5efba9 +1, 112776, 112776, 1024, 2048, 0x8c2e0832 +1, 113800, 113800, 1024, 2048, 0x5c42f66d +0, 2600000, 2600000, 0, 230400, 0x62cdc018 +1, 114824, 114824, 1024, 2048, 0x08e20b1e +1, 115848, 115848, 1024, 2048, 0x4cf7f903 +1, 116872, 116872, 1024, 2048, 0xe6b90794 +0, 2666667, 2666667, 0, 230400, 0xf02c8693 +1, 117896, 117896, 1024, 2048, 0x5956f8e6 +1, 118920, 118920, 1024, 2048, 0x6632ff16 +1, 119944, 119944, 1024, 2048, 0x46c8fe11 +0, 2733333, 2733333, 0, 230400, 0x14436efb +1, 120968, 120968, 1024, 2048, 0x7431f732 +1, 121992, 121992, 1024, 2048, 0xa258049f +1, 123016, 123016, 1024, 2048, 0xdb71f00e +0, 2800000, 2800000, 0, 230400, 0x248ad058 +1, 124040, 124040, 1024, 2048, 0xa89b0359 +1, 125064, 125064, 1024, 2048, 0xe0aff0f2 +1, 126088, 126088, 1024, 2048, 0xc33e0085 +0, 2866667, 2866667, 0, 230400, 0xe87f6c52 +1, 127112, 127112, 1024, 2048, 0x9d09f379 +1, 128136, 128136, 1024, 2048, 0x8c78fd06 +1, 129160, 129160, 1024, 2048, 0x532bfbdd +0, 2933333, 2933333, 0, 230400, 0x6a0c196b +1, 130184, 130184, 1024, 2048, 0xfc36f5cd +1, 131208, 131208, 1024, 2048, 0x2e8f0699 +1, 132232, 132232, 1024, 2048, 0x52382578 +1, 133256, 133256, 1024, 2048, 0x97ed1a28 +1, 134280, 134280, 1024, 2048, 0xabcdf73f +1, 135304, 135304, 1024, 2048, 0x3a24082c +1, 136328, 136328, 1024, 2048, 0xbe1cfc3d +1, 137352, 137352, 1024, 2048, 0xad5800a5 +1, 138376, 138376, 1024, 2048, 0x90b80522 +1, 139400, 139400, 1024, 2048, 0x1fa1f912 +1, 140424, 140424, 1024, 2048, 0x733a0878 +1, 141448, 141448, 1024, 2048, 0x9a3eee47 +1, 142472, 142472, 1024, 2048, 0x5d900759 +1, 143496, 143496, 1024, 2048, 0x1287f540 +1, 144520, 144520, 1024, 2048, 0x941cfe5d +1, 145544, 145544, 1024, 2048, 0x1587f8a9 +1, 146568, 146568, 1024, 2048, 0xb9e7f888 +1, 147592, 147592, 1024, 2048, 0xe9defbe2 +1, 148616, 148616, 1024, 2048, 0x3a5ef312 +1, 149640, 149640, 1024, 2048, 0xdcbe0544 +1, 150664, 150664, 1024, 2048, 0xbe51ecc5 +1, 151688, 151688, 1024, 2048, 0x21a60721 +1, 152712, 152712, 1024, 2048, 0xf29ff318 +1, 153736, 153736, 1024, 2048, 0xcd4c02ea +1, 154760, 154760, 1024, 2048, 0xa424faac +1, 155784, 155784, 1024, 2048, 0xbaedfdab +1, 156808, 156808, 1024, 2048, 0xcbff047c +1, 157832, 157832, 1024, 2048, 0x9ac8f96b +1, 158856, 158856, 1024, 2048, 0x43220bee +1, 159880, 159880, 1024, 2048, 0x547bf351 +1, 160904, 160904, 1024, 2048, 0x7dd10d6e +1, 161928, 161928, 1024, 2048, 0x77cbf603 +1, 162952, 162952, 1024, 2048, 0xb6fcff50 +1, 163976, 163976, 1024, 2048, 0x927bfde5 +1, 165000, 165000, 1024, 2048, 0x5bd0fca5 +1, 166024, 166024, 1024, 2048, 0x672cff2a +1, 167048, 167048, 1024, 2048, 0x3e3ef01c +1, 168072, 168072, 1024, 2048, 0xe52607af +1, 169096, 169096, 1024, 2048, 0x66bceaf5 +1, 170120, 170120, 1024, 2048, 0xe065046b +1, 171144, 171144, 1024, 2048, 0x350bf21f +1, 172168, 172168, 1024, 2048, 0x60b1fca4 +1, 173192, 173192, 1024, 2048, 0x8b1efa55 +1, 174216, 174216, 1024, 2048, 0xf86ff855 +1, 175240, 175240, 1024, 2048, 0x6934061b +1, 176264, 176264, 136, 272, 0x4a458a45 +0, 4000000, 4000000, 0, 230400, 0x88c4d19a +1, 176400, 176400, 1024, 2048, 0xdb0cfe95 +1, 177424, 177424, 1024, 2048, 0xcff3fdf1 +1, 178448, 178448, 1024, 2048, 0x070cf585 +1, 179472, 179472, 1024, 2048, 0xe9b8007f +1, 180496, 180496, 1024, 2048, 0xc51ffd64 +1, 181520, 181520, 1024, 2048, 0xede2fbf9 +0, 4125000, 4125000, 0, 230400, 0x05c1b733 +1, 182544, 182544, 1024, 2048, 0x51510410 +1, 183568, 183568, 1024, 2048, 0x198af498 +1, 184592, 184592, 1024, 2048, 0xae3603a2 +1, 185616, 185616, 1024, 2048, 0x6200f7a1 +1, 186640, 186640, 1024, 2048, 0xe6e3fe32 +0, 4250000, 4250000, 0, 230400, 0x0446ec19 +1, 187664, 187664, 1024, 2048, 0xb2e2fd77 +1, 188688, 188688, 1024, 2048, 0x063dff2f +1, 189712, 189712, 1024, 2048, 0xa89ffe21 +1, 190736, 190736, 1024, 2048, 0x9e6ffa6d +1, 191760, 191760, 1024, 2048, 0x028b004e +1, 192784, 192784, 1024, 2048, 0x57edfa23 +0, 4375000, 4375000, 0, 230400, 0x0f9b1744 +1, 193808, 193808, 1024, 2048, 0x6d8efe1f +1, 194832, 194832, 1024, 2048, 0x774bfe54 +1, 195856, 195856, 1024, 2048, 0xa931fcfb +1, 196880, 196880, 1024, 2048, 0x3505004b +1, 197904, 197904, 1024, 2048, 0x5001f576 +0, 4500000, 4500000, 0, 230400, 0x30cf070a +1, 198928, 198928, 1024, 2048, 0x78ea049b +1, 199952, 199952, 1024, 2048, 0xd45bf733 +1, 200976, 200976, 1024, 2048, 0x6395fead +1, 202000, 202000, 1024, 2048, 0xc126015e +1, 203024, 203024, 1024, 2048, 0xbecff8aa +0, 4625000, 4625000, 0, 230400, 0x9175aaa9 +1, 204048, 204048, 1024, 2048, 0x0fea06c3 +1, 205072, 205072, 1024, 2048, 0xdea6f351 +1, 206096, 206096, 1024, 2048, 0x35b808f0 +1, 207120, 207120, 1024, 2048, 0x5487ee73 +1, 208144, 208144, 1024, 2048, 0xac69050e +1, 209168, 209168, 1024, 2048, 0xcc5ffb00 +0, 4750000, 4750000, 0, 230400, 0x597f5628 +1, 210192, 210192, 1024, 2048, 0x328c00cb +1, 211216, 211216, 1024, 2048, 0xa707fd82 +1, 212240, 212240, 1024, 2048, 0xe442f73d +1, 213264, 213264, 1024, 2048, 0x545c0418 +1, 214288, 214288, 1024, 2048, 0x744ff3f7 +0, 4875000, 4875000, 0, 230400, 0x38a45a85 +1, 215312, 215312, 1024, 2048, 0x01aa04fd +1, 216336, 216336, 1024, 2048, 0xa885f7cd +1, 217360, 217360, 1024, 2048, 0xcfca04f4 +1, 218384, 218384, 1024, 2048, 0x67fdf91b +1, 219408, 219408, 1024, 2048, 0xce2b001d +1, 220432, 220432, 68, 136, 0x33e64a0d