diff mbox series

[FFmpeg-devel,02/18] fftools/ffmpeg_filter: refactor setting input timebase

Message ID 20240306110319.17339-2-anton@khirnov.net
State New
Headers show
Series [FFmpeg-devel,01/18] fftools/cmdutils: fix printing group name in split_commandline() | expand

Checks

Context Check Description
yinshiyou/make_loongarch64 success Make finished
yinshiyou/make_fate_loongarch64 success Make fate finished
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished

Commit Message

Anton Khirnov March 6, 2024, 11:03 a.m. UTC
Treat it analogously to stream parameters like format/dimensions/etc.
This is functionally different from previous code in 2 ways:
* for non-CFR video, the frame timebase (set by the decoder) is used
  rather than the demuxer timebase
* for sub2video, AV_TIME_BASE_Q is used, which is hardcoded by the
  subtitle decoding API

These changes should avoid unnecessary and potentially lossy timestamp
conversions from decoder timebase into the demuxer one.

Changes the timebases used in sub2video tests.
---
 fftools/ffmpeg_filter.c               |  17 ++-
 tests/ref/fate/sub2video_basic        | 182 +++++++++++++-------------
 tests/ref/fate/sub2video_time_limited |   8 +-
 3 files changed, 106 insertions(+), 101 deletions(-)

Comments

Michael Niedermayer March 7, 2024, 8:37 p.m. UTC | #1
On Wed, Mar 06, 2024 at 12:03:03PM +0100, Anton Khirnov wrote:
> Treat it analogously to stream parameters like format/dimensions/etc.
> This is functionally different from previous code in 2 ways:
> * for non-CFR video, the frame timebase (set by the decoder) is used
>   rather than the demuxer timebase
> * for sub2video, AV_TIME_BASE_Q is used, which is hardcoded by the
>   subtitle decoding API
> 
> These changes should avoid unnecessary and potentially lossy timestamp
> conversions from decoder timebase into the demuxer one.
> 
> Changes the timebases used in sub2video tests.
> ---
>  fftools/ffmpeg_filter.c               |  17 ++-
>  tests/ref/fate/sub2video_basic        | 182 +++++++++++++-------------
>  tests/ref/fate/sub2video_time_limited |   8 +-
>  3 files changed, 106 insertions(+), 101 deletions(-)

breaks:

./ffmpeg -i \[a-s\]_full_metal_panic_fumoffu_-_01_-_the_man_from_the_south_-_a_hostage_with_no_compromises__rs2_\[1080p_bd-rip\]\[BBB48A25\].mkv  -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 -y a.avi

Press [q] to stop, [?] for help
[aac @ 0x561991f96340] This stream seems to incorrectly report its last channel as LFE[3], mapping to LFE[0]
[mpeg4 @ 0x561991f7e440] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535
[vost#0:0/mpeg4 @ 0x561991f78f80] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[fc#0 @ 0x561991f4b9c0] Error sending frames to consumers: Invalid argument
[fc#0 @ 0x561991f4b9c0] Task finished with error code: -22 (Invalid argument)
[fc#0 @ 0x561991f4b9c0] Terminating thread with return code -22 (Invalid argument)
[vost#0:0/mpeg4 @ 0x561991f78f80] Could not open encoder before EOF
[vost#0:0/mpeg4 @ 0x561991f78f80] Task finished with error code: -22 (Invalid argument)
[vost#0:0/mpeg4 @ 0x561991f78f80] Terminating thread with return code -22 (Invalid argument)
[libmp3lame @ 0x561992064180] Trying to remove 1152 samples, but the queue is empty
[out#0/avi @ 0x561991f98bc0] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0KiB time=N/A bitrate=N/A speed=N/A
Conversion failed!

ill try to find a smaller testcase

thx


[...]
Anton Khirnov March 8, 2024, 5:34 a.m. UTC | #2
Quoting Michael Niedermayer (2024-03-07 21:37:39)
> On Wed, Mar 06, 2024 at 12:03:03PM +0100, Anton Khirnov wrote:
> > Treat it analogously to stream parameters like format/dimensions/etc.
> > This is functionally different from previous code in 2 ways:
> > * for non-CFR video, the frame timebase (set by the decoder) is used
> >   rather than the demuxer timebase
> > * for sub2video, AV_TIME_BASE_Q is used, which is hardcoded by the
> >   subtitle decoding API
> > 
> > These changes should avoid unnecessary and potentially lossy timestamp
> > conversions from decoder timebase into the demuxer one.
> > 
> > Changes the timebases used in sub2video tests.
> > ---
> >  fftools/ffmpeg_filter.c               |  17 ++-
> >  tests/ref/fate/sub2video_basic        | 182 +++++++++++++-------------
> >  tests/ref/fate/sub2video_time_limited |   8 +-
> >  3 files changed, 106 insertions(+), 101 deletions(-)
> 
> breaks:
> 
> ./ffmpeg -i \[a-s\]_full_metal_panic_fumoffu_-_01_-_the_man_from_the_south_-_a_hostage_with_no_compromises__rs2_\[1080p_bd-rip\]\[BBB48A25\].mkv  -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 -y a.avi
> 

Use a constant framerate.
Michael Niedermayer March 10, 2024, 3:36 a.m. UTC | #3
On Fri, Mar 08, 2024 at 06:34:36AM +0100, Anton Khirnov wrote:
> Quoting Michael Niedermayer (2024-03-07 21:37:39)
> > On Wed, Mar 06, 2024 at 12:03:03PM +0100, Anton Khirnov wrote:
> > > Treat it analogously to stream parameters like format/dimensions/etc.
> > > This is functionally different from previous code in 2 ways:
> > > * for non-CFR video, the frame timebase (set by the decoder) is used
> > >   rather than the demuxer timebase
> > > * for sub2video, AV_TIME_BASE_Q is used, which is hardcoded by the
> > >   subtitle decoding API
> > > 
> > > These changes should avoid unnecessary and potentially lossy timestamp
> > > conversions from decoder timebase into the demuxer one.
> > > 
> > > Changes the timebases used in sub2video tests.
> > > ---
> > >  fftools/ffmpeg_filter.c               |  17 ++-
> > >  tests/ref/fate/sub2video_basic        | 182 +++++++++++++-------------
> > >  tests/ref/fate/sub2video_time_limited |   8 +-
> > >  3 files changed, 106 insertions(+), 101 deletions(-)
> > 
> > breaks:
> > 
> > ./ffmpeg -i \[a-s\]_full_metal_panic_fumoffu_-_01_-_the_man_from_the_south_-_a_hostage_with_no_compromises__rs2_\[1080p_bd-rip\]\[BBB48A25\].mkv  -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 -y a.avi
> > 
> 
> Use a constant framerate.

why not automatically choose a supported timebase  ?

"[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"

?

thx


[...]
Anton Khirnov March 10, 2024, 6:13 a.m. UTC | #4
Quoting Michael Niedermayer (2024-03-10 04:36:29)
> On Fri, Mar 08, 2024 at 06:34:36AM +0100, Anton Khirnov wrote:
> > Quoting Michael Niedermayer (2024-03-07 21:37:39)
> > > On Wed, Mar 06, 2024 at 12:03:03PM +0100, Anton Khirnov wrote:
> > > > Treat it analogously to stream parameters like format/dimensions/etc.
> > > > This is functionally different from previous code in 2 ways:
> > > > * for non-CFR video, the frame timebase (set by the decoder) is used
> > > >   rather than the demuxer timebase
> > > > * for sub2video, AV_TIME_BASE_Q is used, which is hardcoded by the
> > > >   subtitle decoding API
> > > > 
> > > > These changes should avoid unnecessary and potentially lossy timestamp
> > > > conversions from decoder timebase into the demuxer one.
> > > > 
> > > > Changes the timebases used in sub2video tests.
> > > > ---
> > > >  fftools/ffmpeg_filter.c               |  17 ++-
> > > >  tests/ref/fate/sub2video_basic        | 182 +++++++++++++-------------
> > > >  tests/ref/fate/sub2video_time_limited |   8 +-
> > > >  3 files changed, 106 insertions(+), 101 deletions(-)
> > > 
> > > breaks:
> > > 
> > > ./ffmpeg -i \[a-s\]_full_metal_panic_fumoffu_-_01_-_the_man_from_the_south_-_a_hostage_with_no_compromises__rs2_\[1080p_bd-rip\]\[BBB48A25\].mkv  -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 -y a.avi
> > > 
> > 
> > Use a constant framerate.
> 
> why not automatically choose a supported timebase  ?
> 
> "[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"

Because I don't want ffmpeg CLI to have codec-specific code for a codec
that's been obsolete for 15+ years. One could also potentially do it
inside the encoder itself, but it is nontrivial since the computations
are spread across a number of places in mpeg4videoenc.c and
mpegvideo_enc.c. And again, it seems like a waste of time - there is no
reason to encode mpeg4 today.
Michael Niedermayer March 10, 2024, 7:21 p.m. UTC | #5
On Sun, Mar 10, 2024 at 07:13:18AM +0100, Anton Khirnov wrote:
> Quoting Michael Niedermayer (2024-03-10 04:36:29)
> > On Fri, Mar 08, 2024 at 06:34:36AM +0100, Anton Khirnov wrote:
> > > Quoting Michael Niedermayer (2024-03-07 21:37:39)
> > > > On Wed, Mar 06, 2024 at 12:03:03PM +0100, Anton Khirnov wrote:
> > > > > Treat it analogously to stream parameters like format/dimensions/etc.
> > > > > This is functionally different from previous code in 2 ways:
> > > > > * for non-CFR video, the frame timebase (set by the decoder) is used
> > > > >   rather than the demuxer timebase
> > > > > * for sub2video, AV_TIME_BASE_Q is used, which is hardcoded by the
> > > > >   subtitle decoding API
> > > > > 
> > > > > These changes should avoid unnecessary and potentially lossy timestamp
> > > > > conversions from decoder timebase into the demuxer one.
> > > > > 
> > > > > Changes the timebases used in sub2video tests.
> > > > > ---
> > > > >  fftools/ffmpeg_filter.c               |  17 ++-
> > > > >  tests/ref/fate/sub2video_basic        | 182 +++++++++++++-------------
> > > > >  tests/ref/fate/sub2video_time_limited |   8 +-
> > > > >  3 files changed, 106 insertions(+), 101 deletions(-)
> > > > 
> > > > breaks:
> > > > 
> > > > ./ffmpeg -i \[a-s\]_full_metal_panic_fumoffu_-_01_-_the_man_from_the_south_-_a_hostage_with_no_compromises__rs2_\[1080p_bd-rip\]\[BBB48A25\].mkv  -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 -y a.avi
> > > > 
> > > 
> > > Use a constant framerate.
> > 
> > why not automatically choose a supported timebase  ?
> > 
> > "[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"
> 
> Because I don't want ffmpeg CLI to have codec-specific code for a codec
> that's been obsolete for 15+ years. One could also potentially do it
> inside the encoder itself, but it is nontrivial since the computations
> are spread across a number of places in mpeg4videoenc.c and
> mpegvideo_enc.c. And again, it seems like a waste of time - there is no
> reason to encode mpeg4 today.

This is not mpeg4 specific, its just a new additional case that fails

./ffmpeg -i mm-small.mpg test.dv
[dvvideo @ 0x7f868800f100] Found no DV profile for 80x60 yuv420p video. Valid DV profiles are:

IMHO ffmpeg should be able to select supported parameters if the user
indicated thats what he wants

We also do this with pixel formats and not fail and require the user to manually
specify it

thx

[...]
Anton Khirnov March 10, 2024, 10:24 p.m. UTC | #6
Quoting Michael Niedermayer (2024-03-10 20:21:47)
> On Sun, Mar 10, 2024 at 07:13:18AM +0100, Anton Khirnov wrote:
> > Quoting Michael Niedermayer (2024-03-10 04:36:29)
> > > On Fri, Mar 08, 2024 at 06:34:36AM +0100, Anton Khirnov wrote:
> > > > Quoting Michael Niedermayer (2024-03-07 21:37:39)
> > > > > On Wed, Mar 06, 2024 at 12:03:03PM +0100, Anton Khirnov wrote:
> > > > > > Treat it analogously to stream parameters like format/dimensions/etc.
> > > > > > This is functionally different from previous code in 2 ways:
> > > > > > * for non-CFR video, the frame timebase (set by the decoder) is used
> > > > > >   rather than the demuxer timebase
> > > > > > * for sub2video, AV_TIME_BASE_Q is used, which is hardcoded by the
> > > > > >   subtitle decoding API
> > > > > > 
> > > > > > These changes should avoid unnecessary and potentially lossy timestamp
> > > > > > conversions from decoder timebase into the demuxer one.
> > > > > > 
> > > > > > Changes the timebases used in sub2video tests.
> > > > > > ---
> > > > > >  fftools/ffmpeg_filter.c               |  17 ++-
> > > > > >  tests/ref/fate/sub2video_basic        | 182 +++++++++++++-------------
> > > > > >  tests/ref/fate/sub2video_time_limited |   8 +-
> > > > > >  3 files changed, 106 insertions(+), 101 deletions(-)
> > > > > 
> > > > > breaks:
> > > > > 
> > > > > ./ffmpeg -i \[a-s\]_full_metal_panic_fumoffu_-_01_-_the_man_from_the_south_-_a_hostage_with_no_compromises__rs2_\[1080p_bd-rip\]\[BBB48A25\].mkv  -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 -y a.avi
> > > > > 
> > > > 
> > > > Use a constant framerate.
> > > 
> > > why not automatically choose a supported timebase  ?
> > > 
> > > "[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"
> > 
> > Because I don't want ffmpeg CLI to have codec-specific code for a codec
> > that's been obsolete for 15+ years. One could also potentially do it
> > inside the encoder itself, but it is nontrivial since the computations
> > are spread across a number of places in mpeg4videoenc.c and
> > mpegvideo_enc.c. And again, it seems like a waste of time - there is no
> > reason to encode mpeg4 today.
> 
> This is not mpeg4 specific, its just a new additional case that fails

The case you reported is mpeg4 specific.

> ./ffmpeg -i mm-small.mpg test.dv
> [dvvideo @ 0x7f868800f100] Found no DV profile for 80x60 yuv420p video. Valid DV profiles are:

There is no mechanism for an encoder to export supported time bases.

> IMHO ffmpeg should be able to select supported parameters if the user
> indicated thats what he wants
> 
> We also do this with pixel formats and not fail and require the user to manually
> specify it

AFAIK only a tiny number of obsolete encoders place restrictions on the
timebase, so I see little point in spending effort on making them work
automagically.
James Almer March 10, 2024, 10:29 p.m. UTC | #7
On 3/10/2024 7:24 PM, Anton Khirnov wrote:
> Quoting Michael Niedermayer (2024-03-10 20:21:47)
>> On Sun, Mar 10, 2024 at 07:13:18AM +0100, Anton Khirnov wrote:
>>> Quoting Michael Niedermayer (2024-03-10 04:36:29)
>>>> On Fri, Mar 08, 2024 at 06:34:36AM +0100, Anton Khirnov wrote:
>>>>> Quoting Michael Niedermayer (2024-03-07 21:37:39)
>>>>>> On Wed, Mar 06, 2024 at 12:03:03PM +0100, Anton Khirnov wrote:
>>>>>>> Treat it analogously to stream parameters like format/dimensions/etc.
>>>>>>> This is functionally different from previous code in 2 ways:
>>>>>>> * for non-CFR video, the frame timebase (set by the decoder) is used
>>>>>>>    rather than the demuxer timebase
>>>>>>> * for sub2video, AV_TIME_BASE_Q is used, which is hardcoded by the
>>>>>>>    subtitle decoding API
>>>>>>>
>>>>>>> These changes should avoid unnecessary and potentially lossy timestamp
>>>>>>> conversions from decoder timebase into the demuxer one.
>>>>>>>
>>>>>>> Changes the timebases used in sub2video tests.
>>>>>>> ---
>>>>>>>   fftools/ffmpeg_filter.c               |  17 ++-
>>>>>>>   tests/ref/fate/sub2video_basic        | 182 +++++++++++++-------------
>>>>>>>   tests/ref/fate/sub2video_time_limited |   8 +-
>>>>>>>   3 files changed, 106 insertions(+), 101 deletions(-)
>>>>>>
>>>>>> breaks:
>>>>>>
>>>>>> ./ffmpeg -i \[a-s\]_full_metal_panic_fumoffu_-_01_-_the_man_from_the_south_-_a_hostage_with_no_compromises__rs2_\[1080p_bd-rip\]\[BBB48A25\].mkv  -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 -y a.avi
>>>>>>
>>>>>
>>>>> Use a constant framerate.
>>>>
>>>> why not automatically choose a supported timebase  ?
>>>>
>>>> "[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"
>>>
>>> Because I don't want ffmpeg CLI to have codec-specific code for a codec
>>> that's been obsolete for 15+ years. One could also potentially do it
>>> inside the encoder itself, but it is nontrivial since the computations
>>> are spread across a number of places in mpeg4videoenc.c and
>>> mpegvideo_enc.c. And again, it seems like a waste of time - there is no
>>> reason to encode mpeg4 today.
>>
>> This is not mpeg4 specific, its just a new additional case that fails
> 
> The case you reported is mpeg4 specific.
> 
>> ./ffmpeg -i mm-small.mpg test.dv
>> [dvvideo @ 0x7f868800f100] Found no DV profile for 80x60 yuv420p video. Valid DV profiles are:
> 
> There is no mechanism for an encoder to export supported time bases.

Could it be added as an extension to AVProfile, or AVCodec?

> 
>> IMHO ffmpeg should be able to select supported parameters if the user
>> indicated thats what he wants
>>
>> We also do this with pixel formats and not fail and require the user to manually
>> specify it
> 
> AFAIK only a tiny number of obsolete encoders place restrictions on the
> timebase, so I see little point in spending effort on making them work
> automagically.
>
Anton Khirnov March 10, 2024, 10:49 p.m. UTC | #8
Quoting James Almer (2024-03-10 23:29:27)
> On 3/10/2024 7:24 PM, Anton Khirnov wrote:
> > Quoting Michael Niedermayer (2024-03-10 20:21:47)
> >> On Sun, Mar 10, 2024 at 07:13:18AM +0100, Anton Khirnov wrote:
> >>> Quoting Michael Niedermayer (2024-03-10 04:36:29)
> >>>> On Fri, Mar 08, 2024 at 06:34:36AM +0100, Anton Khirnov wrote:
> >>>>> Quoting Michael Niedermayer (2024-03-07 21:37:39)
> >>>>>> On Wed, Mar 06, 2024 at 12:03:03PM +0100, Anton Khirnov wrote:
> >>>>>>> Treat it analogously to stream parameters like format/dimensions/etc.
> >>>>>>> This is functionally different from previous code in 2 ways:
> >>>>>>> * for non-CFR video, the frame timebase (set by the decoder) is used
> >>>>>>>    rather than the demuxer timebase
> >>>>>>> * for sub2video, AV_TIME_BASE_Q is used, which is hardcoded by the
> >>>>>>>    subtitle decoding API
> >>>>>>>
> >>>>>>> These changes should avoid unnecessary and potentially lossy timestamp
> >>>>>>> conversions from decoder timebase into the demuxer one.
> >>>>>>>
> >>>>>>> Changes the timebases used in sub2video tests.
> >>>>>>> ---
> >>>>>>>   fftools/ffmpeg_filter.c               |  17 ++-
> >>>>>>>   tests/ref/fate/sub2video_basic        | 182 +++++++++++++-------------
> >>>>>>>   tests/ref/fate/sub2video_time_limited |   8 +-
> >>>>>>>   3 files changed, 106 insertions(+), 101 deletions(-)
> >>>>>>
> >>>>>> breaks:
> >>>>>>
> >>>>>> ./ffmpeg -i \[a-s\]_full_metal_panic_fumoffu_-_01_-_the_man_from_the_south_-_a_hostage_with_no_compromises__rs2_\[1080p_bd-rip\]\[BBB48A25\].mkv  -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 -y a.avi
> >>>>>>
> >>>>>
> >>>>> Use a constant framerate.
> >>>>
> >>>> why not automatically choose a supported timebase  ?
> >>>>
> >>>> "[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"
> >>>
> >>> Because I don't want ffmpeg CLI to have codec-specific code for a codec
> >>> that's been obsolete for 15+ years. One could also potentially do it
> >>> inside the encoder itself, but it is nontrivial since the computations
> >>> are spread across a number of places in mpeg4videoenc.c and
> >>> mpegvideo_enc.c. And again, it seems like a waste of time - there is no
> >>> reason to encode mpeg4 today.
> >>
> >> This is not mpeg4 specific, its just a new additional case that fails
> > 
> > The case you reported is mpeg4 specific.
> > 
> >> ./ffmpeg -i mm-small.mpg test.dv
> >> [dvvideo @ 0x7f868800f100] Found no DV profile for 80x60 yuv420p video. Valid DV profiles are:
> > 
> > There is no mechanism for an encoder to export supported time bases.
> 
> Could it be added as an extension to AVProfile, or AVCodec?

The two cases are actually pretty different:
* mpeg4 has a constraint on the range of timebases, and actually does
  some perverted computations with the timestamps
* DV just needs your video to be CFR, with a list of supported
  framerates; dvenc should probably read AVCodecContext.framerate
  instead of time_base

But most importantly, is there an actual current use case for either of
those encoders? They have both been obsolete for close to two decades.
It seems silly to add new API that won't actually be useful to anyone.
Michael Niedermayer March 10, 2024, 11:37 p.m. UTC | #9
On Sun, Mar 10, 2024 at 11:49:12PM +0100, Anton Khirnov wrote:
> Quoting James Almer (2024-03-10 23:29:27)
> > On 3/10/2024 7:24 PM, Anton Khirnov wrote:
> > > Quoting Michael Niedermayer (2024-03-10 20:21:47)
> > >> On Sun, Mar 10, 2024 at 07:13:18AM +0100, Anton Khirnov wrote:
> > >>> Quoting Michael Niedermayer (2024-03-10 04:36:29)
> > >>>> On Fri, Mar 08, 2024 at 06:34:36AM +0100, Anton Khirnov wrote:
> > >>>>> Quoting Michael Niedermayer (2024-03-07 21:37:39)
> > >>>>>> On Wed, Mar 06, 2024 at 12:03:03PM +0100, Anton Khirnov wrote:
> > >>>>>>> Treat it analogously to stream parameters like format/dimensions/etc.
> > >>>>>>> This is functionally different from previous code in 2 ways:
> > >>>>>>> * for non-CFR video, the frame timebase (set by the decoder) is used
> > >>>>>>>    rather than the demuxer timebase
> > >>>>>>> * for sub2video, AV_TIME_BASE_Q is used, which is hardcoded by the
> > >>>>>>>    subtitle decoding API
> > >>>>>>>
> > >>>>>>> These changes should avoid unnecessary and potentially lossy timestamp
> > >>>>>>> conversions from decoder timebase into the demuxer one.
> > >>>>>>>
> > >>>>>>> Changes the timebases used in sub2video tests.
> > >>>>>>> ---
> > >>>>>>>   fftools/ffmpeg_filter.c               |  17 ++-
> > >>>>>>>   tests/ref/fate/sub2video_basic        | 182 +++++++++++++-------------
> > >>>>>>>   tests/ref/fate/sub2video_time_limited |   8 +-
> > >>>>>>>   3 files changed, 106 insertions(+), 101 deletions(-)
> > >>>>>>
> > >>>>>> breaks:
> > >>>>>>
> > >>>>>> ./ffmpeg -i \[a-s\]_full_metal_panic_fumoffu_-_01_-_the_man_from_the_south_-_a_hostage_with_no_compromises__rs2_\[1080p_bd-rip\]\[BBB48A25\].mkv  -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 -y a.avi
> > >>>>>>
> > >>>>>
> > >>>>> Use a constant framerate.
> > >>>>
> > >>>> why not automatically choose a supported timebase  ?
> > >>>>
> > >>>> "[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"
> > >>>
> > >>> Because I don't want ffmpeg CLI to have codec-specific code for a codec
> > >>> that's been obsolete for 15+ years. One could also potentially do it
> > >>> inside the encoder itself, but it is nontrivial since the computations
> > >>> are spread across a number of places in mpeg4videoenc.c and
> > >>> mpegvideo_enc.c. And again, it seems like a waste of time - there is no
> > >>> reason to encode mpeg4 today.
> > >>
> > >> This is not mpeg4 specific, its just a new additional case that fails
> > > 
> > > The case you reported is mpeg4 specific.
> > > 
> > >> ./ffmpeg -i mm-small.mpg test.dv
> > >> [dvvideo @ 0x7f868800f100] Found no DV profile for 80x60 yuv420p video. Valid DV profiles are:
> > > 
> > > There is no mechanism for an encoder to export supported time bases.
> > 
> > Could it be added as an extension to AVProfile, or AVCodec?
> 
> The two cases are actually pretty different:
> * mpeg4 has a constraint on the range of timebases, and actually does
>   some perverted computations with the timestamps
> * DV just needs your video to be CFR, with a list of supported
>   framerates; dvenc should probably read AVCodecContext.framerate
>   instead of time_base
> 
> But most importantly, is there an actual current use case for either of
> those encoders? They have both been obsolete for close to two decades.
> It seems silly to add new API that won't actually be useful to anyone.

iam not sugesting to add API specific to mpeg4, rather that maybe
it can be done as part of some more generic solution

thx

[...]
Anton Khirnov March 11, 2024, 6:03 a.m. UTC | #10
Quoting Michael Niedermayer (2024-03-11 00:37:07)
> > > >>> Because I don't want ffmpeg CLI to have codec-specific code for a codec
> > > >>> that's been obsolete for 15+ years. One could also potentially do it
> > > >>> inside the encoder itself, but it is nontrivial since the computations
> > > >>> are spread across a number of places in mpeg4videoenc.c and
> > > >>> mpegvideo_enc.c. And again, it seems like a waste of time - there is no
> > > >>> reason to encode mpeg4 today.
> > > >>
> > > >> This is not mpeg4 specific, its just a new additional case that fails
> > > > 
> > > > The case you reported is mpeg4 specific.
> > > > 
> > > >> ./ffmpeg -i mm-small.mpg test.dv
> > > >> [dvvideo @ 0x7f868800f100] Found no DV profile for 80x60 yuv420p video. Valid DV profiles are:
> > > > 
> > > > There is no mechanism for an encoder to export supported time bases.
> > > 
> > > Could it be added as an extension to AVProfile, or AVCodec?
> > 
> > The two cases are actually pretty different:
> > * mpeg4 has a constraint on the range of timebases, and actually does
> >   some perverted computations with the timestamps
> > * DV just needs your video to be CFR, with a list of supported
> >   framerates; dvenc should probably read AVCodecContext.framerate
> >   instead of time_base
> > 
> > But most importantly, is there an actual current use case for either of
> > those encoders? They have both been obsolete for close to two decades.
> > It seems silly to add new API that won't actually be useful to anyone.
> 
> iam not sugesting to add API specific to mpeg4, rather that maybe
> it can be done as part of some more generic solution

What kind of a solution? As I said above, I don't know of any other
encoders that have similar constraints.
Tobias Rapp March 11, 2024, 10:12 a.m. UTC | #11
On 10/03/2024 23:49, Anton Khirnov wrote:

> Quoting James Almer (2024-03-10 23:29:27)
>> On 3/10/2024 7:24 PM, Anton Khirnov wrote:
>>> Quoting Michael Niedermayer (2024-03-10 20:21:47)
>>>> On Sun, Mar 10, 2024 at 07:13:18AM +0100, Anton Khirnov wrote:
>>>>> Quoting Michael Niedermayer (2024-03-10 04:36:29)
>>>>>> why not automatically choose a supported timebase ?
>>>>>> "[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"
>>>>> Because I don't want ffmpeg CLI to have codec-specific code for a codec
>>>>> that's been obsolete for 15+ years. One could also potentially do it
>>>>> inside the encoder itself, but it is nontrivial since the computations
>>>>> are spread across a number of places in mpeg4videoenc.c and
>>>>> mpegvideo_enc.c. And again, it seems like a waste of time - there is no
>>>>> reason to encode mpeg4 today.
>>>> This is not mpeg4 specific, its just a new additional case that fails
>>> The case you reported is mpeg4 specific.
>>>
>>>> ./ffmpeg -i mm-small.mpg test.dv
>>>> [dvvideo @ 0x7f868800f100] Found no DV profile for 80x60 yuv420p video. Valid DV profiles are:
>>> There is no mechanism for an encoder to export supported time bases.
>> Could it be added as an extension to AVProfile, or AVCodec?
> The two cases are actually pretty different:
> * mpeg4 has a constraint on the range of timebases, and actually does
>    some perverted computations with the timestamps
> * DV just needs your video to be CFR, with a list of supported
>    framerates; dvenc should probably read AVCodecContext.framerate
>    instead of time_base
>
> But most importantly, is there an actual current use case for either of
> those encoders? They have both been obsolete for close to two decades.
> It seems silly to add new API that won't actually be useful to anyone.

Hardware doesn't get outdated as quickly as software. And there are 
people that do not switch their full environment to a new codec every 
decade just to be "in line".

Regards, Tobias
Anton Khirnov March 11, 2024, 12:23 p.m. UTC | #12
Quoting Tobias Rapp (2024-03-11 11:12:38)
> On 10/03/2024 23:49, Anton Khirnov wrote:
> 
> > Quoting James Almer (2024-03-10 23:29:27)
> >> On 3/10/2024 7:24 PM, Anton Khirnov wrote:
> >>> Quoting Michael Niedermayer (2024-03-10 20:21:47)
> >>>> On Sun, Mar 10, 2024 at 07:13:18AM +0100, Anton Khirnov wrote:
> >>>>> Quoting Michael Niedermayer (2024-03-10 04:36:29)
> >>>>>> why not automatically choose a supported timebase ?
> >>>>>> "[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"
> >>>>> Because I don't want ffmpeg CLI to have codec-specific code for a codec
> >>>>> that's been obsolete for 15+ years. One could also potentially do it
> >>>>> inside the encoder itself, but it is nontrivial since the computations
> >>>>> are spread across a number of places in mpeg4videoenc.c and
> >>>>> mpegvideo_enc.c. And again, it seems like a waste of time - there is no
> >>>>> reason to encode mpeg4 today.
> >>>> This is not mpeg4 specific, its just a new additional case that fails
> >>> The case you reported is mpeg4 specific.
> >>>
> >>>> ./ffmpeg -i mm-small.mpg test.dv
> >>>> [dvvideo @ 0x7f868800f100] Found no DV profile for 80x60 yuv420p video. Valid DV profiles are:
> >>> There is no mechanism for an encoder to export supported time bases.
> >> Could it be added as an extension to AVProfile, or AVCodec?
> > The two cases are actually pretty different:
> > * mpeg4 has a constraint on the range of timebases, and actually does
> >    some perverted computations with the timestamps
> > * DV just needs your video to be CFR, with a list of supported
> >    framerates; dvenc should probably read AVCodecContext.framerate
> >    instead of time_base
> >
> > But most importantly, is there an actual current use case for either of
> > those encoders? They have both been obsolete for close to two decades.
> > It seems silly to add new API that won't actually be useful to anyone.
> 
> Hardware doesn't get outdated as quickly as software. And there are 
> people that do not switch their full environment to a new codec every 
> decade just to be "in line".

And your point is...?
Martin Storsjö March 11, 2024, 12:29 p.m. UTC | #13
On Mon, 11 Mar 2024, Anton Khirnov wrote:

> Quoting Tobias Rapp (2024-03-11 11:12:38)
>> On 10/03/2024 23:49, Anton Khirnov wrote:
>>
>>> Quoting James Almer (2024-03-10 23:29:27)
>>>> On 3/10/2024 7:24 PM, Anton Khirnov wrote:
>>>>> Quoting Michael Niedermayer (2024-03-10 20:21:47)
>>>>>> On Sun, Mar 10, 2024 at 07:13:18AM +0100, Anton Khirnov wrote:
>>>>>>> Quoting Michael Niedermayer (2024-03-10 04:36:29)
>>>>>>>> why not automatically choose a supported timebase ?
>>>>>>>> "[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"
>>>>>>> Because I don't want ffmpeg CLI to have codec-specific code for a codec
>>>>>>> that's been obsolete for 15+ years. One could also potentially do it
>>>>>>> inside the encoder itself, but it is nontrivial since the computations
>>>>>>> are spread across a number of places in mpeg4videoenc.c and
>>>>>>> mpegvideo_enc.c. And again, it seems like a waste of time - there is no
>>>>>>> reason to encode mpeg4 today.
>>>>>> This is not mpeg4 specific, its just a new additional case that fails
>>>>> The case you reported is mpeg4 specific.
>>>>>
>>>>>> ./ffmpeg -i mm-small.mpg test.dv
>>>>>> [dvvideo @ 0x7f868800f100] Found no DV profile for 80x60 yuv420p video. Valid DV profiles are:
>>>>> There is no mechanism for an encoder to export supported time bases.
>>>> Could it be added as an extension to AVProfile, or AVCodec?
>>> The two cases are actually pretty different:
>>> * mpeg4 has a constraint on the range of timebases, and actually does
>>>    some perverted computations with the timestamps
>>> * DV just needs your video to be CFR, with a list of supported
>>>    framerates; dvenc should probably read AVCodecContext.framerate
>>>    instead of time_base
>>>
>>> But most importantly, is there an actual current use case for either of
>>> those encoders? They have both been obsolete for close to two decades.
>>> It seems silly to add new API that won't actually be useful to anyone.
>>
>> Hardware doesn't get outdated as quickly as software. And there are
>> people that do not switch their full environment to a new codec every
>> decade just to be "in line".
>
> And your point is...?

I think the point is, that one can't just dismiss that anybody would want 
to encode mpeg4 video any longer, even if it is obsolete. I also would 
like to keep being able to do that.

That said, I haven't followed the discussion closely enough about what to 
do with the time bases.

// Martin
Anton Khirnov March 11, 2024, 1:28 p.m. UTC | #14
Quoting Martin Storsjö (2024-03-11 13:29:15)
> On Mon, 11 Mar 2024, Anton Khirnov wrote:
> 
> > Quoting Tobias Rapp (2024-03-11 11:12:38)
> >> On 10/03/2024 23:49, Anton Khirnov wrote:
> >>
> >>> Quoting James Almer (2024-03-10 23:29:27)
> >>>> On 3/10/2024 7:24 PM, Anton Khirnov wrote:
> >>>>> Quoting Michael Niedermayer (2024-03-10 20:21:47)
> >>>>>> On Sun, Mar 10, 2024 at 07:13:18AM +0100, Anton Khirnov wrote:
> >>>>>>> Quoting Michael Niedermayer (2024-03-10 04:36:29)
> >>>>>>>> why not automatically choose a supported timebase ?
> >>>>>>>> "[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"
> >>>>>>> Because I don't want ffmpeg CLI to have codec-specific code for a codec
> >>>>>>> that's been obsolete for 15+ years. One could also potentially do it
> >>>>>>> inside the encoder itself, but it is nontrivial since the computations
> >>>>>>> are spread across a number of places in mpeg4videoenc.c and
> >>>>>>> mpegvideo_enc.c. And again, it seems like a waste of time - there is no
> >>>>>>> reason to encode mpeg4 today.
> >>>>>> This is not mpeg4 specific, its just a new additional case that fails
> >>>>> The case you reported is mpeg4 specific.
> >>>>>
> >>>>>> ./ffmpeg -i mm-small.mpg test.dv
> >>>>>> [dvvideo @ 0x7f868800f100] Found no DV profile for 80x60 yuv420p video. Valid DV profiles are:
> >>>>> There is no mechanism for an encoder to export supported time bases.
> >>>> Could it be added as an extension to AVProfile, or AVCodec?
> >>> The two cases are actually pretty different:
> >>> * mpeg4 has a constraint on the range of timebases, and actually does
> >>>    some perverted computations with the timestamps
> >>> * DV just needs your video to be CFR, with a list of supported
> >>>    framerates; dvenc should probably read AVCodecContext.framerate
> >>>    instead of time_base
> >>>
> >>> But most importantly, is there an actual current use case for either of
> >>> those encoders? They have both been obsolete for close to two decades.
> >>> It seems silly to add new API that won't actually be useful to anyone.
> >>
> >> Hardware doesn't get outdated as quickly as software. And there are
> >> people that do not switch their full environment to a new codec every
> >> decade just to be "in line".
> >
> > And your point is...?
> 
> I think the point is, that one can't just dismiss that anybody would want 
> to encode mpeg4 video any longer, even if it is obsolete. I also would 
> like to keep being able to do that.

That capability is not going away though, and I'm not arguing that it
should.

> That said, I haven't followed the discussion closely enough about what to 
> do with the time bases.

The only change is that in some rare cases the automatically selected
timebase no longer fits into mpeg4 constraints, so the user has to
specify either the framerate or the timebase explicitly.

Specifically, the commandline used by Michael involves the extremely
obscure case of converting subtitles to video (NOT harsubbing, but
really 1 sub -> 1 video). Since subtitle encoding API is hardcoded to
AV_TIME_BASE_Q, that timebase gets used for encoding, and the mpeg4
encoder rejects it. If it was hardsubbing (i.e. 1 video + 1 sub -> 1
video), the input video timebase should be used, which would probably
work.

I don't think it's that big of a deal to require users to specify the
timebase or framerate explicitly in such a sitation.
Inventing new APIs to cover it automagically seems like a waste of time,
unless somebody has actual (not potential) uses for this.
Martin Storsjö March 11, 2024, 2:03 p.m. UTC | #15
On Mon, 11 Mar 2024, Anton Khirnov wrote:

>> I think the point is, that one can't just dismiss that anybody would want 
>> to encode mpeg4 video any longer, even if it is obsolete. I also would 
>> like to keep being able to do that.
>
> That capability is not going away though, and I'm not arguing that it
> should.

Ok, good. The generally dismissive arguments about mpeg4 encoding being 
obsolete and something that nobody should be doing, could be interpreted 
in such a way.

>> That said, I haven't followed the discussion closely enough about what to 
>> do with the time bases.
>
> The only change is that in some rare cases the automatically selected
> timebase no longer fits into mpeg4 constraints, so the user has to
> specify either the framerate or the timebase explicitly.

Right, I see.

> Specifically, the commandline used by Michael involves the extremely
> obscure case of converting subtitles to video (NOT harsubbing, but
> really 1 sub -> 1 video). Since subtitle encoding API is hardcoded to
> AV_TIME_BASE_Q, that timebase gets used for encoding, and the mpeg4
> encoder rejects it. If it was hardsubbing (i.e. 1 video + 1 sub -> 1
> video), the input video timebase should be used, which would probably
> work.
>
> I don't think it's that big of a deal to require users to specify the
> timebase or framerate explicitly in such a sitation.
> Inventing new APIs to cover it automagically seems like a waste of time,
> unless somebody has actual (not potential) uses for this.

Right, I would agree with this. (If someone else would volunteer to add 
said API I would consider accepting it though.)

Is this a usecase that currently works, but would be go away by getting 
rid of codec specific code in the tools, or is it a nice-to-have new extra 
feature that is being requested?

// Martin
Tobias Rapp March 11, 2024, 2:31 p.m. UTC | #16
On 11/03/2024 14:28, Anton Khirnov wrote:

> Quoting Martin Storsjö (2024-03-11 13:29:15)
>> On Mon, 11 Mar 2024, Anton Khirnov wrote:
>>
>>> Quoting Tobias Rapp (2024-03-11 11:12:38)
>>>> On 10/03/2024 23:49, Anton Khirnov wrote:
>>>>
>>>>> Quoting James Almer (2024-03-10 23:29:27)
>>>>>> On 3/10/2024 7:24 PM, Anton Khirnov wrote:
>>>>>>> Quoting Michael Niedermayer (2024-03-10 20:21:47)
>>>>>>>> On Sun, Mar 10, 2024 at 07:13:18AM +0100, Anton Khirnov wrote:
>>>>>>>>> Quoting Michael Niedermayer (2024-03-10 04:36:29)
>>>>>>>>>> why not automatically choose a supported timebase ?
>>>>>>>>>> "[mpeg4 @ 0x55973c869f00] timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535"
>>>>>>>>> Because I don't want ffmpeg CLI to have codec-specific code for a codec
>>>>>>>>> that's been obsolete for 15+ years. One could also potentially do it
>>>>>>>>> inside the encoder itself, but it is nontrivial since the computations
>>>>>>>>> are spread across a number of places in mpeg4videoenc.c and
>>>>>>>>> mpegvideo_enc.c. And again, it seems like a waste of time - there is no
>>>>>>>>> reason to encode mpeg4 today.
>>>>>>>> This is not mpeg4 specific, its just a new additional case that fails
>>>>>>> The case you reported is mpeg4 specific.
>>>>>>>
>>>>>>>> ./ffmpeg -i mm-small.mpg test.dv
>>>>>>>> [dvvideo @ 0x7f868800f100] Found no DV profile for 80x60 yuv420p video. Valid DV profiles are:
>>>>>>> There is no mechanism for an encoder to export supported time bases.
>>>>>> Could it be added as an extension to AVProfile, or AVCodec?
>>>>> The two cases are actually pretty different:
>>>>> * mpeg4 has a constraint on the range of timebases, and actually does
>>>>>     some perverted computations with the timestamps
>>>>> * DV just needs your video to be CFR, with a list of supported
>>>>>     framerates; dvenc should probably read AVCodecContext.framerate
>>>>>     instead of time_base
>>>>>
>>>>> But most importantly, is there an actual current use case for either of
>>>>> those encoders? They have both been obsolete for close to two decades.
>>>>> It seems silly to add new API that won't actually be useful to anyone.
>>>> Hardware doesn't get outdated as quickly as software. And there are
>>>> people that do not switch their full environment to a new codec every
>>>> decade just to be "in line".
>>> And your point is...?
>> I think the point is, that one can't just dismiss that anybody would want
>> to encode mpeg4 video any longer, even if it is obsolete. I also would
>> like to keep being able to do that.
> That capability is not going away though, and I'm not arguing that it
> should.
>
>> That said, I haven't followed the discussion closely enough about what to
>> do with the time bases.
> The only change is that in some rare cases the automatically selected
> timebase no longer fits into mpeg4 constraints, so the user has to
> specify either the framerate or the timebase explicitly.
>
> [...]


If this is just about having to add some extra parametersin rare 
use-cases then its fine. I got the impression that encoding into MPEG4 
or DV was considered deprecated (or unimportant of additional 
consideration) in general.

Regards, Tobias
Anton Khirnov March 11, 2024, 2:32 p.m. UTC | #17
Quoting Martin Storsjö via ffmpeg-devel (2024-03-11 15:03:15)
> On Mon, 11 Mar 2024, Anton Khirnov wrote:
> 
> >> I think the point is, that one can't just dismiss that anybody would want 
> >> to encode mpeg4 video any longer, even if it is obsolete. I also would 
> >> like to keep being able to do that.
> >
> > That capability is not going away though, and I'm not arguing that it
> > should.
> 
> Ok, good. The generally dismissive arguments about mpeg4 encoding being 
> obsolete and something that nobody should be doing, could be interpreted 
> in such a way.

Well it IS obsolete. AFAIK it was never a particularly popular codec,
and was only really used by the anime and ripping scenes in early 2000s,
and even they dropped it very quickly once x264 appeared. While hardware
"divx players" did exist, I don't think they'd still be used today in
nontrivial numbers. So I don't see any reason to encode it today,
besides testing and academic/historic interest. And again - that does
not mean the capability should be removed, but it does mean that we
shouldn't insist on tuning it for the smoothest user experience, since
this time is then NOT spent doing something actually useful.

> > Specifically, the commandline used by Michael involves the extremely
> > obscure case of converting subtitles to video (NOT harsubbing, but
> > really 1 sub -> 1 video). Since subtitle encoding API is hardcoded to
> > AV_TIME_BASE_Q, that timebase gets used for encoding, and the mpeg4
> > encoder rejects it. If it was hardsubbing (i.e. 1 video + 1 sub -> 1
> > video), the input video timebase should be used, which would probably
> > work.
> >
> > I don't think it's that big of a deal to require users to specify the
> > timebase or framerate explicitly in such a sitation.
> > Inventing new APIs to cover it automagically seems like a waste of time,
> > unless somebody has actual (not potential) uses for this.
> 
> Right, I would agree with this. (If someone else would volunteer to add 
> said API I would consider accepting it though.)
> 
> Is this a usecase that currently works, but would be go away by getting 
> rid of codec specific code in the tools, or is it a nice-to-have new extra 
> feature that is being requested?

The patch is not removing codec-specific code, rather it is changing the
filtering timebase from the demuxer one to the decoder one (which is
needed in this set to avoid the assumption that a filter is necessarily
fed by a demuxer). The two should be the same in most cases, except
* sub2video, where the subtitle decoding API fixes the timebase to
  AV_TIME_BASE_Q
* there is a post-demuxing bsf that changes the timebase, such as setts
Martin Storsjö March 11, 2024, 2:37 p.m. UTC | #18
On Mon, 11 Mar 2024, Anton Khirnov wrote:

> Well it IS obsolete. AFAIK it was never a particularly popular codec,
> and was only really used by the anime and ripping scenes in early 2000s,
> and even they dropped it very quickly once x264 appeared.

Within the scene of mobile HW, they commonly had HW codecs for H263 and 
MPEG4 (or SW codecs), with many but not all also supporting H264. So for 
one specific generation of mobile devices, MPEG4 was the same level of 
lingua franca that H264 is today.

Obviously not a big use case today in nontrivial numbers of course, but 
it is an example of a "scene" where the codec did have a pretty broad 
adoption.

> And again - that does not mean the capability should be removed, but it 
> does mean that we shouldn't insist on tuning it for the smoothest user 
> experience, since this time is then NOT spent doing something actually 
> useful.

I guess that's true.

// Martin
diff mbox series

Patch

diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index 13c5065191..9d9762a599 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -148,6 +148,8 @@  typedef struct InputFilterPriv {
 
     // fallback parameters to use when no input is ever sent
     struct {
+        AVRational          time_base;
+
         int                 format;
 
         int                 width;
@@ -696,6 +698,8 @@  static int ifilter_bind_ist(InputFilter *ifilter, InputStream *ist)
            palettes for all rectangles are identical or compatible */
         ifp->format = AV_PIX_FMT_RGB32;
 
+        ifp->time_base = AV_TIME_BASE_Q;
+
         av_log(fgp, AV_LOG_VERBOSE, "sub2video: using %dx%d canvas\n",
                ifp->width, ifp->height);
     }
@@ -1469,7 +1473,6 @@  static int configure_input_video_filter(FilterGraph *fg, AVFilterGraph *graph,
     AVFilterContext *last_filter;
     const AVFilter *buffer_filt = avfilter_get_by_name("buffer");
     const AVPixFmtDescriptor *desc;
-    InputStream *ist = ifp->ist;
     AVRational fr = ifp->opts.framerate;
     AVRational sar;
     AVBPrint args;
@@ -1482,9 +1485,6 @@  static int configure_input_video_filter(FilterGraph *fg, AVFilterGraph *graph,
     if (ifp->type_src == AVMEDIA_TYPE_SUBTITLE)
         sub2video_prepare(ifp);
 
-    ifp->time_base = (ifp->opts.flags & IFILTER_FLAG_CFR) ?
-                     av_inv_q(ifp->opts.framerate) : ist->st->time_base;
-
     sar = ifp->sample_aspect_ratio;
     if(!sar.den)
         sar = (AVRational){0,1};
@@ -1575,8 +1575,6 @@  static int configure_input_audio_filter(FilterGraph *fg, AVFilterGraph *graph,
     char name[255];
     int ret, pad_idx = 0;
 
-    ifp->time_base = (AVRational){ 1, ifp->sample_rate };
-
     av_bprint_init(&args, 0, AV_BPRINT_SIZE_AUTOMATIC);
     av_bprintf(&args, "time_base=%d/%d:sample_rate=%d:sample_fmt=%s",
                ifp->time_base.num, ifp->time_base.den,
@@ -1804,6 +1802,8 @@  int ifilter_parameters_from_dec(InputFilter *ifilter, const AVCodecContext *dec)
 {
     InputFilterPriv *ifp = ifp_from_ifilter(ifilter);
 
+    ifp->fallback.time_base = dec->pkt_timebase;
+
     if (dec->codec_type == AVMEDIA_TYPE_VIDEO) {
         ifp->fallback.format                 = dec->pix_fmt;
         ifp->fallback.width                  = dec->width;
@@ -1835,6 +1835,10 @@  static int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *fr
     if (ret < 0)
         return ret;
 
+    ifp->time_base = (ifp->type == AVMEDIA_TYPE_AUDIO)    ? (AVRational){ 1, frame->sample_rate } :
+                     (ifp->opts.flags & IFILTER_FLAG_CFR) ? av_inv_q(ifp->opts.framerate)         :
+                     frame->time_base;
+
     ifp->format              = frame->format;
 
     ifp->width               = frame->width;
@@ -2524,6 +2528,7 @@  static int send_eof(FilterGraphThread *fgt, InputFilter *ifilter,
             ifp->sample_aspect_ratio    = ifp->fallback.sample_aspect_ratio;
             ifp->color_space            = ifp->fallback.color_space;
             ifp->color_range            = ifp->fallback.color_range;
+            ifp->time_base              = ifp->fallback.time_base;
 
             ret = av_channel_layout_copy(&ifp->ch_layout,
                                          &ifp->fallback.ch_layout);
diff --git a/tests/ref/fate/sub2video_basic b/tests/ref/fate/sub2video_basic
index a6eb1a34ea..2e4dcb625e 100644
--- a/tests/ref/fate/sub2video_basic
+++ b/tests/ref/fate/sub2video_basic
@@ -1,95 +1,95 @@ 
-#tb 0: 1/1000
+#tb 0: 1/1000000
 #media_type 0: video
 #codec_id 0: rawvideo
 #dimensions 0: 720x480
 #sar 0: 0/1
-0,     132499,     132499,        0,  1382400, 0x00000000
-0,     132499,     132499,        0,  1382400, 0x8c93c2ba
-0,     137459,     137459,        0,  1382400, 0x00000000
-0,     147355,     147355,        0,  1382400, 0xb02e32ca
-0,     152088,     152088,        0,  1382400, 0x00000000
-0,     180797,     180797,        0,  1382400, 0x83b71116
-0,     183357,     183357,        0,  1382400, 0x00000000
-0,     183433,     183433,        0,  1382400, 0x85547fd1
-0,     185799,     185799,        0,  1382400, 0x00000000
-0,     185909,     185909,        0,  1382400, 0x00000000
-0,     185910,     185910,        0,  1382400, 0xb6a8f181
-0,     188606,     188606,        0,  1382400, 0x00000000
-0,     188663,     188663,        0,  1382400, 0xb64d1a2c
-0,     189925,     189925,        0,  1382400, 0x00000000
-0,     190014,     190014,        0,  1382400, 0x7b37ecf3
-0,     191675,     191675,        0,  1382400, 0x00000000
-0,     199724,     199724,        0,  1382400, 0xdc025bd1
-0,     201089,     201089,        0,  1382400, 0x00000000
-0,     201175,     201175,        0,  1382400, 0x688b294d
-0,     202733,     202733,        0,  1382400, 0x00000000
-0,     202819,     202819,        0,  1382400, 0xa2b33d1b
-0,     204684,     204684,        0,  1382400, 0x00000000
-0,     204762,     204762,        0,  1382400, 0xb3e525e3
-0,     206730,     206730,        0,  1382400, 0x00000000
-0,     206806,     206806,        0,  1382400, 0xaa8fbdd7
-0,     208637,     208637,        0,  1382400, 0x00000000
-0,     208716,     208716,        0,  1382400, 0x7b7f26dd
-0,     209978,     209978,        0,  1382400, 0x00000000
-0,     210051,     210051,        0,  1382400, 0x15e2f836
-0,     211575,     211575,        0,  1382400, 0x00000000
-0,     211644,     211644,        0,  1382400, 0x0fee9b0c
-0,     214306,     214306,        0,  1382400, 0x00000000
-0,     214380,     214380,        0,  1382400, 0x89d62791
-0,     217144,     217144,        0,  1382400, 0x00000000
-0,     217225,     217225,        0,  1382400, 0xa6a9fd74
-0,     219591,     219591,        0,  1382400, 0x00000000
-0,     219652,     219652,        0,  1382400, 0x7896178d
-0,     221483,     221483,        0,  1382400, 0x00000000
-0,     223531,     223531,        0,  1382400, 0x01751a52
-0,     225863,     225863,        0,  1382400, 0x00000000
-0,     227510,     227510,        0,  1382400, 0xa3959c6f
-0,     230809,     230809,        0,  1382400, 0x00000000
-0,     230872,     230872,        0,  1382400, 0x3d3ea47b
-0,     233033,     233033,        0,  1382400, 0x00000000
-0,     233124,     233124,        0,  1382400, 0x593f8b24
-0,     237220,     237220,        0,  1382400, 0x00000000
-0,     237303,     237303,        0,  1382400, 0x171f05ba
-0,     240033,     240033,        0,  1382400, 0x00000000
-0,     240106,     240106,        0,  1382400, 0xb014cdf1
-0,     242165,     242165,        0,  1382400, 0x00000000
-0,     273556,     273556,        0,  1382400, 0xd918e667
-0,     275217,     275217,        0,  1382400, 0x00000000
-0,     295445,     295445,        0,  1382400, 0xc9406331
-0,     296776,     296776,        0,  1382400, 0x00000000
-0,     300049,     300049,        0,  1382400, 0xaf08b10d
-0,     301949,     301949,        0,  1382400, 0x00000000
-0,     302034,     302034,        0,  1382400, 0x00000000
-0,     302035,     302035,        0,  1382400, 0x853a9d93
-0,     303559,     303559,        0,  1382400, 0x00000000
-0,     304203,     304203,        0,  1382400, 0x7491a87d
-0,     305898,     305898,        0,  1382400, 0x00000000
-0,     305947,     305947,        0,  1382400, 0xf7383c58
-0,     307881,     307881,        0,  1382400, 0x00000000
-0,     307957,     307957,        0,  1382400, 0xe66be411
-0,     309720,     309720,        0,  1382400, 0x00000000
-0,     321295,     321295,        0,  1382400, 0xd6850362
-0,     323263,     323263,        0,  1382400, 0x00000000
-0,     323356,     323356,        0,  1382400, 0x3e1ed109
-0,     324584,     324584,        0,  1382400, 0x00000000
-0,     324640,     324640,        0,  1382400, 0x39c1b7bd
-0,     326403,     326403,        0,  1382400, 0x00000000
-0,     327193,     327193,        0,  1382400, 0x35b85f2e
-0,     328285,     328285,        0,  1382400, 0x00000000
-0,     328360,     328360,        0,  1382400, 0x00000000
-0,     328361,     328361,        0,  1382400, 0x83f103e5
-0,     329885,     329885,        0,  1382400, 0x00000000
-0,     329946,     329946,        0,  1382400, 0xbc1ca9b3
-0,     331106,     331106,        0,  1382400, 0x00000000
-0,     331230,     331230,        0,  1382400, 0x94d4a51e
-0,     332857,     332857,        0,  1382400, 0x00000000
-0,     332924,     332924,        0,  1382400, 0xf88cdfde
-0,     334687,     334687,        0,  1382400, 0x00000000
-0,     342600,     342600,        0,  1382400, 0xdd51423b
-0,     344431,     344431,        0,  1382400, 0x00000000
-0,     346771,     346771,        0,  1382400, 0x08259fa4
-0,     348329,     348329,        0,  1382400, 0x00000000
-0,     357640,     357640,        0,  1382400, 0x1663fa34
-0,     359767,     359767,        0,  1382400, 0x00000000
-0,     359834,     359834,        0,  1382400, 0xda2ceb55
-0,     361096,     361096,        0,  1382400, 0x00000000
+0,  132499000,  132499000,        0,  1382400, 0x00000000
+0,  132499000,  132499000,        0,  1382400, 0x8c93c2ba
+0,  137459000,  137459000,        0,  1382400, 0x00000000
+0,  147355000,  147355000,        0,  1382400, 0xb02e32ca
+0,  152088000,  152088000,        0,  1382400, 0x00000000
+0,  180797000,  180797000,        0,  1382400, 0x83b71116
+0,  183357000,  183357000,        0,  1382400, 0x00000000
+0,  183433000,  183433000,        0,  1382400, 0x85547fd1
+0,  185799000,  185799000,        0,  1382400, 0x00000000
+0,  185909999,  185909999,        0,  1382400, 0x00000000
+0,  185910000,  185910000,        0,  1382400, 0xb6a8f181
+0,  188606000,  188606000,        0,  1382400, 0x00000000
+0,  188663000,  188663000,        0,  1382400, 0xb64d1a2c
+0,  189925000,  189925000,        0,  1382400, 0x00000000
+0,  190014000,  190014000,        0,  1382400, 0x7b37ecf3
+0,  191675000,  191675000,        0,  1382400, 0x00000000
+0,  199724000,  199724000,        0,  1382400, 0xdc025bd1
+0,  201089000,  201089000,        0,  1382400, 0x00000000
+0,  201175000,  201175000,        0,  1382400, 0x688b294d
+0,  202733000,  202733000,        0,  1382400, 0x00000000
+0,  202819000,  202819000,        0,  1382400, 0xa2b33d1b
+0,  204684000,  204684000,        0,  1382400, 0x00000000
+0,  204762000,  204762000,        0,  1382400, 0xb3e525e3
+0,  206730000,  206730000,        0,  1382400, 0x00000000
+0,  206806000,  206806000,        0,  1382400, 0xaa8fbdd7
+0,  208637000,  208637000,        0,  1382400, 0x00000000
+0,  208716000,  208716000,        0,  1382400, 0x7b7f26dd
+0,  209978000,  209978000,        0,  1382400, 0x00000000
+0,  210051000,  210051000,        0,  1382400, 0x15e2f836
+0,  211575000,  211575000,        0,  1382400, 0x00000000
+0,  211644000,  211644000,        0,  1382400, 0x0fee9b0c
+0,  214306000,  214306000,        0,  1382400, 0x00000000
+0,  214380000,  214380000,        0,  1382400, 0x89d62791
+0,  217144000,  217144000,        0,  1382400, 0x00000000
+0,  217225000,  217225000,        0,  1382400, 0xa6a9fd74
+0,  219591000,  219591000,        0,  1382400, 0x00000000
+0,  219652000,  219652000,        0,  1382400, 0x7896178d
+0,  221483000,  221483000,        0,  1382400, 0x00000000
+0,  223531000,  223531000,        0,  1382400, 0x01751a52
+0,  225863000,  225863000,        0,  1382400, 0x00000000
+0,  227510000,  227510000,        0,  1382400, 0xa3959c6f
+0,  230809000,  230809000,        0,  1382400, 0x00000000
+0,  230872000,  230872000,        0,  1382400, 0x3d3ea47b
+0,  233033000,  233033000,        0,  1382400, 0x00000000
+0,  233124000,  233124000,        0,  1382400, 0x593f8b24
+0,  237220000,  237220000,        0,  1382400, 0x00000000
+0,  237303000,  237303000,        0,  1382400, 0x171f05ba
+0,  240033000,  240033000,        0,  1382400, 0x00000000
+0,  240106000,  240106000,        0,  1382400, 0xb014cdf1
+0,  242165000,  242165000,        0,  1382400, 0x00000000
+0,  273556000,  273556000,        0,  1382400, 0xd918e667
+0,  275217000,  275217000,        0,  1382400, 0x00000000
+0,  295445000,  295445000,        0,  1382400, 0xc9406331
+0,  296776000,  296776000,        0,  1382400, 0x00000000
+0,  300049000,  300049000,        0,  1382400, 0xaf08b10d
+0,  301949000,  301949000,        0,  1382400, 0x00000000
+0,  302034999,  302034999,        0,  1382400, 0x00000000
+0,  302035000,  302035000,        0,  1382400, 0x853a9d93
+0,  303559000,  303559000,        0,  1382400, 0x00000000
+0,  304203000,  304203000,        0,  1382400, 0x7491a87d
+0,  305898000,  305898000,        0,  1382400, 0x00000000
+0,  305947000,  305947000,        0,  1382400, 0xf7383c58
+0,  307881000,  307881000,        0,  1382400, 0x00000000
+0,  307957000,  307957000,        0,  1382400, 0xe66be411
+0,  309720000,  309720000,        0,  1382400, 0x00000000
+0,  321295000,  321295000,        0,  1382400, 0xd6850362
+0,  323263000,  323263000,        0,  1382400, 0x00000000
+0,  323356000,  323356000,        0,  1382400, 0x3e1ed109
+0,  324584000,  324584000,        0,  1382400, 0x00000000
+0,  324640000,  324640000,        0,  1382400, 0x39c1b7bd
+0,  326403000,  326403000,        0,  1382400, 0x00000000
+0,  327193000,  327193000,        0,  1382400, 0x35b85f2e
+0,  328285000,  328285000,        0,  1382400, 0x00000000
+0,  328360999,  328360999,        0,  1382400, 0x00000000
+0,  328361000,  328361000,        0,  1382400, 0x83f103e5
+0,  329885000,  329885000,        0,  1382400, 0x00000000
+0,  329946000,  329946000,        0,  1382400, 0xbc1ca9b3
+0,  331106000,  331106000,        0,  1382400, 0x00000000
+0,  331230000,  331230000,        0,  1382400, 0x94d4a51e
+0,  332857000,  332857000,        0,  1382400, 0x00000000
+0,  332924000,  332924000,        0,  1382400, 0xf88cdfde
+0,  334687000,  334687000,        0,  1382400, 0x00000000
+0,  342600000,  342600000,        0,  1382400, 0xdd51423b
+0,  344431000,  344431000,        0,  1382400, 0x00000000
+0,  346771000,  346771000,        0,  1382400, 0x08259fa4
+0,  348329000,  348329000,        0,  1382400, 0x00000000
+0,  357640000,  357640000,        0,  1382400, 0x1663fa34
+0,  359767000,  359767000,        0,  1382400, 0x00000000
+0,  359834000,  359834000,        0,  1382400, 0xda2ceb55
+0,  361096000,  361096000,        0,  1382400, 0x00000000
diff --git a/tests/ref/fate/sub2video_time_limited b/tests/ref/fate/sub2video_time_limited
index c7d48d639f..07d50fc697 100644
--- a/tests/ref/fate/sub2video_time_limited
+++ b/tests/ref/fate/sub2video_time_limited
@@ -1,8 +1,8 @@ 
-#tb 0: 1/90000
+#tb 0: 1/1000000
 #media_type 0: video
 #codec_id 0: rawvideo
 #dimensions 0: 1920x1080
 #sar 0: 0/1
-0,       6072,       6072,        0,  8294400, 0x00000000
-0,       6072,       6072,        0,  8294400, 0xa87c518f
-0,      36101,      36101,        0,  8294400, 0xa87c518f
+0,      67467,      67467,        0,  8294400, 0x00000000
+0,      67467,      67467,        0,  8294400, 0xa87c518f
+0,     401132,     401132,        0,  8294400, 0xa87c518f