diff mbox series

[FFmpeg-devel,v2,1/2] avformat: add AVFormatContext.first_pkt_wallclock

Message ID 20220625082951.11897-1-ffmpeg@gyani.pro
State New
Headers show
Series [FFmpeg-devel,v2,1/2] avformat: add AVFormatContext.first_pkt_wallclock | expand

Checks

Context Check Description
yinshiyou/make_loongarch64 success Make finished
yinshiyou/make_fate_loongarch64 success Make fate finished
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished

Commit Message

Gyan Doshi June 25, 2022, 8:29 a.m. UTC 
Stores wallclock time for the first packet received.
Used for crude sync offset among inputs.
---
 doc/APIchanges         |  3 +++
 libavformat/avformat.h | 10 ++++++++++
 libavformat/demux.c    |  3 +++
 libavformat/options.c  |  1 +
 libavformat/version.h  |  2 +-
 5 files changed, 18 insertions(+), 1 deletion(-)

Comments

Anton Khirnov June 28, 2022, 5:13 a.m. UTC | #1 
Quoting Gyan Doshi (2022-06-25 10:29:50)
> Stores wallclock time for the first packet received.
> Used for crude sync offset among inputs.
> ---
>  doc/APIchanges         |  3 +++
>  libavformat/avformat.h | 10 ++++++++++
>  libavformat/demux.c    |  3 +++
>  libavformat/options.c  |  1 +
>  libavformat/version.h  |  2 +-
>  5 files changed, 18 insertions(+), 1 deletion(-)

Why should this be in the library? Seems to me this can be just as
easily done by the callers who need it.
Gyan Doshi June 28, 2022, 6:40 a.m. UTC | #2 
On 2022-06-28 10:43 am, Anton Khirnov wrote:
> Quoting Gyan Doshi (2022-06-25 10:29:50)
>> Stores wallclock time for the first packet received.
>> Used for crude sync offset among inputs.
>> ---
>>   doc/APIchanges         |  3 +++
>>   libavformat/avformat.h | 10 ++++++++++
>>   libavformat/demux.c    |  3 +++
>>   libavformat/options.c  |  1 +
>>   libavformat/version.h  |  2 +-
>>   5 files changed, 18 insertions(+), 1 deletion(-)
> Why should this be in the library? Seems to me this can be just as
> easily done by the callers who need it.

To not add some extra latency,  just like how 
`use_wallclock_as_timestamps` was implemented inside lavf.

Regards,
Gyan
Andreas Rheinhardt June 28, 2022, 7:50 a.m. UTC | #3 
Gyan Doshi:
> 
> 
> On 2022-06-28 10:43 am, Anton Khirnov wrote:
>> Quoting Gyan Doshi (2022-06-25 10:29:50)
>>> Stores wallclock time for the first packet received.
>>> Used for crude sync offset among inputs.
>>> ---
>>>   doc/APIchanges         |  3 +++
>>>   libavformat/avformat.h | 10 ++++++++++
>>>   libavformat/demux.c    |  3 +++
>>>   libavformat/options.c  |  1 +
>>>   libavformat/version.h  |  2 +-
>>>   5 files changed, 18 insertions(+), 1 deletion(-)
>> Why should this be in the library? Seems to me this can be just as
>> easily done by the callers who need it.
> 
> To not add some extra latency,  just like how
> `use_wallclock_as_timestamps` was implemented inside lavf.
> 

Why don't you use use_wallclock_as_timestamps and offset the timestamps
by the lowest returned timestamp?

- Andreas
Gyan Doshi June 28, 2022, 8:35 a.m. UTC | #4 
On 2022-06-28 01:20 pm, Andreas Rheinhardt wrote:
> Gyan Doshi:
>>
>> On 2022-06-28 10:43 am, Anton Khirnov wrote:
>>> Quoting Gyan Doshi (2022-06-25 10:29:50)
>>>> Stores wallclock time for the first packet received.
>>>> Used for crude sync offset among inputs.
>>>> ---
>>>>    doc/APIchanges         |  3 +++
>>>>    libavformat/avformat.h | 10 ++++++++++
>>>>    libavformat/demux.c    |  3 +++
>>>>    libavformat/options.c  |  1 +
>>>>    libavformat/version.h  |  2 +-
>>>>    5 files changed, 18 insertions(+), 1 deletion(-)
>>> Why should this be in the library? Seems to me this can be just as
>>> easily done by the callers who need it.
>> To not add some extra latency,  just like how
>> `use_wallclock_as_timestamps` was implemented inside lavf.
>>
> Why don't you use use_wallclock_as_timestamps and offset the timestamps
> by the lowest returned timestamp?

Because that will destroy the original inter-packet timing intervals. 
Those we usually wish to keep, especially for B-frames which are stored 
out of presentation order.

Regards,
Gyan
Anton Khirnov July 1, 2022, 9:50 a.m. UTC | #5 
Quoting Gyan Doshi (2022-06-28 08:40:58)
> 
> 
> On 2022-06-28 10:43 am, Anton Khirnov wrote:
> > Quoting Gyan Doshi (2022-06-25 10:29:50)
> >> Stores wallclock time for the first packet received.
> >> Used for crude sync offset among inputs.
> >> ---
> >>   doc/APIchanges         |  3 +++
> >>   libavformat/avformat.h | 10 ++++++++++
> >>   libavformat/demux.c    |  3 +++
> >>   libavformat/options.c  |  1 +
> >>   libavformat/version.h  |  2 +-
> >>   5 files changed, 18 insertions(+), 1 deletion(-)
> > Why should this be in the library? Seems to me this can be just as
> > easily done by the callers who need it.
> 
> To not add some extra latency,  just like how 
> `use_wallclock_as_timestamps` was implemented inside lavf.

Where would that extra latency come from?
I see no reason for use_wallclock_as_timestamps to exist either, it can
just as well be used from ffmpeg.c or whatever caller needs it.
Gyan Doshi July 1, 2022, 11:07 a.m. UTC | #6 
On 2022-07-01 03:20 pm, Anton Khirnov wrote:
> Quoting Gyan Doshi (2022-06-28 08:40:58)
>>
>> On 2022-06-28 10:43 am, Anton Khirnov wrote:
>>> Quoting Gyan Doshi (2022-06-25 10:29:50)
>>>> Stores wallclock time for the first packet received.
>>>> Used for crude sync offset among inputs.
>>>> ---
>>>>    doc/APIchanges         |  3 +++
>>>>    libavformat/avformat.h | 10 ++++++++++
>>>>    libavformat/demux.c    |  3 +++
>>>>    libavformat/options.c  |  1 +
>>>>    libavformat/version.h  |  2 +-
>>>>    5 files changed, 18 insertions(+), 1 deletion(-)
>>> Why should this be in the library? Seems to me this can be just as
>>> easily done by the callers who need it.
>> To not add some extra latency,  just like how
>> `use_wallclock_as_timestamps` was implemented inside lavf.
> Where would that extra latency come from?

The interval between its current assigment inside ff_read_packet() and 
the chance for assignment in process_input() in ffmpeg.c

Regards,
Gyan
Anton Khirnov July 2, 2022, 8:42 a.m. UTC | #7 
Quoting Gyan Doshi (2022-07-01 13:07:13)
> 
> 
> On 2022-07-01 03:20 pm, Anton Khirnov wrote:
> > Quoting Gyan Doshi (2022-06-28 08:40:58)
> >>
> >> On 2022-06-28 10:43 am, Anton Khirnov wrote:
> >>> Quoting Gyan Doshi (2022-06-25 10:29:50)
> >>>> Stores wallclock time for the first packet received.
> >>>> Used for crude sync offset among inputs.
> >>>> ---
> >>>>    doc/APIchanges         |  3 +++
> >>>>    libavformat/avformat.h | 10 ++++++++++
> >>>>    libavformat/demux.c    |  3 +++
> >>>>    libavformat/options.c  |  1 +
> >>>>    libavformat/version.h  |  2 +-
> >>>>    5 files changed, 18 insertions(+), 1 deletion(-)
> >>> Why should this be in the library? Seems to me this can be just as
> >>> easily done by the callers who need it.
> >> To not add some extra latency,  just like how
> >> `use_wallclock_as_timestamps` was implemented inside lavf.
> > Where would that extra latency come from?
> 
> The interval between its current assigment inside ff_read_packet() and 
> the chance for assignment in process_input() in ffmpeg.c

And why would there be a significant delay there?
Gyan Doshi July 2, 2022, 9:51 a.m. UTC | #8 
On 2022-07-02 02:12 pm, Anton Khirnov wrote:
> Quoting Gyan Doshi (2022-07-01 13:07:13)
>>
>> On 2022-07-01 03:20 pm, Anton Khirnov wrote:
>>> Quoting Gyan Doshi (2022-06-28 08:40:58)
>>>> On 2022-06-28 10:43 am, Anton Khirnov wrote:
>>>>> Quoting Gyan Doshi (2022-06-25 10:29:50)
>>>>>> Stores wallclock time for the first packet received.
>>>>>> Used for crude sync offset among inputs.
>>>>>> ---
>>>>>>     doc/APIchanges         |  3 +++
>>>>>>     libavformat/avformat.h | 10 ++++++++++
>>>>>>     libavformat/demux.c    |  3 +++
>>>>>>     libavformat/options.c  |  1 +
>>>>>>     libavformat/version.h  |  2 +-
>>>>>>     5 files changed, 18 insertions(+), 1 deletion(-)
>>>>> Why should this be in the library? Seems to me this can be just as
>>>>> easily done by the callers who need it.
>>>> To not add some extra latency,  just like how
>>>> `use_wallclock_as_timestamps` was implemented inside lavf.
>>> Where would that extra latency come from?
>> The interval between its current assigment inside ff_read_packet() and
>> the chance for assignment in process_input() in ffmpeg.c
> And why would there be a significant delay there?

With multiple inputs, get_input_packet_mt() will populate each input 
queue in async whereas process_input() will be called on that input at 
best in a round robin fashion depending on choose_output().

Regards,
Gyan
Anton Khirnov July 4, 2022, 4:42 a.m. UTC | #9 
Quoting Gyan Doshi (2022-07-02 11:51:49)
> 
> 
> On 2022-07-02 02:12 pm, Anton Khirnov wrote:
> > Quoting Gyan Doshi (2022-07-01 13:07:13)
> >>
> >> On 2022-07-01 03:20 pm, Anton Khirnov wrote:
> >>> Quoting Gyan Doshi (2022-06-28 08:40:58)
> >>>> On 2022-06-28 10:43 am, Anton Khirnov wrote:
> >>>>> Quoting Gyan Doshi (2022-06-25 10:29:50)
> >>>>>> Stores wallclock time for the first packet received.
> >>>>>> Used for crude sync offset among inputs.
> >>>>>> ---
> >>>>>>     doc/APIchanges         |  3 +++
> >>>>>>     libavformat/avformat.h | 10 ++++++++++
> >>>>>>     libavformat/demux.c    |  3 +++
> >>>>>>     libavformat/options.c  |  1 +
> >>>>>>     libavformat/version.h  |  2 +-
> >>>>>>     5 files changed, 18 insertions(+), 1 deletion(-)
> >>>>> Why should this be in the library? Seems to me this can be just as
> >>>>> easily done by the callers who need it.
> >>>> To not add some extra latency,  just like how
> >>>> `use_wallclock_as_timestamps` was implemented inside lavf.
> >>> Where would that extra latency come from?
> >> The interval between its current assigment inside ff_read_packet() and
> >> the chance for assignment in process_input() in ffmpeg.c
> > And why would there be a significant delay there?
> 
> With multiple inputs, get_input_packet_mt() will populate each input 
> queue in async whereas process_input() will be called on that input at 
> best in a round robin fashion depending on choose_output().

So why should this information not be set by get_input_packet_mt() then?
Gyan Doshi July 4, 2022, 5:23 a.m. UTC | #10 
On 2022-07-04 10:12 am, Anton Khirnov wrote:
> Quoting Gyan Doshi (2022-07-02 11:51:49)
>>
>> On 2022-07-02 02:12 pm, Anton Khirnov wrote:
>>> Quoting Gyan Doshi (2022-07-01 13:07:13)
>>>> On 2022-07-01 03:20 pm, Anton Khirnov wrote:
>>>>> Quoting Gyan Doshi (2022-06-28 08:40:58)
>>>>>> On 2022-06-28 10:43 am, Anton Khirnov wrote:
>>>>>>> Quoting Gyan Doshi (2022-06-25 10:29:50)
>>>>>>>> Stores wallclock time for the first packet received.
>>>>>>>> Used for crude sync offset among inputs.
>>>>>>>> ---
>>>>>>>>      doc/APIchanges         |  3 +++
>>>>>>>>      libavformat/avformat.h | 10 ++++++++++
>>>>>>>>      libavformat/demux.c    |  3 +++
>>>>>>>>      libavformat/options.c  |  1 +
>>>>>>>>      libavformat/version.h  |  2 +-
>>>>>>>>      5 files changed, 18 insertions(+), 1 deletion(-)
>>>>>>> Why should this be in the library? Seems to me this can be just as
>>>>>>> easily done by the callers who need it.
>>>>>> To not add some extra latency,  just like how
>>>>>> `use_wallclock_as_timestamps` was implemented inside lavf.
>>>>> Where would that extra latency come from?
>>>> The interval between its current assigment inside ff_read_packet() and
>>>> the chance for assignment in process_input() in ffmpeg.c
>>> And why would there be a significant delay there?
>> With multiple inputs, get_input_packet_mt() will populate each input
>> queue in async whereas process_input() will be called on that input at
>> best in a round robin fashion depending on choose_output().
> So why should this information not be set by get_input_packet_mt() then?

It can. I wanted to avoid all the latency I could. But I can shift it 
there. Will do.

Regards,
Gyan
diff mbox series

Patch

diff --git a/doc/APIchanges b/doc/APIchanges
index 20b944933a..94c53d283f 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,9 @@  libavutil:     2021-04-27
 
 API changes, most recent first:
 
+2022-06-xx - xxxxxxxxxx - lavf 59.26.100 - avformat.h
+  Add and set AVFormatContext.first_pkt_wallclock field.
+
 2022-06-12 - xxxxxxxxxx - lavf 59.25.100 - avio.h
   Add avio_vprintf(), similar to avio_printf() but allow to use it
   from within a function taking a variable argument list as input.
diff --git a/libavformat/avformat.h b/libavformat/avformat.h
index f12fa7d904..a45777ab39 100644
--- a/libavformat/avformat.h
+++ b/libavformat/avformat.h
@@ -1808,6 +1808,16 @@  typedef struct AVFormatContext {
      */
     int max_probe_packets;
 
+    /**
+     * Wallclock start time of the stream in real world time, in microseconds
+     * since the Unix epoch (00:00 1st January 1970). That is, the first packet
+     * was received at this real world time.
+     * - demuxing: Set by libavformat. Users may want to use start_time_realtime
+     *             if set.
+     * - muxing: unused.
+     */
+    int64_t first_pkt_wallclock;
+
     /**
      * A callback for closing the streams opened with AVFormatContext.io_open().
      *
diff --git a/libavformat/demux.c b/libavformat/demux.c
index e121253dfd..7e0540debf 100644
--- a/libavformat/demux.c
+++ b/libavformat/demux.c
@@ -628,6 +628,9 @@  FF_ENABLE_DEPRECATION_WARNINGS
 
         force_codec_ids(s, st);
 
+        if (s->first_pkt_wallclock == AV_NOPTS_VALUE)
+            s->first_pkt_wallclock = av_gettime();
+
         /* TODO: audio: time filter; video: frame reordering (pts != dts) */
         if (s->use_wallclock_as_timestamps)
             pkt->dts = pkt->pts = av_rescale_q(av_gettime(), AV_TIME_BASE_Q, st->time_base);
diff --git a/libavformat/options.c b/libavformat/options.c
index 0079a06d9a..77d4dfd1d0 100644
--- a/libavformat/options.c
+++ b/libavformat/options.c
@@ -184,6 +184,7 @@  AVFormatContext *avformat_alloc_context(void)
         return NULL;
     }
 
+    s->first_pkt_wallclock = AV_NOPTS_VALUE;
     si->shortest_end = AV_NOPTS_VALUE;
 
     return s;
diff --git a/libavformat/version.h b/libavformat/version.h
index 966ebb7ed3..0708d619c0 100644
--- a/libavformat/version.h
+++ b/libavformat/version.h
@@ -31,7 +31,7 @@ 
 
 #include "version_major.h"
 
-#define LIBAVFORMAT_VERSION_MINOR  25
+#define LIBAVFORMAT_VERSION_MINOR  26
 #define LIBAVFORMAT_VERSION_MICRO 100
 
 #define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \