| Message ID | CAPYw7P4XkouATRLt+YErrU3cKOFqCYYVB_RQRn6vpu7sJjzLaw@mail.gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel] frame durations for framesync | expand |
| Context | Check | Description |
|---|---|---|
| andriy/make_x86 | fail | Make failed |
Paul B Mahol (12023-01-27): > From b4f835c4ef6e0e0bbe6adef8235381e56f3f91df Mon Sep 17 00:00:00 2001 > From: Paul B Mahol <onemda@gmail.com> > Date: Fri, 27 Jan 2023 23:34:02 +0100 > Subject: [PATCH 1/4] avfilter/framesync: calculate frame duration too > > Signed-off-by: Paul B Mahol <onemda@gmail.com> > --- > libavfilter/framesync.c | 20 ++++++++++++++++---- > libavfilter/framesync.h | 15 +++++++++++++++ > 2 files changed, 31 insertions(+), 4 deletions(-) Looking into it right now, but I am pretty sure it cannot work. Regards,
Paul B Mahol (12023-01-27): > From b4f835c4ef6e0e0bbe6adef8235381e56f3f91df Mon Sep 17 00:00:00 2001 > From: Paul B Mahol <onemda@gmail.com> > Date: Fri, 27 Jan 2023 23:34:02 +0100 > Subject: [PATCH 1/4] avfilter/framesync: calculate frame duration too > > Signed-off-by: Paul B Mahol <onemda@gmail.com> > --- > libavfilter/framesync.c | 20 ++++++++++++++++---- > libavfilter/framesync.h | 15 +++++++++++++++ > 2 files changed, 31 insertions(+), 4 deletions(-) It does not work. For example, with the following filter graph: testsrc2=r=1/3[a];testsrc2=r=1/5[b];[a][b]vstack and adding a line of debug at the beginning of process_frame() in vf_stack.c, you get: got event: pts = 0, duration = 3 got event: pts = 3, duration = 3 ← should be 2 got event: pts = 5, duration = 5 ← should be 1 got event: pts = 6, duration = 3 … got event: pts = 9, duration = 3 got event: pts = 10, duration = 5 got event: pts = 12, duration = 3 got event: pts = 15, duration = 3 got event: pts = 18, duration = 3 got event: pts = 20, duration = 5 I was wrong saying it cannot work, you can compute the duration of the new frame without increasing the latency, but you have to duplicate the whole frame synchronization logic. I was against adding the duration field to libavfilter, and this is a very good illustration of the reason. The duration field is redundant with other information, it takes a lot of code to maintain it, and we cannot trust it anyway. For libavfilter, my advice would be to disregard it entirely. Let us just clear it in buffersrc and be done with it. Regards,
On 1/30/23, Nicolas George <george@nsup.org> wrote: > Paul B Mahol (12023-01-27): >> From b4f835c4ef6e0e0bbe6adef8235381e56f3f91df Mon Sep 17 00:00:00 2001 >> From: Paul B Mahol <onemda@gmail.com> >> Date: Fri, 27 Jan 2023 23:34:02 +0100 >> Subject: [PATCH 1/4] avfilter/framesync: calculate frame duration too >> >> Signed-off-by: Paul B Mahol <onemda@gmail.com> >> --- >> libavfilter/framesync.c | 20 ++++++++++++++++---- >> libavfilter/framesync.h | 15 +++++++++++++++ >> 2 files changed, 31 insertions(+), 4 deletions(-) > > It does not work. For example, with the following filter graph: > > testsrc2=r=1/3[a];testsrc2=r=1/5[b];[a][b]vstack > > and adding a line of debug at the beginning of process_frame() in > vf_stack.c, you get: > > got event: pts = 0, duration = 3 > got event: pts = 3, duration = 3 ← should be 2 > got event: pts = 5, duration = 5 ← should be 1 > got event: pts = 6, duration = 3 … > got event: pts = 9, duration = 3 > got event: pts = 10, duration = 5 > got event: pts = 12, duration = 3 > got event: pts = 15, duration = 3 > got event: pts = 18, duration = 3 > got event: pts = 20, duration = 5 > > I was wrong saying it cannot work, you can compute the duration of the > new frame without increasing the latency, but you have to duplicate the > whole frame synchronization logic. > > I was against adding the duration field to libavfilter, and this is a > very good illustration of the reason. The duration field is redundant > with other information, it takes a lot of code to maintain it, and we > cannot trust it anyway. It is not redundant, it describes how much frame lasts. There is no other way to derive it. > > For libavfilter, my advice would be to disregard it entirely. Let us > just clear it in buffersrc and be done with it. > I think you do not understand why frame duration is needed. > Regards, > > -- > Nicolas George > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Paul B Mahol (12023-01-30): > It is not redundant, it describes how much frame lasts. > There is no other way to derive it. The duration of a frame is equal to the difference between its timestamp and the timestamp of the next frame.
On 1/30/23, Nicolas George <george@nsup.org> wrote: > Paul B Mahol (12023-01-30): >> It is not redundant, it describes how much frame lasts. >> There is no other way to derive it. > > The duration of a frame is equal to the difference between its timestamp > and the timestamp of the next frame. Only in special cases.
Paul B Mahol (12023-01-30): > > The duration of a frame is equal to the difference between its timestamp > > and the timestamp of the next frame. > Only in special cases. No, in libavfilter, always; libavfilter does not support overlapping frames nor gaps. And if you want to make a point, you will have to explain your position with a lot more than four words.
On 1/30/23, Nicolas George <george@nsup.org> wrote: > Paul B Mahol (12023-01-30): >> > The duration of a frame is equal to the difference between its >> > timestamp >> > and the timestamp of the next frame. >> Only in special cases. > > No, in libavfilter, always; libavfilter does not support overlapping > frames nor gaps. And if you want to make a point, you will have to > explain your position with a lot more than four words. libavfilter support overlapping frames and frames with gaps, look into aresample filter and its async options. > > -- > Nicolas George > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Paul B Mahol (12023-01-30): > libavfilter support overlapping frames and frames with gaps, look into > aresample filter and its async options. aresample is a very special case. And for audio filters, the duration is carried by nb_samples, not duration.
On 1/30/23, Nicolas George <george@nsup.org> wrote: > Paul B Mahol (12023-01-30): >> libavfilter support overlapping frames and frames with gaps, look into >> aresample filter and its async options. > > aresample is a very special case. And for audio filters, the duration is > carried by nb_samples, not duration. similar relates to video filters. > > -- > Nicolas George > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
From 98ccb736ad5e3a4a65b8ea5840516cef47d3a9c5 Mon Sep 17 00:00:00 2001 From: Paul B Mahol <onemda@gmail.com> Date: Fri, 27 Jan 2023 23:38:38 +0100 Subject: [PATCH 4/4] avfilter/vf_xmedian: set output frame duration Signed-off-by: Paul B Mahol <onemda@gmail.com> --- libavfilter/vf_xmedian.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/libavfilter/vf_xmedian.c b/libavfilter/vf_xmedian.c index 7f7f3de12a..1d11231e61 100644 --- a/libavfilter/vf_xmedian.c +++ b/libavfilter/vf_xmedian.c @@ -221,6 +221,7 @@ static int process_frame(FFFrameSync *fs) if (!out) return AVERROR(ENOMEM); out->pts = av_rescale_q(s->fs.pts, s->fs.time_base, outlink->time_base); + out->duration = av_rescale_q(s->fs.duration, s->fs.time_base, outlink->time_base); if (!ctx->is_disabled) { td.in = in; @@ -426,6 +427,7 @@ static int tmedian_filter_frame(AVFilterLink *inlink, AVFrame *in) if (!out) return AVERROR(ENOMEM); out->pts = s->frames[0]->pts; + out->duration = s->frames[0]->duration; td.out = out; td.in = s->frames; -- 2.39.1