[FFmpeg-devel] lavc: report frame field order in avctx

Submitted by Rodger Combs on Nov. 14, 2016, 11:19 p.m.

Details

Message ID 20161114231925.32943-1-rodger.combs@gmail.com
State New
Headers show

Commit Message

Rodger Combs Nov. 14, 2016, 11:19 p.m.
---
 libavcodec/utils.c                   | 13 +++++++++++++
 tests/api/api-codec-param-test.c     |  3 +++
 tests/fate/matroska.mak              |  2 +-
 tests/ref/fate/api-mjpeg-codec-param |  2 +-
 tests/ref/fate/api-png-codec-param   |  2 +-
 tests/ref/fate/mov-zombie            |  2 +-
 6 files changed, 20 insertions(+), 4 deletions(-)

Comments

Michael Niedermayer Nov. 15, 2016, 9:43 a.m.
On Mon, Nov 14, 2016 at 05:19:25PM -0600, Rodger Combs wrote:
> ---
>  libavcodec/utils.c                   | 13 +++++++++++++
>  tests/api/api-codec-param-test.c     |  3 +++
>  tests/fate/matroska.mak              |  2 +-
>  tests/ref/fate/api-mjpeg-codec-param |  2 +-
>  tests/ref/fate/api-png-codec-param   |  2 +-
>  tests/ref/fate/mov-zombie            |  2 +-
>  6 files changed, 20 insertions(+), 4 deletions(-)
> 
> diff --git a/libavcodec/utils.c b/libavcodec/utils.c
> index d6dca18..b9af880 100644
> --- a/libavcodec/utils.c
> +++ b/libavcodec/utils.c
> @@ -2296,6 +2296,12 @@ fail:
>                                                 guess_correct_pts(avctx,
>                                                                   picture->pts,
>                                                                   picture->pkt_dts));
> +
> +            if (avctx->field_order == AV_FIELD_UNKNOWN) {
> +                avctx->field_order = picture->interlaced_frame
> +                                   ? (picture->top_field_first ? AV_FIELD_TT : AV_FIELD_BB)
> +                                   : AV_FIELD_PROGRESSIVE;
> +            }
>          } else
>              av_frame_unref(picture);
>      } else
> @@ -2895,6 +2901,13 @@ int attribute_align_arg avcodec_receive_frame(AVCodecContext *avctx, AVFrame *fr
>                  av_frame_set_best_effort_timestamp(frame,
>                      guess_correct_pts(avctx, frame->pts, frame->pkt_dts));
>              }
> +
> +            if (avctx->field_order == AV_FIELD_UNKNOWN &&
> +                avctx->codec_type == AVMEDIA_TYPE_VIDEO) {
> +                avctx->field_order = frame->interlaced_frame
> +                                   ? (frame->top_field_first ? AV_FIELD_TT : AV_FIELD_BB)
> +                                   : AV_FIELD_PROGRESSIVE;
> +            }
>          }
>          return ret;
>      }

This doesnt leave any "unknown" option left.
What should a decoder do that knows the field order is unknown

[...]
Rodger Combs Nov. 17, 2016, 10:48 a.m.
> On Nov 15, 2016, at 03:43, Michael Niedermayer <michael@niedermayer.cc> wrote:
> 
> On Mon, Nov 14, 2016 at 05:19:25PM -0600, Rodger Combs wrote:
>> ---
>> libavcodec/utils.c                   | 13 +++++++++++++
>> tests/api/api-codec-param-test.c     |  3 +++
>> tests/fate/matroska.mak              |  2 +-
>> tests/ref/fate/api-mjpeg-codec-param |  2 +-
>> tests/ref/fate/api-png-codec-param   |  2 +-
>> tests/ref/fate/mov-zombie            |  2 +-
>> 6 files changed, 20 insertions(+), 4 deletions(-)
>> 
>> diff --git a/libavcodec/utils.c b/libavcodec/utils.c
>> index d6dca18..b9af880 100644
>> --- a/libavcodec/utils.c
>> +++ b/libavcodec/utils.c
>> @@ -2296,6 +2296,12 @@ fail:
>>                                                guess_correct_pts(avctx,
>>                                                                  picture->pts,
>>                                                                  picture->pkt_dts));
>> +
>> +            if (avctx->field_order == AV_FIELD_UNKNOWN) {
>> +                avctx->field_order = picture->interlaced_frame
>> +                                   ? (picture->top_field_first ? AV_FIELD_TT : AV_FIELD_BB)
>> +                                   : AV_FIELD_PROGRESSIVE;
>> +            }
>>         } else
>>             av_frame_unref(picture);
>>     } else
>> @@ -2895,6 +2901,13 @@ int attribute_align_arg avcodec_receive_frame(AVCodecContext *avctx, AVFrame *fr
>>                 av_frame_set_best_effort_timestamp(frame,
>>                     guess_correct_pts(avctx, frame->pts, frame->pkt_dts));
>>             }
>> +
>> +            if (avctx->field_order == AV_FIELD_UNKNOWN &&
>> +                avctx->codec_type == AVMEDIA_TYPE_VIDEO) {
>> +                avctx->field_order = frame->interlaced_frame
>> +                                   ? (frame->top_field_first ? AV_FIELD_TT : AV_FIELD_BB)
>> +                                   : AV_FIELD_PROGRESSIVE;
>> +            }
>>         }
>>         return ret;
>>     }
> 
> This doesnt leave any "unknown" option left.
> What should a decoder do that knows the field order is unknown

Is this a real case?

What I really need here is a stream-level "is interlaced" indicator; I've considered adding an AV_FIELD_INTERLACED_UNKNOWN value to indicate "This is interlaced, but I don't know the field order".
Thoughts?

> 
> [...]
> -- 
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> What does censorship reveal? It reveals fear. -- Julian Assange
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel <http://ffmpeg.org/mailman/listinfo/ffmpeg-devel>
Michael Niedermayer Nov. 18, 2016, 12:47 a.m.
On Thu, Nov 17, 2016 at 04:48:07AM -0600, Rodger Combs wrote:
> 
> > On Nov 15, 2016, at 03:43, Michael Niedermayer <michael@niedermayer.cc> wrote:
> > 
> > On Mon, Nov 14, 2016 at 05:19:25PM -0600, Rodger Combs wrote:
> >> ---
> >> libavcodec/utils.c                   | 13 +++++++++++++
> >> tests/api/api-codec-param-test.c     |  3 +++
> >> tests/fate/matroska.mak              |  2 +-
> >> tests/ref/fate/api-mjpeg-codec-param |  2 +-
> >> tests/ref/fate/api-png-codec-param   |  2 +-
> >> tests/ref/fate/mov-zombie            |  2 +-
> >> 6 files changed, 20 insertions(+), 4 deletions(-)
> >> 
> >> diff --git a/libavcodec/utils.c b/libavcodec/utils.c
> >> index d6dca18..b9af880 100644
> >> --- a/libavcodec/utils.c
> >> +++ b/libavcodec/utils.c
> >> @@ -2296,6 +2296,12 @@ fail:
> >>                                                guess_correct_pts(avctx,
> >>                                                                  picture->pts,
> >>                                                                  picture->pkt_dts));
> >> +
> >> +            if (avctx->field_order == AV_FIELD_UNKNOWN) {
> >> +                avctx->field_order = picture->interlaced_frame
> >> +                                   ? (picture->top_field_first ? AV_FIELD_TT : AV_FIELD_BB)
> >> +                                   : AV_FIELD_PROGRESSIVE;
> >> +            }
> >>         } else
> >>             av_frame_unref(picture);
> >>     } else
> >> @@ -2895,6 +2901,13 @@ int attribute_align_arg avcodec_receive_frame(AVCodecContext *avctx, AVFrame *fr
> >>                 av_frame_set_best_effort_timestamp(frame,
> >>                     guess_correct_pts(avctx, frame->pts, frame->pkt_dts));
> >>             }
> >> +
> >> +            if (avctx->field_order == AV_FIELD_UNKNOWN &&
> >> +                avctx->codec_type == AVMEDIA_TYPE_VIDEO) {
> >> +                avctx->field_order = frame->interlaced_frame
> >> +                                   ? (frame->top_field_first ? AV_FIELD_TT : AV_FIELD_BB)
> >> +                                   : AV_FIELD_PROGRESSIVE;
> >> +            }
> >>         }
> >>         return ret;
> >>     }
> > 
> > This doesnt leave any "unknown" option left.
> > What should a decoder do that knows the field order is unknown
> 
> Is this a real case?

any codec that doesnt have a interlaced flag couldnt set this
rawvideo and various lossless codecs can store interlaced material
without problems but lack such flags.


> 
> What I really need here is a stream-level "is interlaced" indicator; I've considered adding an AV_FIELD_INTERLACED_UNKNOWN value to indicate "This is interlaced, but I don't know the field order".
> Thoughts?

I think AVFrame lacks a unknown between progressive and interlaced

If you want a video filter to fill in only unknown ones that distinction
would be important


[...]
Rodger Combs Nov. 18, 2016, 10:28 a.m.
> On Nov 17, 2016, at 18:47, Michael Niedermayer <michael@niedermayer.cc> wrote:
> 
> On Thu, Nov 17, 2016 at 04:48:07AM -0600, Rodger Combs wrote:
>> 
>>> On Nov 15, 2016, at 03:43, Michael Niedermayer <michael@niedermayer.cc> wrote:
>>> 
>>> On Mon, Nov 14, 2016 at 05:19:25PM -0600, Rodger Combs wrote:
>>>> ---
>>>> libavcodec/utils.c                   | 13 +++++++++++++
>>>> tests/api/api-codec-param-test.c     |  3 +++
>>>> tests/fate/matroska.mak              |  2 +-
>>>> tests/ref/fate/api-mjpeg-codec-param |  2 +-
>>>> tests/ref/fate/api-png-codec-param   |  2 +-
>>>> tests/ref/fate/mov-zombie            |  2 +-
>>>> 6 files changed, 20 insertions(+), 4 deletions(-)
>>>> 
>>>> diff --git a/libavcodec/utils.c b/libavcodec/utils.c
>>>> index d6dca18..b9af880 100644
>>>> --- a/libavcodec/utils.c
>>>> +++ b/libavcodec/utils.c
>>>> @@ -2296,6 +2296,12 @@ fail:
>>>>                                               guess_correct_pts(avctx,
>>>>                                                                 picture->pts,
>>>>                                                                 picture->pkt_dts));
>>>> +
>>>> +            if (avctx->field_order == AV_FIELD_UNKNOWN) {
>>>> +                avctx->field_order = picture->interlaced_frame
>>>> +                                   ? (picture->top_field_first ? AV_FIELD_TT : AV_FIELD_BB)
>>>> +                                   : AV_FIELD_PROGRESSIVE;
>>>> +            }
>>>>        } else
>>>>            av_frame_unref(picture);
>>>>    } else
>>>> @@ -2895,6 +2901,13 @@ int attribute_align_arg avcodec_receive_frame(AVCodecContext *avctx, AVFrame *fr
>>>>                av_frame_set_best_effort_timestamp(frame,
>>>>                    guess_correct_pts(avctx, frame->pts, frame->pkt_dts));
>>>>            }
>>>> +
>>>> +            if (avctx->field_order == AV_FIELD_UNKNOWN &&
>>>> +                avctx->codec_type == AVMEDIA_TYPE_VIDEO) {
>>>> +                avctx->field_order = frame->interlaced_frame
>>>> +                                   ? (frame->top_field_first ? AV_FIELD_TT : AV_FIELD_BB)
>>>> +                                   : AV_FIELD_PROGRESSIVE;
>>>> +            }
>>>>        }
>>>>        return ret;
>>>>    }
>>> 
>>> This doesnt leave any "unknown" option left.
>>> What should a decoder do that knows the field order is unknown
>> 
>> Is this a real case?
> 
> any codec that doesnt have a interlaced flag couldnt set this
> rawvideo and various lossless codecs can store interlaced material
> without problems but lack such flags.

Eugh, but OK.

> 
> 
>> 
>> What I really need here is a stream-level "is interlaced" indicator; I've considered adding an AV_FIELD_INTERLACED_UNKNOWN value to indicate "This is interlaced, but I don't know the field order".
>> Thoughts?
> 
> I think AVFrame lacks a unknown between progressive and interlaced

Sorry, I meant "interlaced, but with unknown field order". I'd set that when `interlaced_frame` is set, and otherwise leave it as AV_FIELD_UNKNOWN. Does that sound reasonable?

> 
> If you want a video filter to fill in only unknown ones that distinction
> would be important
> 
> 
> [...]
> -- 
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> Breaking DRM is a little like attempting to break through a door even
> though the window is wide open and the only thing in the house is a bunch
> of things you dont want and which you would get tomorrow for free anyway
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel <http://ffmpeg.org/mailman/listinfo/ffmpeg-devel>

Patch hide | download patch | download mbox

diff --git a/libavcodec/utils.c b/libavcodec/utils.c
index d6dca18..b9af880 100644
--- a/libavcodec/utils.c
+++ b/libavcodec/utils.c
@@ -2296,6 +2296,12 @@  fail:
                                                guess_correct_pts(avctx,
                                                                  picture->pts,
                                                                  picture->pkt_dts));
+
+            if (avctx->field_order == AV_FIELD_UNKNOWN) {
+                avctx->field_order = picture->interlaced_frame
+                                   ? (picture->top_field_first ? AV_FIELD_TT : AV_FIELD_BB)
+                                   : AV_FIELD_PROGRESSIVE;
+            }
         } else
             av_frame_unref(picture);
     } else
@@ -2895,6 +2901,13 @@  int attribute_align_arg avcodec_receive_frame(AVCodecContext *avctx, AVFrame *fr
                 av_frame_set_best_effort_timestamp(frame,
                     guess_correct_pts(avctx, frame->pts, frame->pkt_dts));
             }
+
+            if (avctx->field_order == AV_FIELD_UNKNOWN &&
+                avctx->codec_type == AVMEDIA_TYPE_VIDEO) {
+                avctx->field_order = frame->interlaced_frame
+                                   ? (frame->top_field_first ? AV_FIELD_TT : AV_FIELD_BB)
+                                   : AV_FIELD_PROGRESSIVE;
+            }
         }
         return ret;
     }
diff --git a/tests/api/api-codec-param-test.c b/tests/api/api-codec-param-test.c
index 377a5e9..16ba8c6 100644
--- a/tests/api/api-codec-param-test.c
+++ b/tests/api/api-codec-param-test.c
@@ -211,6 +211,9 @@  static int check_video_streams(const AVFormatContext *fmt_ctx1, const AVFormatCo
             if (!strcmp(opt->name, "frame_number"))
                 continue;
 
+            if (!strcmp(opt->name, "field_order"))
+                continue;
+
             av_assert0(av_opt_get(codec_ctx1, opt->name, 0, &str1) >= 0);
             av_assert0(av_opt_get(codec_ctx2, opt->name, 0, &str2) >= 0);
             if (strcmp(str1, str2)) {
diff --git a/tests/fate/matroska.mak b/tests/fate/matroska.mak
index 35ed41f..16397b5 100644
--- a/tests/fate/matroska.mak
+++ b/tests/fate/matroska.mak
@@ -4,6 +4,6 @@ 
 FATE_MATROSKA-$(call DEMMUX, MATROSKA, MATROSKA) += fate-matroska-remux
 fate-matroska-remux: CMD = md5 -i $(TARGET_SAMPLES)/vp9-test-vectors/vp90-2-2pass-akiyo.webm -color_trc 4 -c:v copy -fflags +bitexact -strict -2 -f matroska
 fate-matroska-remux: CMP = oneline
-fate-matroska-remux: REF = 9b8398b42804ba12c39d2f47299a0996
+fate-matroska-remux: REF = cb6cc1cb581e31c98dd77173c7b4d221
 
 FATE_SAMPLES_AVCONV += $(FATE_MATROSKA-yes)
diff --git a/tests/ref/fate/api-mjpeg-codec-param b/tests/ref/fate/api-mjpeg-codec-param
index c67d1b1..c7e8da2 100644
--- a/tests/ref/fate/api-mjpeg-codec-param
+++ b/tests/ref/fate/api-mjpeg-codec-param
@@ -307,7 +307,7 @@  stream=0, decode=1
     refcounted_frames=false
     side_data_only_packets=true
     skip_alpha=false
-    field_order=0
+    field_order=1
     dump_separator=
     codec_whitelist=
     pixel_format=yuvj422p
diff --git a/tests/ref/fate/api-png-codec-param b/tests/ref/fate/api-png-codec-param
index bd53441..cba634e 100644
--- a/tests/ref/fate/api-png-codec-param
+++ b/tests/ref/fate/api-png-codec-param
@@ -307,7 +307,7 @@  stream=0, decode=1
     refcounted_frames=false
     side_data_only_packets=true
     skip_alpha=false
-    field_order=0
+    field_order=1
     dump_separator=
     codec_whitelist=
     pixel_format=rgba
diff --git a/tests/ref/fate/mov-zombie b/tests/ref/fate/mov-zombie
index 42e3a6f..c9cf7db 100644
--- a/tests/ref/fate/mov-zombie
+++ b/tests/ref/fate/mov-zombie
@@ -129,5 +129,5 @@  packet|codec_type=video|stream_index=0|pts=188623|pts_time=2.095811|dts=188622|d
 frame|media_type=video|stream_index=0|key_frame=0|pkt_pts=188623|pkt_pts_time=2.095811|pkt_dts=188622|pkt_dts_time=2.095800|best_effort_timestamp=188623|best_effort_timestamp_time=2.095811|pkt_duration=3003|pkt_duration_time=0.033367|pkt_pos=100846|pkt_size=974|width=160|height=240|pix_fmt=yuv420p|sample_aspect_ratio=2:1|pict_type=B|coded_picture_number=64|display_picture_number=0|interlaced_frame=0|top_field_first=0|repeat_pict=0
 packet|codec_type=video|stream_index=0|pts=197632|pts_time=2.195911|dts=191625|dts_time=2.129167|duration=3003|duration_time=0.033367|convergence_duration=N/A|convergence_duration_time=N/A|size=580|pos=101820|flags=__
 frame|media_type=video|stream_index=0|key_frame=0|pkt_pts=191626|pkt_pts_time=2.129178|pkt_dts=N/A|pkt_dts_time=N/A|best_effort_timestamp=191626|best_effort_timestamp_time=2.129178|pkt_duration=3003|pkt_duration_time=0.033367|pkt_pos=99180|pkt_size=1666|width=160|height=240|pix_fmt=yuv420p|sample_aspect_ratio=2:1|pict_type=P|coded_picture_number=63|display_picture_number=0|interlaced_frame=0|top_field_first=0|repeat_pict=0
-stream|index=0|codec_name=h264|profile=77|codec_type=video|codec_time_base=212521/12744000|codec_tag_string=avc1|codec_tag=0x31637661|width=160|height=240|coded_width=160|coded_height=240|has_b_frames=0|sample_aspect_ratio=2:1|display_aspect_ratio=4:3|pix_fmt=yuv420p|level=12|color_range=tv|color_space=smpte170m|color_transfer=bt709|color_primaries=smpte170m|chroma_location=topleft|field_order=unknown|timecode=N/A|refs=2|is_avc=true|nal_length_size=4|id=N/A|r_frame_rate=30000/1001|avg_frame_rate=6372000/212521|time_base=1/90000|start_pts=0|start_time=0.000000|duration_ts=2125200|duration=23.613333|bit_rate=333874|max_bit_rate=N/A|bits_per_raw_sample=8|nb_frames=708|nb_read_frames=65|nb_read_packets=66|disposition:default=1|disposition:dub=0|disposition:original=0|disposition:comment=0|disposition:lyrics=0|disposition:karaoke=0|disposition:forced=0|disposition:hearing_impaired=0|disposition:visual_impaired=0|disposition:clean_effects=0|disposition:attached_pic=0|disposition:timed_thumbnails=0|tag:rotate=0|tag:creation_time=2008-05-12T20:59:27.000000Z|tag:language=eng|tag:handler_name=Apple Alias Data Handler|tag:encoder=H.264
+stream|index=0|codec_name=h264|profile=77|codec_type=video|codec_time_base=212521/12744000|codec_tag_string=avc1|codec_tag=0x31637661|width=160|height=240|coded_width=160|coded_height=240|has_b_frames=0|sample_aspect_ratio=2:1|display_aspect_ratio=4:3|pix_fmt=yuv420p|level=12|color_range=tv|color_space=smpte170m|color_transfer=bt709|color_primaries=smpte170m|chroma_location=topleft|field_order=progressive|timecode=N/A|refs=2|is_avc=true|nal_length_size=4|id=N/A|r_frame_rate=30000/1001|avg_frame_rate=6372000/212521|time_base=1/90000|start_pts=0|start_time=0.000000|duration_ts=2125200|duration=23.613333|bit_rate=333874|max_bit_rate=N/A|bits_per_raw_sample=8|nb_frames=708|nb_read_frames=65|nb_read_packets=66|disposition:default=1|disposition:dub=0|disposition:original=0|disposition:comment=0|disposition:lyrics=0|disposition:karaoke=0|disposition:forced=0|disposition:hearing_impaired=0|disposition:visual_impaired=0|disposition:clean_effects=0|disposition:attached_pic=0|disposition:timed_thumbnails=0|tag:rotate=0|tag:creation_time=2008-05-12T20:59:27.000000Z|tag:language=eng|tag:handler_name=Apple Alias Data Handler|tag:encoder=H.264
 side_data|side_data_type=Display Matrix|side_data_size=36|displaymatrix=\n00000000:       131072           0           0\n00000001:            0       65536           0\n00000002:            0           0  1073741824\n|rotation=0