| Message ID | 20240709190547.99246-1-joshua.allmann@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI | expand |
| Context | Check | Description |
|---|---|---|
| yinshiyou/make_loongarch64 | success | Make finished |
| yinshiyou/make_fate_loongarch64 | fail | Make fate failed |
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | success | Make fate finished |
On Tue, 9 Jul 2024 at 12:06, Josh Allmann <joshua.allmann@gmail.com> wrote: > > Encoders may emit a buffering period SEI without a corresponding > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > During Annex B conversion, this may result in the SPS/PPS being > inserted *after* the buffering period SEI but before the IDR NAL. > > Since the buffering period SEI references the SPS, the SPS/PPS > needs to come first. > --- > > Notes: > v2: Updated FATE test refs > > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > 4 files changed, 29 insertions(+), 16 deletions(-) > Ping for review. Looking at the FATE output, this patch fixes a number of things - see [1] for details [1] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-July/330964.html > diff --git a/libavcodec/bsf/h264_mp4toannexb.c b/libavcodec/bsf/h264_mp4toannexb.c > index 92af6a6881..6607f1e91a 100644 > --- a/libavcodec/bsf/h264_mp4toannexb.c > +++ b/libavcodec/bsf/h264_mp4toannexb.c > @@ -363,6 +363,19 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) > if (!new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80)) > new_idr = 1; > > + /* If this is a buffering period SEI without a corresponding sps/pps > + * then prepend any existing sps/pps before the SEI */ > + if (unit_type == H264_NAL_SEI && buf[1] == 0 && !sps_seen && !pps_seen) { > + if (s->sps_size) { > + count_or_copy(&out, &out_size, s->sps, s->sps_size, PS_OUT_OF_BAND, j); > + sps_seen = 1; > + } > + if (s->pps_size) { > + count_or_copy(&out, &out_size, s->pps, s->pps_size, PS_OUT_OF_BAND, j); > + pps_seen = 1; > + } > + } > + > /* prepend only to the first type 5 NAL unit of an IDR picture, if no sps/pps are already present */ > if (new_idr && unit_type == H264_NAL_IDR_SLICE && !sps_seen && !pps_seen) { > if (s->sps_size) > diff --git a/tests/ref/fate/h264-bsf-mp4toannexb b/tests/ref/fate/h264-bsf-mp4toannexb > index 2049f39701..81ff568f3d 100644 > --- a/tests/ref/fate/h264-bsf-mp4toannexb > +++ b/tests/ref/fate/h264-bsf-mp4toannexb > @@ -1 +1 @@ > -5f04c27cc6ee8625fe2405fb0f7da9a3 > +ff2551123909f54c382294baa1bb4364 > diff --git a/tests/ref/fate/h264_mp4toannexb_ticket2991 b/tests/ref/fate/h264_mp4toannexb_ticket2991 > index f8e3e920d4..9a1fbf2f8c 100644 > --- a/tests/ref/fate/h264_mp4toannexb_ticket2991 > +++ b/tests/ref/fate/h264_mp4toannexb_ticket2991 > @@ -1,4 +1,4 @@ > -05d66e60ab22ee004720e0051af0fe74 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 > +b6ff5910928ad0b2a7eec481dcc41594 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 > 1985815 tests/data/fate/h264_mp4toannexb_ticket2991.h264 > #extradata 0: 47, 0x3a590d55 > #tb 0: 1/1200000 > @@ -6,7 +6,7 @@ > #codec_id 0: h264 > #dimensions 0: 1280x720 > #sar 0: 3/4 > -0, 0, 0, 40040, 37126, 0xb020184c > +0, 0, 0, 40040, 37126, 0x515c184c > 0, 40040, 40040, 40040, 6920, 0x8512361a, F=0x0 > 0, 80081, 80081, 40040, 7550, 0x1bc56ed4, F=0x0 > 0, 120121, 120121, 40040, 8752, 0xb8c6f0a1, F=0x0 > @@ -21,7 +21,7 @@ > 0, 480485, 480485, 40040, 11234, 0x83cbd9fd, F=0x0 > 0, 520525, 520525, 40040, 17616, 0xfdf95104, F=0x0 > 0, 560566, 560566, 40040, 10689, 0x9633d32b, F=0x0 > -0, 600606, 600606, 40040, 45291, 0x543c2cf6 > +0, 600606, 600606, 40040, 45291, 0xa8292cf6 > 0, 640646, 640646, 40040, 20837, 0x051abfab, F=0x0 > 0, 680687, 680687, 40040, 21418, 0xe2a59d70, F=0x0 > 0, 720727, 720727, 40040, 15643, 0x15cf2cec, F=0x0 > @@ -36,7 +36,7 @@ > 0, 1081091, 1081091, 40040, 13130, 0xcbb6bb8e, F=0x0 > 0, 1121131, 1121131, 40040, 16180, 0x5d188a7a, F=0x0 > 0, 1161172, 1161172, 40040, 14961, 0x9ff2f463, F=0x0 > -0, 1201212, 1201212, 40040, 54296, 0xe6ec30ed > +0, 1201212, 1201212, 40040, 54296, 0x3ae830ed > 0, 1241252, 1241252, 40040, 11500, 0x8c4852c9, F=0x0 > 0, 1281293, 1281293, 40040, 12065, 0xfb7954c3, F=0x0 > 0, 1321333, 1321333, 40040, 12532, 0xf0a935d3, F=0x0 > @@ -51,7 +51,7 @@ > 0, 1681697, 1681697, 40040, 13250, 0xfed0deb8, F=0x0 > 0, 1721737, 1721737, 40040, 13360, 0xbf92d476, F=0x0 > 0, 1761778, 1761778, 40040, 11749, 0x3041eaf1, F=0x0 > -0, 1801818, 1801818, 40040, 23997, 0xdbe6d5c4 > +0, 1801818, 1801818, 40040, 23997, 0x2fe2d5c4 > 0, 1841858, 1841858, 40040, 16065, 0xe8f715b7, F=0x0 > 0, 1881899, 1881899, 40040, 16441, 0x0a4e060f, F=0x0 > 0, 1921939, 1921939, 40040, 17395, 0xa8edecc2, F=0x0 > @@ -66,7 +66,7 @@ > 0, 2282303, 2282303, 40040, 13748, 0xed26aeb4, F=0x0 > 0, 2322343, 2322343, 40040, 15092, 0x3c983538, F=0x0 > 0, 2362384, 2362384, 40040, 14636, 0x9b278a6c, F=0x0 > -0, 2402424, 2402424, 40040, 29134, 0xf784be18 > +0, 2402424, 2402424, 40040, 29134, 0x4b80be18 > 0, 2442464, 2442464, 40040, 10232, 0x5408e15b, F=0x0 > 0, 2482505, 2482505, 40040, 9769, 0xc93cb7f9, F=0x0 > 0, 2522545, 2522545, 40040, 14454, 0x45230dbe, F=0x0 > @@ -81,7 +81,7 @@ > 0, 2882909, 2882909, 40040, 14801, 0x40bae016, F=0x0 > 0, 2922949, 2922949, 40040, 17303, 0x9ce1fd31, F=0x0 > 0, 2962990, 2962990, 40040, 17678, 0x9bd66141, F=0x0 > -0, 3003030, 3003030, 40040, 48672, 0x3215ce46 > +0, 3003030, 3003030, 40040, 48672, 0x8602ce46 > 0, 3043070, 3043070, 40040, 11894, 0x12e1fece, F=0x0 > 0, 3083111, 3083111, 40040, 16514, 0xc57aed05, F=0x0 > 0, 3123151, 3123151, 40040, 13044, 0x61914fa0, F=0x0 > @@ -96,7 +96,7 @@ > 0, 3483515, 3483515, 40040, 12208, 0x81a587c0, F=0x0 > 0, 3523555, 3523555, 40040, 14709, 0x5dffbe04, F=0x0 > 0, 3563596, 3563596, 40040, 14390, 0xbfd1e041, F=0x0 > -0, 3603636, 3603636, 40040, 37236, 0xe7f924b1 > +0, 3603636, 3603636, 40040, 37236, 0x3bf524b1 > 0, 3643676, 3643676, 40040, 14056, 0x24714c7c, F=0x0 > 0, 3683717, 3683717, 40040, 19438, 0x0c50dcd5, F=0x0 > 0, 3723757, 3723757, 40040, 21728, 0x7eea4a11, F=0x0 > @@ -111,7 +111,7 @@ > 0, 4084121, 4084121, 40040, 16878, 0x98efbae2, F=0x0 > 0, 4124161, 4124161, 40040, 14685, 0x1bf78d65, F=0x0 > 0, 4164202, 4164202, 40040, 13127, 0x0b91881d, F=0x0 > -0, 4204242, 4204242, 40040, 29390, 0xf6a5ed6b > +0, 4204242, 4204242, 40040, 29390, 0x4aa1ed6b > 0, 4244282, 4244282, 40040, 12576, 0xe9845ded, F=0x0 > 0, 4284323, 4284323, 40040, 12599, 0x96a79ab8, F=0x0 > 0, 4324363, 4324363, 40040, 16134, 0xb4c36d3f, F=0x0 > diff --git a/tests/ref/fate/segment-mp4-to-ts b/tests/ref/fate/segment-mp4-to-ts > index 2994416270..54b59a09cc 100644 > --- a/tests/ref/fate/segment-mp4-to-ts > +++ b/tests/ref/fate/segment-mp4-to-ts > @@ -4,7 +4,7 @@ > #codec_id 0: h264 > #dimensions 0: 640x360 > #sar 0: 1/1 > -0, -7200, 0, 3600, 22630, 0x9b109541, S=1, 1 > +0, -7200, 0, 3600, 22630, 0xee579541, S=1, 1 > 0, -3600, 14400, 3600, 4021, 0xbf7cdb02, F=0x0, S=1, 1 > 0, 0, 7200, 3600, 1096, 0x4f162690, F=0x0, S=1, 1 > 0, 3600, 3600, 3600, 687, 0x00394b95, F=0x0, S=1, 1 > @@ -25,7 +25,7 @@ > 0, 57600, 64800, 3600, 1182, 0xbe1a4847, F=0x0, S=1, 1 > 0, 61200, 61200, 3600, 809, 0x8d948a4e, F=0x0, S=1, 1 > 0, 64800, 68400, 3600, 656, 0x4fa03c2b, F=0x0, S=1, 1 > -0, 68400, 86400, 3600, 26555, 0x5629b584, S=1, 1 > +0, 68400, 86400, 3600, 26606, 0x0f75c37d, S=1, 1 > 0, 72000, 79200, 3600, 1141, 0x761b31e8, F=0x0, S=1, 1 > 0, 75600, 75600, 3600, 717, 0x57746351, F=0x0, S=1, 1 > 0, 79200, 82800, 3600, 693, 0x78b24263, F=0x0, S=1, 1 > @@ -49,7 +49,7 @@ > 0, 144000, 151200, 3600, 1271, 0x46006870, F=0x0, S=1, 1 > 0, 147600, 147600, 3600, 849, 0x94dc99c7, F=0x2, S=1, 1 > 0, 151200, 154800, 3600, 753, 0xf4236cab, F=0x0, S=1, 1 > -0, 154800, 172800, 3600, 25825, 0xd5464dee, S=1, 1 > +0, 154800, 172800, 3600, 25876, 0xb4305be7, S=1, 1 > 0, 158400, 165600, 3600, 1206, 0x8ce84344, F=0x0, S=1, 1 > 0, 162000, 162000, 3600, 867, 0x312fa07d, F=0x0, S=1, 1 > 0, 165600, 169200, 3600, 719, 0x810666d1, F=0x0, S=1, 1 > @@ -73,7 +73,7 @@ > 0, 230400, 237600, 3600, 1545, 0x0099fc98, F=0x0, S=1, 1 > 0, 234000, 234000, 3600, 929, 0xfd72d049, F=0x2, S=1, 1 > 0, 237600, 241200, 3600, 829, 0xcfda9e96, F=0x0, S=1, 1 > -0, 241200, 259200, 3600, 24220, 0x5ca21d71, S=1, 1 > +0, 241200, 259200, 3600, 24271, 0x9c472b6a, S=1, 1 > 0, 244800, 252000, 3600, 1422, 0xcde6cc34, F=0x0, S=1, 1 > 0, 248400, 248400, 3600, 883, 0xedacbe25, F=0x0, S=1, 1 > 0, 252000, 255600, 3600, 768, 0x89d774bc, F=0x0, S=1, 1 > @@ -97,7 +97,7 @@ > 0, 316800, 324000, 3600, 1501, 0xb3b8f001, F=0x0, S=1, 1 > 0, 320400, 320400, 3600, 941, 0x92b0cb18, F=0x2, S=1, 1 > 0, 324000, 327600, 3600, 823, 0x3d548355, F=0x0, S=1, 1 > -0, 327600, 345600, 3600, 24042, 0x441e94fb, S=1, 1 > +0, 327600, 345600, 3600, 24093, 0xcc0ba2f4, S=1, 1 > 0, 331200, 338400, 3600, 1582, 0x4f5d1049, F=0x0, S=1, 1 > 0, 334800, 334800, 3600, 945, 0x4f3cc9e8, F=0x0, S=1, 1 > 0, 338400, 342000, 3600, 815, 0x0ca790a4, F=0x0, S=1, 1 > @@ -121,7 +121,7 @@ > 0, 403200, 410400, 3600, 359, 0x11bdae52, F=0x0, S=1, 1 > 0, 406800, 406800, 3600, 235, 0xbec26964, F=0x2, S=1, 1 > 0, 410400, 414000, 3600, 221, 0x8380682c, F=0x0, S=1, 1 > -0, 414000, 432000, 3600, 22588, 0xf0ecf072, S=1, 1 > +0, 414000, 432000, 3600, 22639, 0x1809fe6b, S=1, 1 > 0, 417600, 424800, 3600, 383, 0x4f3bb571, F=0x0, S=1, 1 > 0, 421200, 421200, 3600, 257, 0x22e87802, F=0x0, S=1, 1 > 0, 424800, 428400, 3600, 261, 0xdb988134, F=0x0, S=1, 1 > -- > 2.39.2 >
On Mon, 15 Jul 2024 at 10:48, Josh Allmann <joshua.allmann@gmail.com> wrote: > > On Tue, 9 Jul 2024 at 12:06, Josh Allmann <joshua.allmann@gmail.com> wrote: > > > > Encoders may emit a buffering period SEI without a corresponding > > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > > > During Annex B conversion, this may result in the SPS/PPS being > > inserted *after* the buffering period SEI but before the IDR NAL. > > > > Since the buffering period SEI references the SPS, the SPS/PPS > > needs to come first. > > --- > > > > Notes: > > v2: Updated FATE test refs > > > > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > > 4 files changed, 29 insertions(+), 16 deletions(-) > > > > Ping for review. Looking at the FATE output, this patch fixes a number > of things - see [1] for details > > [1] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-July/330964.html Pasted the wrong link - please see this for a review of the FATE test changes https://ffmpeg.org//pipermail/ffmpeg-devel/2024-July/330912.html
On Tue, 9 Jul 2024 at 12:06, Josh Allmann <joshua.allmann@gmail.com> wrote: > > Encoders may emit a buffering period SEI without a corresponding > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > During Annex B conversion, this may result in the SPS/PPS being > inserted *after* the buffering period SEI but before the IDR NAL. > > Since the buffering period SEI references the SPS, the SPS/PPS > needs to come first. > --- > > Notes: > v2: Updated FATE test refs > > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > 4 files changed, 29 insertions(+), 16 deletions(-) > Ping again for review. Looking at the FATE output, this patch fixes a number of things - see [1] for details [1] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-July/330912.html
On 23/07/2024 01:01, Josh Allmann wrote: > On Tue, 9 Jul 2024 at 12:06, Josh Allmann <joshua.allmann@gmail.com> wrote: >> >> Encoders may emit a buffering period SEI without a corresponding >> SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. >> >> During Annex B conversion, this may result in the SPS/PPS being >> inserted *after* the buffering period SEI but before the IDR NAL. >> >> Since the buffering period SEI references the SPS, the SPS/PPS >> needs to come first. >> --- >> >> Notes: >> v2: Updated FATE test refs >> >> libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ >> tests/ref/fate/h264-bsf-mp4toannexb | 2 +- >> tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- >> tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ >> 4 files changed, 29 insertions(+), 16 deletions(-) >> > > Ping again for review. Looking at the FATE output, this patch fixes a number > of things - see [1] for details patch generally looks good to me, but I'm not closely familiar with the code there.
On Mon, 22 Jul 2024 at 17:17, Timo Rothenpieler <timo@rothenpieler.org> wrote: > > On 23/07/2024 01:01, Josh Allmann wrote: > > On Tue, 9 Jul 2024 at 12:06, Josh Allmann <joshua.allmann@gmail.com> wrote: > >> > >> Encoders may emit a buffering period SEI without a corresponding > >> SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > >> > >> During Annex B conversion, this may result in the SPS/PPS being > >> inserted *after* the buffering period SEI but before the IDR NAL. > >> > >> Since the buffering period SEI references the SPS, the SPS/PPS > >> needs to come first. > >> --- > >> > >> Notes: > >> v2: Updated FATE test refs > >> > >> libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > >> tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > >> tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > >> tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > >> 4 files changed, 29 insertions(+), 16 deletions(-) > >> > > > > Ping again for review. Looking at the FATE output, this patch fixes a number > > of things - see [1] for details > > patch generally looks good to me, but I'm not closely familiar with the > code there. > Thanks, is there anyone else more familiar with the code who can also sign off on this patch? Josh
Quoting Josh Allmann (2024-07-09 21:05:47) > Encoders may emit a buffering period SEI without a corresponding > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > During Annex B conversion, this may result in the SPS/PPS being > inserted *after* the buffering period SEI but before the IDR NAL. > > Since the buffering period SEI references the SPS, the SPS/PPS > needs to come first. > --- > > Notes: > v2: Updated FATE test refs > > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > 4 files changed, 29 insertions(+), 16 deletions(-) > > diff --git a/libavcodec/bsf/h264_mp4toannexb.c b/libavcodec/bsf/h264_mp4toannexb.c > index 92af6a6881..6607f1e91a 100644 > --- a/libavcodec/bsf/h264_mp4toannexb.c > +++ b/libavcodec/bsf/h264_mp4toannexb.c > @@ -363,6 +363,19 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) > if (!new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80)) > new_idr = 1; > > + /* If this is a buffering period SEI without a corresponding sps/pps > + * then prepend any existing sps/pps before the SEI */ > + if (unit_type == H264_NAL_SEI && buf[1] == 0 && !sps_seen && !pps_seen) { That 0 should be SEI_TYPE_BUFFERING_PERIOD, right? > + if (s->sps_size) { > + count_or_copy(&out, &out_size, s->sps, s->sps_size, PS_OUT_OF_BAND, j); > + sps_seen = 1; > + } > + if (s->pps_size) { > + count_or_copy(&out, &out_size, s->pps, s->pps_size, PS_OUT_OF_BAND, j); > + pps_seen = 1; > + } Is there a reason to insert the PPS? IIUC only the SPS is needed.
On Thu, 1 Aug 2024 at 00:58, Anton Khirnov <anton@khirnov.net> wrote: > Thanks for the review. > Quoting Josh Allmann (2024-07-09 21:05:47) > > Encoders may emit a buffering period SEI without a corresponding > > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > > > During Annex B conversion, this may result in the SPS/PPS being > > inserted *after* the buffering period SEI but before the IDR NAL. > > > > Since the buffering period SEI references the SPS, the SPS/PPS > > needs to come first. > > --- > > > > Notes: > > v2: Updated FATE test refs > > > > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > > 4 files changed, 29 insertions(+), 16 deletions(-) > > > > diff --git a/libavcodec/bsf/h264_mp4toannexb.c b/libavcodec/bsf/h264_mp4toannexb.c > > index 92af6a6881..6607f1e91a 100644 > > --- a/libavcodec/bsf/h264_mp4toannexb.c > > +++ b/libavcodec/bsf/h264_mp4toannexb.c > > @@ -363,6 +363,19 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) > > if (!new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80)) > > new_idr = 1; > > > > + /* If this is a buffering period SEI without a corresponding sps/pps > > + * then prepend any existing sps/pps before the SEI */ > > + if (unit_type == H264_NAL_SEI && buf[1] == 0 && !sps_seen && !pps_seen) { > > That 0 should be SEI_TYPE_BUFFERING_PERIOD, right? > Yes - fixed > > + if (s->sps_size) { > > + count_or_copy(&out, &out_size, s->sps, s->sps_size, PS_OUT_OF_BAND, j); > > + sps_seen = 1; > > + } > > + if (s->pps_size) { > > + count_or_copy(&out, &out_size, s->pps, s->pps_size, PS_OUT_OF_BAND, j); > > + pps_seen = 1; > > + } > > Is there a reason to insert the PPS? IIUC only the SPS is needed. > I believe it would be needed if using this bsf with the segment muxer, and segmentation happens on a recovery point (with a buffering period), eg in the test fate-segment-mp4-to-ts . Granted it is kind of incidental that this patch actually fixes that specific case. I have never seen a SPS without a PPS though. Josh
diff --git a/libavcodec/bsf/h264_mp4toannexb.c b/libavcodec/bsf/h264_mp4toannexb.c index 92af6a6881..6607f1e91a 100644 --- a/libavcodec/bsf/h264_mp4toannexb.c +++ b/libavcodec/bsf/h264_mp4toannexb.c @@ -363,6 +363,19 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) if (!new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80)) new_idr = 1; + /* If this is a buffering period SEI without a corresponding sps/pps + * then prepend any existing sps/pps before the SEI */ + if (unit_type == H264_NAL_SEI && buf[1] == 0 && !sps_seen && !pps_seen) { + if (s->sps_size) { + count_or_copy(&out, &out_size, s->sps, s->sps_size, PS_OUT_OF_BAND, j); + sps_seen = 1; + } + if (s->pps_size) { + count_or_copy(&out, &out_size, s->pps, s->pps_size, PS_OUT_OF_BAND, j); + pps_seen = 1; + } + } + /* prepend only to the first type 5 NAL unit of an IDR picture, if no sps/pps are already present */ if (new_idr && unit_type == H264_NAL_IDR_SLICE && !sps_seen && !pps_seen) { if (s->sps_size) diff --git a/tests/ref/fate/h264-bsf-mp4toannexb b/tests/ref/fate/h264-bsf-mp4toannexb index 2049f39701..81ff568f3d 100644 --- a/tests/ref/fate/h264-bsf-mp4toannexb +++ b/tests/ref/fate/h264-bsf-mp4toannexb @@ -1 +1 @@ -5f04c27cc6ee8625fe2405fb0f7da9a3 +ff2551123909f54c382294baa1bb4364 diff --git a/tests/ref/fate/h264_mp4toannexb_ticket2991 b/tests/ref/fate/h264_mp4toannexb_ticket2991 index f8e3e920d4..9a1fbf2f8c 100644 --- a/tests/ref/fate/h264_mp4toannexb_ticket2991 +++ b/tests/ref/fate/h264_mp4toannexb_ticket2991 @@ -1,4 +1,4 @@ -05d66e60ab22ee004720e0051af0fe74 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 +b6ff5910928ad0b2a7eec481dcc41594 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 1985815 tests/data/fate/h264_mp4toannexb_ticket2991.h264 #extradata 0: 47, 0x3a590d55 #tb 0: 1/1200000 @@ -6,7 +6,7 @@ #codec_id 0: h264 #dimensions 0: 1280x720 #sar 0: 3/4 -0, 0, 0, 40040, 37126, 0xb020184c +0, 0, 0, 40040, 37126, 0x515c184c 0, 40040, 40040, 40040, 6920, 0x8512361a, F=0x0 0, 80081, 80081, 40040, 7550, 0x1bc56ed4, F=0x0 0, 120121, 120121, 40040, 8752, 0xb8c6f0a1, F=0x0 @@ -21,7 +21,7 @@ 0, 480485, 480485, 40040, 11234, 0x83cbd9fd, F=0x0 0, 520525, 520525, 40040, 17616, 0xfdf95104, F=0x0 0, 560566, 560566, 40040, 10689, 0x9633d32b, F=0x0 -0, 600606, 600606, 40040, 45291, 0x543c2cf6 +0, 600606, 600606, 40040, 45291, 0xa8292cf6 0, 640646, 640646, 40040, 20837, 0x051abfab, F=0x0 0, 680687, 680687, 40040, 21418, 0xe2a59d70, F=0x0 0, 720727, 720727, 40040, 15643, 0x15cf2cec, F=0x0 @@ -36,7 +36,7 @@ 0, 1081091, 1081091, 40040, 13130, 0xcbb6bb8e, F=0x0 0, 1121131, 1121131, 40040, 16180, 0x5d188a7a, F=0x0 0, 1161172, 1161172, 40040, 14961, 0x9ff2f463, F=0x0 -0, 1201212, 1201212, 40040, 54296, 0xe6ec30ed +0, 1201212, 1201212, 40040, 54296, 0x3ae830ed 0, 1241252, 1241252, 40040, 11500, 0x8c4852c9, F=0x0 0, 1281293, 1281293, 40040, 12065, 0xfb7954c3, F=0x0 0, 1321333, 1321333, 40040, 12532, 0xf0a935d3, F=0x0 @@ -51,7 +51,7 @@ 0, 1681697, 1681697, 40040, 13250, 0xfed0deb8, F=0x0 0, 1721737, 1721737, 40040, 13360, 0xbf92d476, F=0x0 0, 1761778, 1761778, 40040, 11749, 0x3041eaf1, F=0x0 -0, 1801818, 1801818, 40040, 23997, 0xdbe6d5c4 +0, 1801818, 1801818, 40040, 23997, 0x2fe2d5c4 0, 1841858, 1841858, 40040, 16065, 0xe8f715b7, F=0x0 0, 1881899, 1881899, 40040, 16441, 0x0a4e060f, F=0x0 0, 1921939, 1921939, 40040, 17395, 0xa8edecc2, F=0x0 @@ -66,7 +66,7 @@ 0, 2282303, 2282303, 40040, 13748, 0xed26aeb4, F=0x0 0, 2322343, 2322343, 40040, 15092, 0x3c983538, F=0x0 0, 2362384, 2362384, 40040, 14636, 0x9b278a6c, F=0x0 -0, 2402424, 2402424, 40040, 29134, 0xf784be18 +0, 2402424, 2402424, 40040, 29134, 0x4b80be18 0, 2442464, 2442464, 40040, 10232, 0x5408e15b, F=0x0 0, 2482505, 2482505, 40040, 9769, 0xc93cb7f9, F=0x0 0, 2522545, 2522545, 40040, 14454, 0x45230dbe, F=0x0 @@ -81,7 +81,7 @@ 0, 2882909, 2882909, 40040, 14801, 0x40bae016, F=0x0 0, 2922949, 2922949, 40040, 17303, 0x9ce1fd31, F=0x0 0, 2962990, 2962990, 40040, 17678, 0x9bd66141, F=0x0 -0, 3003030, 3003030, 40040, 48672, 0x3215ce46 +0, 3003030, 3003030, 40040, 48672, 0x8602ce46 0, 3043070, 3043070, 40040, 11894, 0x12e1fece, F=0x0 0, 3083111, 3083111, 40040, 16514, 0xc57aed05, F=0x0 0, 3123151, 3123151, 40040, 13044, 0x61914fa0, F=0x0 @@ -96,7 +96,7 @@ 0, 3483515, 3483515, 40040, 12208, 0x81a587c0, F=0x0 0, 3523555, 3523555, 40040, 14709, 0x5dffbe04, F=0x0 0, 3563596, 3563596, 40040, 14390, 0xbfd1e041, F=0x0 -0, 3603636, 3603636, 40040, 37236, 0xe7f924b1 +0, 3603636, 3603636, 40040, 37236, 0x3bf524b1 0, 3643676, 3643676, 40040, 14056, 0x24714c7c, F=0x0 0, 3683717, 3683717, 40040, 19438, 0x0c50dcd5, F=0x0 0, 3723757, 3723757, 40040, 21728, 0x7eea4a11, F=0x0 @@ -111,7 +111,7 @@ 0, 4084121, 4084121, 40040, 16878, 0x98efbae2, F=0x0 0, 4124161, 4124161, 40040, 14685, 0x1bf78d65, F=0x0 0, 4164202, 4164202, 40040, 13127, 0x0b91881d, F=0x0 -0, 4204242, 4204242, 40040, 29390, 0xf6a5ed6b +0, 4204242, 4204242, 40040, 29390, 0x4aa1ed6b 0, 4244282, 4244282, 40040, 12576, 0xe9845ded, F=0x0 0, 4284323, 4284323, 40040, 12599, 0x96a79ab8, F=0x0 0, 4324363, 4324363, 40040, 16134, 0xb4c36d3f, F=0x0 diff --git a/tests/ref/fate/segment-mp4-to-ts b/tests/ref/fate/segment-mp4-to-ts index 2994416270..54b59a09cc 100644 --- a/tests/ref/fate/segment-mp4-to-ts +++ b/tests/ref/fate/segment-mp4-to-ts @@ -4,7 +4,7 @@ #codec_id 0: h264 #dimensions 0: 640x360 #sar 0: 1/1 -0, -7200, 0, 3600, 22630, 0x9b109541, S=1, 1 +0, -7200, 0, 3600, 22630, 0xee579541, S=1, 1 0, -3600, 14400, 3600, 4021, 0xbf7cdb02, F=0x0, S=1, 1 0, 0, 7200, 3600, 1096, 0x4f162690, F=0x0, S=1, 1 0, 3600, 3600, 3600, 687, 0x00394b95, F=0x0, S=1, 1 @@ -25,7 +25,7 @@ 0, 57600, 64800, 3600, 1182, 0xbe1a4847, F=0x0, S=1, 1 0, 61200, 61200, 3600, 809, 0x8d948a4e, F=0x0, S=1, 1 0, 64800, 68400, 3600, 656, 0x4fa03c2b, F=0x0, S=1, 1 -0, 68400, 86400, 3600, 26555, 0x5629b584, S=1, 1 +0, 68400, 86400, 3600, 26606, 0x0f75c37d, S=1, 1 0, 72000, 79200, 3600, 1141, 0x761b31e8, F=0x0, S=1, 1 0, 75600, 75600, 3600, 717, 0x57746351, F=0x0, S=1, 1 0, 79200, 82800, 3600, 693, 0x78b24263, F=0x0, S=1, 1 @@ -49,7 +49,7 @@ 0, 144000, 151200, 3600, 1271, 0x46006870, F=0x0, S=1, 1 0, 147600, 147600, 3600, 849, 0x94dc99c7, F=0x2, S=1, 1 0, 151200, 154800, 3600, 753, 0xf4236cab, F=0x0, S=1, 1 -0, 154800, 172800, 3600, 25825, 0xd5464dee, S=1, 1 +0, 154800, 172800, 3600, 25876, 0xb4305be7, S=1, 1 0, 158400, 165600, 3600, 1206, 0x8ce84344, F=0x0, S=1, 1 0, 162000, 162000, 3600, 867, 0x312fa07d, F=0x0, S=1, 1 0, 165600, 169200, 3600, 719, 0x810666d1, F=0x0, S=1, 1 @@ -73,7 +73,7 @@ 0, 230400, 237600, 3600, 1545, 0x0099fc98, F=0x0, S=1, 1 0, 234000, 234000, 3600, 929, 0xfd72d049, F=0x2, S=1, 1 0, 237600, 241200, 3600, 829, 0xcfda9e96, F=0x0, S=1, 1 -0, 241200, 259200, 3600, 24220, 0x5ca21d71, S=1, 1 +0, 241200, 259200, 3600, 24271, 0x9c472b6a, S=1, 1 0, 244800, 252000, 3600, 1422, 0xcde6cc34, F=0x0, S=1, 1 0, 248400, 248400, 3600, 883, 0xedacbe25, F=0x0, S=1, 1 0, 252000, 255600, 3600, 768, 0x89d774bc, F=0x0, S=1, 1 @@ -97,7 +97,7 @@ 0, 316800, 324000, 3600, 1501, 0xb3b8f001, F=0x0, S=1, 1 0, 320400, 320400, 3600, 941, 0x92b0cb18, F=0x2, S=1, 1 0, 324000, 327600, 3600, 823, 0x3d548355, F=0x0, S=1, 1 -0, 327600, 345600, 3600, 24042, 0x441e94fb, S=1, 1 +0, 327600, 345600, 3600, 24093, 0xcc0ba2f4, S=1, 1 0, 331200, 338400, 3600, 1582, 0x4f5d1049, F=0x0, S=1, 1 0, 334800, 334800, 3600, 945, 0x4f3cc9e8, F=0x0, S=1, 1 0, 338400, 342000, 3600, 815, 0x0ca790a4, F=0x0, S=1, 1 @@ -121,7 +121,7 @@ 0, 403200, 410400, 3600, 359, 0x11bdae52, F=0x0, S=1, 1 0, 406800, 406800, 3600, 235, 0xbec26964, F=0x2, S=1, 1 0, 410400, 414000, 3600, 221, 0x8380682c, F=0x0, S=1, 1 -0, 414000, 432000, 3600, 22588, 0xf0ecf072, S=1, 1 +0, 414000, 432000, 3600, 22639, 0x1809fe6b, S=1, 1 0, 417600, 424800, 3600, 383, 0x4f3bb571, F=0x0, S=1, 1 0, 421200, 421200, 3600, 257, 0x22e87802, F=0x0, S=1, 1 0, 424800, 428400, 3600, 261, 0xdb988134, F=0x0, S=1, 1
Encoders may emit a buffering period SEI without a corresponding SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. During Annex B conversion, this may result in the SPS/PPS being inserted *after* the buffering period SEI but before the IDR NAL. Since the buffering period SEI references the SPS, the SPS/PPS needs to come first. --- Notes: v2: Updated FATE test refs libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ tests/ref/fate/h264-bsf-mp4toannexb | 2 +- tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ 4 files changed, 29 insertions(+), 16 deletions(-)