[FFmpeg-devel,v1] avcodec/h264_metadata_bsf: add option to delete SEI user data

From: Limin Wang <lance.lmwang@gmail.com>

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
---
 doc/bitstream_filters.texi     |  3 +++
 libavcodec/h264_metadata_bsf.c | 19 +++++++++++++++++++
 2 files changed, 22 insertions(+)

On Mon, Apr 20, 2020 at 07:01:09PM +0800, lance.lmwang@gmail.com wrote:
> From: Limin Wang <lance.lmwang@gmail.com>
> 
> Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> ---
>  doc/bitstream_filters.texi     |  3 +++
>  libavcodec/h264_metadata_bsf.c | 19 +++++++++++++++++++
>  2 files changed, 22 insertions(+)
> 
> diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi
> index 8fe5b3ad75..652eb4620f 100644
> --- a/doc/bitstream_filters.texi
> +++ b/doc/bitstream_filters.texi
> @@ -273,6 +273,9 @@ possibly separated by hyphens, and the string can be anything.
>  For example, @samp{086f3693-b7b3-4f2c-9653-21492feee5b8+hello} will
>  insert the string ``hello'' associated with the given UUID.
>  
> +@item delete_user_data
> +Deletes all SEI user data messages.
> +
>  @item delete_filler
>  Deletes both filler NAL units and filler SEI messages.
>  
> diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c
> index d96a50dbf7..8b42faff1b 100644
> --- a/libavcodec/h264_metadata_bsf.c
> +++ b/libavcodec/h264_metadata_bsf.c
> @@ -76,6 +76,7 @@ typedef struct H264MetadataContext {
>      int crop_bottom;
>  
>      const char *sei_user_data;
> +    int delete_user_data;
>  
>      int delete_filler;
>  
> @@ -361,6 +362,21 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt)
>          }
>      }
>  
> +    if (ctx->delete_user_data) {
> +        for (i = au->nb_units - 1; i >= 0; i--) {
> +            if (au->units[i].type == H264_NAL_SEI) {
> +                H264RawSEI *sei = au->units[i].content;
> +
> +                for (j = sei->payload_count - 1; j >= 0; j--) {
> +                    if (sei->payload[j].payload_type ==
> +                        H264_SEI_TYPE_USER_DATA_UNREGISTERED)
> +                        ff_cbs_h264_delete_sei_message(ctx->cbc, au,
> +                                                       &au->units[i], j);
> +                }
> +            }
> +        }
> +    }
> +
>      // Only insert the SEI in access units containing SPSs, and also
>      // unconditionally in the first access unit we ever see.
>      if (ctx->sei_user_data && (has_sps || !ctx->done_first_au)) {
> @@ -684,6 +700,9 @@ static const AVOption h264_metadata_options[] = {
>      { "sei_user_data", "Insert SEI user data (UUID+string)",
>          OFFSET(sei_user_data), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = FLAGS },
>  
> +    { "delete_user_data", "Delete all SEI user data",
> +        OFFSET(delete_user_data), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, FLAGS},
> +
>      { "delete_filler", "Delete all filler (both NAL and SEI)",
>          OFFSET(delete_filler), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, FLAGS},
>  
> -- 
> 2.21.0

ping

>

On 20/04/2020 12:01, lance.lmwang@gmail.com wrote:
> From: Limin Wang <lance.lmwang@gmail.com>
> 
> Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> ---
>  doc/bitstream_filters.texi     |  3 +++
>  libavcodec/h264_metadata_bsf.c | 19 +++++++++++++++++++
>  2 files changed, 22 insertions(+)
> 
> diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi
> index 8fe5b3ad75..652eb4620f 100644
> --- a/doc/bitstream_filters.texi
> +++ b/doc/bitstream_filters.texi
> @@ -273,6 +273,9 @@ possibly separated by hyphens, and the string can be anything.
>  For example, @samp{086f3693-b7b3-4f2c-9653-21492feee5b8+hello} will
>  insert the string ``hello'' associated with the given UUID.
>  
> +@item delete_user_data
> +Deletes all SEI user data messages.
> +
>  @item delete_filler
>  Deletes both filler NAL units and filler SEI messages.
>  
> diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c
> index d96a50dbf7..8b42faff1b 100644
> --- a/libavcodec/h264_metadata_bsf.c
> +++ b/libavcodec/h264_metadata_bsf.c
> @@ -76,6 +76,7 @@ typedef struct H264MetadataContext {
>      int crop_bottom;
>  
>      const char *sei_user_data;
> +    int delete_user_data;
>  
>      int delete_filler;
>  
> @@ -361,6 +362,21 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt)
>          }
>      }
>  
> +    if (ctx->delete_user_data) {
> +        for (i = au->nb_units - 1; i >= 0; i--) {
> +            if (au->units[i].type == H264_NAL_SEI) {
> +                H264RawSEI *sei = au->units[i].content;
> +
> +                for (j = sei->payload_count - 1; j >= 0; j--) {
> +                    if (sei->payload[j].payload_type ==
> +                        H264_SEI_TYPE_USER_DATA_UNREGISTERED)
> +                        ff_cbs_h264_delete_sei_message(ctx->cbc, au,
> +                                                       &au->units[i], j);
> +                }
> +            }
> +        }
> +    }
> +
>      // Only insert the SEI in access units containing SPSs, and also
>      // unconditionally in the first access unit we ever see.
>      if (ctx->sei_user_data && (has_sps || !ctx->done_first_au)) {
> @@ -684,6 +700,9 @@ static const AVOption h264_metadata_options[] = {
>      { "sei_user_data", "Insert SEI user data (UUID+string)",
>          OFFSET(sei_user_data), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = FLAGS },
>  
> +    { "delete_user_data", "Delete all SEI user data",
> +        OFFSET(delete_user_data), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, FLAGS},
> +
>      { "delete_filler", "Delete all filler (both NAL and SEI)",
>          OFFSET(delete_filler), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, FLAGS},
>  
> 

This approach is too general, because stripping everything includes notes which may be required to correctly decode the stream.  An option like this was suggested a while ago and rejected because of the old nonstandard libx264 streams which have decoding broken by it.

Can you explain your intended use-case?  Perhaps a delete option which takes a UUID argument and removes any SEI UDU matching it?

- Mark

On Sun, May 03, 2020 at 03:56:58PM +0100, Mark Thompson wrote:
> On 20/04/2020 12:01, lance.lmwang@gmail.com wrote:
> > From: Limin Wang <lance.lmwang@gmail.com>
> > 
> > Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> > ---
> >  doc/bitstream_filters.texi     |  3 +++
> >  libavcodec/h264_metadata_bsf.c | 19 +++++++++++++++++++
> >  2 files changed, 22 insertions(+)
> > 
> > diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi
> > index 8fe5b3ad75..652eb4620f 100644
> > --- a/doc/bitstream_filters.texi
> > +++ b/doc/bitstream_filters.texi
> > @@ -273,6 +273,9 @@ possibly separated by hyphens, and the string can be anything.
> >  For example, @samp{086f3693-b7b3-4f2c-9653-21492feee5b8+hello} will
> >  insert the string ``hello'' associated with the given UUID.
> >  
> > +@item delete_user_data
> > +Deletes all SEI user data messages.
> > +
> >  @item delete_filler
> >  Deletes both filler NAL units and filler SEI messages.
> >  
> > diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c
> > index d96a50dbf7..8b42faff1b 100644
> > --- a/libavcodec/h264_metadata_bsf.c
> > +++ b/libavcodec/h264_metadata_bsf.c
> > @@ -76,6 +76,7 @@ typedef struct H264MetadataContext {
> >      int crop_bottom;
> >  
> >      const char *sei_user_data;
> > +    int delete_user_data;
> >  
> >      int delete_filler;
> >  
> > @@ -361,6 +362,21 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt)
> >          }
> >      }
> >  
> > +    if (ctx->delete_user_data) {
> > +        for (i = au->nb_units - 1; i >= 0; i--) {
> > +            if (au->units[i].type == H264_NAL_SEI) {
> > +                H264RawSEI *sei = au->units[i].content;
> > +
> > +                for (j = sei->payload_count - 1; j >= 0; j--) {
> > +                    if (sei->payload[j].payload_type ==
> > +                        H264_SEI_TYPE_USER_DATA_UNREGISTERED)
> > +                        ff_cbs_h264_delete_sei_message(ctx->cbc, au,
> > +                                                       &au->units[i], j);
> > +                }
> > +            }
> > +        }
> > +    }
> > +
> >      // Only insert the SEI in access units containing SPSs, and also
> >      // unconditionally in the first access unit we ever see.
> >      if (ctx->sei_user_data && (has_sps || !ctx->done_first_au)) {
> > @@ -684,6 +700,9 @@ static const AVOption h264_metadata_options[] = {
> >      { "sei_user_data", "Insert SEI user data (UUID+string)",
> >          OFFSET(sei_user_data), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = FLAGS },
> >  
> > +    { "delete_user_data", "Delete all SEI user data",
> > +        OFFSET(delete_user_data), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, FLAGS},
> > +
> >      { "delete_filler", "Delete all filler (both NAL and SEI)",
> >          OFFSET(delete_filler), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, FLAGS},
> >  
> > 
> 
> This approach is too general, because stripping everything includes notes which may be required to correctly decode the stream.  An option like this was suggested a while ago and rejected because of the old nonstandard libx264 streams which have decoding broken by it.

I haven't notice other alike patch have submit and reject. I'll try to search the thread to check.

> 
> Can you explain your intended use-case?  Perhaps a delete option which takes a UUID argument and removes any SEI UDU matching it?

We may do stream copy for some h264 stream, so we don't want to keep all of the old SEI
user data. If delete one of the UUID matching SEI, it's better to take a UUID
options, also user had to know the UUID first.  

> 
> - Mark
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".