diff mbox

[FFmpeg-devel,v2,2/2] avcodec/h265_metadata_bsf: add option to insert a string as SEI unregistered user data

Message ID 20191226010103.29008-1-lance.lmwang@gmail.com
State New
Headers show

Commit Message

Lance Wang Dec. 26, 2019, 1:01 a.m. UTC
From: Limin Wang <lance.lmwang@gmail.com>

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
---
update to add keyframe NAL checking as H.264 patch for global header

 doc/bitstream_filters.texi     |  8 ++++
 libavcodec/h265_metadata_bsf.c | 68 +++++++++++++++++++++++++++++++++-
 2 files changed, 75 insertions(+), 1 deletion(-)

Comments

Mark Thompson Dec. 27, 2019, 11:32 p.m. UTC | #1
On 26/12/2019 01:01, lance.lmwang@gmail.com wrote:
> From: Limin Wang <lance.lmwang@gmail.com>
> 
> Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> ---
> update to add keyframe NAL checking as H.264 patch for global header

This case is much easier in H.265, though - the extradata can include SEI which applies globally, so just put it there rather than splicing it into the stream.

- Mark
Lance Wang Dec. 28, 2019, 1:38 a.m. UTC | #2
On Fri, Dec 27, 2019 at 11:32:40PM +0000, Mark Thompson wrote:
> On 26/12/2019 01:01, lance.lmwang@gmail.com wrote:
> > From: Limin Wang <lance.lmwang@gmail.com>
> > 
> > Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> > ---
> > update to add keyframe NAL checking as H.264 patch for global header
> 
> This case is much easier in H.265, though - the extradata can include SEI which applies globally, so just put it there rather than splicing it into the stream.

Mark, are you say put the user data into the extradata? For my case, the string
in the user data will be updated for every key frame runtime. 

> 
> - Mark
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Mark Thompson Jan. 6, 2020, 9:51 p.m. UTC | #3
On 28/12/2019 01:38, Limin Wang wrote:
> On Fri, Dec 27, 2019 at 11:32:40PM +0000, Mark Thompson wrote:
>> On 26/12/2019 01:01, lance.lmwang@gmail.com wrote:
>>> From: Limin Wang <lance.lmwang@gmail.com>
>>>
>>> Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
>>> ---
>>> update to add keyframe NAL checking as H.264 patch for global header
>>
>> This case is much easier in H.265, though - the extradata can include SEI which applies globally, so just put it there rather than splicing it into the stream.
> 
> Mark, are you say put the user data into the extradata? For my case, the string
> in the user data will be updated for every key frame runtime. 

Yes - if you put it in the extradata then it isn't necessary to update any frames in the stream at all.  That ends up being far simpler for global-header streams (MP4 / MKV), and in flat streams (Annex B, TS) the extradata will be spliced in at every seek point by the writer.

- Mark
Lance Wang Jan. 7, 2020, 12:44 a.m. UTC | #4
On Mon, Jan 06, 2020 at 09:51:34PM +0000, Mark Thompson wrote:
> On 28/12/2019 01:38, Limin Wang wrote:
> > On Fri, Dec 27, 2019 at 11:32:40PM +0000, Mark Thompson wrote:
> >> On 26/12/2019 01:01, lance.lmwang@gmail.com wrote:
> >>> From: Limin Wang <lance.lmwang@gmail.com>
> >>>
> >>> Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> >>> ---
> >>> update to add keyframe NAL checking as H.264 patch for global header
> >>
> >> This case is much easier in H.265, though - the extradata can include SEI which applies globally, so just put it there rather than splicing it into the stream.
> > 
> > Mark, are you say put the user data into the extradata? For my case, the string
> > in the user data will be updated for every key frame runtime. 
> 
> Yes - if you put it in the extradata then it isn't necessary to update any frames in the stream at all.  That ends up being far simpler for global-header streams (MP4 / MKV), and in flat streams (Annex B, TS) the extradata will be spliced in at every seek point by the writer.

Sorry, my description isn't clear before, for my case, The user data may have time
related string, so it'll updated for every insertion.
Also, it'll be used for real time streaming, RTMP protocol etc. You had to use
API to get the function, FFmpeg cli can't test it.


> 
> - Mark
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff mbox

Patch

diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi
index 8fe5b3ad75..81b41d70b3 100644
--- a/doc/bitstream_filters.texi
+++ b/doc/bitstream_filters.texi
@@ -376,6 +376,14 @@  The argument must be the name of a level (for example, @samp{5.1}), a
 or the special name @samp{auto} indicating that the filter should
 attempt to guess the level from the input stream properties.
 
+@item sei_user_data
+Insert a string as SEI unregistered user data.  The argument must
+be of the form @emph{UUID+string}, where the UUID is as a 32-character
+(16 bytes) hexadecimal string possibly separated by hyphens, and the
+string can be anything.
+
+For example, @samp{086f3693-b7b3-4f2c-9653-21492feee5b8+hello} will
+insert the string ``hello'' associated with the given 32-bit UUID.
 @end table
 
 @section hevc_mp4toannexb
diff --git a/libavcodec/h265_metadata_bsf.c b/libavcodec/h265_metadata_bsf.c
index b3a1fda144..615a701b37 100644
--- a/libavcodec/h265_metadata_bsf.c
+++ b/libavcodec/h265_metadata_bsf.c
@@ -16,6 +16,7 @@ 
  * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
  */
 
+#include "libavutil/avstring.h"
 #include "libavutil/common.h"
 #include "libavutil/opt.h"
 
@@ -23,6 +24,7 @@ 
 #include "cbs.h"
 #include "cbs_h265.h"
 #include "hevc.h"
+#include "hevc_sei.h"
 #include "h265_profile_level.h"
 
 enum {
@@ -65,6 +67,8 @@  typedef struct H265MetadataContext {
     int crop_top;
     int crop_bottom;
 
+    const char *sei_user_data;
+
     int level;
     int level_guess;
     int level_warned;
@@ -340,7 +344,7 @@  static int h265_metadata_filter(AVBSFContext *bsf, AVPacket *pkt)
 {
     H265MetadataContext *ctx = bsf->priv_data;
     CodedBitstreamFragment *au = &ctx->access_unit;
-    int err, i;
+    int err, i, j, has_sps = 0, has_vps = 0, is_keyframe = 0;
 
     err = ff_bsf_get_packet_ref(bsf, pkt);
     if (err < 0)
@@ -410,11 +414,70 @@  static int h265_metadata_filter(AVBSFContext *bsf, AVPacket *pkt)
             err = h265_metadata_update_vps(bsf, au->units[i].content);
             if (err < 0)
                 goto fail;
+            has_vps = 1;
         }
         if (au->units[i].type == HEVC_NAL_SPS) {
             err = h265_metadata_update_sps(bsf, au->units[i].content);
             if (err < 0)
                 goto fail;
+            has_sps = 1;
+        }
+        if (au->units[i].type >= HEVC_NAL_BLA_W_LP &&
+            au->units[i].type <= HEVC_NAL_IRAP_VCL23)
+            is_keyframe = 1;
+    }
+
+    // Only insert the SEI in access units containing SPSs or VPSs or Keyframe
+    if (ctx->sei_user_data && (has_sps || has_vps || is_keyframe)) {
+        H265RawSEIPayload payload = {
+            .payload_type = HEVC_SEI_TYPE_USER_DATA_UNREGISTERED,
+        };
+        H265RawSEIUserDataUnregistered *udu =
+            &payload.payload.user_data_unregistered;
+
+        for (i = j = 0; j < 32 && ctx->sei_user_data[i]; i++) {
+            int c, v;
+            c = ctx->sei_user_data[i];
+            if (c == '-') {
+                continue;
+            } else if (av_isxdigit(c)) {
+                c = av_tolower(c);
+                v = (c <= '9' ? c - '0' : c - 'a' + 10);
+            } else {
+                goto invalid_user_data;
+            }
+            if (i & 1)
+                udu->uuid_iso_iec_11578[j / 2] |= v;
+            else
+                udu->uuid_iso_iec_11578[j / 2] = v << 4;
+            ++j;
+        }
+        if (j == 32 && ctx->sei_user_data[i] == '+') {
+            size_t len = strlen(ctx->sei_user_data + i + 1);
+
+            udu->data_ref = av_buffer_alloc(len + 1);
+            if (!udu->data_ref) {
+                err = AVERROR(ENOMEM);
+                goto fail;
+            }
+
+            udu->data        = udu->data_ref->data;
+            udu->data_length = len + 1;
+            memcpy(udu->data, ctx->sei_user_data + i + 1, len + 1);
+
+            err = ff_cbs_h265_add_sei_prefix_message(ctx->cbc, au, &payload);
+            if (err < 0) {
+                av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI "
+                       "message to access unit.\n");
+                goto fail;
+            }
+
+        } else {
+invalid_user_data:
+            av_log(bsf, AV_LOG_ERROR, "Invalid user data: "
+                   "must be \"UUID+string\".\n");
+            err = AVERROR(EINVAL);
+            goto fail;
         }
     }
 
@@ -547,6 +610,9 @@  static const AVOption h265_metadata_options[] = {
         OFFSET(crop_bottom), AV_OPT_TYPE_INT,
         { .i64 = -1 }, -1, HEVC_MAX_HEIGHT, FLAGS },
 
+    { "sei_user_data", "Insert SEI user data (UUID+string)",
+        OFFSET(sei_user_data), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = FLAGS },
+
     { "level", "Set level (tables A.6 and A.7)",
         OFFSET(level), AV_OPT_TYPE_INT,
         { .i64 = LEVEL_UNSET }, LEVEL_UNSET, 0xff, FLAGS, "level" },