diff mbox series

[FFmpeg-devel,v2,2/2] avformat/flvdec: support demux HEVC in Enhancing FLV

Message ID 20230406122416.93934-1-lq@chinaffmpeg.org
State New
Headers show
Series None | expand

Commit Message

Steven Liu April 6, 2023, 12:24 p.m. UTC
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
---
 libavformat/flvdec.c | 63 +++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 56 insertions(+), 7 deletions(-)

Comments

Martin Storsjö April 6, 2023, 12:30 p.m. UTC | #1
On Thu, 6 Apr 2023, Steven Liu wrote:

> Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
> ---
> libavformat/flvdec.c | 63 +++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 56 insertions(+), 7 deletions(-)

Even if the title of the linked document is "Enhancing RTMP, FLV" I kinda 
would expect the name of the concept/format to be more like "Enhanced FLV" 
(as the name of the repo is "enhanced-rtmp", not "enhancing-rtmp").

So I'd kinda prefer to refer to it in that form in the commit subject and 
in the comment in the text. If you want to use literally the name used in 
the document, I guess that's "Enhancing RTMP, FLV" though.

// Martin
Steven Liu April 6, 2023, 12:38 p.m. UTC | #2
Martin Storsjö <martin@martin.st> 于2023年4月6日周四 20:31写道:
>
> On Thu, 6 Apr 2023, Steven Liu wrote:
>
> > Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
> > ---
> > libavformat/flvdec.c | 63 +++++++++++++++++++++++++++++++++++++++-----
> > 1 file changed, 56 insertions(+), 7 deletions(-)
>
> Even if the title of the linked document is "Enhancing RTMP, FLV" I kinda
> would expect the name of the concept/format to be more like "Enhanced FLV"
> (as the name of the repo is "enhanced-rtmp", not "enhancing-rtmp").
>
> So I'd kinda prefer to refer to it in that form in the commit subject and
> in the comment in the text. If you want to use literally the name used in
> the document, I guess that's "Enhancing RTMP, FLV" though.
Or submit three patches, BTW supprt the Enhanced rtmp?
>
> // Martin
>

Thanks Martin


Steven
Martin Storsjö April 6, 2023, 1:19 p.m. UTC | #3
On Thu, 6 Apr 2023, Steven Liu wrote:

> Martin Storsjö <martin@martin.st> 于2023年4月6日周四 20:31写道:
>>
>> On Thu, 6 Apr 2023, Steven Liu wrote:
>>
>> > Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
>> > ---
>> > libavformat/flvdec.c | 63 +++++++++++++++++++++++++++++++++++++++-----
>> > 1 file changed, 56 insertions(+), 7 deletions(-)
>>
>> Even if the title of the linked document is "Enhancing RTMP, FLV" I kinda
>> would expect the name of the concept/format to be more like "Enhanced FLV"
>> (as the name of the repo is "enhanced-rtmp", not "enhancing-rtmp").
>>
>> So I'd kinda prefer to refer to it in that form in the commit subject and
>> in the comment in the text. If you want to use literally the name used in
>> the document, I guess that's "Enhancing RTMP, FLV" though.

> Or submit three patches, BTW supprt the Enhanced rtmp?

I don't really understand what you mean here and how this is a response to 
my comment?

Please respond with a bit more elaborate answer.

// Martin
Steven Liu April 7, 2023, 2:51 a.m. UTC | #4
Martin Storsjö <martin@martin.st> 于2023年4月6日周四 21:20写道:
>
> On Thu, 6 Apr 2023, Steven Liu wrote:
>
> > Martin Storsjö <martin@martin.st> 于2023年4月6日周四 20:31写道:
> >>
> >> On Thu, 6 Apr 2023, Steven Liu wrote:
> >>
> >> > Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
> >> > ---
> >> > libavformat/flvdec.c | 63 +++++++++++++++++++++++++++++++++++++++-----
> >> > 1 file changed, 56 insertions(+), 7 deletions(-)
> >>
> >> Even if the title of the linked document is "Enhancing RTMP, FLV" I kinda
> >> would expect the name of the concept/format to be more like "Enhanced FLV"
> >> (as the name of the repo is "enhanced-rtmp", not "enhancing-rtmp").
> >>
> >> So I'd kinda prefer to refer to it in that form in the commit subject and
> >> in the comment in the text. If you want to use literally the name used in
> >> the document, I guess that's "Enhancing RTMP, FLV" though.
>
> > Or submit three patches, BTW supprt the Enhanced rtmp?
>
> I don't really understand what you mean here and how this is a response to
> my comment?
I mean, What  should i do? Resubmit some patches for support the full
functions which describe in documents?
Or just modify comments in patch commit messages?


Thanks
Steven
Martin Storsjö April 7, 2023, 6:54 p.m. UTC | #5
On Fri, 7 Apr 2023, Steven Liu wrote:

> Martin Storsjö <martin@martin.st> 于2023年4月6日周四 21:20写道:
>>
>> On Thu, 6 Apr 2023, Steven Liu wrote:
>>
>> > Martin Storsjö <martin@martin.st> 于2023年4月6日周四 20:31写道:
>> >>
>> >> On Thu, 6 Apr 2023, Steven Liu wrote:
>> >>
>> >> > Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
>> >> > ---
>> >> > libavformat/flvdec.c | 63 +++++++++++++++++++++++++++++++++++++++-----
>> >> > 1 file changed, 56 insertions(+), 7 deletions(-)
>> >>
>> >> Even if the title of the linked document is "Enhancing RTMP, FLV" I kinda
>> >> would expect the name of the concept/format to be more like "Enhanced FLV"
>> >> (as the name of the repo is "enhanced-rtmp", not "enhancing-rtmp").
>> >>
>> >> So I'd kinda prefer to refer to it in that form in the commit subject and
>> >> in the comment in the text. If you want to use literally the name used in
>> >> the document, I guess that's "Enhancing RTMP, FLV" though.
>>
>> > Or submit three patches, BTW supprt the Enhanced rtmp?
>>
>> I don't really understand what you mean here and how this is a response to
>> my comment?
>
> I mean, What  should i do? Resubmit some patches for support the full
> functions which describe in documents?
> Or just modify comments in patch commit messages?

I'm just saying you should replace "Enhancing" with "Enhanced" in the 
commit message and code comment.

// Martin
diff mbox series

Patch

diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c
index d83edff727..8060c43772 100644
--- a/libavformat/flvdec.c
+++ b/libavformat/flvdec.c
@@ -79,6 +79,8 @@  typedef struct FLVContext {
     int64_t last_ts;
     int64_t time_offset;
     int64_t time_pos;
+
+    uint8_t exheader;
 } FLVContext;
 
 /* AMF date type */
@@ -302,13 +304,26 @@  static void flv_set_audio_codec(AVFormatContext *s, AVStream *astream,
     }
 }
 
-static int flv_same_video_codec(AVCodecParameters *vpar, int flags)
+static int flv_same_video_codec(AVFormatContext *s, AVCodecParameters *vpar, int flags)
 {
     int flv_codecid = flags & FLV_VIDEO_CODECID_MASK;
+    FLVContext *flv = s->priv_data;
 
     if (!vpar->codec_id && !vpar->codec_tag)
         return 1;
 
+    if (flv->exheader) {
+        uint8_t *codec_id_str = (uint8_t *)s->pb->buf_ptr;
+        uint32_t codec_id = codec_id_str[3] | codec_id_str[2] << 8 | codec_id_str[1] << 16 | codec_id_str[0] << 24;
+        switch(codec_id) {
+            case MKBETAG('h', 'v', 'c', '1'):
+                flv_codecid = FLV_CODECID_HEVC;
+                break;
+            default:
+                break;
+        }
+    }
+
     switch (flv_codecid) {
     case FLV_CODECID_H263:
         return vpar->codec_id == AV_CODEC_ID_FLV1;
@@ -322,6 +337,8 @@  static int flv_same_video_codec(AVCodecParameters *vpar, int flags)
         return vpar->codec_id == AV_CODEC_ID_VP6A;
     case FLV_CODECID_H264:
         return vpar->codec_id == AV_CODEC_ID_H264;
+    case FLV_CODECID_HEVC:
+        return vpar->codec_id == AV_CODEC_ID_HEVC;
     default:
         return vpar->codec_tag == flv_codecid;
     }
@@ -331,9 +348,23 @@  static int flv_set_video_codec(AVFormatContext *s, AVStream *vstream,
                                int flv_codecid, int read)
 {
     FFStream *const vstreami = ffstream(vstream);
+    FLVContext *flv = s->priv_data;
     int ret = 0;
     AVCodecParameters *par = vstream->codecpar;
     enum AVCodecID old_codec_id = vstream->codecpar->codec_id;
+    flv_codecid &= FLV_VIDEO_CODECID_MASK;
+
+    if (flv->exheader) {
+        uint32_t codec_id = avio_rb32(s->pb);
+
+        switch(codec_id) {
+            case MKBETAG('h', 'v', 'c', '1'):
+                flv_codecid = FLV_CODECID_HEVC;
+                break;
+            default:
+                break;
+        }
+    }
     switch (flv_codecid) {
     case FLV_CODECID_H263:
         par->codec_id = AV_CODEC_ID_FLV1;
@@ -372,6 +403,11 @@  static int flv_set_video_codec(AVFormatContext *s, AVStream *vstream,
         par->codec_id = AV_CODEC_ID_MPEG4;
         ret = 3;
         break;
+    case FLV_CODECID_HEVC:
+        par->codec_id = AV_CODEC_ID_HEVC;
+        vstreami->need_parsing = AVSTREAM_PARSE_HEADERS;
+        ret = 3;
+        break;
     default:
         avpriv_request_sample(s, "Video codec (%x)", flv_codecid);
         par->codec_tag = flv_codecid;
@@ -796,6 +832,7 @@  static int flv_read_header(AVFormatContext *s)
     s->start_time = 0;
     flv->sum_flv_tag_size = 0;
     flv->last_keyframe_stream_index = -1;
+    flv->exheader = 0;
 
     return 0;
 }
@@ -1071,6 +1108,11 @@  retry:
     } else if (type == FLV_TAG_TYPE_VIDEO) {
         stream_type = FLV_STREAM_TYPE_VIDEO;
         flags    = avio_r8(s->pb);
+        /*
+         * Reference Enhancing FLV 2023-03-v1.0.0-B.8
+         * https://github.com/veovera/enhanced-rtmp/blob/main/enhanced-rtmp-v1.pdf
+         * */
+        flv->exheader = (flags >> 7) & 1;
         size--;
         if ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_VIDEO_INFO_CMD)
             goto skip;
@@ -1129,7 +1171,7 @@  skip:
                 break;
         } else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
             if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO &&
-                (s->video_codec_id || flv_same_video_codec(st->codecpar, flags)))
+                (s->video_codec_id || flv_same_video_codec(s, st->codecpar, flags)))
                 break;
         } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
             if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE)
@@ -1230,7 +1272,7 @@  retry_duration:
             avcodec_parameters_free(&par);
         }
     } else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
-        int ret = flv_set_video_codec(s, st, flags & FLV_VIDEO_CODECID_MASK, 1);
+        int ret = flv_set_video_codec(s, st, flags, 1);
         if (ret < 0)
             return ret;
         size -= ret;
@@ -1242,8 +1284,14 @@  retry_duration:
 
     if (st->codecpar->codec_id == AV_CODEC_ID_AAC ||
         st->codecpar->codec_id == AV_CODEC_ID_H264 ||
-        st->codecpar->codec_id == AV_CODEC_ID_MPEG4) {
-        int type = avio_r8(s->pb);
+        st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
+        st->codecpar->codec_id == AV_CODEC_ID_HEVC) {
+        int type = 0;
+        if (flv->exheader) {
+            type = flags & 0x0F;
+        } else {
+            type = avio_r8(s->pb);
+        }
         size--;
 
         if (size < 0) {
@@ -1251,7 +1299,8 @@  retry_duration:
             goto leave;
         }
 
-        if (st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_MPEG4) {
+        if (st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_MPEG4
+                || (st->codecpar->codec_id == AV_CODEC_ID_HEVC && (flags & 0x0F))) {
             // sign extension
             int32_t cts = (avio_rb24(s->pb) + 0xff800000) ^ 0xff800000;
             pts = av_sat_add64(dts, cts);
@@ -1267,7 +1316,7 @@  retry_duration:
             }
         }
         if (type == 0 && (!st->codecpar->extradata || st->codecpar->codec_id == AV_CODEC_ID_AAC ||
-            st->codecpar->codec_id == AV_CODEC_ID_H264)) {
+            st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_HEVC)) {
             AVDictionaryEntry *t;
 
             if (st->codecpar->extradata) {