[FFmpeg-devel] avformat/flvdec: added support for KUX container

Submitted by swarajhota353@gmail.com on March 19, 2019, 5:11 p.m.

Details

Message ID 20190319171131.15567-1-swarajhota353@gmail.com
State New
Headers show

Commit Message

swarajhota353@gmail.com March 19, 2019, 5:11 p.m.
From: Swaraj Hota <swarajhota353@gmail.com>

Fixes ticket #4519.
---
This is my qualification task for GSoC 2019.
Please suggest any more changes if required.

Signed-off-by: Swaraj Hota <swarajhota353@gmail.com>
---
 Changelog                |  1 +
 libavformat/allformats.c |  1 +
 libavformat/flvdec.c     | 38 ++++++++++++++++++++++++++++++++++++++
 libavformat/version.h    |  4 ++--
 4 files changed, 42 insertions(+), 2 deletions(-)

Comments

Carl Eugen Hoyos March 20, 2019, 9:06 a.m.
2019-03-19 18:11 GMT+01:00, swarajhota353@gmail.com <swarajhota353@gmail.com>:
> From: Swaraj Hota <swarajhota353@gmail.com>
>
> Fixes ticket #4519.
> ---
> This is my qualification task for GSoC 2019.
> Please suggest any more changes if required.

There is a zipped section in the files starting at 0xe00004,
the password is "meta". Is there anything in this metadata
block that may be useful for a player?

Carl Eugen
swarajhota353@gmail.com March 20, 2019, 11:04 p.m.
Okay, I found out the metadata block contains fields like:
logo (a url to thumbnail), tags (in Chinese), title (in Chinese as well),
and other information which mostly seem to be related to streaming.

I'm not sure though if anything is much useful. Please do suggest
if you think any of these can be used.
Carl Eugen Hoyos March 20, 2019, 11:13 p.m.
2019-03-21 0:04 GMT+01:00, Swaraj Hota <swarajhota353@gmail.com>:
> Okay, I found out the metadata block contains fields like:
> logo (a url to thumbnail), tags (in Chinese),

> title (in Chinese as well),

Can you convert the title to something that your terminal
(set to utf8) can display?
There is "title" metadata, so this can be useful.

Carl Eugen
swarajhota353@gmail.com March 21, 2019, 8:41 p.m.
The title is in unicode so I guess we can just encode it to utf8.
I can write a simple C code to do that.
What should I change in the patch though regarding this?
Can you give me more details on what exactly to implement?
Carl Eugen Hoyos March 21, 2019, 10:18 p.m.
2019-03-21 21:41 GMT+01:00, Swaraj Hota <swarajhota353@gmail.com>:
> The title is in unicode so I guess we can just encode it to utf8.
> I can write a simple C code to do that.
> What should I change in the patch though regarding this?
> Can you give me more details on what exactly to implement?

There is metadata in FFmpeg, for example the tag "title" is known
and shown by "ffmpeg -i".
You can use av_dict_set() (as done elsewhere in flvdec.c) to set
the tag "title" to whatever is in the unicode block we both see.

Carl Eugen
swarajhota353@gmail.com March 22, 2019, 6:29 p.m.
I have the meta chunk loaded into a buffer.
I searched a lot but I couldn't find a way to decompress
a password protected zip buffer.

I saw zlib is used in the code base, so went through it.
Found a function "decompress()" but it didn't take a
password as argument. I found another function
under minizip/unzip.h called unzOpenCurrentFilePassword()
which does take password but it takes a file as an argument
and not a buffer.

I am not sure what to do. I thought of using mmap and pass
it to unzOpenCurrentFilePassword() but I guess its not
portable and might not work with windows. Or will it?
Please suggest something I can do here.
swarajhota353@gmail.com March 22, 2019, 7:42 p.m.
Okay I found "fmemopen()". I'll see if I can use it with
unzOpenCurrentFilePassword(). Please do tell if there
is a better solution though.
Carl Eugen Hoyos March 22, 2019, 10:32 p.m.
2019-03-22 19:29 GMT+01:00, Swaraj Hota <swarajhota353@gmail.com>:
> I have the meta chunk loaded into a buffer.
> I searched a lot but I couldn't find a way to decompress
> a password protected zip buffer.
>
> I saw zlib is used in the code base, so went through it.
> Found a function "decompress()" but it didn't take a
> password as argument.

Then this currently can't be done.

I will look at your original patch again tomorrow and
likely commit it.

Carl Eugen
swarajhota353@gmail.com March 23, 2019, 9:19 a.m.
Oh, I really wished to do it ':D
Anyway, thanks :) Tell me if any other change can be made.
Carl Eugen Hoyos March 25, 2019, 11:24 p.m.
2019-03-19 18:11 GMT+01:00, swarajhota353@gmail.com <swarajhota353@gmail.com>:
> From: Swaraj Hota <swarajhota353@gmail.com>
>
> Fixes ticket #4519.

I will push this if there are no objections.

Carl Eugen
Moritz Barsnick March 26, 2019, 10:18 a.m.
On Tue, Mar 26, 2019 at 00:24:40 +0100, Carl Eugen Hoyos wrote:
> > Fixes ticket #4519.
> I will push this if there are no objections.

While trying to see if the proposed XV demuxer[1] can be simplified, I
tripped over the fact that this KUX patch introduces a new demuxer for
what looks like "only" a minor variant of FLV. Could the patch not just
as well extend the flv probe and the extension list?

Of course, this would need to be implemented differently (perhaps a
flag within the Class?):

> if(!strcmp(s->iformat->name, "kux"))

Or is the difference between "a variant" and "a demuxer of its own" too
small to justify the former? Just wondering.

And I personally would have only bumped micro, not minor, but that's
just a gut feeling, I haven't checked what is usually done.

Cheers,
Moritz

P.S.: I will subsequently propose a similar initial XV patch. Either as
this KUX patch does, or as I suggested above.

[1] http://ffmpeg.org/pipermail/ffmpeg-devel/2019-March/241483.html
Carl Eugen Hoyos March 26, 2019, 10:22 a.m.
2019-03-26 11:18 GMT+01:00, Moritz Barsnick <barsnick@gmx.net>:
> On Tue, Mar 26, 2019 at 00:24:40 +0100, Carl Eugen Hoyos wrote:
>> > Fixes ticket #4519.
>> I will push this if there are no objections.
>
> While trying to see if the proposed XV demuxer[1] can be simplified, I
> tripped over the fact that this KUX patch introduces a new demuxer for
> what looks like "only" a minor variant of FLV. Could the patch not just
> as well extend the flv probe and the extension list?

What would be the advantage?

[...]

> And I personally would have only bumped micro, not minor, but that's
> just a gut feeling, I haven't checked what is usually done.

Minor is what is usually done for a new demuxer.

Carl Eugen
Moritz Barsnick March 26, 2019, 11:17 a.m.
On more remark:

On Tue, Mar 19, 2019 at 22:41:32 +0530, swarajhota353@gmail.com wrote:
> +static int kux_probe(AVProbeData *p)

I get a warning if this isn't declared "const AVProbeData *p".

Moritz
Moritz Barsnick March 26, 2019, 11:28 a.m.
On Tue, Mar 26, 2019 at 11:22:58 +0100, Carl Eugen Hoyos wrote:
> > While trying to see if the proposed XV demuxer[1] can be simplified, I
> > tripped over the fact that this KUX patch introduces a new demuxer for
> > what looks like "only" a minor variant of FLV. Could the patch not just
> > as well extend the flv probe and the extension list?
>
> What would be the advantage?

Not exploding the list of supported formats by many minor subvariants.
(It seems KUX is 99% FLV, with a different header tag and an offset.)
I know it's a special example, but imagine "mov,mp4,m4a,3gp,3g2,mj2"
implemented as six separate formats, six separate probes, ...

Just my $0.02.

> Minor is what is usually done for a new demuxer.

Okay, I will do so for XV as well.

Thanks,
Moritz
Carl Eugen Hoyos March 26, 2019, 11:34 a.m.
2019-03-26 12:28 GMT+01:00, Moritz Barsnick <barsnick@gmx.net>:
> On Tue, Mar 26, 2019 at 11:22:58 +0100, Carl Eugen Hoyos wrote:
>> > While trying to see if the proposed XV demuxer[1] can be simplified, I
>> > tripped over the fact that this KUX patch introduces a new demuxer for
>> > what looks like "only" a minor variant of FLV. Could the patch not just
>> > as well extend the flv probe and the extension list?
>>
>> What would be the advantage?
>
> Not exploding the list of supported formats by many minor subvariants.
> (It seems KUX is 99% FLV, with a different header tag and an offset.)

> I know it's a special example, but imagine "mov,mp4,m4a,3gp,3g2,mj2"
> implemented as six separate formats, six separate probes, ...

Not sure how easy it would be to separate them via probe.

Did you receive my private mail?

Carl Eugen
Carl Eugen Hoyos April 6, 2019, 1:58 p.m.
2019-03-19 18:11 GMT+01:00, swarajhota353@gmail.com <swarajhota353@gmail.com>:
> From: Swaraj Hota <swarajhota353@gmail.com>
>
> Fixes ticket #4519.
> ---
> This is my qualification task for GSoC 2019.

Patch applied.

Thank you, Carl Eugen

Patch hide | download patch | download mbox

diff --git a/Changelog b/Changelog
index 4d80e5b54f..8ec6b88d66 100644
--- a/Changelog
+++ b/Changelog
@@ -19,6 +19,7 @@  version <next>:
 - ARBC decoder
 - libaribb24 based ARIB STD-B24 caption support (profiles A and C)
 - Support decoding of HEVC 4:4:4 content in nvdec and cuviddec
+- KUX demuxer
 
 
 version 4.1:
diff --git a/libavformat/allformats.c b/libavformat/allformats.c
index 06844986f3..a7bea4efe2 100644
--- a/libavformat/allformats.c
+++ b/libavformat/allformats.c
@@ -210,6 +210,7 @@  extern AVInputFormat  ff_ivr_demuxer;
 extern AVInputFormat  ff_jacosub_demuxer;
 extern AVOutputFormat ff_jacosub_muxer;
 extern AVInputFormat  ff_jv_demuxer;
+extern AVInputFormat  ff_kux_demuxer;
 extern AVOutputFormat ff_latm_muxer;
 extern AVInputFormat  ff_lmlm4_demuxer;
 extern AVInputFormat  ff_loas_demuxer;
diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c
index 972e333313..090de3db5c 100644
--- a/libavformat/flvdec.c
+++ b/libavformat/flvdec.c
@@ -106,6 +106,20 @@  static int live_flv_probe(AVProbeData *p)
     return probe(p, 1);
 }
 
+static int kux_probe(AVProbeData *p)
+{
+    const uint8_t *d = p->buf;
+
+    if (d[0] == 'K' &&
+        d[1] == 'D' &&
+        d[2] == 'K' &&
+        d[3] == 0 &&
+        d[4] == 0) {
+        return AVPROBE_SCORE_MAX;
+    }
+    return 0;
+}
+
 static void add_keyframes_index(AVFormatContext *s)
 {
     FLVContext *flv   = s->priv_data;
@@ -718,6 +732,10 @@  static int flv_read_header(AVFormatContext *s)
     int offset;
     int pre_tag_size = 0;
 
+    /* Actual FLV data at 0xe40000 in KUX file */
+    if(!strcmp(s->iformat->name, "kux"))
+        avio_skip(s->pb, 0xe40000);
+
     avio_skip(s->pb, 4);
     flags = avio_r8(s->pb);
 
@@ -1366,3 +1384,23 @@  AVInputFormat ff_live_flv_demuxer = {
     .priv_class     = &live_flv_class,
     .flags          = AVFMT_TS_DISCONT
 };
+
+static const AVClass kux_class = {
+    .class_name = "kuxdec",
+    .item_name  = av_default_item_name,
+    .option     = options,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+AVInputFormat ff_kux_demuxer = {
+    .name           = "kux",
+    .long_name      = NULL_IF_CONFIG_SMALL("KUX (YouKu)"),
+    .priv_data_size = sizeof(FLVContext),
+    .read_probe     = kux_probe,
+    .read_header    = flv_read_header,
+    .read_packet    = flv_read_packet,
+    .read_seek      = flv_read_seek,
+    .read_close     = flv_read_close,
+    .extensions     = "kux",
+    .priv_class     = &kux_class,
+};
diff --git a/libavformat/version.h b/libavformat/version.h
index 2e83eb4f23..7efc774b7d 100644
--- a/libavformat/version.h
+++ b/libavformat/version.h
@@ -32,8 +32,8 @@ 
 // Major bumping may affect Ticket5467, 5421, 5451(compatibility with Chromium)
 // Also please add any ticket numbers that you believe might be affected here
 #define LIBAVFORMAT_VERSION_MAJOR  58
-#define LIBAVFORMAT_VERSION_MINOR  26
-#define LIBAVFORMAT_VERSION_MICRO 101
+#define LIBAVFORMAT_VERSION_MINOR  27
+#define LIBAVFORMAT_VERSION_MICRO 100
 
 #define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \
                                                LIBAVFORMAT_VERSION_MINOR, \