[FFmpeg-devel] patch for failing on WavPack DSD files

Submitted by David Bryant on Nov. 21, 2018, 5:23 a.m.

Details

Message ID 23717d64-cffd-23ab-dbde-28b08dba7c68@wavpack.com
State New
Headers show

Commit Message

David Bryant Nov. 21, 2018, 5:23 a.m.
Hi,

Was made aware of this problem on Kodi:

https://github.com/xbmc/xbmc/issues/14771

I'm going to try to add full WavPack DSD support, but thought in the meantime it would be a good idea to detect and error out early.

Thanks!

David

Comments

Peter Ross Nov. 21, 2018, 6:58 a.m.
On Tue, Nov 20, 2018 at 09:23:03PM -0800, David Bryant wrote:
> Hi,
> 
> Was made aware of this problem on Kodi:
> 
> https://github.com/xbmc/xbmc/issues/14771
> 
> I'm going to try to add full WavPack DSD support, but thought in the meantime it would be a good idea to detect and error out early.
> 
> Thanks!

cool. is this dst-based, or your own algorithm?

-- Peter
(A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
David Bryant Nov. 22, 2018, 5:50 a.m.
On 11/20/18 10:58 PM, Peter Ross wrote:
> On Tue, Nov 20, 2018 at 09:23:03PM -0800, David Bryant wrote:
>> Hi,
>>
>> Was made aware of this problem on Kodi:
>>
>> https://github.com/xbmc/xbmc/issues/14771
>>
>> I'm going to try to add full WavPack DSD support, but thought in the meantime it would be a good idea to detect and error out early.
>>
>> Thanks!
> cool. is this dst-based, or your own algorithm?
>
> -- Peter
> (A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

This is my own algorithm. Actually there are two: one is a byte-based algorithm that is very fast and the other is bit-based
(like DST) that is slower but compresses better (and still uses less CPU than DST). If I had to make a rough estimation of the
average compression ratios, it would be about 0.6 for the fast mode, 0.5 for the high mode, and 0.4 for DST.

-David
David Bryant Dec. 24, 2018, 12:02 a.m.
On 11/21/18 9:50 PM, David Bryant wrote:
> On 11/20/18 10:58 PM, Peter Ross wrote:
>> On Tue, Nov 20, 2018 at 09:23:03PM -0800, David Bryant wrote:
>>> Hi,
>>>
>>> Was made aware of this problem on Kodi:
>>>
>>> https://github.com/xbmc/xbmc/issues/14771
>>>
>>> I'm going to try to add full WavPack DSD support, but thought in the meantime it would be a good idea to detect and error out early.
>>>
>>> Thanks!
>> cool. is this dst-based, or your own algorithm?
>>
>> -- Peter
>> (A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> This is my own algorithm. Actually there are two: one is a byte-based algorithm that is very fast and the other is bit-based
> (like DST) that is slower but compresses better (and still uses less CPU than DST). If I had to make a rough estimation of the
> average compression ratios, it would be about 0.6 for the fast mode, 0.5 for the high mode, and 0.4 for DST.
>
> -David
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Hi. Is there anything else I can provide to help this along?

The behavior of FFmpeg encountering WavPack DSD files without this patch is pretty ugly...

-David
Paul B Mahol Dec. 24, 2018, 8:21 a.m.
On 12/24/18, David Bryant <david@wavpack.com> wrote:
> On 11/21/18 9:50 PM, David Bryant wrote:
>> On 11/20/18 10:58 PM, Peter Ross wrote:
>>> On Tue, Nov 20, 2018 at 09:23:03PM -0800, David Bryant wrote:
>>>> Hi,
>>>>
>>>> Was made aware of this problem on Kodi:
>>>>
>>>> https://github.com/xbmc/xbmc/issues/14771
>>>>
>>>> I'm going to try to add full WavPack DSD support, but thought in the
>>>> meantime it would be a good idea to detect and error out early.
>>>>
>>>> Thanks!
>>> cool. is this dst-based, or your own algorithm?
>>>
>>> -- Peter
>>> (A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
>>>
>>> _______________________________________________
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel@ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> This is my own algorithm. Actually there are two: one is a byte-based
>> algorithm that is very fast and the other is bit-based
>> (like DST) that is slower but compresses better (and still uses less CPU
>> than DST). If I had to make a rough estimation of the
>> average compression ratios, it would be about 0.6 for the fast mode, 0.5
>> for the high mode, and 0.4 for DST.
>>
>> -David
>>
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> Hi. Is there anything else I can provide to help this along?
>
> The behavior of FFmpeg encountering WavPack DSD files without this patch is
> pretty ugly...

Yes, post full patch that adds DSD support in WavPack.
David Bryant Dec. 24, 2018, 5:47 p.m.
On 12/24/18 12:21 AM, Paul B Mahol wrote:
> On 12/24/18, David Bryant <david@wavpack.com> wrote:
>> On 11/21/18 9:50 PM, David Bryant wrote:
>>> On 11/20/18 10:58 PM, Peter Ross wrote:
>>>> On Tue, Nov 20, 2018 at 09:23:03PM -0800, David Bryant wrote:
>>>>> Hi,
>>>>>
>>>>> Was made aware of this problem on Kodi:
>>>>>
>>>>> https://github.com/xbmc/xbmc/issues/14771
>>>>>
>>>>> I'm going to try to add full WavPack DSD support, but thought in the
>>>>> meantime it would be a good idea to detect and error out early.
>>>>>
>>>>> Thanks!
>>>> cool. is this dst-based, or your own algorithm?
>>>>
>>>> -- Peter
>>>> (A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
>>>>
>>>> _______________________________________________
>>>> ffmpeg-devel mailing list
>>>> ffmpeg-devel@ffmpeg.org
>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>> This is my own algorithm. Actually there are two: one is a byte-based
>>> algorithm that is very fast and the other is bit-based
>>> (like DST) that is slower but compresses better (and still uses less CPU
>>> than DST). If I had to make a rough estimation of the
>>> average compression ratios, it would be about 0.6 for the fast mode, 0.5
>>> for the high mode, and 0.4 for DST.
>>>
>>> -David
>>>
>>>
>>> _______________________________________________
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel@ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> Hi. Is there anything else I can provide to help this along?
>>
>> The behavior of FFmpeg encountering WavPack DSD files without this patch is
>> pretty ugly...
> Yes, post full patch that adds DSD support in WavPack.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

I want to do that, but am swamped at work right now, so it will probably be a few months before I can get to that.

In the meantime, I think this patch would be a safe stopgap (and prevent the Kodi crash).

Thanks!

-David
Derek Buitenhuis Dec. 24, 2018, 7:02 p.m.
On 24/12/2018 17:47, David Bryant wrote:
> I want to do that, but am swamped at work right now, so it will probably be a few months before I can get to that.
> 
> In the meantime, I think this patch would be a safe stopgap (and prevent the Kodi crash).

I think it's OK for the meantime (my opinion).

- Derek
Paul B Mahol Dec. 28, 2018, 11:56 a.m.
On 12/24/18, Derek Buitenhuis <derek.buitenhuis@gmail.com> wrote:
> On 24/12/2018 17:47, David Bryant wrote:
>> I want to do that, but am swamped at work right now, so it will probably
>> be a few months before I can get to that.
>>
>> In the meantime, I think this patch would be a safe stopgap (and prevent
>> the Kodi crash).
>
> I think it's OK for the meantime (my opinion).

Applied.
David Bryant Jan. 3, 2019, 5:19 a.m.
On 12/28/18 3:56 AM, Paul B Mahol wrote:
> On 12/24/18, Derek Buitenhuis <derek.buitenhuis@gmail.com> wrote:
>> On 24/12/2018 17:47, David Bryant wrote:
>>> I want to do that, but am swamped at work right now, so it will probably
>>> be a few months before I can get to that.
>>>
>>> In the meantime, I think this patch would be a safe stopgap (and prevent
>>> the Kodi crash).
>> I think it's OK for the meantime (my opinion).
> Applied.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Thanks!

The guys at Kodi asked me to request that this patch be backported to 4.0.

I don't really know if that's a thing, but if it is then that would be great.

-David
Michael Niedermayer Jan. 3, 2019, 10:42 a.m.
On Wed, Jan 02, 2019 at 09:19:30PM -0800, David Bryant wrote:
> On 12/28/18 3:56 AM, Paul B Mahol wrote:
> > On 12/24/18, Derek Buitenhuis <derek.buitenhuis@gmail.com> wrote:
> >> On 24/12/2018 17:47, David Bryant wrote:
> >>> I want to do that, but am swamped at work right now, so it will probably
> >>> be a few months before I can get to that.
> >>>
> >>> In the meantime, I think this patch would be a safe stopgap (and prevent
> >>> the Kodi crash).
> >> I think it's OK for the meantime (my opinion).
> > Applied.
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> Thanks!
> 
> The guys at Kodi asked me to request that this patch be backported to 4.0.
> 
> I don't really know if that's a thing, but if it is then that would be great.

locally backported, should be part of my next push to release/4.0

[...]
Carl Eugen Hoyos Jan. 6, 2019, 12:43 p.m.
2019-01-03 6:19 GMT+01:00, David Bryant <david@wavpack.com>:
> On 12/28/18 3:56 AM, Paul B Mahol wrote:
>> On 12/24/18, Derek Buitenhuis <derek.buitenhuis@gmail.com> wrote:
>>> On 24/12/2018 17:47, David Bryant wrote:
>>>> I want to do that, but am swamped at work right now, so it will probably
>>>> be a few months before I can get to that.
>>>>
>>>> In the meantime, I think this patch would be a safe stopgap (and prevent
>>>> the Kodi crash).
>>> I think it's OK for the meantime (my opinion).
>> Applied.

> The guys at Kodi asked me to request that this patch be backported to 4.0.

While this may be ok, somebody should tell the Kodi developers
to fix the crash as I would expect it's possible to create files
that are still tried to be decoded but produce identical output
as before.

Carl Eugen
David Bryant Jan. 8, 2019, 7:05 a.m.
On 1/6/19 4:43 AM, Carl Eugen Hoyos wrote:
> 2019-01-03 6:19 GMT+01:00, David Bryant <david@wavpack.com>:
>> On 12/28/18 3:56 AM, Paul B Mahol wrote:
>>> On 12/24/18, Derek Buitenhuis <derek.buitenhuis@gmail.com> wrote:
>>>> On 24/12/2018 17:47, David Bryant wrote:
>>>>> I want to do that, but am swamped at work right now, so it will probably
>>>>> be a few months before I can get to that.
>>>>>
>>>>> In the meantime, I think this patch would be a safe stopgap (and prevent
>>>>> the Kodi crash).
>>>> I think it's OK for the meantime (my opinion).
>>> Applied.
>> The guys at Kodi asked me to request that this patch be backported to 4.0.
> While this may be ok, somebody should tell the Kodi developers
> to fix the crash as I would expect it's possible to create files
> that are still tried to be decoded but produce identical output
> as before.
>
> Carl Eugen
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

I agree. In fact, take a WavPack DSD file and clear the bit that this patch checks for, and I'll bet you'll get exactly the old
behavior.

It should be investigated as to why Kodi crashes with these files while FFmpeg just spews a bunch of error messages.

-David

Patch hide | download patch | download mbox

From c86aacdf98c3d34a3f8d63233e01c4a3ab55577e Mon Sep 17 00:00:00 2001
From: David Bryant <david@wavpack.com>
Date: Tue, 20 Nov 2018 21:00:47 -0800
Subject: [PATCH] detect and error out on WavPack DSD files (which are not
 currently supported)

---
 libavformat/wvdec.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/libavformat/wvdec.c b/libavformat/wvdec.c
index 8252656..2060523 100644
--- a/libavformat/wvdec.c
+++ b/libavformat/wvdec.c
@@ -40,6 +40,7 @@  enum WV_FLAGS {
     WV_HBAL   = 0x0400,
     WV_MCINIT = 0x0800,
     WV_MCEND  = 0x1000,
+    WV_DSD    = 0x80000000,
 };
 
 static const int wv_rates[16] = {
@@ -97,6 +98,11 @@  static int wv_read_block_header(AVFormatContext *ctx, AVIOContext *pb)
         return ret;
     }
 
+    if (wc->header.flags & WV_DSD) {
+        avpriv_report_missing_feature(ctx, "WV DSD");
+        return AVERROR_PATCHWELCOME;
+    }
+
     if (wc->header.version < 0x402 || wc->header.version > 0x410) {
         avpriv_report_missing_feature(ctx, "WV version 0x%03X",
                                       wc->header.version);
-- 
1.9.1