diff mbox

[FFmpeg-devel] avformat: Fix probing on some JPEGs

Message ID CANCrOAkGuW8RohVitRWGv2F=5=NhafZTLgTWK2B9Ud5Bsuv4fw@mail.gmail.com
State Superseded
Headers show

Commit Message

Niki Bowe Aug. 21, 2019, 8:58 p.m. UTC 
On Mon, Aug 19, 2019 at 7:22 PM Carl Eugen Hoyos <ceffmpeg@gmail.com> wrote:

>
> This score would mean that mjpeg can never be detected.
> I suspect you have to reduce one of the demuxers to "- 1".
>
>
Thanks Carl.
Attached patch to reduce mpeg probe by -1, which also fixes the issue.

Alternatively I could bump both jpeg and mjpeg?
mpegps_probe looks like it has some heuristics to come up with its scores,
which may make it brittle.

Comments

Michael Niedermayer Aug. 22, 2019, 7:24 a.m. UTC | #1 
On Wed, Aug 21, 2019 at 01:58:52PM -0700, Niki Bowe wrote:
> On Mon, Aug 19, 2019 at 7:22 PM Carl Eugen Hoyos <ceffmpeg@gmail.com> wrote:
> 
> >
> > This score would mean that mjpeg can never be detected.
> > I suspect you have to reduce one of the demuxers to "- 1".
> >
> >
> Thanks Carl.
> Attached patch to reduce mpeg probe by -1, which also fixes the issue.
> 
> Alternatively I could bump both jpeg and mjpeg?
> mpegps_probe looks like it has some heuristics to come up with its scores,
> which may make it brittle.
> 
> -- 
> 
> Nikolas Bowe |  SWE |  nbowe@google.com |  408-565-5137

>  mpeg.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 9bfc9441a3a3a2c060005d42fda9128ff39406ef  0001-avformat-Fix-probing-on-some-JPEGs.patch
> From ab42e2041ec1469f43bccbf8c3e06084bbb7985a Mon Sep 17 00:00:00 2001
> From: Nikolas Bowe <nbowe@google.com>
> Date: Thu, 8 Aug 2019 15:32:51 -0700
> Subject: [PATCH] avformat: Fix probing on some JPEGs
> 
> Fixes "Invalid data found when processing input" on some JPEGs.
> 
> Some large extensionless JPEGs can get probe score collisions with mpeg
> eg
> $ ffprobe -loglevel trace  /tmp/foo
> [NULL @ 0x55c130ab04c0] Opening '/tmp/foo' for reading
> [file @ 0x55c130ab0f40] Setting default whitelist 'file,crypto'
> Probing jpeg_pipe score:6 size:2048
> Probing jpeg_pipe score:6 size:4096
> Probing jpeg_pipe score:6 size:8192
> Probing jpeg_pipe score:6 size:16384
> Probing jpeg_pipe score:25 size:32768
> Probing jpeg_pipe score:25 size:65536
> Probing jpeg_pipe score:25 size:131072
> Probing jpeg_pipe score:25 size:262144
> Probing jpeg_pipe score:25 size:524288
> Probing mpeg score:25 size:1048576
> Probing jpeg_pipe score:25 size:1048576
> [AVIOContext @ 0x55c130ab9300] Statistics: 1048576 bytes read, 0 seeks
> /tmp/foo: Invalid data found when processing input
> 
> This patch fixes this by reducing probe score for mpeg
> ---
>  libavformat/mpeg.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavformat/mpeg.c b/libavformat/mpeg.c
> index 3205f209e6..7a7de54894 100644
> --- a/libavformat/mpeg.c
> +++ b/libavformat/mpeg.c
> @@ -100,7 +100,7 @@ static int mpegps_probe(const AVProbeData *p)
>      }
>  
>      if (vid + audio > invalid + 1) /* invalid VDR files nd short PES streams */
> -        score = AVPROBE_SCORE_EXTENSION / 2;
> +        score = AVPROBE_SCORE_EXTENSION / 2 - 1; // 1 less than jpeg in SOS
>  

jpeg and mpeg are quite different on the syntax side. Have you checked
if that jpeg file violates some easy checkable property that mpeg files
have ?
it would be better to make the detection better and more robust.
tuning the scores has a good chance of just moving the problem elsewhere

thx
[...]
Carl Eugen Hoyos Aug. 22, 2019, 9:18 a.m. UTC | #2 
Am Mi., 21. Aug. 2019 um 23:05 Uhr schrieb Niki Bowe
<nbowe-at-google.com@ffmpeg.org>:
>
> On Mon, Aug 19, 2019 at 7:22 PM Carl Eugen Hoyos <ceffmpeg@gmail.com> wrote:
>
> >
> > This score would mean that mjpeg can never be detected.
> > I suspect you have to reduce one of the demuxers to "- 1".
> >
> >
> Thanks Carl.
> Attached patch to reduce mpeg probe by -1, which also fixes the issue.

Sorry, I misread the original report, it looked to me as if mJpeg was
the culprit.

Imo, the mpeg probing should be fixed (return a smaller value) for your sample
by detecting that it is not mpeg, not by returning a smaller value for
all samples.

Carl Eugen
Paul B Mahol Aug. 22, 2019, 9:29 a.m. UTC | #3 
On Thu, Aug 22, 2019 at 11:19 AM Carl Eugen Hoyos <ceffmpeg@gmail.com>
wrote:

> Am Mi., 21. Aug. 2019 um 23:05 Uhr schrieb Niki Bowe
> <nbowe-at-google.com@ffmpeg.org>:
> >
> > On Mon, Aug 19, 2019 at 7:22 PM Carl Eugen Hoyos <ceffmpeg@gmail.com>
> wrote:
> >
> > >
> > > This score would mean that mjpeg can never be detected.
> > > I suspect you have to reduce one of the demuxers to "- 1".
> > >
> > >
> > Thanks Carl.
> > Attached patch to reduce mpeg probe by -1, which also fixes the issue.
>
> Sorry, I misread the original report, it looked to me as if mJpeg was
> the culprit.
>
> Imo, the mpeg probing should be fixed (return a smaller value) for your
> sample
> by detecting that it is not mpeg, not by returning a smaller value for
> all samples.
>

1000% agree.


> Carl Eugen
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff mbox

Patch

From ab42e2041ec1469f43bccbf8c3e06084bbb7985a Mon Sep 17 00:00:00 2001
From: Nikolas Bowe <nbowe@google.com>
Date: Thu, 8 Aug 2019 15:32:51 -0700
Subject: [PATCH] avformat: Fix probing on some JPEGs

Fixes "Invalid data found when processing input" on some JPEGs.

Some large extensionless JPEGs can get probe score collisions with mpeg
eg
$ ffprobe -loglevel trace  /tmp/foo
[NULL @ 0x55c130ab04c0] Opening '/tmp/foo' for reading
[file @ 0x55c130ab0f40] Setting default whitelist 'file,crypto'
Probing jpeg_pipe score:6 size:2048
Probing jpeg_pipe score:6 size:4096
Probing jpeg_pipe score:6 size:8192
Probing jpeg_pipe score:6 size:16384
Probing jpeg_pipe score:25 size:32768
Probing jpeg_pipe score:25 size:65536
Probing jpeg_pipe score:25 size:131072
Probing jpeg_pipe score:25 size:262144
Probing jpeg_pipe score:25 size:524288
Probing mpeg score:25 size:1048576
Probing jpeg_pipe score:25 size:1048576
[AVIOContext @ 0x55c130ab9300] Statistics: 1048576 bytes read, 0 seeks
/tmp/foo: Invalid data found when processing input

This patch fixes this by reducing probe score for mpeg
---
 libavformat/mpeg.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavformat/mpeg.c b/libavformat/mpeg.c
index 3205f209e6..7a7de54894 100644
--- a/libavformat/mpeg.c
+++ b/libavformat/mpeg.c
@@ -100,7 +100,7 @@  static int mpegps_probe(const AVProbeData *p)
     }
 
     if (vid + audio > invalid + 1) /* invalid VDR files nd short PES streams */
-        score = AVPROBE_SCORE_EXTENSION / 2;
+        score = AVPROBE_SCORE_EXTENSION / 2 - 1; // 1 less than jpeg in SOS
 
 //     av_log(NULL, AV_LOG_ERROR, "vid:%d aud:%d sys:%d pspack:%d invalid:%d size:%d \n",
 //            vid, audio, sys, pspack, invalid, p->buf_size);
-- 
2.23.0.rc1.153.gdeed80330f-goog