From patchwork Fri Jul 21 13:22:11 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 4416 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.1.76 with SMTP id 73csp919740vsb; Fri, 21 Jul 2017 06:23:03 -0700 (PDT) X-Received: by 10.28.8.144 with SMTP id 138mr5319889wmi.8.1500643383649; Fri, 21 Jul 2017 06:23:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1500643383; cv=none; d=google.com; s=arc-20160816; b=oYQn3flwtMIoTLkNusrzDLrM7r0H/pxK5XfsApL19LkFY9Gi4WRMa3hCtSdgHiPUgv DXmqtEt8Y86mL5Ta9eoGNhHn8HKa0jYtOb8NwnYAfuc3lIZ0SqWmuxj8bvAQdGt7MwlA NSpwhH3D6euFmvokC/+LiikntkF8nkpHS0MzA5fDokc3WYai2sQhPfWXFeiJfv+wf6BI 18l2t3pJZZRZqMXzrzWVbTWOiYD0R8avx9NdaiG2Xv2mLPeP5q/xXLH01ypho9il57/F QdPrbvUoT3GN6755GgfZWLQBNFFDLIuSavSTaoeynTtgBklOdJp6qSdYWxxwpU3OSMPK gH2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject:user-agent :in-reply-to:mime-version:references:message-id:to:from:date :delivered-to:arc-authentication-results; bh=EmjP/Xu0fNNkQy8OXkGFHVe9CV9lSVAF7Il4X1eLv4Y=; b=J6lFGyHePnCznNESfte0MEbu0pOCYIuCmwWeeWjLNrH/i215eK+OawyCCieH67TMER UppUYNBYN1nyJx5xUA9iX1wY/IF5QuLi6OeKW0IgCZuv5vJkzSiSulir3VCWLtsbThwO LAWpVOt8LmjUij5DU4tDnfX3R/WxCdckWCodWcjLh88Zz8y3tzSpVVYSBUZrQb2U2BGF We0tfWGBMI2amDK+v4wChSvRT9zX21NnGyVCy+lnqF3EMmTNHyA+b4mE0kFp7OW0BY2I KNB1N5rN1D4ZK2MGOvKtfzlRQ3A1NsmoxMVl6PvbfjfNj7QIyuiiTiI2WK89+y4NRGwY P40A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a193si1032883wmd.151.2017.07.21.06.23.03; Fri, 21 Jul 2017 06:23:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9F1A6689D37; Fri, 21 Jul 2017 16:22:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E6A93689B03 for ; Fri, 21 Jul 2017 16:22:46 +0300 (EEST) Received: from mfilter41-d.gandi.net (mfilter41-d.gandi.net [217.70.178.173]) by relay5-d.mail.gandi.net (Postfix) with ESMTP id 25A0F41C07F for ; Fri, 21 Jul 2017 15:22:55 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mfilter41-d.gandi.net Received: from relay5-d.mail.gandi.net ([IPv6:::ffff:217.70.183.197]) by mfilter41-d.gandi.net (mfilter41-d.gandi.net [::ffff:10.0.15.180]) (amavisd-new, port 10024) with ESMTP id nQtgsbEQ0pBh for ; Fri, 21 Jul 2017 15:22:53 +0200 (CEST) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay5-d.mail.gandi.net (Postfix) with ESMTPSA id 2224941C07E for ; Fri, 21 Jul 2017 15:22:52 +0200 (CEST) Date: Fri, 21 Jul 2017 15:22:11 +0200 From: Michael Niedermayer To: FFmpeg development discussions and patches Message-ID: <20170721132211.GT3740@nb4> References: <20170718230105.5764A174178@albiero.videolan.org> <20170720003348.GB3693@nb4> MIME-Version: 1.0 In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [FFmpeg-devel] [PATCH] pixdesc: Explicitly handle invalid arguments to av_find_best_pix_fmt_of_2() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On Thu, Jul 20, 2017 at 09:54:05PM +0100, Mark Thompson wrote: > --- > On 20/07/17 01:33, Michael Niedermayer wrote: > > Hi > > > > On Tue, Jul 18, 2017 at 11:01:01PM +0000, Mark Thompson wrote: > >> ffmpeg | branch: master | Mark Thompson | Thu Jul 6 22:50:35 2017 +0100| [8a442d7a8a687a469ca502a18a0c68f5302b15e0] | committer: Mark Thompson > >> > >> pixdesc: Improve scoring for opaque/unknown pixel formats > >> > >> Hardware pixel formats do not tell you anything about their actual > >> contents, but should still score higher than formats with completely > >> unknown properties, which in turn should score higher than invalid > >> formats. > >> > >> Do not return an AVERROR code as a score. > >> > >> Fixes a hang in libavfilter where format negotiation gets stuck in a > >> loop because AV_PIX_FMT_NONE scores more highly than all other > >> possibilities. > >> > >>> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=8a442d7a8a687a469ca502a18a0c68f5302b15e0 > >> --- > >> > >> libavutil/pixdesc.c | 16 ++++++++++++---- > >> 1 file changed, 12 insertions(+), 4 deletions(-) > > > > This still breaks > > valgrind ./ffmpeg_g -i ~/videos/matrixbench_mpeg2.mpg -y -t 4 -acodec libmp3lame http://127.0.0.1:8192/feed1.ffm > > [mpeg @ 0x24823f20] start time for stream 0 is not set in estimate_timings_from_pts > > Input #0, mpeg, from '/home/michael/videos/matrixbench_mpeg2.mpg': > > Duration: 00:03:07.66, start: 0.220000, bitrate: 5633 kb/s > > Stream #0:0[0x1bf]: Data: dvd_nav_packet > > Stream #0:1[0x1e0]: Video: mpeg2video (Main), yuv420p(tv, bt470bg/bt470m/bt470m, bottom first), 720x576 [SAR 16:15 DAR 4:3], 25 fps, 25 tbr, 90k tbn, 50 tbc > > Stream #0:2[0x1c0]: Audio: mp2, 48000 Hz, stereo, s16p, 384 kb/s > > ==17852== Invalid read of size 1 > > ==17852== at 0x104871B: av_find_best_pix_fmt_of_2 (in ffmpeg/ffmpeg_g) > > ==17852== by 0x4BD23A: choose_pixel_fmt (in ffmpeg/ffmpeg_g) > > ==17852== by 0x4BBB2C: open_output_file (in ffmpeg/ffmpeg_g) > > ==17852== by 0x4BC696: ffmpeg_parse_options (in ffmpeg/ffmpeg_g) > > ==17852== by 0x4A8A7D: main (in ffmpeg/ffmpeg_g) > > ==17852== Address 0x9 is not stack'd, malloc'd or (recently) free'd > > ==17852== > > ==17852== > > ==17852== Process terminating with default action of signal 11 (SIGSEGV) > > ==17852== Access not within mapped region at address 0x9 > > ==17852== at 0x104871B: av_find_best_pix_fmt_of_2 (in ffmpeg/ffmpeg_g) > > ==17852== by 0x4BD23A: choose_pixel_fmt (in ffmpeg/ffmpeg_g) > > ==17852== by 0x4BBB2C: open_output_file (in ffmpeg/ffmpeg_g) > > ==17852== by 0x4BC696: ffmpeg_parse_options (in ffmpeg/ffmpeg_g) > > ==17852== by 0x4A8A7D: main (in ffmpeg/ffmpeg_g) > > ==17852== If you believe this happened as a result of a stack > > ==17852== overflow in your program's main thread (unlikely but > > ==17852== possible), you can try to increase the size of the > > ==17852== main thread stack using the --main-stacksize= flag. > > ==17852== The main thread stack size used in this run was 8388608. > > > > > > the receiver side of the connection can be setup with > > valgrind ./ffserver_g -f ~/videos/ffserver.conf > > > > ffserver.conf attached > > > > [...] > > > > Right - so someone does call find_best() with an invalid source format, which will give the same score to all possibilities and then barf if one of them is invalid. > > This change makes the invalid format handling more explicit, and fixes your case. > > Thanks, > > - Mark > > > libavutil/pixdesc.c | 32 +++++++++++++++++++------------- > 1 file changed, 19 insertions(+), 13 deletions(-) LGTM thx PS: heres the patch without whitespace changes (much more readable) commit a0f2f597f0922a37486732a961c94af0507af234 Author: Mark Thompson Date: Thu Jul 20 21:54:05 2017 +0100 pixdesc: Explicitly handle invalid arguments to av_find_best_pix_fmt_of_2() Signed-off-by: Michael Niedermayer [...] diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index 1983ce9ef5..a9dd11a498 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2633,6 +2633,11 @@ enum AVPixelFormat av_find_best_pix_fmt_of_2(enum AVPixelFormat dst_pix_fmt1, en const AVPixFmtDescriptor *desc2 = av_pix_fmt_desc_get(dst_pix_fmt2); int score1, score2; + if (!desc1) + dst_pix_fmt = dst_pix_fmt2; + else if (!desc2) + dst_pix_fmt = dst_pix_fmt1; + else { loss_mask= loss_ptr?~*loss_ptr:~0; /* use loss mask if provided */ if(!has_alpha) loss_mask &= ~FF_LOSS_ALPHA; @@ -2649,6 +2654,7 @@ enum AVPixelFormat av_find_best_pix_fmt_of_2(enum AVPixelFormat dst_pix_fmt1, en } else { dst_pix_fmt = score1 < score2 ? dst_pix_fmt2 : dst_pix_fmt1; } + } if (loss_ptr) *loss_ptr = av_get_pix_fmt_loss(dst_pix_fmt, src_pix_fmt, has_alpha)