From patchwork Thu Jul 6 21:59:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 4245 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.1.76 with SMTP id 73csp2715363vsb; Thu, 6 Jul 2017 14:59:34 -0700 (PDT) X-Received: by 10.223.176.124 with SMTP id g57mr39941546wra.190.1499378374912; Thu, 06 Jul 2017 14:59:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1499378374; cv=none; d=google.com; s=arc-20160816; b=iqcvELjAue27Urh087q2VSpXt6NJEoSHwvl8JChGG6ZOY6C15l9ZR2ly/2hpx+tKYL 20SgwjG4Jrz6AMzhIYYh1haqr3Us6T83qdQ1T/fVW2YPIycsDF861mAlNH1MOLcm5FWK Q268mpCDiMjmBoJnJwLS8hhsv2U29oWefNbXhLDkWdvmmttPjTKPrg4hOSCg6XwM+5S8 Oi0/IWMwccmR6c09oFgo/POX95WkmpidYEOp0dSvZh5gq5xkWuMQjcaMxU3mKRmjEf6d MgMBvKztUBO3kSkfJmK4dwfX5Y2sKWA9BNsufgSvIWUH8BSiehAIp59UFBWS9/XOpx2w yGCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:user-agent:date:message-id:from:to :dkim-signature:delivered-to:arc-authentication-results; bh=Xs2rksEH3F34N2vGQ4y7hPODmaFfh9uYpykISc2OJ3A=; b=k6nVsd3yktZ75OpUEsyBx6I9lZ81dPp18YJkbfGpPgpxFsFOzzK7mNZdlcO5RNJ+hj yGUgSUvGYDL36WgdPtIwxrghm6s/LxgyVba6+W/ejkzpa7RSkEZN8Erf3nqKB+uZDJ2u TfgqLBSDIAy9s0RTLVoys+DNuY/h/kp7/RNb/oLeYB4WdiV6/dTYZGn1/enPvp8gHdkH qzaC7Q24swJJstuo59wAs0T/G/hZrvYhwr52aB3by8+DGx6ugbshgr/h6FScTcglfegH ZK9PH7eTFiOnEfJ0rUGlb4YQqHO7FJzmxqXZC4z6f+U0HLVdDT0z2Uvpdd2hCqETWpej GPeQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.b=TqcXGHA1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p3si1238896wmp.33.2017.07.06.14.59.34; Thu, 06 Jul 2017 14:59:34 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.b=TqcXGHA1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 887DC689A53; Fri, 7 Jul 2017 00:59:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f174.google.com (mail-wr0-f174.google.com [209.85.128.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8B4B7689898 for ; Fri, 7 Jul 2017 00:59:23 +0300 (EEST) Received: by mail-wr0-f174.google.com with SMTP id r103so21390774wrb.0 for ; Thu, 06 Jul 2017 14:59:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=lP7CJPsEdzkj6fbtjTUEJrkiSMd517jbO5D0ejrezxM=; b=TqcXGHA1Abwvw/yZ9nQYHRI8cBzgicK29LWxl2BfaDCAXJtwPxNL8G3RHytDzaNyNg vjkV5v60gFR5ak53rZyHAa8PTfmsL4sAlL8rEXESbizJca/gdHC4fupOsf2zk0Tul4Vv 4g7CqvqhOQ+1y5pGWLJ7t8TS5hvkoQ5vT917ByKEr5mqezfSBLrDC/Dc4YDdGLg1q0/S JL1TSfr0D/6hsl50VhhmpVuBXwAg6JiZIW6Lcbx3hPdhB97unoyKUrwDTq2vtWJ/DXrj Iga/W8RPAOPx4xv2LLgkhX5a7zD7WIjYOSZwBlI5X9X7QIuZvKWwJn9F9TEIpRUkFxFM Hqjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=lP7CJPsEdzkj6fbtjTUEJrkiSMd517jbO5D0ejrezxM=; b=JidHV+iXJ0bKI1OU8uB/FpczBV2i5A0SQUpW1BOLaJT0Fpq6qX3dDf8LWial44N5eX T/sFP8nPYHDFQDZyBk0TOd+PQYJ2JfHeB2SJKpGS4Ny+njAXiG0i2biOG2P8+HggOyqZ Lp2Lc5KH42hT4/jOqoW3Kjx6r/mKzPSqhMQYaL7MFiL8KE2t4d6OW5ywGNxaAicr3/SB yXo2/hHwiwkTmQtS+rqIy2PvdtgEgMIMygLcOChK6fDNgY5PvV9qtMylpVnq2ZP/gtG0 8KCHRCKZb71qYEyhw5Z2p3fh/J2PKNuqtfwi/TQWeREGvS+WeP7gT37OfgzKx9+qpVxB YPAg== X-Gm-Message-State: AIVw111wSTaocil4Tpn5gJVUJqLrFSlhfVzTp15TM5zFapxvvWFZVsG7 xo9viUSOgGPur5EPIyM= X-Received: by 10.28.50.70 with SMTP id y67mr44342wmy.62.1499378365698; Thu, 06 Jul 2017 14:59:25 -0700 (PDT) Received: from [192.168.0.8] (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id q17sm1581057wmd.4.2017.07.06.14.59.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Jul 2017 14:59:24 -0700 (PDT) To: FFmpeg development discussions and patches From: Mark Thompson Message-ID: <8ef60c68-b260-83a2-8765-87737879e0b3@jkqxz.net> Date: Thu, 6 Jul 2017 22:59:24 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] pixdesc: Improve scoring for opaque/unknown pixel formats X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hardware pixel formats do not tell you anything about their actual contents, but should still score higher than formats with completely unknown properties, which in turn should score higher than invalid formats. Do not return an AVERROR code as a score. Fixes a hang in libavfilter where format negotiation gets stuck in a loop because AV_PIX_FMT_NONE scores more highly than all other possibilities. --- The hang in libavfilter happens when trying to choose a pixfmt for output from a list of software formats when a hardware format is the input (the hwdownload filter does this). The matching begins with AV_PIX_FMT_NONE as an invalid value and compares against each possibility in turn, but unfortunately it scores -1 when considered as a conversion from an opaque hardware format, higher than the AVERROR(EINVAL) (== -22 on Linux) scored for all of the real formats. Hence the format selection code chooses AV_PIX_FMT_NONE and thinks it is making forward progress, but actually hasn't and therefore gets stuck in an infinite loop. libavutil/pixdesc.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index 46a7eff06d..35b63f43c6 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2512,7 +2512,11 @@ static int get_pix_fmt_score(enum AVPixelFormat dst_pix_fmt, int score = INT_MAX - 1; if (dst_pix_fmt >= AV_PIX_FMT_NB || dst_pix_fmt <= AV_PIX_FMT_NONE) - return ~0; + return -2; + + if ((src_desc->flags & AV_PIX_FMT_FLAG_HWACCEL) || + (dst_desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) + return 0; /* compute loss */ *lossp = loss = 0; @@ -2521,9 +2525,9 @@ static int get_pix_fmt_score(enum AVPixelFormat dst_pix_fmt, return INT_MAX; if ((ret = get_pix_fmt_depth(&src_min_depth, &src_max_depth, src_pix_fmt)) < 0) - return ret; + return -1; if ((ret = get_pix_fmt_depth(&dst_min_depth, &dst_max_depth, dst_pix_fmt)) < 0) - return ret; + return -1; src_color = get_color_type(src_desc); dst_color = get_color_type(dst_desc);