From patchwork Tue Mar 22 19:10:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 34908 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:ab0:5fda:0:0:0:0:0 with SMTP id g26csp729951uaj; Tue, 22 Mar 2022 12:11:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwGptDRmMbpxGJi7rMwmH4BJe2rEs1deUofn+p3wm8/GflDWO4ijWSU8YmMu01+EnSVwCQp X-Received: by 2002:a17:906:7751:b0:6ce:e3c:81a6 with SMTP id o17-20020a170906775100b006ce0e3c81a6mr25933810ejn.278.1647976289241; Tue, 22 Mar 2022 12:11:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1647976289; cv=none; d=google.com; s=arc-20160816; b=mX+hnzZrpKR0q8DWmXg8oRWCfXWGvOpRMyNZCv5njyNhZrQWnrNRJQGR+ZlHX7SyEH /FNaBdc0aH7okpgw1VLDortptJEDER+5URBtWUVDTFHntetc5xlEnXSypJNadQBH7GjN x42cG7hBv9bgF3aSs6/sWN+63ORsggglJ4ryjHqai6LaSYd35calbMNoGCq/gfDHiv+M 3rzLeTTbYsem4+V887zUJTx9GsviTmpHoQLrSH7xN3MfeUB9J2PTS3MAD8h33GLD70/E zazTSHhM+6KruLu9dWM0Y4S45n0UGMkNwiJ4tpSiCtpbEDLMncHDQBcEr49sZdjmmQ9S SNGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=JdiD6gxcytqKjFc+VkcdAaKQoI66KxvukxvdfHOw2nM=; b=jxDMjzPYFy0rVBhKrbbkKm1ACKozdkIN7QkL1dyKO2pYOImG+B/6aw9FK78VpEAMYt 8lC4fHJdM/pd9hWCwridJCPXfml4NDFsZznqNMmXNL/3qHFrnoJ8A5/wx3/NaR01+KD3 0YZU5xqmToLy6qJSZrX5orIyPPDZ1fqxYNv0emvWWQS7fKsuWFZ+0mDHXIxcnCFK3I+3 GX4QBWHI8dUJMO05cQXRNCaMDXfa7zgbgufc+bS4eV8qxZ75hdsCc7vGQ4cSp+wP+sLh YQ1nURt63+TPYV8aUCTt0Y9Sq9Gqiyqz2j/aTUedxAiEjgW65hhtm+b64GpIWGu9KdCA rTTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=FHT0jdRV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w19-20020a50f113000000b00418c2b5bd2fsi14139323edl.17.2022.03.22.12.11.28; Tue, 22 Mar 2022 12:11:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=FHT0jdRV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0171D68B185; Tue, 22 Mar 2022 21:11:26 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oo1-f47.google.com (mail-oo1-f47.google.com [209.85.161.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 95DF468049F for ; Tue, 22 Mar 2022 21:11:18 +0200 (EET) Received: by mail-oo1-f47.google.com with SMTP id i8-20020a4a6f48000000b00324ada4b9d9so7188oof.11 for ; Tue, 22 Mar 2022 12:11:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=mjy1phbPWM3ifEmrsJM373sTGO39uPbdf4rPKh22oAE=; b=FHT0jdRVsO4f9QHOcQg7K+MFm+8VfkZD6vzD8PBYFuSLIqChYvmykSL9mL5vlIOQdL cXcTgEvEm6NbxDEUhY9qrgdr1XWBamjX4UyuWRRbk/mzFJBKxgBkx9T+xUGsUf4DGZaP bLSgtpwQG0yMiaG60RiliodMFTfDjgjZ/9TdpLRTmTDdHWo89El02Q0e+4j8Qh7bPHiQ 63Lqc2ysFQiVIGDj6HZTLrz1t4lO4KwHqYy6rUZD1+GSoUo5OZOTkepfoUYCL3u8dopb sN99NiLxcl3wVIwkGSv0acHX50Pq5B111gTBFrSDUiW3mgMx2SaiI/Mrb7QpSehf1ew9 9zjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=mjy1phbPWM3ifEmrsJM373sTGO39uPbdf4rPKh22oAE=; b=cgv79gPWQmswmDr+yCsrccQSb8gk0/pSHK17S0yLE5AYk8lqVabLjBOGf80T22fs1i HTxgQUzzzxtTTwrWXXolXv2bT+bX+bii/TbBgZ6/9w0CUFo+Y5wsVf4z9OSfg0MYJed5 yefzTUd1zhJH969Ybwe2gRP0f0kkTkAL/g7+uy0YWXk0L53QRk+MPUTyBf5lvUmTBrOL cNk+XGki3XPF4Mdovxw13XwXqyvR8HQJ/1q9XfbUR3xRyG3nls/nbinltkwJWBpRxZpq 5RhAQiXSUNeLcOTs1XQVBGWa/8ndIZb1ZVKW+2O8gsoFPb8eiiDuWNepcqdHvq7xCK0R SrJg== X-Gm-Message-State: AOAM532AAgkmBDNdwuOKf0rWVQ9+GhRNakULLs1Yq+bx0OTX6ongkkUO Y+7qrU6bubVVs5x/alGPsmxbSAn/Ne9sNA== X-Received: by 2002:a4a:b5c5:0:b0:321:248:bd50 with SMTP id u5-20020a4ab5c5000000b003210248bd50mr8751718ooo.50.1647976276659; Tue, 22 Mar 2022 12:11:16 -0700 (PDT) Received: from localhost.localdomain ([186.136.131.95]) by smtp.gmail.com with ESMTPSA id 37-20020a9d0328000000b005b2265711fcsm9053129otv.16.2022.03.22.12.11.15 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Mar 2022 12:11:16 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Mar 2022 16:10:59 -0300 Message-Id: <20220322191100.3071-1-jamrial@gmail.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avcodec/av1: only set the private context pix_fmt field if get_pixel_format() succeeds X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: nBOyCU6pPQ0S Otherwise get_pixel_format() will not be called when parsing a subsequent Sequence Header in non hwaccel enabled scenarios, allowing frame parsing when it shouldn't. This prevents the scenario seqhdr -> frame_hdr/redundant_frame_hdr -> seqhdr -> redundant_frame_hdr from having the latter redundant frame header parsed as if it was a frame header by the decoder because the former was discarded. Since CBS did not discard it, the latter redundant frame header is output with a zeroed AV1RawFrameHeader struct, which can have undesired results, like division by zero with fields normally guaranteed to be anything else. Signed-off-by: James Almer --- libavcodec/av1dec.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index 613efc5d11..727d829e5b 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -506,9 +506,8 @@ static int get_pixel_format(AVCodecContext *avctx) if (pix_fmt == AV_PIX_FMT_NONE) return -1; - s->pix_fmt = pix_fmt; - switch (s->pix_fmt) { + switch (pix_fmt) { case AV_PIX_FMT_YUV420P: #if CONFIG_AV1_DXVA2_HWACCEL *fmtp++ = AV_PIX_FMT_DXVA2_VLD; @@ -551,7 +550,7 @@ static int get_pixel_format(AVCodecContext *avctx) break; } - *fmtp++ = s->pix_fmt; + *fmtp++ = pix_fmt; *fmtp = AV_PIX_FMT_NONE; ret = ff_thread_get_format(avctx, pix_fmts); @@ -569,7 +568,7 @@ static int get_pixel_format(AVCodecContext *avctx) return AVERROR(ENOSYS); } - avctx->pix_fmt = ret; + avctx->pix_fmt = s->pix_fmt = ret; return 0; } From patchwork Tue Mar 22 19:11:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 34909 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:ab0:5fda:0:0:0:0:0 with SMTP id g26csp729992uaj; Tue, 22 Mar 2022 12:11:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx+0qkz9C9Svg2RKlP3h0v05IsZyprNIvc1IxlU8v0RQFYc4VKsw+1qUTnvQ1QSeZXWSjkI X-Received: by 2002:a17:906:5804:b0:6ce:3f17:bf35 with SMTP id m4-20020a170906580400b006ce3f17bf35mr26703166ejq.346.1647976297587; Tue, 22 Mar 2022 12:11:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1647976297; cv=none; d=google.com; s=arc-20160816; b=auEoc6+CVI1jkduSt6qa69FKWKxN9jU2W/sPI0V5X78W3hqaFK/GWz40Aj/U/c+KFM JfkboZkNqJZSjUpjBHTpm9xYNPjkc8hkH5Z/WK9yK121iP2Hw0U48PyU6ralLlbA261j hXMjwzr9ExwU826mBHE9VclUQ0yTiua/bcl81X/+FZMgMbEHPY5PPBU6Zqrua/t8Lp1S i8jvBzJQC07MAasNYKngyThdGUVp9B+vfSRm3/VL0kspeihzMRylT+w8e+AO6GG1pEa7 Yt//yVs5vnWnQHeirKqTbIMyr3w7hMMMhW/jk5elZJtiVqsxuUhIqy2JI2JwxwuhGaOC 3WAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=b+gzude4woLlL4h9r9vMfYl9mnGCjz0dQy46tkdWHsY=; b=0efiHkWuRlCwp/A6v+jW6Sl6bblHFazTlbX+1Z62ipTNONt5NxFV/m9TS6CZpIw/95 ip30/2ZIrJT9+QgiLKJieawkdfHCHsrK7mPT99OP7LdUtoK0YoPXSOJOYR1sZTYfZapQ Boz64Rq1+kMzUwZhw0hPy3MWIgpZl6U66gfkbWjrtx5iyGBTMcWIx2GVi2vBSIkI/ClA pjxZGiW/O715ayCJsJsjRkNjbku7zGoaHT0cUh8idN5qEd5U3CeNrMe9/4tDOUmL5oyk tnWp9BfYFyuHMX3p0YYHQuWQHqpyJ3rR0VYUTUjCYF5Lt29XqRt+vhhYVwZ5y91Xhn9L foLg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=blhZ09K9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 24-20020a170906019800b006df9ffbf08dsi10526163ejb.365.2022.03.22.12.11.37; Tue, 22 Mar 2022 12:11:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=blhZ09K9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1DB7F68B1A5; Tue, 22 Mar 2022 21:11:28 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oa1-f42.google.com (mail-oa1-f42.google.com [209.85.160.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9FB4968049F for ; Tue, 22 Mar 2022 21:11:19 +0200 (EET) Received: by mail-oa1-f42.google.com with SMTP id 586e51a60fabf-d39f741ba0so2658540fac.13 for ; Tue, 22 Mar 2022 12:11:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=DGSnepF6JzYf6dbfyOfCMQ8t77JTsqSuitsVCSvYnQA=; b=blhZ09K9sfAfQTzSvYResyw4MP47EM+6tsaoutEkszhNh2wdsNWrj5hVWelfTgfw9s LOGIFEKpFzeqtBdeh7zouAcXW126Y15m9xokM+cxYOgFl3W9oIY7hBv4VQISfHAWnMwj PgqBg/0aGsEtugu8tRsLlTDRlr1bjjmbmSaOvSSddhjSAujHMSZ+JGtMyE0AsI6UKCJU PxD5F0MqxX7krThLkP7bwASNQd156CPeckwMt6o0wG7KrFkeR5HZ9rRNFj1WXeIB3iP4 trZFT5gu5ih0ZFK4ztoWQzNyFLyihWFgroLdNCTQQuijpiSVl3MAc2ZRWEb7Ery3U3lC FpSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DGSnepF6JzYf6dbfyOfCMQ8t77JTsqSuitsVCSvYnQA=; b=L/gzWqH7lVM0lgj+YJrzjRbJrG8dJeJRLd+YUSFuI9OrbYnZYiPuxXKN775tu0ov4m XyC4z1bt0sY51tdwsDTv5ltAi3TywzIkV30ITxp/llht07q7a5hv8eGQq0lpsYtGlzfC 7Flg7205CB9QEWprmp3vp4257WNvcsryoYAf8vKCpXT+QtetXSy9wUwNFV+fQzNrANWE pcvMLSkw05pRnSPfHrNTgyGiFJCr/OMadRoybifzqfBaDHvF6/zWmVyUwQO/k/5kDCiU qw2vQQNMojZ7fKlpaNwu/OBHunylftpv7QJRvLoQvS0guwbgodRfQkUjXcfjfkyj/nop 0tkw== X-Gm-Message-State: AOAM530IWf7IbPgTiN6KnKIJcBRj5gNshgByDoL7h3gav/Ecnbna52uZ LYg20Zo8Nss0KhBgs/BtbQIm3lseUztGmw== X-Received: by 2002:a05:6870:310d:b0:da:f5e5:5b5b with SMTP id v13-20020a056870310d00b000daf5e55b5bmr2402409oaa.297.1647976278014; Tue, 22 Mar 2022 12:11:18 -0700 (PDT) Received: from localhost.localdomain ([186.136.131.95]) by smtp.gmail.com with ESMTPSA id 37-20020a9d0328000000b005b2265711fcsm9053129otv.16.2022.03.22.12.11.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Mar 2022 12:11:17 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Mar 2022 16:11:00 -0300 Message-Id: <20220322191100.3071-2-jamrial@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220322191100.3071-1-jamrial@gmail.com> References: <20220322191100.3071-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] Revert "avcodec/av1dec: Check tile_cols" X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: GyN/34CkmVA7 This reverts commit 4fdfe430c12225e819f68993474bd2f473c7e8b1. The previous commit ensures this field will never be zero. --- libavcodec/av1dec.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index 727d829e5b..0543452b06 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -398,9 +398,6 @@ static int get_tiles_info(AVCodecContext *avctx, const AV1RawTileGroup *tile_gro s->tg_start = tile_group->tg_start; s->tg_end = tile_group->tg_end; - if (s->raw_frame_header->tile_cols == 0) - return AVERROR_INVALIDDATA; - for (tile_num = tile_group->tg_start; tile_num <= tile_group->tg_end; tile_num++) { tile_row = tile_num / s->raw_frame_header->tile_cols; tile_col = tile_num % s->raw_frame_header->tile_cols;