From patchwork Fri Nov 18 13:28:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ronald S. Bultje" X-Patchwork-Id: 39324 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp319187pzb; Fri, 18 Nov 2022 05:29:16 -0800 (PST) X-Google-Smtp-Source: AA0mqf6qpIuD5B7NuKaPLLqzdhBme1/SPFTRBMchDvexuUlI8s2Di6sLmU5rKNoUInmk8BjgP3j3 X-Received: by 2002:a17:906:d211:b0:79d:f5f2:6f55 with SMTP id w17-20020a170906d21100b0079df5f26f55mr5616924ejz.531.1668778155973; Fri, 18 Nov 2022 05:29:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668778155; cv=none; d=google.com; s=arc-20160816; b=aXmIHhV1MTingAcu6E8wRNF9nQqK5sYnZePTNJIuYR34LRurtZlTf+X1uPxf41VKMA M0WGRvS4t4w6+BUG7McDEgAveQEJVpwYfPEwGIganML+caV/wUlPjMdquJZufT/aYa6M hu/nT+nh4SiVoGGGgJP99uWxbyGdEa55HdZUDctqBEJeAuVCkRrZYBLpXE0WvDeLhk3h dNBhf3tu204PW7pgbo9jhzr1eL79wgkh5Zq0fdU7hiR7eSbZL0MNGKCevThly31Ww92O feLgVuqCq9zIqMsCK9aJmBRK5pccxEM+VyNewwwSBZ4aJ1ZTns+v2vd/B8+XBMZPOxAC 5c0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=y/8JaLi/ihElLdR94sokUinaBF0vimOpvscNOmcPiaM=; b=LGrd6F1NXdqITk2uCh76bi1FUqzYEZA/fbVAiKdOsI+WyrsCzcf8vBJtuRAFC3fEuu k8gFzvI0j83YaGMm4weY570tZBzh8hqZyMaxraEMdCHRhAanoK4xWALVMjn5GTPBRu2V qyeeIwQn19U3ZnolHTpq3jVTIYe1iYZDieKZNp/pO40m+DtXPvX0L37xn3D1976ANbWw nVcPnvjbXf7ddewBqf30lfS/0JlV6czkrdv/2OfbCrfws9jA8dnGY8kmpW3oaoT276cv ozWCjQL5hi5sJWfyW1/JigGCl4+gwvw4St4ejbRDT/B5ddBW4sITonotwREWBxXphrGf 5UsQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=XxzW2TL2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hp36-20020a1709073e2400b007aedb84056csi3353988ejc.718.2022.11.18.05.29.15; Fri, 18 Nov 2022 05:29:15 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=XxzW2TL2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 61D6168B37A; Fri, 18 Nov 2022 15:29:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3C88D68B76F for ; Fri, 18 Nov 2022 15:29:06 +0200 (EET) Received: by mail-qk1-f181.google.com with SMTP id s20so3362687qkg.5 for ; Fri, 18 Nov 2022 05:29:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tKThwgQ3lXyXumDEh1ceUebylLvTo66rmo2XZtvnXpI=; b=XxzW2TL2cQJuQv7ZqI0lJETC7y2R7mjt2on+631VDYM5fyeenCMftsuiKsxSBdkuV6 8bP5c8i5ZApGe3zDzj58VS7CxgiweSydzsotcjktXwckhxYGVWxwQsZ4bp1ftHiVkW8s ywGi1dnL7E9140jXXk1u90O9fSc8vPA6A9ABZDn2CnX2O2ZL1sDsfIwa2wkZFxywScZ/ y6ui0UC/EgR5K5M2ydZBynGf2SF03QgPWbSY0KOhJnwuXzxjqbCQ9JTknrm+neog4BKu w1mpyzE0VwBw7DEe9DL9BXUlVoAa8tF5iUClDNNBDTS+B2cjoygnGFckYsDBeJI6Nntn q6Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tKThwgQ3lXyXumDEh1ceUebylLvTo66rmo2XZtvnXpI=; b=e3MeiwONd/cakx0Nz7sP1rrJxXCMmIRgC55hrJz2banoqVstSL+CT1wuH2ww+3laN7 JvY2Axp9ETkc7BaQJLf/ZfSIXtVURxYnFCX6ob+tf521ziFW/ikETcE92lRm7BKnlSX+ xoU6XeN+aEQNmK52NgaJdtuj6Er08gxXbuTLK+TBTpxFXtw/iicmrpXUhf3yv2u9vkTY 29CHmjCHENly9Qj++WhGUh3SNSqfk4tVkZb6lC0h1B3zvtRfFpxT8ChifYm4Tn9GYWzd D61NGZg+4hCz5+tPzcsexJi77WagGsB9W25FreI6WzmPWEZF43DqNczrxo8aZSri+W+m 4RXg== X-Gm-Message-State: ANoB5pmBNjVNlEmsHxwnWYGxO7mPfb2VihwgaT9nhROSWDm8V3KGdGjC 7tz+G1GPVyRcBbAEQAJh8G/gHPAhY5o= X-Received: by 2002:a05:620a:2e2:b0:6ec:5385:8e98 with SMTP id a2-20020a05620a02e200b006ec53858e98mr5719001qko.267.1668778144246; Fri, 18 Nov 2022 05:29:04 -0800 (PST) Received: from Ronalds-MBP.lan (2603-7000-3a00-0b4d-3d0e-a149-0c91-0a3a.res6.spectrum.com. [2603:7000:3a00:b4d:3d0e:a149:c91:a3a]) by smtp.gmail.com with ESMTPSA id k21-20020a05620a415500b006eecc4a0de9sm2414822qko.62.2022.11.18.05.29.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 05:29:02 -0800 (PST) From: "Ronald S. Bultje" To: ffmpeg-devel@ffmpeg.org Date: Fri, 18 Nov 2022 08:28:50 -0500 Message-Id: <20221118132850.5020-1-rsbultje@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221116084412.2537585-1-hiroh@chromium.org> References: <20221116084412.2537585-1-hiroh@chromium.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/vp8: Fix wrong vpx_rac_is_end() check X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Ronald S . Bultje" , Hirokazu Honda Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: nvOwF3br+9yu From: Hirokazu Honda The check of vpx_rac_is_end check(s) are added originally from 1afd246960202917e244c844c534e9c1e3c323f5. It causes a regression of some vp8 stream. b6b9ac5698c8f911841b469af77199153278c55c fixes the regression by a sort of band-aid way. This fixes the wrongness of the original commit. vpx_rac_is_end() should be called against the bool decoder for the vp8 headr context, not one for each coefficient. Reference is vp8_dixie_tokens_process_row() in token.c in spec 20.16. Fixes: Ticket 8069 Fixes: regression of 1afd246960202917e244c844c534e9c1e3c323f5. Fixes: b6b9ac5698c8f911841b469af77199153278c55c Signed-off-by: Hirokazu Honda Signed-off-by: Ronald S. Bultje --- libavcodec/vp8.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/libavcodec/vp8.c b/libavcodec/vp8.c index 67f36d8933..db2419deaf 100644 --- a/libavcodec/vp8.c +++ b/libavcodec/vp8.c @@ -2404,7 +2404,8 @@ static av_always_inline int decode_mb_row_no_filter(AVCodecContext *avctx, void int num_jobs = s->num_jobs; const VP8Frame *prev_frame = s->prev_frame; VP8Frame *curframe = s->curframe; - VPXRangeCoder *c = &s->coeff_partition[mb_y & (s->num_coeff_partitions - 1)]; + VPXRangeCoder *coeff_c = &s->coeff_partition[mb_y & (s->num_coeff_partitions - 1)]; + VP8Macroblock *mb; uint8_t *dst[3] = { curframe->tf.f->data[0] + 16 * mb_y * s->linesize, @@ -2412,7 +2413,7 @@ static av_always_inline int decode_mb_row_no_filter(AVCodecContext *avctx, void curframe->tf.f->data[2] + 8 * mb_y * s->uvlinesize }; - if (vpx_rac_is_end(c)) + if (vpx_rac_is_end(&s->c)) return AVERROR_INVALIDDATA; if (mb_y == 0) @@ -2443,7 +2444,7 @@ static av_always_inline int decode_mb_row_no_filter(AVCodecContext *avctx, void td->mv_bounds.mv_max.x = ((s->mb_width - 1) << 6) + MARGIN; for (mb_x = 0; mb_x < s->mb_width; mb_x++, mb_xy++, mb++) { - if (vpx_rac_is_end(c)) + if (vpx_rac_is_end(&s->c)) return AVERROR_INVALIDDATA; // Wait for previous thread to read mb_x+2, and reach mb_y-1. if (prev_td != td) { @@ -2470,8 +2471,11 @@ static av_always_inline int decode_mb_row_no_filter(AVCodecContext *avctx, void prefetch_motion(s, mb, mb_x, mb_y, mb_xy, VP8_FRAME_PREVIOUS); - if (!mb->skip) - decode_mb_coeffs(s, td, c, mb, s->top_nnz[mb_x], td->left_nnz, is_vp7); + if (!mb->skip) { + if (vpx_rac_is_end(coeff_c)) + return AVERROR_INVALIDDATA; + decode_mb_coeffs(s, td, coeff_c, mb, s->top_nnz[mb_x], td->left_nnz, is_vp7); + } if (mb->mode <= MODE_I4x4) intra_predict(s, td, dst, mb, mb_x, mb_y, is_vp7);