From patchwork Fri Nov 18 13:37:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ronald S. Bultje" X-Patchwork-Id: 39325 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp324743pzb; Fri, 18 Nov 2022 05:37:53 -0800 (PST) X-Google-Smtp-Source: AA0mqf7EaHj25ygnXdBi3p3jdqj20cT4TYjPB0YIrWT6aXa1ZrS0c2LZlbAzupcSz7fIvmM6M/3r X-Received: by 2002:a17:906:9c96:b0:73d:7112:4601 with SMTP id fj22-20020a1709069c9600b0073d71124601mr5940495ejc.239.1668778673172; Fri, 18 Nov 2022 05:37:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668778673; cv=none; d=google.com; s=arc-20160816; b=FTnu0yi+w5w1YnUcOQlnjOrJkdemZQLTjkEIumOk4tF3/Ji4I2qIJev1acXhkAe7rU eagLmn141D7295/n7LK04Q79RjX0XC0B6M5idXyXBNtwylx7II14Ex7y0K3Ch2AMmKkq GgHvOMb7/HUXea+QQfMUAAvD16gK5igyNOavvUk3kt0G9FcV5X9G5QZN3Vwk/IzYKyrb GRUVZR18Kvg6waSUtzRF3WB5L2/6+lVUdOVb6KtoIEUyqfZdSpw0z2iXxeAhnfgFsjwF StT1UkpMhNP8F6T/jBn4A8UpdvZwn0oUhTUIRqUvgXzQ3lRqrfRPj/j02RoVlSwcpm62 fxeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=zGwux2JoUhknyMMahNPKFPbjymvrslYqu00PBViyNcM=; b=g5kWYMfYYx/KbcctSIDfe5CViS8eiKugCLW/IcRUHKBVNQ8z4hTIJwBao12Iwfkv6M fh7pdfSEW8cs62urOZzsZ7/A+03kpwOYo45bOZQFnpwogQf+2JsC8zvhSMuNt/Xp7qFU VAzJ9VysaBd0/Wqp/lfOgr8Uwaood2l5e0xOVWLZSits1tAFhN4mAwVnWJqhIGsAGcvq 0Sx1XciImSiGmwcdrzDPxD2M4kgymtuYg/ABaImfKc8m6LMyuQEAq8Rtrw4KhXcPxDfE ZiZnI+jVz+mMDaEy6udI12kUldOn+w5MvYIqWNv1tGTDUtlzx8LLeUO09z4AAJ2Ga+bZ 017g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=X1z7syu0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sz20-20020a1709078b1400b007316ac034a1si2877504ejc.831.2022.11.18.05.37.52; Fri, 18 Nov 2022 05:37:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=X1z7syu0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3D77168BC5F; Fri, 18 Nov 2022 15:37:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0FA0368B8BF for ; Fri, 18 Nov 2022 15:37:43 +0200 (EET) Received: by mail-qt1-f170.google.com with SMTP id w4so3122683qts.0 for ; Fri, 18 Nov 2022 05:37:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MeEqAlrk2o1MCBViFI2MouHF3CsoqYrZsypGJIfWkac=; b=X1z7syu0VxiqqvmYEhAJLC9PK24CthCrmenScHDWhgPnJ+yf8Ox75S7j+IYzbADWRU R4mCByKAlJoi02AVtcTIlHqabOWmYtKw90tmfgj5ddyLY0pHV5QR9AaEThnDvEuKf2Or ZjJOzg/W6IOHGnpwsvyvID/g/9HIn7+IbOo87U+lzQMI0M8HiPwhaTFWAo/9UBASZKCJ jitZLwRWNP05CeEHqeYw67sf+Qf3XvyALi8o9ewJUvCEfrFcocAD/sNUdPMsiUUgX4G2 z+VooSjbM/cAlQVL6thOcleK8gB52X8foT8eG0eqSvw7Zv0eEmu+S3Gc7VCAdRYlTWsp dl/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MeEqAlrk2o1MCBViFI2MouHF3CsoqYrZsypGJIfWkac=; b=8MUq3QG4VzxC9iy+k7HYNTvhCwXFl3Na3meA+lQSekC9xLtyLFro/9GVbWLxbbw09g IJOVYJtttrCUi3VzUO8Xf7qTry30GdPV90PBUII5DCXnhziCHZWKRLDwrSL9RDlwTzVK I2NH1dhB+Mj+IWAr/TOOJglHTA4x60MZNzZFZzW9kjBVS2CjNmLWPAp3OYihUfMl0t1z wEVmwKvuoUDCfIdGBZblE8ZV2puscIDpSEbzZ2o9DoZPdFhifhwE1TnBIB5zmjgqWl4B txL0CGdESjIutpywfD8XZ/cnH8pEXTwsti4x+pcIQutsbXh81hXWWwQntPJkGeSl4jzh SPXw== X-Gm-Message-State: ANoB5pn4ifd4T9mgt1LMYZqQ8CO0l9fP6Ibrehb1XwV+we43odllvLO/ sXywzjeRit+sg9fr67uQmgvAMPFx0gw= X-Received: by 2002:ac8:691:0:b0:3a5:2968:21d5 with SMTP id f17-20020ac80691000000b003a5296821d5mr6663160qth.359.1668778661223; Fri, 18 Nov 2022 05:37:41 -0800 (PST) Received: from Ronalds-MBP.lan (2603-7000-3a00-0b4d-3d0e-a149-0c91-0a3a.res6.spectrum.com. [2603:7000:3a00:b4d:3d0e:a149:c91:a3a]) by smtp.gmail.com with ESMTPSA id h4-20020a05620a284400b006f9e103260dsm2386873qkp.91.2022.11.18.05.37.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 05:37:40 -0800 (PST) From: "Ronald S. Bultje" To: ffmpeg-devel@ffmpeg.org Date: Fri, 18 Nov 2022 08:37:38 -0500 Message-Id: <20221118133738.5065-1-rsbultje@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221118132850.5020-1-rsbultje@gmail.com> References: <20221118132850.5020-1-rsbultje@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/vp8: Fix wrong vpx_rac_is_end() check X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Ronald S . Bultje" , Hirokazu Honda Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ByDrovB7wQvT From: Hirokazu Honda The check of vpx_rac_is_end check(s) are added originally from 1afd246960202917e244c844c534e9c1e3c323f5. It causes a regression of some vp8 stream. b6b9ac5698c8f911841b469af77199153278c55c fixes the regression by a sort of band-aid way. This fixes the wrongness of the original commit. vpx_rac_is_end() should be called against the bool decoder for the vp8 headr context, not one for each coefficient. Reference is vp8_dixie_tokens_process_row() in token.c in spec 20.16. Fixes: Ticket 8069 Fixes: regression of 1afd246960202917e244c844c534e9c1e3c323f5. Fixes: b6b9ac5698c8f911841b469af77199153278c55c Co-authored-by: Ronald S. Bultje Signed-off-by: Hirokazu Honda Signed-off-by: Ronald S. Bultje --- libavcodec/vp8.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/libavcodec/vp8.c b/libavcodec/vp8.c index 67f36d8933..db2419deaf 100644 --- a/libavcodec/vp8.c +++ b/libavcodec/vp8.c @@ -2404,7 +2404,8 @@ static av_always_inline int decode_mb_row_no_filter(AVCodecContext *avctx, void int num_jobs = s->num_jobs; const VP8Frame *prev_frame = s->prev_frame; VP8Frame *curframe = s->curframe; - VPXRangeCoder *c = &s->coeff_partition[mb_y & (s->num_coeff_partitions - 1)]; + VPXRangeCoder *coeff_c = &s->coeff_partition[mb_y & (s->num_coeff_partitions - 1)]; + VP8Macroblock *mb; uint8_t *dst[3] = { curframe->tf.f->data[0] + 16 * mb_y * s->linesize, @@ -2412,7 +2413,7 @@ static av_always_inline int decode_mb_row_no_filter(AVCodecContext *avctx, void curframe->tf.f->data[2] + 8 * mb_y * s->uvlinesize }; - if (vpx_rac_is_end(c)) + if (vpx_rac_is_end(&s->c)) return AVERROR_INVALIDDATA; if (mb_y == 0) @@ -2443,7 +2444,7 @@ static av_always_inline int decode_mb_row_no_filter(AVCodecContext *avctx, void td->mv_bounds.mv_max.x = ((s->mb_width - 1) << 6) + MARGIN; for (mb_x = 0; mb_x < s->mb_width; mb_x++, mb_xy++, mb++) { - if (vpx_rac_is_end(c)) + if (vpx_rac_is_end(&s->c)) return AVERROR_INVALIDDATA; // Wait for previous thread to read mb_x+2, and reach mb_y-1. if (prev_td != td) { @@ -2470,8 +2471,11 @@ static av_always_inline int decode_mb_row_no_filter(AVCodecContext *avctx, void prefetch_motion(s, mb, mb_x, mb_y, mb_xy, VP8_FRAME_PREVIOUS); - if (!mb->skip) - decode_mb_coeffs(s, td, c, mb, s->top_nnz[mb_x], td->left_nnz, is_vp7); + if (!mb->skip) { + if (vpx_rac_is_end(coeff_c)) + return AVERROR_INVALIDDATA; + decode_mb_coeffs(s, td, coeff_c, mb, s->top_nnz[mb_x], td->left_nnz, is_vp7); + } if (mb->mode <= MODE_I4x4) intra_predict(s, td, dst, mb, mb_x, mb_y, is_vp7);