From patchwork Fri Sep 15 13:11:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 43744 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bca2:b0:149:dfde:5c0a with SMTP id fx34csp305409pzb; Fri, 15 Sep 2023 06:12:23 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGehSLEE7663OY2OjW+JncIiv4qNX5JFPxzPqIGMfFEChfLLz0c3KvcP9elU6NPY5zJFvDz X-Received: by 2002:a05:6402:8c5:b0:525:53b6:3ef9 with SMTP id d5-20020a05640208c500b0052553b63ef9mr1363185edz.35.1694783543010; Fri, 15 Sep 2023 06:12:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694783542; cv=none; d=google.com; s=arc-20160816; b=gfNhZlZm5zelaiCDsBg6Fu7RB5NC5uFdrUqN15r4yb9lKHUtCM76zZVNp7qjHXJ1YC /Cz4M8tzmPrUw0122kbpmPYNxkT0kWPzAq+ovWCuj6+L9dKkQW7ImlW22QdeBhnjBall NG98Ye7JiB5ExEtlPgT8yR73uWCsS2UCd6CpM/I2/yRlYSiCjXcwpjtValuVRxvOR1ZB h0NnbhTu7bluOznJQInFrabL1bcsWWqoUGVQfVChnvVjqoFd3AEGajaV0ZnTTOBzXE+3 gOGvSSx1REFpb/DfXqLm1tARJ+1Vhh1cYlTNCz/T+RPc+ecniGnu3pB01whFWHOfopFe L1EA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to; bh=GI9nIDLnqgzotb7Ac+oyEkGZ2wyQyF6OkwaXUuGbPlY=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=TjvmcOcrLz1hMLj5FQ0zg2wuP+h04sD2ESmPeZB9whlG5laW+Ef+jR5dz3OdAhd/uE +8Eu9uvLl+dHJN/KT4cM8mzWeMgYhlU0IfaVRPMmi1WowUNEAJe+Ai8Do3gdB3dD7ypD Thpk0FLY4gJ8dPoOusuAfe8ylT6OjHbrskBCwDZHlh8MsjtZDDobF/PcfLMXoCDRJat+ xoWR/buacwXhE7oNR38rlixTcc/jn63X7hHQC5pSWW5bEBBRTDDQ1iZwnIp8czHQqqen S0sp5x4z/fK8g5+ds6YViS6zJnAUgp3H6+OssFm3SXNIYmMOFm28TO5rUSxmlSChKde9 SXxQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n15-20020a056402514f00b0052a40523c4fsi3125103edd.539.2023.09.15.06.11.59; Fri, 15 Sep 2023 06:12:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1E1E068C768; Fri, 15 Sep 2023 16:11:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CB4D668C6B3 for ; Fri, 15 Sep 2023 16:11:48 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id E534CC0006 for ; Fri, 15 Sep 2023 13:11:47 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Fri, 15 Sep 2023 15:11:44 +0200 Message-Id: <20230915131147.5945-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 1/4] avcodec/evc_ps: Check cpb_cnt_minus1 and propagate error X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4Mt0wEQm9+MQ Fixes: out of array access Fixes: 60949/clusterfuzz-testcase-minimized-ffmpeg_dem_EVC_fuzzer-5959738853294080 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/evc_ps.c | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/libavcodec/evc_ps.c b/libavcodec/evc_ps.c index 64384a392c2..7fe13fd32f0 100644 --- a/libavcodec/evc_ps.c +++ b/libavcodec/evc_ps.c @@ -53,6 +53,9 @@ static int ref_pic_list_struct(GetBitContext *gb, RefPicListStruct *rpl) static int hrd_parameters(GetBitContext *gb, HRDParameters *hrd) { hrd->cpb_cnt_minus1 = get_ue_golomb_31(gb); + if (hrd->cpb_cnt_minus1 >= FF_ARRAY_ELEMS(hrd->cpb_size_value_minus1)) + return AVERROR_INVALIDDATA; + hrd->bit_rate_scale = get_bits(gb, 4); hrd->cpb_size_scale = get_bits(gb, 4); for (int SchedSelIdx = 0; SchedSelIdx <= hrd->cpb_cnt_minus1; SchedSelIdx++) { @@ -71,6 +74,8 @@ static int hrd_parameters(GetBitContext *gb, HRDParameters *hrd) // @see ISO_IEC_23094-1 (E.2.1 VUI parameters syntax) static int vui_parameters(GetBitContext *gb, VUIParameters *vui) { + int ret; + vui->aspect_ratio_info_present_flag = get_bits(gb, 1); if (vui->aspect_ratio_info_present_flag) { vui->aspect_ratio_idc = get_bits(gb, 8); @@ -109,11 +114,18 @@ static int vui_parameters(GetBitContext *gb, VUIParameters *vui) vui->fixed_pic_rate_flag = get_bits(gb, 1); } vui->nal_hrd_parameters_present_flag = get_bits(gb, 1); - if (vui->nal_hrd_parameters_present_flag) - hrd_parameters(gb, &vui->hrd_parameters); + if (vui->nal_hrd_parameters_present_flag) { + ret = hrd_parameters(gb, &vui->hrd_parameters); + if (ret < 0) + return ret; + } + vui->vcl_hrd_parameters_present_flag = get_bits(gb, 1); - if (vui->vcl_hrd_parameters_present_flag) - hrd_parameters(gb, &vui->hrd_parameters); + if (vui->vcl_hrd_parameters_present_flag) { + ret = hrd_parameters(gb, &vui->hrd_parameters); + if (ret < 0) + return ret; + } if (vui->nal_hrd_parameters_present_flag || vui->vcl_hrd_parameters_present_flag) vui->low_delay_hrd_flag = get_bits(gb, 1); vui->pic_struct_present_flag = get_bits(gb, 1); @@ -292,8 +304,11 @@ int ff_evc_parse_sps(GetBitContext *gb, EVCParamSets *ps) } sps->vui_parameters_present_flag = get_bits1(gb); - if (sps->vui_parameters_present_flag) - vui_parameters(gb, &(sps->vui_parameters)); + if (sps->vui_parameters_present_flag) { + ret = vui_parameters(gb, &(sps->vui_parameters)); + if (ret < 0) + goto fail; + } // @note // If necessary, add the missing fields to the EVCParserSPS structure From patchwork Fri Sep 15 13:11:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 43745 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bca2:b0:149:dfde:5c0a with SMTP id fx34csp305571pzb; Fri, 15 Sep 2023 06:12:38 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFAVBPTBREzOWpd+NF2m86LTHxJ0lGiC9fWaPlmd7uDMpQxLqzchWsjwVqsN1q0tI+tvPSV X-Received: by 2002:aa7:c642:0:b0:522:4764:8baa with SMTP id z2-20020aa7c642000000b0052247648baamr2417500edr.12.1694783556510; Fri, 15 Sep 2023 06:12:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694783556; cv=none; d=google.com; s=arc-20160816; b=N0xUNuZosqlJt9x4lpvuVUoK1Mdq9GTN2FyeKrigakTBhmddwkoKty4q1PkMvdGUtd pNU4ReQkPXasFejQX1M1wxQqeUw/vHNj+Ax4RUHwPVwManYdKfgQ70l+9EsGf+/OwRtX 3iQDPHsquWhjMRRXbGXdK3heJVtK1iYsjDwc0SSCROueAYfJD8CDLkaJUm+416qHjNvu Hs37dwvn5xxRIDTcm4UA7i3yRxWKrd+Qvy2h+pw5KRbz/qnjIpJ+jhih1JXa6lCxh1tb xD6dJKe9ZWU+eFTZyF+pWYKROJIXl7mxaYJnQgCRQbMM5aqXaeaV6y0hcd5ktXXfeT5O jG0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to; bh=LgAJC/GTceEP2QXcQ6lV6GGnp2chh2kVYt65KlrUA/I=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=hyuIPEZ6YOHGOQ6Jo7+YifoAefDU57xJp1psomnGEFg7tVlS4NoARXICE9ZTT8FgNj vynW8KpMe2Vsz/o0LLW8hevRjJeJEQaFWsxuclBdvOMGaPCf7+02ZVHEejkRO1Es34pM 4kJGTUgAfJiGMn9zbvYfXTfrfONnfLeAIyGSM9qvToyD7aHzcoFRwY6uN1bUd70eTc3P 3c0V9GIYl0sYmh2MXZBfyNJwm8YRqQx2f/RuY0u3qKHJyLLUopu6NsXks/ADBvl2fVPO UEMfpIxLgU/GRlVXhwZaZXbeyuM1LP+jbypIQE+y41k7LLn7J58vY1oByf4orGZ5BOGh BYqw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c12-20020aa7d60c000000b0052a40d398dfsi3132179edr.691.2023.09.15.06.12.13; Fri, 15 Sep 2023 06:12:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 46A6668C7BB; Fri, 15 Sep 2023 16:11:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0FDEF68C765 for ; Fri, 15 Sep 2023 16:11:50 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 3C89EFF80F for ; Fri, 15 Sep 2023 13:11:48 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Fri, 15 Sep 2023 15:11:45 +0200 Message-Id: <20230915131147.5945-2-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230915131147.5945-1-michael@niedermayer.cc> References: <20230915131147.5945-1-michael@niedermayer.cc> X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 2/4] avcodec/evc_ps: Check ref_pic_num and sps_max_dec_pic_buffering_minus1 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: nzU0cx4+u8Y3 Fixes: out of array write Found-by: dongsookim@korea.ac.kr Signed-off-by: Michael Niedermayer --- libavcodec/evc_ps.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/libavcodec/evc_ps.c b/libavcodec/evc_ps.c index 7fe13fd32f0..96237ed2911 100644 --- a/libavcodec/evc_ps.c +++ b/libavcodec/evc_ps.c @@ -22,12 +22,15 @@ #include "evc_ps.h" #define EXTENDED_SAR 255 - // @see ISO_IEC_23094-1 (7.3.7 Reference picture list structure syntax) -static int ref_pic_list_struct(GetBitContext *gb, RefPicListStruct *rpl) +static int ref_pic_list_struct(EVCParserSPS *sps, GetBitContext *gb, RefPicListStruct *rpl) { uint32_t delta_poc_st, strp_entry_sign_flag = 0; rpl->ref_pic_num = get_ue_golomb_long(gb); + + if ((unsigned)rpl->ref_pic_num > sps->sps_max_dec_pic_buffering_minus1) + return AVERROR_INVALIDDATA; + if (rpl->ref_pic_num > 0) { delta_poc_st = get_ue_golomb_long(gb); @@ -251,6 +254,8 @@ int ff_evc_parse_sps(GetBitContext *gb, EVCParamSets *ps) sps->max_num_tid0_ref_pics = get_ue_golomb_31(gb); else { sps->sps_max_dec_pic_buffering_minus1 = get_ue_golomb_long(gb); + if ((unsigned)sps->sps_max_dec_pic_buffering_minus1 > 16 - 1) + return AVERROR_INVALIDDATA; sps->long_term_ref_pic_flag = get_bits1(gb); sps->rpl1_same_as_rpl0_flag = get_bits1(gb); sps->num_ref_pic_list_in_sps[0] = get_ue_golomb(gb); @@ -261,7 +266,7 @@ int ff_evc_parse_sps(GetBitContext *gb, EVCParamSets *ps) } for (int i = 0; i < sps->num_ref_pic_list_in_sps[0]; ++i) - ref_pic_list_struct(gb, &sps->rpls[0][i]); + ref_pic_list_struct(sps, gb, &sps->rpls[0][i]); if (!sps->rpl1_same_as_rpl0_flag) { sps->num_ref_pic_list_in_sps[1] = get_ue_golomb(gb); @@ -270,7 +275,7 @@ int ff_evc_parse_sps(GetBitContext *gb, EVCParamSets *ps) goto fail; } for (int i = 0; i < sps->num_ref_pic_list_in_sps[1]; ++i) - ref_pic_list_struct(gb, &sps->rpls[1][i]); + ref_pic_list_struct(sps, gb, &sps->rpls[1][i]); } } From patchwork Fri Sep 15 13:11:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 43746 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bca2:b0:149:dfde:5c0a with SMTP id fx34csp305583pzb; Fri, 15 Sep 2023 06:12:38 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEV+J1os1m142MOfbCISnJghBpOHFQ6+UIEDv48SGfseyALbcB6ViN4ZRSde6AejRaLtBzK X-Received: by 2002:a17:907:1de9:b0:9ad:e0c9:8566 with SMTP id og41-20020a1709071de900b009ade0c98566mr392251ejc.76.1694783556485; Fri, 15 Sep 2023 06:12:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694783556; cv=none; d=google.com; s=arc-20160816; b=lcnuzPETyt56Gfu33mP2I5ofOPtUVOBTpLERYO7JQOIFSA1h4KzzckFDNNVqZtfCyB olCOHFIQs4C0aHcRuoYj4iGg2hHiGoqNiLUKP6RO9PpT1R+vI58F1R1BRZv5qh/Yn/EP XvXM6hCa/iMqR0hpt5NhytsK4zc9JZ6X4r8INTAysm95W9XHMZ4plRkm3Dh5x5jw3nGp 8WOC93gyO+Nr2IfmsJsGYMF5lQcadPVSMaJRdNon42Rkf9ZpPcnQZTKuNNE0vL2PKM/e XhtTdD2q+XFvmSr8ULEbs7Rsozj7wHUZg7fCFcTgh157XsgFKdiXiYkaZWOTZMdOl+Gn Uz6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to; bh=mseYpqojD1aLcGpPBJrzpqd6uZLQzbYKrBIdZx5AxVU=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=yJOrL+AJHAgIVPVi76oa7ZMOnv2pXCeFQRnIOORyp0M7+sVVJkJ6ix7MCT1eQ1Tweb zGlL0HqOAMg2nO1iM3fpzodMkrYZP08Zw5pAgU9NZwCr89+ZFGPjM90Bnftr03QuqF5M eFnJUqzHBGTSgFyfLV1CDU63oNUabsbM+cQkJ51QYhD/6KUOLyTEpClvEz7tz7fuYyCn 2I8CUqJH54k/SaZQGG0YfN2IhPacZL7YuxhPi6ceMyxJPCMqKZBBfOsWAADH45U4G/fA MJVyzW5UrYpzdEM0lDZnLGlmhvUTyvPQDVnzT2TDLQhXFRRokkL9jVJOeJfhDeIQZ4ak hJag== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id js4-20020a17090797c400b009adc768b7e5si2424772ejc.939.2023.09.15.06.12.25; Fri, 15 Sep 2023 06:12:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9761268C77C; Fri, 15 Sep 2023 16:12:00 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3DD3368C7B8 for ; Fri, 15 Sep 2023 16:11:51 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 5C7AD240002 for ; Fri, 15 Sep 2023 13:11:49 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Fri, 15 Sep 2023 15:11:46 +0200 Message-Id: <20230915131147.5945-3-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230915131147.5945-1-michael@niedermayer.cc> References: <20230915131147.5945-1-michael@niedermayer.cc> X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 3/4] avutil/tx_template: Fix some signed integer overflows in DECL_FFT5() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +PI8KqgAleJV Fixes: signed integer overflow: -1364715454 + -1468954671 cannot be represented in type 'int' Fixes: 62093/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AAC_FIXED_fuzzer-5538774254485504 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavutil/tx_template.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/libavutil/tx_template.c b/libavutil/tx_template.c index 6e3b3dad338..8dc3d2519c1 100644 --- a/libavutil/tx_template.c +++ b/libavutil/tx_template.c @@ -222,8 +222,8 @@ static av_always_inline void NAME(TXComplex *out, TXComplex *in, \ BF(t[3].im, t[2].re, in[2].re, in[3].re); \ BF(t[3].re, t[2].im, in[2].im, in[3].im); \ \ - out[D0*stride].re = dc.re + t[0].re + t[2].re; \ - out[D0*stride].im = dc.im + t[0].im + t[2].im; \ + out[D0*stride].re = dc.re + (TXUSample)t[0].re + t[2].re; \ + out[D0*stride].im = dc.im + (TXUSample)t[0].im + t[2].im; \ \ SMUL(t[4].re, t[0].re, tab[0], tab[2], t[2].re, t[0].re); \ SMUL(t[4].im, t[0].im, tab[0], tab[2], t[2].im, t[0].im); \ @@ -235,14 +235,14 @@ static av_always_inline void NAME(TXComplex *out, TXComplex *in, \ BF(z0[2].re, z0[1].re, t[4].re, t[5].re); \ BF(z0[2].im, z0[1].im, t[4].im, t[5].im); \ \ - out[D1*stride].re = dc.re + z0[3].re; \ - out[D1*stride].im = dc.im + z0[0].im; \ - out[D2*stride].re = dc.re + z0[2].re; \ - out[D2*stride].im = dc.im + z0[1].im; \ - out[D3*stride].re = dc.re + z0[1].re; \ - out[D3*stride].im = dc.im + z0[2].im; \ - out[D4*stride].re = dc.re + z0[0].re; \ - out[D4*stride].im = dc.im + z0[3].im; \ + out[D1*stride].re = dc.re + (TXUSample)z0[3].re; \ + out[D1*stride].im = dc.im + (TXUSample)z0[0].im; \ + out[D2*stride].re = dc.re + (TXUSample)z0[2].re; \ + out[D2*stride].im = dc.im + (TXUSample)z0[1].im; \ + out[D3*stride].re = dc.re + (TXUSample)z0[1].re; \ + out[D3*stride].im = dc.im + (TXUSample)z0[2].im; \ + out[D4*stride].re = dc.re + (TXUSample)z0[0].re; \ + out[D4*stride].im = dc.im + (TXUSample)z0[3].im; \ } DECL_FFT5(fft5, 0, 1, 2, 3, 4) From patchwork Fri Sep 15 13:11:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 43747 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bca2:b0:149:dfde:5c0a with SMTP id fx34csp305726pzb; Fri, 15 Sep 2023 06:12:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF5BVARcXLtV8eIp0rtuUjTC6S3v2/Xk4sJyTVBVo0DC9T/mY0uDDg/yCCIjNRItYm7WnuD X-Received: by 2002:aa7:c454:0:b0:51e:3558:5eb6 with SMTP id n20-20020aa7c454000000b0051e35585eb6mr1528371edr.3.1694783568882; Fri, 15 Sep 2023 06:12:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694783568; cv=none; d=google.com; s=arc-20160816; b=cdvLFUiRjTtn5i+cmOlC+WqIqq/Y8AIQQGbNdYSIIlZ4Ll+Sa79fQkr4Vc4Y9MvKJT 7cJQ0p2ZEJSVKig1Vy/zPyDjEAqHOLyr74I+ezg2e6nBpcrlNYN2zN3SZrctNKgkNOas MgBCNFlmriabiRKSs7q000YCxxVVjJhFOeC3hYIzEGbr/kp1w5JP1BaFAGMjbU8ypZcA XnrVv15KypIq4fVaIvVQodUBrc3urvzuw4HpjzfBygS/3NadBuyu/r7qRXKV7HNkH0+R OZAERxJw3ZLMu2Ld+LXxYe/4FveEBlLCtohF4bY/IMn3/AyaeOjmWJLbSMC0sWkQhMbL XDUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to; bh=5JPshfNlHyBcEw0IS4okXqoeZ6ymM0SFrmUsnUrA+FA=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=ZNoHiWC4xR6v0rfV55mETPxCqguto8iBmYChygM6R6/YkC4WCYyxtKZYGdVgGF8sxa jWm1bWo0cJFHNhtof9TE5VHuXWXkmylvYQrX3ovgMZOM3tGtopg2Drlv7cH10JlxIeaF 2wV5OKum3Q6TH/FVVaDze3To1Z79bWioxYUthiD0A0eA2zWYanb9iWqPKQMxzy4f3Yr3 Y3U4AzKe8+bLtgTllEu0F+1nyGAqT4+n3cxm7acz0Q8rg2rjgu/9xFQ8bZmVzB2kJsa/ f5fh+vykROSSHCma3UQmBknlBoPzrR8MdnPlQInI807Juel7/n5iggODhLFyhPx3wtUU ozNQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s22-20020aa7cb16000000b005259d57fb82si3451267edt.3.2023.09.15.06.12.36; Fri, 15 Sep 2023 06:12:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9E86768C808; Fri, 15 Sep 2023 16:12:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C474968C7C2 for ; Fri, 15 Sep 2023 16:11:51 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 2968BFF807 for ; Fri, 15 Sep 2023 13:11:51 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Fri, 15 Sep 2023 15:11:47 +0200 Message-Id: <20230915131147.5945-4-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230915131147.5945-1-michael@niedermayer.cc> References: <20230915131147.5945-1-michael@niedermayer.cc> X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 4/4] avcodec/osq: avoid using too large numbers for shifts and integers in update_residue_parameter() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: qRaVYKHprJ+9 The code should be changed to not use floats in the VLC parameters This patch merely fixes undefined behavior Fixes: 2.96539e+09 is outside the range of representable values of type 'int' Fixes: Assertion n>=0 && n<=32 failed at libavcodec/get_bits.h:423 Fixes: 62241/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_OSQ_fuzzer-4525761925873664 Signed-off-by: Michael Niedermayer --- libavcodec/osq.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/libavcodec/osq.c b/libavcodec/osq.c index e2a84657fb4..e7f11691d2e 100644 --- a/libavcodec/osq.c +++ b/libavcodec/osq.c @@ -152,11 +152,15 @@ static int update_residue_parameter(OSQChannel *cb) sum = cb->sum; x = sum / cb->count; - rice_k = av_ceil_log2(x); + rice_k = ceil(log2(x)); if (rice_k >= 30) { - rice_k = floor(sum / 1.4426952 + 0.5); - if (rice_k < 1) + double f = floor(sum / 1.4426952 + 0.5); + if (f <= 1) { rice_k = 1; + } else if (f >= 31) { + rice_k = 31; + } else + rice_k = f; } return rice_k;