From patchwork Tue Mar 29 12:44:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Martin_Storsj=C3=B6?= X-Patchwork-Id: 35045 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c05:b0:7a:e998:b410 with SMTP id bw5csp161116pzb; Tue, 29 Mar 2022 05:45:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz+IjMN6pYTBtvS9zNHWfmywwlWU+2pC/g8FW5jDlC+nqHs4oaU5i0NJCmXXdMdji5dX2ZY X-Received: by 2002:a17:906:c307:b0:6df:c7d0:90e8 with SMTP id s7-20020a170906c30700b006dfc7d090e8mr33631173ejz.421.1648557900256; Tue, 29 Mar 2022 05:45:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1648557900; cv=none; d=google.com; s=arc-20160816; b=lTuS6QfqPg52Ng2slS08crxyrMZlfBC0FZj4VPtozzwiCGj7xG70qFtQznf3c23Q6O QBM+E6HXasFB5RhR76koPiJffLLpiyQJM2jwdLozr/90UBgFtWyStrCz6C6njWPYkQJD FzazQ1tlYYvHfAqgZR/RvyjveHx6nampN2eGn+gazdbO/1onAHDapLVEftEQW9H+UhIX 0dYsBXXdLo3cO9Bzx39PIvK6+WkCr+G3wXqLcNTOXjUNs8QXGXbtEqheZhVuzTqVUjBl gTtM8dAXtp+2hYMpShzw5D0LCuJs603Y+mhJ/YYDi7rydHifdedwuU6IdjHhwPHVQMZO DStA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=pQqpN3M34BqMXzaWcPvwtFcFzY1JFY4AKiPb7f/NZmU=; b=fn+VruGFK5k4ilaQqIbRq0NSdOK/g9mKQ/MsN7Jq0L0bWtMUEInk1hY1CzsMNtvzGF RqJssIFN5+4zoQnBl9kop8ypxvcFquRVLzqopJMX6PjOfjKzJbrJ3JV7Fc4v5rh1Gi3H Uk+w1NDQw57neSQktLOsy3V85QBn+aQuG12H6aPvxjpJQTYQXIq64cWnlxEHaYOntKJj Lz4w/Yfo9/swjd+4iG027DVeu6O2VJvTp2lHwzMwU5mMO/4apWgEGEObobXiatZe4PQd LTTj8Epv9ynvrvDN/G3DZHzfBnZbo/yLd08ilIEmmzvArvcegYxbn7DawXTrhJVR5AnT sz4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@martin-st.20210112.gappssmtp.com header.s=20210112 header.b=0kydRedb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i16-20020a170906a29000b006df76385c26si18035324ejz.198.2022.03.29.05.44.59; Tue, 29 Mar 2022 05:45:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@martin-st.20210112.gappssmtp.com header.s=20210112 header.b=0kydRedb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7954768B26B; Tue, 29 Mar 2022 15:44:57 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f182.google.com (mail-lj1-f182.google.com [209.85.208.182]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E81F968B195 for ; Tue, 29 Mar 2022 15:44:50 +0300 (EEST) Received: by mail-lj1-f182.google.com with SMTP id q5so23256082ljb.11 for ; Tue, 29 Mar 2022 05:44:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=martin-st.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=c7x/4ILq28FLrbrN9oHMc2x4xqVDSAXppy+Widz68iM=; b=0kydRedb2FJf+NlhtTAHKJZLpo5gVBtEvl8pBO1aUtKE5+7a9JPITQt463XiCoqfuQ 6jOf2nXHlq6BLAMUWaMgHnFyBAAjmqcMD2CnnQ5Ngs2uNHYT5I2tjVaeuwVhHDM+aPIz CaCpS55xDMJe0/QYsdO5kET9c/kuNWd91Enkj+Nv02lyT/g9H8L0RdGtYL9SMniupCWJ A5A226awrbw/tdSJC3KN/x1BPSiO/wHJECPUS1u5Z0vbJZJBLRBOTPcdvDJ3uIS8MeDu ODZqnLiEsi7jfODfP1hakXTN7UaHWr4L4jdDlNiDPZxbzVxJrKzrdXoVzFgs6+EIuLcr 8gUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=c7x/4ILq28FLrbrN9oHMc2x4xqVDSAXppy+Widz68iM=; b=tZcAFFZoyfRMm4CicapfjHoiN+iVgV27qMVjkzCYzQOecEUrmTORqxdDp9QbCaj0EL wLPjfwTYYVGV/fKTWVEqyETYYIlOFpFEGqnL5rFcJDlu49xSCYk/GXF6w8fSCQ1qbaef o7DsM03eoNX1rTKKJo/iMhvg4jXzNpaddi1V2HYSfqCRps5RLfgdX4qpumCLVoP8VpKv jRwWZ8rQZDzuv094wgSbI9mrhbok7skLTWy4r68jRJSbz6fMhvFyo6luHUCnQIxFyYg1 W7RwazkF+h3swA7tKyS8pDeGjQabzADcjqXc/3yqqgK5L4pMvB8tpEi1v5KnjjdmvuQO yCyA== X-Gm-Message-State: AOAM530iLDfi61p3x3aI0Wg0yxsnPzDYl7eO6beOWSLdshvY87oPwl5K 95oS+xUQVWsPua0ZPyIxzZkRYDQ6/vO65/HV X-Received: by 2002:a2e:5753:0:b0:249:7bc1:3934 with SMTP id r19-20020a2e5753000000b002497bc13934mr2407596ljd.414.1648557890188; Tue, 29 Mar 2022 05:44:50 -0700 (PDT) Received: from localhost.localdomain (dsl-tkubng21-58c01c-243.dhcp.inet.fi. [88.192.28.243]) by smtp.gmail.com with ESMTPSA id 24-20020ac24d58000000b0044a6a1c7b0asm1727431lfp.139.2022.03.29.05.44.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Mar 2022 05:44:49 -0700 (PDT) From: =?utf-8?q?Martin_Storsj=C3=B6?= To: ffmpeg-devel@ffmpeg.org Date: Tue, 29 Mar 2022 15:44:48 +0300 Message-Id: <20220329124448.2593145-1-martin@martin.st> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] vc1dsp: Change remaining stride parameters to ptrdiff_t X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Ben Avison , =?utf-8?q?Martin_Storsj=C3=B6?= Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: NazgeXu8MiV1 The existing x86 assembly for loop filters uses the stride as a full register without clearing/sign extending the upper half of the registers on x86_64. This avoids crashes if the caller would have passed nonzero bits in the previously undefined upper 32 bits of the parameters. Signed-off-by: Martin Storsjö --- libavcodec/vc1dsp.c | 20 ++++++++++---------- libavcodec/vc1dsp.h | 16 ++++++++-------- libavcodec/x86/vc1dsp_init.c | 16 ++++++++-------- 3 files changed, 26 insertions(+), 26 deletions(-) diff --git a/libavcodec/vc1dsp.c b/libavcodec/vc1dsp.c index a29b91bf3d..c567bc43e8 100644 --- a/libavcodec/vc1dsp.c +++ b/libavcodec/vc1dsp.c @@ -36,7 +36,7 @@ #include "startcode.h" /* Apply overlap transform to horizontal edge */ -static void vc1_v_overlap_c(uint8_t *src, int stride) +static void vc1_v_overlap_c(uint8_t *src, ptrdiff_t stride) { int i; int a, b, c, d; @@ -60,7 +60,7 @@ static void vc1_v_overlap_c(uint8_t *src, int stride) } /* Apply overlap transform to vertical edge */ -static void vc1_h_overlap_c(uint8_t *src, int stride) +static void vc1_h_overlap_c(uint8_t *src, ptrdiff_t stride) { int i; int a, b, c, d; @@ -146,7 +146,7 @@ static void vc1_h_s_overlap_c(int16_t *left, int16_t *right, int left_stride, in * @return whether other 3 pairs should be filtered or not * @see 8.6 */ -static av_always_inline int vc1_filter_line(uint8_t *src, int stride, int pq) +static av_always_inline int vc1_filter_line(uint8_t *src, ptrdiff_t stride, int pq) { int a0 = (2 * (src[-2 * stride] - src[1 * stride]) - 5 * (src[-1 * stride] - src[0 * stride]) + 4) >> 3; @@ -195,7 +195,7 @@ static av_always_inline int vc1_filter_line(uint8_t *src, int stride, int pq) * @param pq block quantizer * @see 8.6 */ -static inline void vc1_loop_filter(uint8_t *src, int step, int stride, +static inline void vc1_loop_filter(uint8_t *src, int step, ptrdiff_t stride, int len, int pq) { int i; @@ -212,32 +212,32 @@ static inline void vc1_loop_filter(uint8_t *src, int step, int stride, } } -static void vc1_v_loop_filter4_c(uint8_t *src, int stride, int pq) +static void vc1_v_loop_filter4_c(uint8_t *src, ptrdiff_t stride, int pq) { vc1_loop_filter(src, 1, stride, 4, pq); } -static void vc1_h_loop_filter4_c(uint8_t *src, int stride, int pq) +static void vc1_h_loop_filter4_c(uint8_t *src, ptrdiff_t stride, int pq) { vc1_loop_filter(src, stride, 1, 4, pq); } -static void vc1_v_loop_filter8_c(uint8_t *src, int stride, int pq) +static void vc1_v_loop_filter8_c(uint8_t *src, ptrdiff_t stride, int pq) { vc1_loop_filter(src, 1, stride, 8, pq); } -static void vc1_h_loop_filter8_c(uint8_t *src, int stride, int pq) +static void vc1_h_loop_filter8_c(uint8_t *src, ptrdiff_t stride, int pq) { vc1_loop_filter(src, stride, 1, 8, pq); } -static void vc1_v_loop_filter16_c(uint8_t *src, int stride, int pq) +static void vc1_v_loop_filter16_c(uint8_t *src, ptrdiff_t stride, int pq) { vc1_loop_filter(src, 1, stride, 16, pq); } -static void vc1_h_loop_filter16_c(uint8_t *src, int stride, int pq) +static void vc1_h_loop_filter16_c(uint8_t *src, ptrdiff_t stride, int pq) { vc1_loop_filter(src, stride, 1, 16, pq); } diff --git a/libavcodec/vc1dsp.h b/libavcodec/vc1dsp.h index c6443acb20..34f3c89d85 100644 --- a/libavcodec/vc1dsp.h +++ b/libavcodec/vc1dsp.h @@ -42,16 +42,16 @@ typedef struct VC1DSPContext { void (*vc1_inv_trans_8x4_dc)(uint8_t *dest, ptrdiff_t stride, int16_t *block); void (*vc1_inv_trans_4x8_dc)(uint8_t *dest, ptrdiff_t stride, int16_t *block); void (*vc1_inv_trans_4x4_dc)(uint8_t *dest, ptrdiff_t stride, int16_t *block); - void (*vc1_v_overlap)(uint8_t *src, int stride); - void (*vc1_h_overlap)(uint8_t *src, int stride); + void (*vc1_v_overlap)(uint8_t *src, ptrdiff_t stride); + void (*vc1_h_overlap)(uint8_t *src, ptrdiff_t stride); void (*vc1_v_s_overlap)(int16_t *top, int16_t *bottom); void (*vc1_h_s_overlap)(int16_t *left, int16_t *right, int left_stride, int right_stride, int flags); - void (*vc1_v_loop_filter4)(uint8_t *src, int stride, int pq); - void (*vc1_h_loop_filter4)(uint8_t *src, int stride, int pq); - void (*vc1_v_loop_filter8)(uint8_t *src, int stride, int pq); - void (*vc1_h_loop_filter8)(uint8_t *src, int stride, int pq); - void (*vc1_v_loop_filter16)(uint8_t *src, int stride, int pq); - void (*vc1_h_loop_filter16)(uint8_t *src, int stride, int pq); + void (*vc1_v_loop_filter4)(uint8_t *src, ptrdiff_t stride, int pq); + void (*vc1_h_loop_filter4)(uint8_t *src, ptrdiff_t stride, int pq); + void (*vc1_v_loop_filter8)(uint8_t *src, ptrdiff_t stride, int pq); + void (*vc1_h_loop_filter8)(uint8_t *src, ptrdiff_t stride, int pq); + void (*vc1_v_loop_filter16)(uint8_t *src, ptrdiff_t stride, int pq); + void (*vc1_h_loop_filter16)(uint8_t *src, ptrdiff_t stride, int pq); /* put 8x8 block with bicubic interpolation and quarterpel precision * last argument is actually round value instead of height diff --git a/libavcodec/x86/vc1dsp_init.c b/libavcodec/x86/vc1dsp_init.c index 8e0c284cc0..2fbf0b3a74 100644 --- a/libavcodec/x86/vc1dsp_init.c +++ b/libavcodec/x86/vc1dsp_init.c @@ -34,18 +34,18 @@ #include "config.h" #define LOOP_FILTER(EXT) \ -void ff_vc1_v_loop_filter4_ ## EXT(uint8_t *src, int stride, int pq); \ -void ff_vc1_h_loop_filter4_ ## EXT(uint8_t *src, int stride, int pq); \ -void ff_vc1_v_loop_filter8_ ## EXT(uint8_t *src, int stride, int pq); \ -void ff_vc1_h_loop_filter8_ ## EXT(uint8_t *src, int stride, int pq); \ +void ff_vc1_v_loop_filter4_ ## EXT(uint8_t *src, ptrdiff_t stride, int pq); \ +void ff_vc1_h_loop_filter4_ ## EXT(uint8_t *src, ptrdiff_t stride, int pq); \ +void ff_vc1_v_loop_filter8_ ## EXT(uint8_t *src, ptrdiff_t stride, int pq); \ +void ff_vc1_h_loop_filter8_ ## EXT(uint8_t *src, ptrdiff_t stride, int pq); \ \ -static void vc1_v_loop_filter16_ ## EXT(uint8_t *src, int stride, int pq) \ +static void vc1_v_loop_filter16_ ## EXT(uint8_t *src, ptrdiff_t stride, int pq) \ { \ ff_vc1_v_loop_filter8_ ## EXT(src, stride, pq); \ ff_vc1_v_loop_filter8_ ## EXT(src+8, stride, pq); \ } \ \ -static void vc1_h_loop_filter16_ ## EXT(uint8_t *src, int stride, int pq) \ +static void vc1_h_loop_filter16_ ## EXT(uint8_t *src, ptrdiff_t stride, int pq) \ { \ ff_vc1_h_loop_filter8_ ## EXT(src, stride, pq); \ ff_vc1_h_loop_filter8_ ## EXT(src+8*stride, stride, pq); \ @@ -56,9 +56,9 @@ LOOP_FILTER(mmxext) LOOP_FILTER(sse2) LOOP_FILTER(ssse3) -void ff_vc1_h_loop_filter8_sse4(uint8_t *src, int stride, int pq); +void ff_vc1_h_loop_filter8_sse4(uint8_t *src, ptrdiff_t stride, int pq); -static void vc1_h_loop_filter16_sse4(uint8_t *src, int stride, int pq) +static void vc1_h_loop_filter16_sse4(uint8_t *src, ptrdiff_t stride, int pq) { ff_vc1_h_loop_filter8_sse4(src, stride, pq); ff_vc1_h_loop_filter8_sse4(src+8*stride, stride, pq);