From patchwork Tue Aug 1 13:46:22 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Cowgill X-Patchwork-Id: 4573 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.153.134 with SMTP id b128csp3402960vse; Tue, 1 Aug 2017 06:46:45 -0700 (PDT) X-Received: by 10.223.160.26 with SMTP id k26mr14048097wrk.121.1501595204927; Tue, 01 Aug 2017 06:46:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501595204; cv=none; d=google.com; s=arc-20160816; b=CGlEkQ2uaHLxTndZsTWMOenHJKBxJx4SfDFuGCkEQAUD7su6uzdfMdP5uuFG6vMQ9w sElQfZVulDd0OS9WvkXvJcYI3nRDljaBHVQa441Hej2HZ9Ymbq8m55mEaCfotQY89kMu o2mQYA89kZBVjoKlj3UI6ebcoosHna0iRX8xdNZzGxYM2Fx0iFLsC0jWeZ8P5JQk4XYh ufS+kUF7EcGkyQY3jh+JIYZBVwM1RBoB5HCTuwBzwom67BxExWfhI1E0uutaxyptGEGa 6KNvKCQYrDwoNXJDbsJDH5nstKs+zm5Ybl3tDlvmjWHSfe6V9SKRGj9bfAt6TJKehIqU duoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to:arc-authentication-results; bh=TuZMM2WfKAKwziZynRkCjvDA/Efr72cNSh4TndVyLsQ=; b=UnjvNpzT+4X0e3/mBVA80GQIb/UA7aT+VMRWENwh4YGImR1OfIS8e3KVyMnGEAz3Sn wEoKMPXpzmeCyh2jHr5OxnjU0Gmy+uIH7mANP52j/1N3TO2EK7AtVMuLTuwUS+j26wiz qb4WMjXRDB1/jnDBCllxQCkzrfsWqfQZwJ4JVNhF8BKdoHLm6rfAfwUuH34BkK6M3ec0 MmJ0qk1Q7MI2WdhlaTIm7tr8s/v93DE3on4QWJvIjZ+lKe+B3lfVz4u81j4uyOnLbDME fX78X03gWHZD4lwzuIti6b1FYJOiexUd8fhi8t6W9NxE4sF/j2JBE3yPeR84WmvqS6QD +XPQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a141si1335347wma.43.2017.08.01.06.46.44; Tue, 01 Aug 2017 06:46:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 98C3A68A457; Tue, 1 Aug 2017 16:46:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mailapp01.imgtec.com (mailapp01.imgtec.com [195.59.15.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A631B688396 for ; Tue, 1 Aug 2017 16:46:33 +0300 (EEST) Received: from HHMAIL01.hh.imgtec.org (unknown [10.100.10.19]) by Forcepoint Email with ESMTPS id 6ABC3AB9DE80B for ; Tue, 1 Aug 2017 14:46:31 +0100 (IST) Received: from LDT-J-COWGILL.le.imgtec.org (10.150.130.85) by HHMAIL01.hh.imgtec.org (10.100.10.21) with Microsoft SMTP Server (TLS) id 14.3.294.0; Tue, 1 Aug 2017 14:46:35 +0100 From: James Cowgill To: Date: Tue, 1 Aug 2017 14:46:22 +0100 Message-ID: <20170801134622.632-1-James.Cowgill@imgtec.com> X-Mailer: git-send-email 2.13.3 MIME-Version: 1.0 X-Originating-IP: [10.150.130.85] Subject: [FFmpeg-devel] [PATCH] swscale: fix gbrap16 alpha channel issues X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes filter-pixfmts-scale test failing on big-endian systems due to alpSrc not being cast to (const int32_t**). Also fixes distortions in the output alpha channel values by copying the alpha channel code from the rgba64 case found elsewhere in output.c. Fixes ticket 6555. Signed-off-by: James Cowgill --- libswscale/output.c | 15 ++++++++------- tests/ref/fate/filter-pixfmts-scale | 4 ++-- 2 files changed, 10 insertions(+), 9 deletions(-) diff --git a/libswscale/output.c b/libswscale/output.c index 9774e9f327..8e5ec0a256 100644 --- a/libswscale/output.c +++ b/libswscale/output.c @@ -2026,17 +2026,18 @@ yuv2gbrp16_full_X_c(SwsContext *c, const int16_t *lumFilter, const int16_t **lumSrcx, int lumFilterSize, const int16_t *chrFilter, const int16_t **chrUSrcx, const int16_t **chrVSrcx, int chrFilterSize, - const int16_t **alpSrc, uint8_t **dest, + const int16_t **alpSrcx, uint8_t **dest, int dstW, int y) { const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(c->dstFormat); int i; - int hasAlpha = (desc->flags & AV_PIX_FMT_FLAG_ALPHA) && alpSrc; + int hasAlpha = (desc->flags & AV_PIX_FMT_FLAG_ALPHA) && alpSrcx; uint16_t **dest16 = (uint16_t**)dest; const int32_t **lumSrc = (const int32_t**)lumSrcx; const int32_t **chrUSrc = (const int32_t**)chrUSrcx; const int32_t **chrVSrc = (const int32_t**)chrVSrcx; - int A = 0; // init to silence warning + const int32_t **alpSrc = (const int32_t**)alpSrcx; + int A = 0xFFFF << 14; for (i = 0; i < dstW; i++) { int j; @@ -2059,13 +2060,13 @@ yuv2gbrp16_full_X_c(SwsContext *c, const int16_t *lumFilter, V >>= 14; if (hasAlpha) { - A = 1 << 18; + A = -0x40000000; for (j = 0; j < lumFilterSize; j++) A += alpSrc[j][i] * lumFilter[j]; - if (A & 0xF8000000) - A = av_clip_uintp2(A, 27); + A >>= 1; + A += 0x20002000; } Y -= c->yuv2rgb_y_offset; @@ -2083,7 +2084,7 @@ yuv2gbrp16_full_X_c(SwsContext *c, const int16_t *lumFilter, dest16[1][i] = B >> 14; dest16[2][i] = R >> 14; if (hasAlpha) - dest16[3][i] = A >> 11; + dest16[3][i] = av_clip_uintp2(A, 30) >> 14; } if ((!isBE(c->dstFormat)) != (!HAVE_BIGENDIAN)) { for (i = 0; i < dstW; i++) { diff --git a/tests/ref/fate/filter-pixfmts-scale b/tests/ref/fate/filter-pixfmts-scale index 9b601b71da..dcc34bd4d1 100644 --- a/tests/ref/fate/filter-pixfmts-scale +++ b/tests/ref/fate/filter-pixfmts-scale @@ -23,8 +23,8 @@ gbrap10be 6d89abb9248006c3e9017545e9474654 gbrap10le cf974e23f485a10740f5de74a5c8c3df gbrap12be 1d9b57766ba9c2192403f43967cb9af0 gbrap12le bb1ba1c157717db3dd612a76d38a018e -gbrap16be 81542b96575d1fe3b239d23899f5ece3 -gbrap16le 6feb8b9da131917abe867e0eaaf07b90 +gbrap16be c72b935a6e57a8e1c37bff08c2db55b1 +gbrap16le 13eb0e62b1ac9c1c86c81521eaefab5f gbrp dc3387f925f972c61aae7eb23cdc19f0 gbrp10be 0277d4c3a8498d75e2783fb81379e481 gbrp10le f3d70f8ab845c3c9b8f7452e4a6e285a