From patchwork Thu Dec 27 19:28:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Mogurenko X-Patchwork-Id: 11567 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 6A32544C32A for ; Thu, 27 Dec 2018 21:34:24 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 10ECD68AB29; Thu, 27 Dec 2018 21:34:21 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f193.google.com (mail-lj1-f193.google.com [209.85.208.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9B93368AAFB for ; Thu, 27 Dec 2018 21:34:14 +0200 (EET) Received: by mail-lj1-f193.google.com with SMTP id v1-v6so17072766ljd.0 for ; Thu, 27 Dec 2018 11:34:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=y4U00ZlbNpwfL37XhGy//Gr/+NbhrvjDYqtY6ed9erM=; b=LKZotBsc5r2BbQNpGSTdT9nMxTGZMu0FicqUlL5pqS63hKVQeiFt6vZeoA+VC9qEL6 VjC8OHX5l8hnS7+7FDkJbgnwaGeMFlPSntk+yXbhI6334r4hXFaLf1gW2JWg+PTUuc7C Xh58/O9YrAziumiSBSGxGOTtNO8ZPt6edtAHIQpqbQpzSIol3zdkavXRFH/ew2MgTiwi +MAUs3Bwwt1mVafSrcSqjDzc5XTsvZPgjbIjrETRZ1HKxpigKMudvNEgyboxw3NvPh1S HJM7riQ3A8aO111bjXiJVpp7RWanJ90bAx1twMjCr5Cg9K3baeyca2XaT6uWLVxEzSCd Edlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:subject:date:message-id :mime-version:content-transfer-encoding; bh=y4U00ZlbNpwfL37XhGy//Gr/+NbhrvjDYqtY6ed9erM=; b=HuQxv1rvX+1YdPY72rhT9wulMGW/9930YxaV8Tj/hIxPKSnhvxuPmkeiX0FPORYkRB 5fekHiPcqtFbUh5IK6NPhk9gAP0x93Bw9YwWw8+klKkqgbCjXbHKXfGqAOnz6EyARpQp u5O7MboFPD6t9nVvvy08ffo1AmJhVYuiUjLB1l6Nq2kEEOfXJTaHhrFZSSnlqI88qve+ UuT7FVfnCKh8P2xaehPkvaD+feywBM6dgf5w5HyeizMl/ZlGmqFAk8b8vxghtnDVydJ9 cQostmbr9NvfbUl67AobLwYAbDTlgnC+33u70ti7xSeyxkRJf9II5kUsAKCLJTrTauHl UtYw== X-Gm-Message-State: AJcUukdDzAv1r7bOWbbbO09JdYJzQxVILKTL5JDnZGSWeOEM8VGpTRHX nbzp36qbP/Ua4h4mMsGHJ/x0e3oX X-Google-Smtp-Source: ALg8bN6NeM3UdGB6MUmF5nN1dcrONJd/I3515n3LtKazJnWOA0UyGS49fpC+vE/1y8O3xEzQiD1jYw== X-Received: by 2002:a2e:4942:: with SMTP id b2-v6mr14106853ljd.168.1545938905054; Thu, 27 Dec 2018 11:28:25 -0800 (PST) Received: from localhost.localdomain ([77.121.132.178]) by smtp.gmail.com with ESMTPSA id h203sm7725838lfe.44.2018.12.27.11.28.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 27 Dec 2018 11:28:24 -0800 (PST) From: Alex Mogurenko To: ffmpeg-devel@ffmpeg.org Date: Thu, 27 Dec 2018 21:28:06 +0200 Message-Id: <20181227192806.8152-1-alex@mogurenko.com> X-Mailer: git-send-email 2.19.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] fixed luma quantizing when q >= MAX_STORED_Q X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/proresenc_kostya.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/libavcodec/proresenc_kostya.c b/libavcodec/proresenc_kostya.c index 9a77d24fb6..e045a972f1 100644 --- a/libavcodec/proresenc_kostya.c +++ b/libavcodec/proresenc_kostya.c @@ -222,6 +222,7 @@ typedef struct ProresThreadData { DECLARE_ALIGNED(16, int16_t, blocks)[MAX_PLANES][64 * 4 * MAX_MBS_PER_SLICE]; DECLARE_ALIGNED(16, uint16_t, emu_buf)[16 * 16]; int16_t custom_q[64]; + int16_t custom_chroma_q[64]; struct TrellisNode *nodes; } ProresThreadData; @@ -232,6 +233,7 @@ typedef struct ProresContext { int16_t quants[MAX_STORED_Q][64]; int16_t quants_chroma[MAX_STORED_Q][64]; int16_t custom_q[64]; + int16_t custom_chroma_q[64]; const uint8_t *quant_mat; const uint8_t *quant_chroma_mat; const uint8_t *scantable; @@ -574,7 +576,7 @@ static int encode_slice(AVCodecContext *avctx, const AVFrame *pic, qmat_chroma = ctx->quants_chroma[quant]; } else { qmat = ctx->custom_q; - qmat_chroma = ctx->custom_q; + qmat_chroma = ctx->custom_chroma_q; for (i = 0; i < 64; i++) { qmat[i] = ctx->quant_mat[i] * quant; qmat_chroma[i] = ctx->quant_chroma_mat[i] * quant; @@ -902,7 +904,7 @@ static int find_slice_quant(AVCodecContext *avctx, qmat_chroma = ctx->quants_chroma[q]; } else { qmat = td->custom_q; - qmat_chroma = td->custom_q; + qmat_chroma = td->custom_chroma_q; for (i = 0; i < 64; i++) { qmat[i] = ctx->quant_mat[i] * q; qmat_chroma[i] = ctx->quant_chroma_mat[i] * q;