From patchwork Sun Jan 21 14:18:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jianfeng Zheng X-Patchwork-Id: 45690 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a402:b0:199:de12:6fa6 with SMTP id z2csp943859pzk; Sun, 21 Jan 2024 06:19:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IHfMpGEsX1PPEU1+Eg/SZn0J4E0q60nxX1upD2ppb6xMPZhdBD3hnSr9vslNenB+9uFiM6C X-Received: by 2002:a5d:56c2:0:b0:339:2bf5:7efa with SMTP id m2-20020a5d56c2000000b003392bf57efamr2128834wrw.6.1705846790313; Sun, 21 Jan 2024 06:19:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1705846790; cv=none; d=google.com; s=arc-20160816; b=F9p0+x+mlVmZTScqqazfDYkg0M8z8H4PrP2Q2kiYhzOQF3e4Xt5aj/kfKNRRGr1N2M SgVNia6UP9nAOyS+opQrGnwcfUe9fJwqlG3eFVEuzeI4T1jjnJaxvRqFAsLT7FHeyilD YETGYe7MCeYtqLngXMJRckpoKOsDjD6crV4w8cXKbLhxKognzPku4o/5MMNDLZzBEBtR 7nFRS/xcwcvFxDQyK5aZRdFHlNcNBmm8XhJ7RNxvfJGI7AYRxKwJvGnq+R28vQ1eTLy9 Vu5toC2ZGSWvjGjO3RA9tUXHi096m1tEB1pZONVBFxGosml7OJs1RdrfFXI3puLpRBKA iEKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=hFoNwueKXp/vKwZNfM6EyzLdpL/oAc9617NsIqCiR2Y=; fh=iiAW4vjlCGdXV3Yszi/eNCPr6t9/oDQbs1mdNL59KvI=; b=rn9D3hLBK7n250j6BgGRsBK8lLJOOedTFHVmGEB8vAikM0YYZqklXMFAXlWNVJPARZ eKTlfWMobHo8uRa4FuDZOWBHxFiHnbgHRJEjSgBnYIVhgt7OYfqFYy4bI2z8qSGMElGC uGlU8Jru6cTHyjg0gCoCR8/dnor9XK4tZ4VTOGcKcyp/VMt3DIhExpcHYN66kPzOu7sa 6zwLHuiNjMMph8U9cK6KxsuBezqtwQaeH46/bnEG7+YeHtUUfrAtNuI013z26S3ukYns 0OvdS68fioAn1lLYwZWC4XppHcHZ9DwjL+AZhImQiDB47NKcQgeg6bZE58dlbRBwC0pk tA9w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=GAj53RGB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n17-20020a170906119100b00a2cd76d079fsi8697528eja.773.2024.01.21.06.19.50; Sun, 21 Jan 2024 06:19:50 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=GAj53RGB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7D82868D0BA; Sun, 21 Jan 2024 16:19:26 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D580968CBFD for ; Sun, 21 Jan 2024 16:19:19 +0200 (EET) Received: by mail-pg1-f172.google.com with SMTP id 41be03b00d2f7-5ceb3fe708eso925740a12.3 for ; Sun, 21 Jan 2024 06:19:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705846758; x=1706451558; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iFPuLOLB2v4JBf6SNnr2wuhaR8jMdEL9C7VTl0UVBv4=; b=GAj53RGBZtpCdpNUE/bSoHL4wLrVSX1Qjrxy345Nyl+R0C7aG9yWaHobdHIwG70caL TmqnqSrKG178OFZk5TYCp1IooZEr1WmyK3Z2CEZIgHT6YvZzyfNSxf8RGfADV21xAVkK 2Bqx6Juc3m3KiEwrlXmVYbkfJ2ifb31WldqlxNkAes59zRJeGnJ+cP7oBfhTWvClVob0 IHtzZp5/BoUaIUpNy4AARaGJPxKy/3bTX97gVRulKO9XVqst3e/xT/GYyt7kF3P7v1j/ hKilEGQ11s2mrefFHDGO7Twe/sSp2DOY+NZJ5KX8dziGa5A/sqyNLzftTM6uiERsKfxl iKWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705846758; x=1706451558; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iFPuLOLB2v4JBf6SNnr2wuhaR8jMdEL9C7VTl0UVBv4=; b=viaS/h9j2gqV507BzMGqvuZIqdNiKzb/gSKMkfggg3iPR/moZLXbCXBzbI9mQBWwUv BOXLfM17U3Nbp4cMHnQExLJOmadYCdCBtjPQN6KC6q0y1UV1BNd7pczAS8YE5sUer7bn xUI0R2S32uezg0kqwMrmEqoxhS343Zu3IAl3KkEXTtZmiBeMsSaydwM7Z5C1GrRks9W0 wrJpzw7LasKV6lzkl6bm8crk+yYFVYfojGbvhDDaHAe+qcy6ZU7NaJ60zvtZaMtv29CY 6vkBFFYEctoOcpZipB3kPLeKpKfTvKUwq7MYzAIzwgT5DonrXphgJiiuALwvRSS16X71 iCJg== X-Gm-Message-State: AOJu0YzJivHjuKUVlSs8R65Gl/zfyZ2nJg8qtkhoyVJwhG2oVeR1ZBdn hwUW/dXoantv0M3BiL6qnYG2uy3km1jwNErj2rhEXBxsCJC5YX0YRILnyskY/oA= X-Received: by 2002:a17:902:7448:b0:1d4:47d4:82b4 with SMTP id e8-20020a170902744800b001d447d482b4mr1038899plt.15.1705846757653; Sun, 21 Jan 2024 06:19:17 -0800 (PST) Received: from localhost.localdomain ([183.242.63.6]) by smtp.gmail.com with ESMTPSA id bg4-20020a1709028e8400b001d4b73f60e4sm5840726plb.149.2024.01.21.06.19.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Jan 2024 06:19:17 -0800 (PST) From: "jianfeng.zheng" X-Google-Original-From: "jianfeng.zheng" To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Jan 2024 22:18:44 +0800 Message-Id: <20240121141846.4077778-5-jianfeng.zheng@mthreads.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240121141846.4077778-1-jianfeng.zheng@mthreads.com> References: <20240121141846.4077778-1-jianfeng.zheng@mthreads.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [vaapi-cavs 5/7] cavs: decode wqm and slice weighting for future usage X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "jianfeng.zheng" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IlHpyUsWsZmX Signed-off-by: jianfeng.zheng --- libavcodec/cavs.h | 26 +++++++- libavcodec/cavsdec.c | 142 +++++++++++++++++++++++++++++++++++++------ 2 files changed, 147 insertions(+), 21 deletions(-) diff --git a/libavcodec/cavs.h b/libavcodec/cavs.h index f490657959..33ef10e850 100644 --- a/libavcodec/cavs.h +++ b/libavcodec/cavs.h @@ -186,12 +186,36 @@ typedef struct AVSContext { int mb_width, mb_height; int width, height; int stream_revision; ///<0 for samples from 2006, 1 for rm52j encoder - int progressive; + int progressive_seq; + int progressive_frame; int pic_structure; int skip_mode_flag; ///< select between skip_count or one skip_flag per MB int loop_filter_disable; int alpha_offset, beta_offset; int ref_flag; + + /** \defgroup guangdian profile + * @{ + */ + int aec_flag; + int weight_quant_flag; + int chroma_quant_param_delta_cb; + int chroma_quant_param_delta_cr; + uint8_t wqm_8x8[64]; + /**@}*/ + + /** \defgroup slice weighting + * FFmpeg don't support slice weighting natively, but maybe needed for HWaccel. + * @{ + */ + uint32_t slice_weight_pred_flag : 1; + uint32_t mb_weight_pred_flag : 1; + uint8_t luma_scale[4]; + int8_t luma_shift[4]; + uint8_t chroma_scale[4]; + int8_t chroma_shift[4]; + /**@}*/ + int mbx, mby, mbidx; ///< macroblock coordinates int flags; ///< availability flags of neighbouring macroblocks int stc; ///< last start code diff --git a/libavcodec/cavsdec.c b/libavcodec/cavsdec.c index 6f462d861c..8d3ba530a6 100644 --- a/libavcodec/cavsdec.c +++ b/libavcodec/cavsdec.c @@ -30,6 +30,7 @@ #include "avcodec.h" #include "get_bits.h" #include "golomb.h" +#include "profiles.h" #include "cavs.h" #include "codec_internal.h" #include "decode.h" @@ -37,6 +38,43 @@ #include "mpeg12data.h" #include "startcode.h" +static const uint8_t default_wq_param[4][6] = { + {128, 98, 106, 116, 116, 128}, + {135, 143, 143, 160, 160, 213}, + {128, 98, 106, 116, 116, 128}, + {128, 128, 128, 128, 128, 128}, +}; +static const uint8_t wq_model_2_param[4][64] = { + { + 0, 0, 0, 4, 4, 4, 5, 5, + 0, 0, 3, 3, 3, 3, 5, 5, + 0, 3, 2, 2, 1, 1, 5, 5, + 4, 3, 2, 2, 1, 5, 5, 5, + 4, 3, 1, 1, 5, 5, 5, 5, + 4, 3, 1, 5, 5, 5, 5, 5, + 5, 5, 5, 5, 5, 5, 5, 5, + 5, 5, 5, 5, 5, 5, 5, 5, + }, { + 0, 0, 0, 4, 4, 4, 5, 5, + 0, 0, 4, 4, 4, 4, 5, 5, + 0, 3, 2, 2, 2, 1, 5, 5, + 3, 3, 2, 2, 1, 5, 5, 5, + 3, 3, 2, 1, 5, 5, 5, 5, + 3, 3, 1, 5, 5, 5, 5, 5, + 5, 5, 5, 5, 5, 5, 5, 5, + 5, 5, 5, 5, 5, 5, 5, 5, + }, { + 0, 0, 0, 4, 4, 3, 5, 5, + 0, 0, 4, 4, 3, 2, 5, 5, + 0, 4, 4, 3, 2, 1, 5, 5, + 4, 4, 3, 2, 1, 5, 5, 5, + 4, 3, 2, 1, 5, 5, 5, 5, + 3, 2, 1, 5, 5, 5, 5, 5, + 5, 5, 5, 5, 5, 5, 5, 5, + 5, 5, 5, 5, 5, 5, 5, 5, + } +}; + static const uint8_t mv_scan[4] = { MV_FWD_X0, MV_FWD_X1, MV_FWD_X2, MV_FWD_X3 @@ -927,7 +965,11 @@ static int decode_mb_b(AVSContext *h, enum cavs_mb mb_type) static inline int decode_slice_header(AVSContext *h, GetBitContext *gb) { - if (h->stc > 0xAF) + int i, nref; + + av_log(h->avctx, AV_LOG_TRACE, "slice start code 0x%02x\n", h->stc); + + if (h->stc > SLICE_MAX_START_CODE) av_log(h->avctx, AV_LOG_ERROR, "unexpected start code 0x%02x\n", h->stc); if (h->stc >= h->mb_height) { @@ -946,11 +988,29 @@ static inline int decode_slice_header(AVSContext *h, GetBitContext *gb) } /* inter frame or second slice can have weighting params */ if ((h->cur.f->pict_type != AV_PICTURE_TYPE_I) || - (!h->pic_structure && h->mby >= h->mb_width / 2)) - if (get_bits1(gb)) { //slice_weighting_flag + (!h->pic_structure && h->mby >= h->mb_height / 2)) { + h->slice_weight_pred_flag = get_bits1(gb); + if (h->slice_weight_pred_flag) { + nref = h->cur.f->pict_type == AV_PICTURE_TYPE_I ? 1 : (h->pic_structure ? 2 : 4); + for (i = 0; i < nref; i++) { + h->luma_scale[i] = get_bits(gb, 8); + h->luma_shift[i] = get_sbits(gb, 8); + skip_bits1(gb); + h->chroma_scale[i] = get_bits(gb, 8); + h->chroma_shift[i] = get_sbits(gb, 8); + skip_bits1(gb); + } + h->mb_weight_pred_flag = get_bits1(gb); + if (!h->avctx->hwaccel) { av_log(h->avctx, AV_LOG_ERROR, "weighted prediction not yet supported\n"); } + } + } + if (h->aec_flag) { + align_get_bits(gb); + } + return 0; } @@ -1108,7 +1168,11 @@ static int decode_pic(AVSContext *h) cavs_frame_unref(&h->cur); - skip_bits(&h->gb, 16);//bbv_dwlay + skip_bits(&h->gb, 16);//bbv_delay + if (h->profile == AV_PROFILE_CAVS_GUANGDIAN) { + skip_bits(&h->gb, 8);//bbv_dwlay_extension + } + if (h->stc == PIC_PB_START_CODE) { h->cur.f->pict_type = get_bits(&h->gb, 2) + AV_PICTURE_TYPE_I; if (h->cur.f->pict_type > AV_PICTURE_TYPE_B) { @@ -1192,9 +1256,9 @@ static int decode_pic(AVSContext *h) if (h->low_delay) get_ue_golomb(&h->gb); //bbv_check_times - h->progressive = get_bits1(&h->gb); + h->progressive_frame = get_bits1(&h->gb); h->pic_structure = 1; - if (!h->progressive) + if (!h->progressive_frame) h->pic_structure = get_bits1(&h->gb); if (!h->pic_structure && h->stc == PIC_PB_START_CODE) skip_bits1(&h->gb); //advanced_pred_mode_disable @@ -1203,9 +1267,11 @@ static int decode_pic(AVSContext *h) h->pic_qp_fixed = h->qp_fixed = get_bits1(&h->gb); h->qp = get_bits(&h->gb, 6); + h->skip_mode_flag = 0; + h->ref_flag = 0; if (h->cur.f->pict_type == AV_PICTURE_TYPE_I) { - if (!h->progressive && !h->pic_structure) - skip_bits1(&h->gb);//what is this? + if (!h->progressive_frame && !h->pic_structure) + h->skip_mode_flag = get_bits1(&h->gb); skip_bits(&h->gb, 4); //reserved bits } else { if (!(h->cur.f->pict_type == AV_PICTURE_TYPE_B && h->pic_structure == 1)) @@ -1220,12 +1286,46 @@ static int decode_pic(AVSContext *h) if ( h->alpha_offset < -64 || h->alpha_offset > 64 || h-> beta_offset < -64 || h-> beta_offset > 64) { h->alpha_offset = h->beta_offset = 0; + av_log(h->avctx, AV_LOG_ERROR, "invalid loop filter params\n"); return AVERROR_INVALIDDATA; } } else { h->alpha_offset = h->beta_offset = 0; } + h->weight_quant_flag = 0; + if (h->profile == AV_PROFILE_CAVS_GUANGDIAN) { + h->weight_quant_flag = get_bits1(&h->gb); + if (h->weight_quant_flag) { + int wq_param[6] = {128, 128, 128, 128, 128, 128}; + int i, wqp_index, wq_model; + const uint8_t *m2p; + + skip_bits1(&h->gb); + if (!get_bits1(&h->gb)) { + h->chroma_quant_param_delta_cb = get_se_golomb(&h->gb); + h->chroma_quant_param_delta_cr = get_se_golomb(&h->gb); + } + wqp_index = get_bits(&h->gb, 2); + wq_model = get_bits(&h->gb, 2); + m2p = wq_model_2_param[wq_model]; + + for (i = 0; i < 6; i++) { + int delta = (wqp_index == 1 || wqp_index == 2) ? get_se_golomb(&h->gb) : 0; + wq_param[i] = default_wq_param[wqp_index][i] + delta; + av_log(h->avctx, AV_LOG_DEBUG, "wqp[%d]=%d\n", i, wq_param[i]); + } + for (i = 0; i < 64; i++) { + h->wqm_8x8[i] = wq_param[ m2p[i] ]; + } + } else { + memset(h->wqm_8x8, 128, sizeof(h->wqm_8x8)); + } + h->aec_flag = get_bits1(&h->gb); + av_log(h->avctx, AV_LOG_DEBUG, "wq_flag=%d, aec_flag=%d\n", + h->weight_quant_flag, h->aec_flag); + } + if (h->stream_revision > 0) { skip_stuffing_bits(h); skip_extension_and_user_data(h); @@ -1311,13 +1411,8 @@ static int decode_seq_header(AVSContext *h) int ret; h->profile = get_bits(&h->gb, 8); - if (h->profile != 0x20) { - avpriv_report_missing_feature(h->avctx, - "only supprt JiZhun profile"); - return AVERROR_PATCHWELCOME; - } h->level = get_bits(&h->gb, 8); - skip_bits1(&h->gb); //progressive sequence + h->progressive_seq = get_bits1(&h->gb); width = get_bits(&h->gb, 14); height = get_bits(&h->gb, 14); @@ -1344,6 +1439,9 @@ static int decode_seq_header(AVSContext *h) skip_bits1(&h->gb); //marker_bit skip_bits(&h->gb, 12); //bit_rate_upper h->low_delay = get_bits1(&h->gb); + av_log(h->avctx, AV_LOG_DEBUG, + "seq: profile=0x%02x, level=0x%02x, size=%dx%d, low_delay=%d\n", + h->profile, h->level, width, height, h->low_delay); ret = ff_set_dimensions(h->avctx, width, height); if (ret < 0) @@ -1369,8 +1467,6 @@ static int cavs_decode_frame(AVCodecContext *avctx, AVFrame *rframe, int *got_frame, AVPacket *avpkt) { AVSContext *h = avctx->priv_data; - const uint8_t *buf = avpkt->data; - int buf_size = avpkt->size; uint32_t stc = -1; int input_size, ret; const uint8_t *buf_end; @@ -1388,16 +1484,18 @@ static int cavs_decode_frame(AVCodecContext *avctx, AVFrame *rframe, h->stc = 0; - buf_ptr = buf; - buf_end = buf + buf_size; - for(;;) { + buf_ptr = avpkt->data; + buf_end = avpkt->data + avpkt->size; + for(; buf_ptr < buf_end;) { buf_ptr = avpriv_find_start_code(buf_ptr, buf_end, &stc); if ((stc & 0xFFFFFE00) || buf_ptr == buf_end) { if (!h->stc) av_log(h->avctx, AV_LOG_WARNING, "no frame decoded\n"); - return FFMAX(0, buf_ptr - buf); + return FFMAX(0, buf_ptr - avpkt->data); } input_size = (buf_end - buf_ptr) * 8; + av_log(h->avctx, AV_LOG_TRACE, "Found start code 0x%04x, sz=%d\n", + stc, input_size / 8); switch (stc) { case CAVS_START_CODE: init_get_bits(&h->gb, buf_ptr, input_size); @@ -1464,12 +1562,16 @@ static int cavs_decode_frame(AVCodecContext *avctx, AVFrame *rframe, break; default: if (stc <= SLICE_MAX_START_CODE) { + h->stc = stc & 0xff; init_get_bits(&h->gb, buf_ptr, input_size); decode_slice_header(h, &h->gb); + } else { + av_log(h->avctx, AV_LOG_WARNING, "Skip unsupported start code 0x%04X\n", stc); } break; } } + return (buf_ptr - avpkt->data); } const FFCodec ff_cavs_decoder = {