From patchwork Mon May 11 09:32:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 19620 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E9A6844A33C for ; Mon, 11 May 2020 12:36:47 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C0BE3687F26; Mon, 11 May 2020 12:36:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6A1586808BB for ; Mon, 11 May 2020 12:36:41 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id DEE8E28A163 for ; Mon, 11 May 2020 11:36:40 +0200 (CEST) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id hJimN6hhqiHn for ; Mon, 11 May 2020 11:36:39 +0200 (CEST) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id 701C1289F2E for ; Mon, 11 May 2020 11:36:39 +0200 (CEST) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id BE49E2051E for ; Mon, 11 May 2020 11:36:38 +0200 (CEST) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id jFMqjX3ASMNI for ; Mon, 11 May 2020 11:36:36 +0200 (CEST) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id 49B482027A for ; Mon, 11 May 2020 11:36:36 +0200 (CEST) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id A06E620E00A8; Mon, 11 May 2020 11:32:53 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Mon, 11 May 2020 11:32:50 +0200 Message-Id: <20200511093250.19433-1-anton@khirnov.net> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200418101415.26409-10-anton@khirnov.net> References: <20200418101415.26409-10-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] vp9dec: support exporting QP tables through the AVVideoEncParams API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- Now the nb_block_structure reset is moved to decode_frame_header(), which fixes block-structure export with frame threading. No idea how I didn't notice this before --- libavcodec/vp9.c | 72 ++++++++++++++++++++++++++++++++++++ libavcodec/vp9block.c | 8 ++++ libavcodec/vp9dec.h | 7 ++++ libavutil/video_enc_params.h | 15 ++++++++ 4 files changed, 102 insertions(+) diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c index 6c72168deb..fd0bab14a2 100644 --- a/libavcodec/vp9.c +++ b/libavcodec/vp9.c @@ -34,6 +34,7 @@ #include "vp9dec.h" #include "libavutil/avassert.h" #include "libavutil/pixdesc.h" +#include "libavutil/video_enc_params.h" #define VP9_SYNCCODE 0x498342 @@ -97,6 +98,7 @@ static void vp9_tile_data_free(VP9TileData *td) { av_freep(&td->b_base); av_freep(&td->block_base); + av_freep(&td->block_structure); } static void vp9_frame_unref(AVCodecContext *avctx, VP9Frame *f) @@ -326,6 +328,12 @@ static int update_block_buffers(AVCodecContext *avctx) td->eob_base = (uint8_t *) (td->uvblock_base[1] + sbs * chroma_blocks * bytesperpixel); td->uveob_base[0] = td->eob_base + 16 * 16 * sbs; td->uveob_base[1] = td->uveob_base[0] + chroma_eobs * sbs; + + if (avctx->export_side_data & AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS) { + td->block_structure = av_malloc_array(s->cols * s->rows, sizeof(*td->block_structure)); + if (!td->block_structure) + return AVERROR(ENOMEM); + } } else { for (i = 1; i < s->active_tile_cols; i++) vp9_tile_data_free(&s->td[i]); @@ -341,6 +349,12 @@ static int update_block_buffers(AVCodecContext *avctx) s->td[i].eob_base = (uint8_t *) (s->td[i].uvblock_base[1] + chroma_blocks * bytesperpixel); s->td[i].uveob_base[0] = s->td[i].eob_base + 16 * 16; s->td[i].uveob_base[1] = s->td[i].uveob_base[0] + chroma_eobs; + + if (avctx->export_side_data & AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS) { + s->td[i].block_structure = av_malloc_array(s->cols * s->rows, sizeof(*td->block_structure)); + if (!s->td[i].block_structure) + return AVERROR(ENOMEM); + } } } s->block_alloc_using_2pass = s->s.frames[CUR_FRAME].uses_2pass; @@ -880,6 +894,7 @@ static int decode_frame_header(AVCodecContext *avctx, } else { memset(&s->td[i].counts, 0, sizeof(s->td[0].counts)); } + s->td[i].nb_block_structure = 0; } /* FIXME is it faster to not copy here, but do it down in the fw updates @@ -1481,6 +1496,58 @@ int loopfilter_proc(AVCodecContext *avctx) } #endif +static int vp9_export_enc_params(VP9Context *s, VP9Frame *frame) +{ + AVVideoEncParams *par; + unsigned int tile, nb_blocks = 0; + + if (s->s.h.segmentation.enabled) { + for (tile = 0; tile < s->active_tile_cols; tile++) + nb_blocks += s->td[tile].nb_block_structure; + } + + par = av_video_enc_params_create_side_data(frame->tf.f, + AV_VIDEO_ENC_PARAMS_VP9, nb_blocks); + if (!par) + return AVERROR(ENOMEM); + + par->qp = s->s.h.yac_qi; + par->delta_qp[0][0] = s->s.h.ydc_qdelta; + par->delta_qp[1][0] = s->s.h.uvdc_qdelta; + par->delta_qp[2][0] = s->s.h.uvdc_qdelta; + par->delta_qp[1][1] = s->s.h.uvac_qdelta; + par->delta_qp[2][1] = s->s.h.uvac_qdelta; + + if (nb_blocks) { + unsigned int block = 0; + unsigned int tile, block_tile; + + for (tile = 0; tile < s->active_tile_cols; tile++) { + VP9TileData *td = &s->td[tile]; + + for (block_tile = 0; block_tile < td->nb_block_structure; block_tile++) { + AVVideoBlockParams *b = av_video_enc_params_block(par, block++); + unsigned int row = td->block_structure[block_tile].row; + unsigned int col = td->block_structure[block_tile].col; + uint8_t seg_id = frame->segmentation_map[row * 8 * s->sb_cols + col]; + + b->src_x = col * 8; + b->src_y = row * 8; + b->w = 1 << (3 + td->block_structure[block_tile].block_size_idx_x); + b->h = 1 << (3 + td->block_structure[block_tile].block_size_idx_y); + + if (s->s.h.segmentation.feat[seg_id].q_enabled) { + b->delta_qp = s->s.h.segmentation.feat[seg_id].q_val; + if (s->s.h.segmentation.absolute_vals) + b->delta_qp -= par->qp; + } + } + } + } + + return 0; +} + static int vp9_decode_frame(AVCodecContext *avctx, void *frame, int *got_frame, AVPacket *pkt) { @@ -1689,6 +1756,11 @@ FF_ENABLE_DEPRECATION_WARNINGS s->td->error_info = 0; return AVERROR_INVALIDDATA; } + if (avctx->export_side_data & AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS) { + ret = vp9_export_enc_params(s, &s->s.frames[CUR_FRAME]); + if (ret < 0) + return ret; + } finish: // ref frame setup diff --git a/libavcodec/vp9block.c b/libavcodec/vp9block.c index 1c3f7a7225..ec16e26c69 100644 --- a/libavcodec/vp9block.c +++ b/libavcodec/vp9block.c @@ -1290,6 +1290,14 @@ void ff_vp9_decode_block(VP9TileData *td, int row, int col, b->uvtx = b->tx - ((s->ss_h && w4 * 2 == (1 << b->tx)) || (s->ss_v && h4 * 2 == (1 << b->tx))); + if (td->block_structure) { + td->block_structure[td->nb_block_structure].row = row; + td->block_structure[td->nb_block_structure].col = col; + td->block_structure[td->nb_block_structure].block_size_idx_x = av_log2(w4); + td->block_structure[td->nb_block_structure].block_size_idx_y = av_log2(h4); + td->nb_block_structure++; + } + if (!b->skip) { int has_coeffs; diff --git a/libavcodec/vp9dec.h b/libavcodec/vp9dec.h index e203666bb4..cc2440b854 100644 --- a/libavcodec/vp9dec.h +++ b/libavcodec/vp9dec.h @@ -226,6 +226,13 @@ struct VP9TileData { // error message int error_info; + struct { + unsigned int row:13; + unsigned int col:13; + unsigned int block_size_idx_x:2; + unsigned int block_size_idx_y:2; + } *block_structure; + unsigned int nb_block_structure; }; void ff_vp9_fill_mv(VP9TileData *td, VP56mv *mv, int mode, int sb); diff --git a/libavutil/video_enc_params.h b/libavutil/video_enc_params.h index f64864caae..0cf79c435c 100644 --- a/libavutil/video_enc_params.h +++ b/libavutil/video_enc_params.h @@ -27,6 +27,21 @@ enum AVVideoEncParamsType { AV_VIDEO_ENC_PARAMS_NONE = -1, + /** + * VP9 stores: + * - per-frame base (luma AC) quantizer index, exported as AVVideoEncParams.qp + * - deltas for luma DC, chroma AC and chroma DC, exported in the + * corresponding entries in AVVideoEncParams.delta_qp + * - per-segment delta, exported as for each block as AVVideoBlockParams.delta_qp + * + * To compute the resulting quantizer index for a block: + * - for luma AC, add the base qp and the per-block delta_qp, saturating to + * unsigned 8-bit. + * - for luma DC and chroma AC/DC, add the corresponding + * AVVideoBlockParams.delta_qp to the luma AC index, again saturating to + * unsigned 8-bit. + */ + AV_VIDEO_ENC_PARAMS_VP9, }; /**