From patchwork Wed Jul 1 17:42:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongle Lin X-Patchwork-Id: 20763 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B5C5D44B781 for ; Wed, 1 Jul 2020 21:06:17 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 916D068A4D4; Wed, 1 Jul 2020 21:06:17 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f195.google.com (mail-oi1-f195.google.com [209.85.167.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EB425689226 for ; Wed, 1 Jul 2020 21:06:11 +0300 (EEST) Received: by mail-oi1-f195.google.com with SMTP id k6so17495261oij.11 for ; Wed, 01 Jul 2020 11:06:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=J/78RTqskXLlzrGIugTqkYF50q3lsSOzbMVqdNfZXKk=; b=XsiU4BqP6OalJvIydrpQVdLVnmdWrZ3XS7OHSUe6E2SZa+xFD/PxS0YR0ll1m2GBOq xMWZ09/Hi0smjeNqaxBTs4/Btt8qJEYP4JDeuJ4sqRwMih7W8aZ78s4n4bja4grDg4aw sCQ0UMGud0KQ9RSPOx7PKVpE9/5Oc+U+xsw0rpf62hyIyVjIQ2B3PvezKusCKcMSmXko ApwL1Ihhj/n6mpz0E+udwD9Bb4dxnDpmo2rGQn1YWUP6rZx9CO3YhXXnta6D5HVJP4Es 8Ic5iuh5CS9gaBq9kOUC6H6jnvWZu/1NmC9Us0f9vHh5xIxZwN80l+GJKEKtqV0XvdQZ DOUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=J/78RTqskXLlzrGIugTqkYF50q3lsSOzbMVqdNfZXKk=; b=BdMBpSZhjqfflF/fw3E7CzHy5cQ2Ev0F6giPPJrByqjKAqWI3iQB4syLiQv08Zz6JX TgXxI+xP2EAwHES4rzSFear7ju4a9mbPwJpFasxSQ8ETT9msdTHq4YUImG4wtcm74yXL qcOxADYnhDuQBeWjO+qbJHJf7owoXPrHD+xwyr3pwd+PULtjvGWnHQD4wUcwDufR7H7j mG/YuYx5/2cXPTrN9dTDG8UfeqgSMUUv1KYDJ/Zp2LMHDJpRoIOLqzYc9lG+mT6tkCYf CpQek2Swh+0jApZ7VocHxWqZnnYsXJ1MynmEtbSbpIY0dhhjdrTVZ7Bfzlc2RzLeyMCB rXzQ== X-Gm-Message-State: AOAM530IoGbV70SKvnqVr9ev0tvMUfNNjX/hORlRkvkh/1YXGKV9TKZa Xvm7lIP+OJiroQEhVzL/NiwRiZGq X-Google-Smtp-Source: ABdhPJz0cc55ZpjvIlQ+xEMIAgJWq/Bq9YNvkrEM0D2fBVpgHYqf0tPilWg1qwxqmxcUocpgP30EOw== X-Received: by 2002:a17:90a:df11:: with SMTP id gp17mr26997995pjb.188.1593625420566; Wed, 01 Jul 2020 10:43:40 -0700 (PDT) Received: from yonglel.c.googlers.com.com (219.29.83.34.bc.googleusercontent.com. [34.83.29.219]) by smtp.gmail.com with ESMTPSA id t188sm6776343pfc.198.2020.07.01.10.43.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Jul 2020 10:43:40 -0700 (PDT) From: Yongle Lin X-Google-Original-From: Yongle Lin To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 Jul 2020 17:42:48 +0000 Message-Id: <20200701174248.1364594-1-yonglel@google.com> X-Mailer: git-send-email 2.27.0.212.ge8ba1cc988-goog MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] libavfilter/vf_codecview: add block structure visualization X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Yongle Lin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" example command line to visualize block decomposition: ./ffmpeg -export_side_data +venc_params -i input.webm -vf codecview=bs=true output.webm --- doc/filters.texi | 3 +++ libavcodec/vp9.c | 8 +++----- libavfilter/vf_codecview.c | 41 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 47 insertions(+), 5 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 84567dec16..db2c80b1e9 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -7285,6 +7285,9 @@ backward predicted MVs of B-frames @item qp Display quantization parameters using the chroma planes. +@item bs +Display block structure using the luma plane. + @item mv_type, mvt Set motion vectors type to visualize. Includes MVs from all frames unless specified by @var{frame_type} option. diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c index fd0bab14a2..e700def70e 100644 --- a/libavcodec/vp9.c +++ b/libavcodec/vp9.c @@ -1501,10 +1501,8 @@ static int vp9_export_enc_params(VP9Context *s, VP9Frame *frame) AVVideoEncParams *par; unsigned int tile, nb_blocks = 0; - if (s->s.h.segmentation.enabled) { - for (tile = 0; tile < s->active_tile_cols; tile++) - nb_blocks += s->td[tile].nb_block_structure; - } + for (tile = 0; tile < s->active_tile_cols; tile++) + nb_blocks += s->td[tile].nb_block_structure; par = av_video_enc_params_create_side_data(frame->tf.f, AV_VIDEO_ENC_PARAMS_VP9, nb_blocks); @@ -1536,7 +1534,7 @@ static int vp9_export_enc_params(VP9Context *s, VP9Frame *frame) b->w = 1 << (3 + td->block_structure[block_tile].block_size_idx_x); b->h = 1 << (3 + td->block_structure[block_tile].block_size_idx_y); - if (s->s.h.segmentation.feat[seg_id].q_enabled) { + if (s->s.h.segmentation.enabled && s->s.h.segmentation.feat[seg_id].q_enabled) { b->delta_qp = s->s.h.segmentation.feat[seg_id].q_val; if (s->s.h.segmentation.absolute_vals) b->delta_qp -= par->qp; diff --git a/libavfilter/vf_codecview.c b/libavfilter/vf_codecview.c index 331bfba777..db06625d70 100644 --- a/libavfilter/vf_codecview.c +++ b/libavfilter/vf_codecview.c @@ -34,6 +34,7 @@ #include "libavutil/opt.h" #include "avfilter.h" #include "internal.h" +#include "libavutil/video_enc_params.h" #define MV_P_FOR (1<<0) #define MV_B_FOR (1<<1) @@ -51,6 +52,7 @@ typedef struct CodecViewContext { unsigned mv_type; int hsub, vsub; int qp; + int bs; } CodecViewContext; #define OFFSET(x) offsetof(CodecViewContext, x) @@ -63,6 +65,7 @@ static const AVOption codecview_options[] = { CONST("bf", "forward predicted MVs of B-frames", MV_B_FOR, "mv"), CONST("bb", "backward predicted MVs of B-frames", MV_B_BACK, "mv"), { "qp", NULL, OFFSET(qp), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, .flags = FLAGS }, + { "bs", "set block structure to visualize", OFFSET(bs), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, .flags = FLAGS }, { "mv_type", "set motion vectors type", OFFSET(mv_type), AV_OPT_TYPE_FLAGS, {.i64=0}, 0, INT_MAX, FLAGS, "mv_type" }, { "mvt", "set motion vectors type", OFFSET(mv_type), AV_OPT_TYPE_FLAGS, {.i64=0}, 0, INT_MAX, FLAGS, "mv_type" }, CONST("fp", "forward predicted MVs", MV_TYPE_FOR, "mv_type"), @@ -212,6 +215,30 @@ static void draw_arrow(uint8_t *buf, int sx, int sy, int ex, draw_line(buf, sx, sy, ex, ey, w, h, stride, color); } +static void draw_block_border(AVFrame *frame, AVVideoBlockParams *b) +{ + const int lzy = frame->linesize[0]; + uint8_t *py = frame->data[0] + b->src_y * lzy; + + for (int x = b->src_x; x < b->src_x + b->w; x++) { + if (x >= frame->width) + break; + py[x] = py[x] * 3 / 4; + } + for (int y = b->src_y; y < b->src_y + b->h; y++) { + if (y >= frame->height) + break; + py[b->src_x] = py[b->src_x] * 3 / 4; + py[b->src_x + b->w - 1] = py[b->src_x + b->w - 1] * 3 / 4; + py += lzy; + } + for (int x = b->src_x; x < b->src_x + b->w; x++) { + if (x >= frame->width) + break; + py[x] = py[x] * 3 / 4; + } +} + static int filter_frame(AVFilterLink *inlink, AVFrame *frame) { AVFilterContext *ctx = inlink->dst; @@ -242,6 +269,20 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) } } + if (s->bs) { + AVFrameSideData *sd = av_frame_get_side_data(frame, AV_FRAME_DATA_VIDEO_ENC_PARAMS); + if (sd) { + AVVideoEncParams *par = (AVVideoEncParams*)sd->data; + + if (par->nb_blocks) { + for (int i = 0; i < par->nb_blocks; i++) { + AVVideoBlockParams *b = av_video_enc_params_block(par, i); + draw_block_border(frame, b); + } + } + } + } + if (s->mv || s->mv_type) { AVFrameSideData *sd = av_frame_get_side_data(frame, AV_FRAME_DATA_MOTION_VECTORS); if (sd) {