From patchwork Fri Oct 27 20:03:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 5727 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.90 with SMTP id m26csp1110253jah; Fri, 27 Oct 2017 13:12:09 -0700 (PDT) X-Google-Smtp-Source: ABhQp+SZ9TaJBiazHQpgo8WeXs7ZsdZ2j1aqp9qDP4JCj06DizWfjcntF1Kio5aoErfI70TRVzt2 X-Received: by 10.28.128.212 with SMTP id b203mr1216699wmd.82.1509135129014; Fri, 27 Oct 2017 13:12:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1509135128; cv=none; d=google.com; s=arc-20160816; b=UofUZJ7OL3p+mlzYCpQh2rmOS+jSDw4mdJMQ00oxyXoRjVQn2lMvdSAhn3dnNUuNPj GZ3M2RBIbh5lp1J00hEijlMWsdECPxa9oFAJnl/epBsSaNulL2d19ICAuAcsVMH+I21n valcePKeEnXJC+fn1ukkPWm3cUNUWq6JE7NGdxsaK3QAYQk/cxIynEw79gB05WUeWFJ1 UeGO3YBrv6BPuq7deH8CcyFZ4AyBGCXxcoRfUeJt2KdVlh8Dw41ZvcoySFsLRzGJ2c6S fVepKOgGTmq7F2hH/urFYq2w0EG8GW19YBLrl9bHB3vR0Ula63oNIVNsmp3Y1mDgE9hO FkjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=WbNBO4sUC3K8cliuctpjHZqxomuzSaIdlhRLPtUiHqM=; b=1BjTZToxx21yJjApgaUWG2qld02YBcFtLc1l3F6rf5hQe0SjvFsOwRUFNfizeuta3J 25MevRhFQQTlbVIG0hole24zGt9IBixYqxhtYxo/Z2hwn5Xda+xmwnJLcM3wz2TR2jLK LM/Y2WGoaNyICNL1bnS6zkMmwhhXu/m9kHd2CtF4PLxt5GadR59LWw6MiJkbkrynQzBD dEVGoGhmvtuvUazahqFHf4HIuLj4tGHT9yTKSKW5+YTaqKt3jSde7siP1sZGjCWh8CFu Dp0UoMerxdj/MCSWiomML4wc5kWWM7+/CrHgmvmMHKbxRbf2CegCJdx88CGi9X8GmtHU dEZQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=aeD+z+HZ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w189si1673882wmd.34.2017.10.27.13.12.08; Fri, 27 Oct 2017 13:12:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=aeD+z+HZ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4901F68A468; Fri, 27 Oct 2017 23:11:59 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f65.google.com (mail-wm0-f65.google.com [74.125.82.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 87C1468A429 for ; Fri, 27 Oct 2017 23:11:52 +0300 (EEST) Received: by mail-wm0-f65.google.com with SMTP id m72so5408494wmc.1 for ; Fri, 27 Oct 2017 13:12:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=78XhFpXExtvVnl3AKYs2ilvg/P+X40LN58EDd/Ru/gQ=; b=aeD+z+HZQC1LnBRWjpShOXIMo3lC5PvRgxdWzHSQxgJpe2UnmUH+x3AtZhf/+cDOGW DC4Z/oPd3+KNu/Dm90NuirJTqzyOaKyQT+IPbnJ8EJHpFs3zNiuLhxo+bd2/frCl2Cv4 TObhKxFa1kgR5Y824bexNVLXJtLOBYm25Fo5+oMAMLErlGUTIOY5nrFSRG5bzri5tCMM +g+kcMFu1o8fszfeswGsEaccqFyMNG3EjAaG2CjkDgxwFiK1nbNrPTVCxVO+pZysWPag c/GG6SiPvQkY5LmDI5UCvQkZht5aXTvfH2VndjrsbrqgIglo3YxCmqCnQwt2teH4ycQL 6ZuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=78XhFpXExtvVnl3AKYs2ilvg/P+X40LN58EDd/Ru/gQ=; b=LJcGcMQHPW699NS/jnUIPcDtQ9G8Xf6myECvSd7XHp1jWnqw17WrgLxs1qILNkdwEv OB1ymPtLFvEnvYt8OnyMW+qsTaBk4vU1tPs26NpxhSzidk7m1dj5XJdRUMxOCTrGCTpf QBgnuCH6PKXd+DM3O6Xb/LaLEqh9Ih8/wX6V1ZmyRMwVZtnZ0rbt1ifNBLQbPyp57fzz GMUcUGvta7UWIDCiq+a2xm5THA5t974Zx0k/lW/D6JZcwW/PWlo+BjCl6nYy0Cixw6+/ ldaWpKuP9etPphh1km+/OIZp5W92Iw2OSwGTxevBoAuPQsYjPLQKBA6tW0WFHmUq5+sQ ABKA== X-Gm-Message-State: AMCzsaX8MCqbTTA4X8TetjJIzEzmFPzM/vyfOmBN+gusHRFksbnSzY48 F8+Bl3XOF5GoCg4H70N6yYl/Mw== X-Received: by 10.28.48.150 with SMTP id w144mr1205478wmw.23.1509134678076; Fri, 27 Oct 2017 13:04:38 -0700 (PDT) Received: from localhost.localdomain ([94.250.174.60]) by smtp.gmail.com with ESMTPSA id f84sm1997024wmh.47.2017.10.27.13.04.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Oct 2017 13:04:37 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Fri, 27 Oct 2017 22:03:54 +0200 Message-Id: <20171027200354.31214-1-onemda@gmail.com> X-Mailer: git-send-email 2.11.0 Subject: [FFmpeg-devel] [PATCH] avfilter: initial macroblock types export and visualization X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- libavcodec/mpegvideo.c | 10 +++++ libavfilter/vf_codecview.c | 105 +++++++++++++++++++++++++++++++++++++++++++++ libavutil/frame.h | 4 ++ 3 files changed, 119 insertions(+) diff --git a/libavcodec/mpegvideo.c b/libavcodec/mpegvideo.c index 2f5793b9a4..ce6108d094 100644 --- a/libavcodec/mpegvideo.c +++ b/libavcodec/mpegvideo.c @@ -1603,6 +1603,16 @@ void ff_print_debug_info2(AVCodecContext *avctx, AVFrame *pict, uint8_t *mbskip_ } } + if (mb_height && mb_width) { + AVFrameSideData *sd; + + av_log(avctx, AV_LOG_DEBUG, "Adding %d MB types info to frame %d\n", mb_width * mb_height, avctx->frame_number); + sd = av_frame_new_side_data(pict, AV_FRAME_DATA_MACROBLOCK_TYPES, mb_width * mb_height * sizeof(uint32_t)); + if (!sd) + return; + memcpy(sd->data, mbtype_table, mb_width * mb_height * sizeof(uint32_t)); + } + #if FF_API_DEBUG_MV if ((avctx->debug & (FF_DEBUG_VIS_QP | FF_DEBUG_VIS_MB_TYPE)) || (avctx->debug_mv)) { diff --git a/libavfilter/vf_codecview.c b/libavfilter/vf_codecview.c index 331bfba777..40fb8dfb7a 100644 --- a/libavfilter/vf_codecview.c +++ b/libavfilter/vf_codecview.c @@ -29,6 +29,7 @@ * TODO: segmentation */ +#include "libavutil/avassert.h" #include "libavutil/imgutils.h" #include "libavutil/motion_vector.h" #include "libavutil/opt.h" @@ -44,9 +45,23 @@ #define FRAME_TYPE_P (1<<1) #define FRAME_TYPE_B (1<<2) +#define IS_PCM(a) ((a) & (1 << 2)) +#define IS_INTRA(a) ((a) & 7) +#define IS_ACPRED(a) ((a) & (1 << 9)) +#define IS_INTRA16x16(a) ((a) & (1 << 1)) +#define IS_INTRA4x4(a) ((a) & (1 << 0)) +#define IS_DIRECT(a) ((a) & (1 << 8)) +#define IS_SKIP(a) ((a) & (1 << 11)) +#define IS_GMC(a) ((a) & (1 << 10)) +#define USES_LIST(a, list) ((a) & (((1 << 12) | (1 << 13)) << (2 * (list)))) +#define IS_8X8(a) ((a) & ((1 << 6))) +#define IS_16X8(a) ((a) & ((1 << 4))) +#define IS_8X16(a) ((a) & ((1 << 5))) + typedef struct CodecViewContext { const AVClass *class; unsigned mv; + unsigned mbtypes; unsigned frame_type; unsigned mv_type; int hsub, vsub; @@ -72,6 +87,7 @@ static const AVOption codecview_options[] = { CONST("if", "I-frames", FRAME_TYPE_I, "frame_type"), CONST("pf", "P-frames", FRAME_TYPE_P, "frame_type"), CONST("bf", "B-frames", FRAME_TYPE_B, "frame_type"), + { "mb", "visualize macroblock types", OFFSET(mbtypes), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS, }, { NULL } }; @@ -277,6 +293,95 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) } } + if (s->mbtypes) { + AVFrameSideData *sd = av_frame_get_side_data(frame, AV_FRAME_DATA_MACROBLOCK_TYPES); + if (sd) { + int mb_height = (frame->height + 31) / 32 * 2; + int mb_width = (frame->width + 15) / 16; + int block_height = 16 >> 1; + int mb_stride = mb_width + 1; + int mb_y, mb_x; + uint32_t *mbtype_table = (uint32_t *)sd->data; + + for (mb_y = 0; mb_y < mb_height; mb_y++) { + for (mb_x = 0; mb_x < mb_width; mb_x++) { + const int mb_index = mb_x + mb_y * mb_stride; + /*{ + uint64_t c = (qscale_table[mb_index] * 128 / 31) * + 0x0101010101010101ULL; + int y; + for (y = 0; y < block_height; y++) { + *(uint64_t *)(frame->data[1] + 8 * mb_x + + (block_height * mb_y + y) * + frame->linesize[1]) = c; + *(uint64_t *)(frame->data[2] + 8 * mb_x + + (block_height * mb_y + y) * + frame->linesize[2]) = c; + } + } */ + { + int mb_type = mbtype_table[mb_index]; + uint64_t u,v; + int y; + +#define COLOR(theta, r) \ + u = (int)(128 + r * cos(theta * M_PI / 180)); \ + v = (int)(128 + r * sin(theta * M_PI / 180)); + + u = v = 128; + if (IS_PCM(mb_type)) { + COLOR(120, 48) + } else if ((IS_INTRA(mb_type) && IS_ACPRED(mb_type)) || + IS_INTRA16x16(mb_type)) { + COLOR(30, 48) + } else if (IS_INTRA4x4(mb_type)) { + COLOR(90, 48) + } else if (IS_DIRECT(mb_type) && IS_SKIP(mb_type)) { + COLOR(120, 48) + } else if (IS_DIRECT(mb_type)) { + COLOR(150, 48) + } else if (IS_GMC(mb_type) && IS_SKIP(mb_type)) { + COLOR(170, 48) + } else if (IS_GMC(mb_type)) { + COLOR(190, 48) + } else if (IS_SKIP(mb_type)) { + COLOR(180, 48) + } else if (!USES_LIST(mb_type, 1)) { + COLOR(240, 48) + } else if (!USES_LIST(mb_type, 0)) { + COLOR(0, 48) + } else { + av_assert2(USES_LIST(mb_type, 0) && USES_LIST(mb_type, 1)); + COLOR(300,48) + } + + u *= 0x0101010101010101ULL; + v *= 0x0101010101010101ULL; + for (y = 0; y < block_height; y++) { + *(uint64_t *)(frame->data[1] + 8 * mb_x + + (block_height * mb_y + y) * frame->linesize[1]) = u; + *(uint64_t *)(frame->data[2] + 8 * mb_x + + (block_height * mb_y + y) * frame->linesize[2]) = v; + } + + // segmentation + if (IS_8X8(mb_type) || IS_16X8(mb_type)) { + *(uint64_t *)(frame->data[0] + 16 * mb_x + 0 + + (16 * mb_y + 8) * frame->linesize[0]) ^= 0x8080808080808080ULL; + *(uint64_t *)(frame->data[0] + 16 * mb_x + 8 + + (16 * mb_y + 8) * frame->linesize[0]) ^= 0x8080808080808080ULL; + } + if (IS_8X8(mb_type) || IS_8X16(mb_type)) { + for (y = 0; y < 16; y++) + frame->data[0][16 * mb_x + 8 + (16 * mb_y + y) * + frame->linesize[0]] ^= 0x80; + } + } + } + } + } + } + return ff_filter_frame(outlink, frame); } diff --git a/libavutil/frame.h b/libavutil/frame.h index fef558ea2f..8481dc080b 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -141,6 +141,10 @@ enum AVFrameSideDataType { * metadata key entry "name". */ AV_FRAME_DATA_ICC_PROFILE, + /** + * Macroblock types exported by some codecs. + */ + AV_FRAME_DATA_MACROBLOCK_TYPES, }; enum AVActiveFormatDescription {