From patchwork Thu Dec 26 11:47:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 16980 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 43B2744A875 for ; Thu, 26 Dec 2019 13:47:52 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2669168A41E; Thu, 26 Dec 2019 13:47:52 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E417468813A for ; Thu, 26 Dec 2019 13:47:44 +0200 (EET) Received: by mail-wm1-f43.google.com with SMTP id q9so4158454wmj.5 for ; Thu, 26 Dec 2019 03:47:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=sThIjxOGYrdaHnG7tDctvRavg6Hb6O0Fo7Zfkm1dNY8=; b=DfkphRHRHCdjXqTVHnHiMnJDzjVMJQGbE78ivmgFXYcPir9PMgG5MQRSy1t1dffJSt BMWCjs6bPJhLXYrwKm/z/hgrIYCKDDKabGK/YlGhoM+JPFRFqCl3/oInoWFDSQHRo1mA Y4YJsaiVwvaX9f6SMDiKzBChj1NNpPn7rYmizWj/aJ+tNDxuV3RvP6+MUcNeqfHMJWlt z4MZmEEh07YByM+bbD0OvYOBXX46SkC8sAluo0/0FrzIHpLTesFhoK47C2Wznd2Mwh/W XpyuTyeaa6km4FWSM7yXhWktMMFxAXH2M6AwV+39BnsvtBkXu2642rtr9coxHVSksEPa dCCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=sThIjxOGYrdaHnG7tDctvRavg6Hb6O0Fo7Zfkm1dNY8=; b=T+qWtiRtnZFQsAEBYQ89LtHHqJ0PrD5HQcaj+JN6RannityheK+tHHd+665IjdH82Q OtrqTmK1NZiJwT5nyQ/YeZNV2kxXf5j5RHNcYS8AcDsZwORAH4llCQeyb+LFY3k9sIT7 jSBsPsEGc5YHqKybnliqYbhSjKaxN+yND1CxPglHGVZS0d1AZBWmVEwPrqhC4IZkmgqf I64zvV0vbQOHjVhl5m7pmwq5mvnZztAn4DxH9+/r4DSqm3rws2MDR67/sW5w6mCWpBqD /h68b/miTouFMl/h3Ltmqxhte6OgRWhCWeHsngDNf75SmwD9XTlJ4or7Kg9ZYawq1PGz 4srw== X-Gm-Message-State: APjAAAUX77iDDYe/g3SGGwVx9R/PUkTqvJsNdAcVk8hmmg8KVShqjGqd LizYnuy6sVB5Iq+EZ45r0FUAiHqP X-Google-Smtp-Source: APXvYqynWV1Phd39RUw/0eBpj8Hhs0te61VRucZtY8+MGKwmBFIfEvUCO9+xQJ2iLOxFWUK4mQGsuA== X-Received: by 2002:a05:600c:1007:: with SMTP id c7mr14492399wmc.158.1577360863796; Thu, 26 Dec 2019 03:47:43 -0800 (PST) Received: from localhost.localdomain ([94.250.180.106]) by smtp.gmail.com with ESMTPSA id q19sm8127884wmc.12.2019.12.26.03.47.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Dec 2019 03:47:42 -0800 (PST) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Thu, 26 Dec 2019 12:47:35 +0100 Message-Id: <20191226114735.30404-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add thistogram video filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 46 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_thistogram.c | 358 ++++++++++++++++++++++++++++++++++++ 4 files changed, 406 insertions(+) create mode 100644 libavfilter/vf_thistogram.c diff --git a/doc/filters.texi b/doc/filters.texi index 8c5d3a5760..4468351fc4 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -11674,6 +11674,7 @@ the histogram. Possible values are @code{none}, @code{weak} or @code{strong}. It defaults to @code{none}. @end table +@anchor{histogram} @section histogram Compute and draw a color distribution histogram for the input video. @@ -17717,6 +17718,51 @@ PAL output (25i): 16p: 33333334 @end example +@section thistogram + +Compute and draw a color distribution histogram for the input video across time. + +Unlike @ref{histogram} video filter which only shows histogram of single input frame +at certain time, this filter shows also past histograms of number of frames defined +by @code{width} option. + +The computed histogram is a representation of the color component +distribution in an image. + +The filter accepts the following options: + +@table @option +@item width +Set width of single color component output. Default value is @code{1080}. +This also set number of passed histograms to keep. +Allowed range is [50, 8192]. + +@item display_mode, d +Set display mode. +It accepts the following values: +@table @samp +@item stack +Per color component graphs are placed below each other. + +@item parade +Per color component graphs are placed side by side. + +@item overlay +Presents information identical to that in the @code{parade}, except +that the graphs representing color components are superimposed directly +over one another. +@end table +Default is @code{stack}. + +@item levels_mode, m +Set mode. Can be either @code{linear}, or @code{logarithmic}. +Default is @code{linear}. + +@item components, c +Set what color components to display. +Default is @code{7}. +@end table + @section threshold Apply threshold effect to video stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 37d4eee858..cb024f95ff 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -401,6 +401,7 @@ OBJS-$(CONFIG_SWAPRECT_FILTER) += vf_swaprect.o OBJS-$(CONFIG_SWAPUV_FILTER) += vf_swapuv.o OBJS-$(CONFIG_TBLEND_FILTER) += vf_blend.o framesync.o OBJS-$(CONFIG_TELECINE_FILTER) += vf_telecine.o +OBJS-$(CONFIG_THISTOGRAM_FILTER) += vf_thistogram.o OBJS-$(CONFIG_THRESHOLD_FILTER) += vf_threshold.o framesync.o OBJS-$(CONFIG_THUMBNAIL_FILTER) += vf_thumbnail.o OBJS-$(CONFIG_THUMBNAIL_CUDA_FILTER) += vf_thumbnail_cuda.o vf_thumbnail_cuda.ptx.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index c295f8e403..9f2080f857 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -382,6 +382,7 @@ extern AVFilter ff_vf_swaprect; extern AVFilter ff_vf_swapuv; extern AVFilter ff_vf_tblend; extern AVFilter ff_vf_telecine; +extern AVFilter ff_vf_thistogram; extern AVFilter ff_vf_threshold; extern AVFilter ff_vf_thumbnail; extern AVFilter ff_vf_thumbnail_cuda; diff --git a/libavfilter/vf_thistogram.c b/libavfilter/vf_thistogram.c new file mode 100644 index 0000000000..c29d8f8876 --- /dev/null +++ b/libavfilter/vf_thistogram.c @@ -0,0 +1,358 @@ +/* + * Copyright (c) 2019 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/opt.h" +#include "libavutil/parseutils.h" +#include "libavutil/pixdesc.h" +#include "libavutil/imgutils.h" +#include "libavutil/intreadwrite.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" + +typedef struct THistogramContext { + const AVClass *class; ///< AVClass context for log and options purpose + unsigned histogram[65536]; + int histogram_size; + int x_pos; + int mult; + int ncomp; + int dncomp; + int width; + int display_mode; + int levels_mode; + const AVPixFmtDescriptor *desc, *odesc; + int components; + int planewidth[4]; + int planeheight[4]; + uint8_t bg_color[4]; + AVFrame *out; +} THistogramContext; + +#define OFFSET(x) offsetof(THistogramContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM + +static const AVOption thistogram_options[] = { + { "width", "set width", OFFSET(width), AV_OPT_TYPE_INT, {.i64=1080}, 50, 8192, FLAGS}, + { "display_mode", "set display mode", OFFSET(display_mode), AV_OPT_TYPE_INT, {.i64=2}, 0, 2, FLAGS, "display_mode"}, + { "d", "set display mode", OFFSET(display_mode), AV_OPT_TYPE_INT, {.i64=2}, 0, 2, FLAGS, "display_mode"}, + { "overlay", NULL, 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, FLAGS, "display_mode" }, + { "parade", NULL, 0, AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, FLAGS, "display_mode" }, + { "stack", NULL, 0, AV_OPT_TYPE_CONST, {.i64=2}, 0, 0, FLAGS, "display_mode" }, + { "levels_mode", "set levels mode", OFFSET(levels_mode), AV_OPT_TYPE_INT, {.i64=0}, 0, 1, FLAGS, "levels_mode"}, + { "m", "set levels mode", OFFSET(levels_mode), AV_OPT_TYPE_INT, {.i64=0}, 0, 1, FLAGS, "levels_mode"}, + { "linear", NULL, 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, FLAGS, "levels_mode" }, + { "logarithmic", NULL, 0, AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, FLAGS, "levels_mode" }, + { "components", "set color components to display", OFFSET(components), AV_OPT_TYPE_INT, {.i64=7}, 1, 15, FLAGS}, + { "c", "set color components to display", OFFSET(components), AV_OPT_TYPE_INT, {.i64=7}, 1, 15, FLAGS}, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(thistogram); + +static const enum AVPixelFormat levels_in_pix_fmts[] = { + AV_PIX_FMT_YUVA420P, AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUVJ420P, + AV_PIX_FMT_YUVA422P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUVJ422P, + AV_PIX_FMT_YUV411P, AV_PIX_FMT_YUVJ411P, + AV_PIX_FMT_YUV440P, AV_PIX_FMT_YUV410P, + AV_PIX_FMT_YUVA444P, AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUVJ444P, + AV_PIX_FMT_YUV420P9, AV_PIX_FMT_YUV422P9, AV_PIX_FMT_YUV444P9, + AV_PIX_FMT_YUVA420P9, AV_PIX_FMT_YUVA422P9, AV_PIX_FMT_YUVA444P9, + AV_PIX_FMT_YUV420P10, AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV444P10, + AV_PIX_FMT_YUVA420P10, AV_PIX_FMT_YUVA422P10, AV_PIX_FMT_YUVA444P10, + AV_PIX_FMT_YUV420P12, AV_PIX_FMT_YUV422P12, AV_PIX_FMT_YUV444P12, AV_PIX_FMT_YUV440P12, + AV_PIX_FMT_GBRAP, AV_PIX_FMT_GBRP, + AV_PIX_FMT_GBRP9, AV_PIX_FMT_GBRP10, AV_PIX_FMT_GBRAP10, + AV_PIX_FMT_GBRP12, AV_PIX_FMT_GBRAP12, + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_NONE +}; + +static const enum AVPixelFormat levels_out_yuv8_pix_fmts[] = { + AV_PIX_FMT_YUV444P, + AV_PIX_FMT_NONE +}; + +static const enum AVPixelFormat levels_out_yuv9_pix_fmts[] = { + AV_PIX_FMT_YUV444P9, + AV_PIX_FMT_NONE +}; + +static const enum AVPixelFormat levels_out_yuv10_pix_fmts[] = { + AV_PIX_FMT_YUV444P10, + AV_PIX_FMT_NONE +}; + +static const enum AVPixelFormat levels_out_yuv12_pix_fmts[] = { + AV_PIX_FMT_YUV444P12, + AV_PIX_FMT_NONE +}; + +static const enum AVPixelFormat levels_out_rgb8_pix_fmts[] = { + AV_PIX_FMT_GBRP, + AV_PIX_FMT_NONE +}; + +static const enum AVPixelFormat levels_out_rgb9_pix_fmts[] = { + AV_PIX_FMT_GBRP9, + AV_PIX_FMT_NONE +}; + +static const enum AVPixelFormat levels_out_rgb10_pix_fmts[] = { + AV_PIX_FMT_GBRP10, + AV_PIX_FMT_NONE +}; + +static const enum AVPixelFormat levels_out_rgb12_pix_fmts[] = { + AV_PIX_FMT_GBRP12, + AV_PIX_FMT_NONE +}; + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *avff; + const AVPixFmtDescriptor *desc; + const enum AVPixelFormat *out_pix_fmts; + int rgb, i, bits; + int ret; + + if (!ctx->inputs[0]->in_formats || + !ctx->inputs[0]->in_formats->nb_formats) { + return AVERROR(EAGAIN); + } + + if (!ctx->inputs[0]->out_formats) + if ((ret = ff_formats_ref(ff_make_format_list(levels_in_pix_fmts), &ctx->inputs[0]->out_formats)) < 0) + return ret; + avff = ctx->inputs[0]->in_formats; + desc = av_pix_fmt_desc_get(avff->formats[0]); + rgb = desc->flags & AV_PIX_FMT_FLAG_RGB; + bits = desc->comp[0].depth; + for (i = 1; i < avff->nb_formats; i++) { + desc = av_pix_fmt_desc_get(avff->formats[i]); + if ((rgb != (desc->flags & AV_PIX_FMT_FLAG_RGB)) || + (bits != desc->comp[0].depth)) + return AVERROR(EAGAIN); + } + + if (rgb && bits == 8) + out_pix_fmts = levels_out_rgb8_pix_fmts; + else if (rgb && bits == 9) + out_pix_fmts = levels_out_rgb9_pix_fmts; + else if (rgb && bits == 10) + out_pix_fmts = levels_out_rgb10_pix_fmts; + else if (rgb && bits == 12) + out_pix_fmts = levels_out_rgb12_pix_fmts; + else if (bits == 8) + out_pix_fmts = levels_out_yuv8_pix_fmts; + else if (bits == 9) + out_pix_fmts = levels_out_yuv9_pix_fmts; + else if (bits == 10) + out_pix_fmts = levels_out_yuv10_pix_fmts; + else if (bits == 12) + out_pix_fmts = levels_out_yuv12_pix_fmts; + else + return AVERROR(EAGAIN); + if ((ret = ff_formats_ref(ff_make_format_list(out_pix_fmts), &ctx->outputs[0]->in_formats)) < 0) + return ret; + + return 0; +} + +static const uint8_t black_yuva_color[4] = { 0, 127, 127, 255 }; +static const uint8_t black_gbrp_color[4] = { 0, 0, 0, 255 }; + +static int config_input(AVFilterLink *inlink) +{ + THistogramContext *s = inlink->dst->priv; + + s->desc = av_pix_fmt_desc_get(inlink->format); + s->ncomp = s->desc->nb_components; + s->histogram_size = 1 << s->desc->comp[0].depth; + s->mult = s->histogram_size / 256; + + switch (inlink->format) { + case AV_PIX_FMT_GBRAP12: + case AV_PIX_FMT_GBRP12: + case AV_PIX_FMT_GBRAP10: + case AV_PIX_FMT_GBRP10: + case AV_PIX_FMT_GBRP9: + case AV_PIX_FMT_GBRAP: + case AV_PIX_FMT_GBRP: + memcpy(s->bg_color, black_gbrp_color, 4); + break; + default: + memcpy(s->bg_color, black_yuva_color, 4); + } + + s->planeheight[1] = s->planeheight[2] = AV_CEIL_RSHIFT(inlink->h, s->desc->log2_chroma_h); + s->planeheight[0] = s->planeheight[3] = inlink->h; + s->planewidth[1] = s->planewidth[2] = AV_CEIL_RSHIFT(inlink->w, s->desc->log2_chroma_w); + s->planewidth[0] = s->planewidth[3] = inlink->w; + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + THistogramContext *s = ctx->priv; + int ncomp = 0, i; + + for (i = 0; i < s->ncomp; i++) { + if ((1 << i) & s->components) + ncomp++; + } + outlink->w = s->width * FFMAX(ncomp * (s->display_mode == 1), 1); + outlink->h = s->histogram_size * FFMAX(ncomp * (s->display_mode == 2), 1); + + s->odesc = av_pix_fmt_desc_get(outlink->format); + s->dncomp = s->odesc->nb_components; + outlink->sample_aspect_ratio = (AVRational){1,1}; + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + THistogramContext *s = inlink->dst->priv; + AVFilterContext *ctx = inlink->dst; + AVFilterLink *outlink = ctx->outputs[0]; + int k, m; + + if (!s->out) { + s->out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!s->out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + + for (k = 0; k < 4 && s->out->data[k]; k++) { + const int is_chroma = (k == 1 || k == 2); + const int dst_h = AV_CEIL_RSHIFT(outlink->h, (is_chroma ? s->odesc->log2_chroma_h : 0)); + const int dst_w = AV_CEIL_RSHIFT(outlink->w, (is_chroma ? s->odesc->log2_chroma_w : 0)); + + if (s->histogram_size <= 256) { + for (int i = 0; i < dst_h ; i++) + memset(s->out->data[s->odesc->comp[k].plane] + + i * s->out->linesize[s->odesc->comp[k].plane], + s->bg_color[k], dst_w); + } else { + const int mult = s->mult; + + for (int i = 0; i < dst_h ; i++) + for (int j = 0; j < dst_w; j++) + AV_WN16(s->out->data[s->odesc->comp[k].plane] + + i * s->out->linesize[s->odesc->comp[k].plane] + j * 2, + s->bg_color[k] * mult); + } + } + } + + for (m = 0, k = 0; k < s->ncomp; k++) { + const int p = s->desc->comp[k].plane; + const int height = s->planeheight[p]; + const int width = s->planewidth[p]; + double max_hval_log; + unsigned max_hval = 0; + int starty, startx; + + if (!((1 << k) & s->components)) + continue; + starty = m * s->histogram_size * (s->display_mode == 2); + startx = m++ * s->width * (s->display_mode == 1); + + if (s->histogram_size <= 256) { + for (int i = 0; i < height; i++) { + const uint8_t *src = in->data[p] + i * in->linesize[p]; + for (int j = 0; j < width; j++) + s->histogram[src[j]]++; + } + } else { + for (int i = 0; i < height; i++) { + const uint16_t *src = (const uint16_t *)(in->data[p] + i * in->linesize[p]); + for (int j = 0; j < width; j++) + s->histogram[src[j]]++; + } + } + + for (int i = 0; i < s->histogram_size; i++) + max_hval = FFMAX(max_hval, s->histogram[i]); + max_hval_log = log2(max_hval + 1); + + for (int i = 0; i < s->histogram_size; i++) { + int idx = s->histogram_size - i - 1; + int value; + + if (s->levels_mode) + value = lrint((s->histogram_size - 1) * (log2(s->histogram[idx] + 1) / max_hval_log)); + else + value = lrint((s->histogram_size - 1) * s->histogram[idx] / (float)max_hval); + + if (s->histogram_size <= 256) { + s->out->data[p][(i + starty) * s->out->linesize[p] + startx + s->x_pos] = value; + } else { + AV_WN16(s->out->data[p] + (i + starty) * s->out->linesize[p] + startx * 2 + s->x_pos * 2, value); + } + } + + memset(s->histogram, 0, s->histogram_size * sizeof(unsigned)); + } + + s->out->pts = in->pts; + + av_frame_free(&in); + s->x_pos++; + if (s->x_pos >= s->width) + s->x_pos = 0; + + return ff_filter_frame(outlink, av_frame_clone(s->out)); +} + +static const AVFilterPad thistogram_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, + { NULL } +}; + +static const AVFilterPad thistogram_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, + { NULL } +}; + +AVFilter ff_vf_thistogram = { + .name = "thistogram", + .description = NULL_IF_CONFIG_SMALL("Compute and draw a temporal histogram."), + .priv_size = sizeof(THistogramContext), + .query_formats = query_formats, + .inputs = thistogram_inputs, + .outputs = thistogram_outputs, + .priv_class = &thistogram_class, +};