From patchwork Wed Nov 28 06:08:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ruiling Song X-Patchwork-Id: 11196 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 093CA44E09E for ; Wed, 28 Nov 2018 08:11:23 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A1F5B689ED4; Wed, 28 Nov 2018 08:11:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4C70C68830E for ; Wed, 28 Nov 2018 08:11:15 +0200 (EET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Nov 2018 22:11:18 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,289,1539673200"; d="scan'208";a="112813913" Received: from ruiling-skl2.sh.intel.com ([10.239.160.154]) by orsmga002.jf.intel.com with ESMTP; 27 Nov 2018 22:11:17 -0800 From: Ruiling Song To: ffmpeg-devel@ffmpeg.org Date: Wed, 28 Nov 2018 14:08:44 +0800 Message-Id: <1543385324-9456-1-git-send-email-ruiling.song@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [FFmpeg-devel] [PATCH] lavfi/tonemap_opencl: reuse matrix calculation from vf_colorspace X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Ruiling Song MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" As these functions are moved to shared file, other colorspace-related filters could also leverage the code. Signed-off-by: Ruiling Song --- libavfilter/colorspace.c | 71 +++++++++++++++++++++++++++++ libavfilter/colorspace.h | 4 ++ libavfilter/opencl/colorspace_common.cl | 25 ----------- libavfilter/vf_colorspace.c | 80 ++------------------------------- libavfilter/vf_tonemap_opencl.c | 62 +++++++++++-------------- 5 files changed, 106 insertions(+), 136 deletions(-) diff --git a/libavfilter/colorspace.c b/libavfilter/colorspace.c index c668221..19616e4 100644 --- a/libavfilter/colorspace.c +++ b/libavfilter/colorspace.c @@ -93,6 +93,77 @@ void ff_fill_rgb2xyz_table(const struct PrimaryCoefficients *coeffs, rgb2xyz[2][1] *= sg; rgb2xyz[2][2] *= sb; } +static const double ycgco_matrix[3][3] = +{ + { 0.25, 0.5, 0.25 }, + { -0.25, 0.5, -0.25 }, + { 0.5, 0, -0.5 }, +}; + +static const double gbr_matrix[3][3] = +{ + { 0, 1, 0 }, + { 0, -0.5, 0.5 }, + { 0.5, -0.5, 0 }, +}; + +/* + * All constants explained in e.g. https://linuxtv.org/downloads/v4l-dvb-apis/ch02s06.html + * The older ones (bt470bg/m) are also explained in their respective ITU docs + * (e.g. https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.470-5-199802-S!!PDF-E.pdf) + * whereas the newer ones can typically be copied directly from wikipedia :) + */ +static const struct LumaCoefficients luma_coefficients[AVCOL_SPC_NB] = { + [AVCOL_SPC_FCC] = { 0.30, 0.59, 0.11 }, + [AVCOL_SPC_BT470BG] = { 0.299, 0.587, 0.114 }, + [AVCOL_SPC_SMPTE170M] = { 0.299, 0.587, 0.114 }, + [AVCOL_SPC_BT709] = { 0.2126, 0.7152, 0.0722 }, + [AVCOL_SPC_SMPTE240M] = { 0.212, 0.701, 0.087 }, + [AVCOL_SPC_YCOCG] = { 0.25, 0.5, 0.25 }, + [AVCOL_SPC_RGB] = { 1, 1, 1 }, + [AVCOL_SPC_BT2020_NCL] = { 0.2627, 0.6780, 0.0593 }, + [AVCOL_SPC_BT2020_CL] = { 0.2627, 0.6780, 0.0593 }, +}; + +const struct LumaCoefficients *ff_get_luma_coefficients(enum AVColorSpace csp) +{ + const struct LumaCoefficients *coeffs; + + if (csp >= AVCOL_SPC_NB) + return NULL; + coeffs = &luma_coefficients[csp]; + if (!coeffs->cr) + return NULL; + + return coeffs; +} + +void ff_fill_rgb2yuv_table(const struct LumaCoefficients *coeffs, + double rgb2yuv[3][3]) +{ + double bscale, rscale; + + // special ycgco matrix + if (coeffs->cr == 0.25 && coeffs->cg == 0.5 && coeffs->cb == 0.25) { + memcpy(rgb2yuv, ycgco_matrix, sizeof(double) * 9); + return; + } else if (coeffs->cr == 1 && coeffs->cg == 1 && coeffs->cb == 1) { + memcpy(rgb2yuv, gbr_matrix, sizeof(double) * 9); + return; + } + + rgb2yuv[0][0] = coeffs->cr; + rgb2yuv[0][1] = coeffs->cg; + rgb2yuv[0][2] = coeffs->cb; + bscale = 0.5 / (coeffs->cb - 1.0); + rscale = 0.5 / (coeffs->cr - 1.0); + rgb2yuv[1][0] = bscale * coeffs->cr; + rgb2yuv[1][1] = bscale * coeffs->cg; + rgb2yuv[1][2] = 0.5; + rgb2yuv[2][0] = 0.5; + rgb2yuv[2][1] = rscale * coeffs->cg; + rgb2yuv[2][2] = rscale * coeffs->cb; +} double ff_determine_signal_peak(AVFrame *in) { diff --git a/libavfilter/colorspace.h b/libavfilter/colorspace.h index 9366818..459a5df 100644 --- a/libavfilter/colorspace.h +++ b/libavfilter/colorspace.h @@ -44,6 +44,10 @@ void ff_fill_rgb2xyz_table(const struct PrimaryCoefficients *coeffs, const struct WhitepointCoefficients *wp, double rgb2xyz[3][3]); +const struct LumaCoefficients *ff_get_luma_coefficients(enum AVColorSpace csp); +void ff_fill_rgb2yuv_table(const struct LumaCoefficients *coeffs, + double rgb2yuv[3][3]); + double ff_determine_signal_peak(AVFrame *in); void ff_update_hdr_metadata(AVFrame *in, double peak); diff --git a/libavfilter/opencl/colorspace_common.cl b/libavfilter/opencl/colorspace_common.cl index 94a4dd0..1d68a54 100644 --- a/libavfilter/opencl/colorspace_common.cl +++ b/libavfilter/opencl/colorspace_common.cl @@ -39,31 +39,6 @@ constant const float ST2084_C1 = 0.8359375f; constant const float ST2084_C2 = 18.8515625f; constant const float ST2084_C3 = 18.6875f; -__constant float yuv2rgb_bt2020[] = { - 1.0f, 0.0f, 1.4746f, - 1.0f, -0.16455f, -0.57135f, - 1.0f, 1.8814f, 0.0f -}; - -__constant float yuv2rgb_bt709[] = { - 1.0f, 0.0f, 1.5748f, - 1.0f, -0.18732f, -0.46812f, - 1.0f, 1.8556f, 0.0f -}; - -__constant float rgb2yuv_bt709[] = { - 0.2126f, 0.7152f, 0.0722f, - -0.11457f, -0.38543f, 0.5f, - 0.5f, -0.45415f, -0.04585f -}; - -__constant float rgb2yuv_bt2020[] ={ - 0.2627f, 0.678f, 0.0593f, - -0.1396f, -0.36037f, 0.5f, - 0.5f, -0.4598f, -0.0402f, -}; - - float get_luma_dst(float3 c) { return luma_dst.x * c.x + luma_dst.y * c.y + luma_dst.z * c.z; } diff --git a/libavfilter/vf_colorspace.c b/libavfilter/vf_colorspace.c index f8d1ecd..2120199 100644 --- a/libavfilter/vf_colorspace.c +++ b/libavfilter/vf_colorspace.c @@ -170,78 +170,6 @@ typedef struct ColorSpaceContext { // FIXME dithering if bitdepth goes down? // FIXME bitexact for fate integration? -static const double ycgco_matrix[3][3] = -{ - { 0.25, 0.5, 0.25 }, - { -0.25, 0.5, -0.25 }, - { 0.5, 0, -0.5 }, -}; - -static const double gbr_matrix[3][3] = -{ - { 0, 1, 0 }, - { 0, -0.5, 0.5 }, - { 0.5, -0.5, 0 }, -}; - -/* - * All constants explained in e.g. https://linuxtv.org/downloads/v4l-dvb-apis/ch02s06.html - * The older ones (bt470bg/m) are also explained in their respective ITU docs - * (e.g. https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.470-5-199802-S!!PDF-E.pdf) - * whereas the newer ones can typically be copied directly from wikipedia :) - */ -static const struct LumaCoefficients luma_coefficients[AVCOL_SPC_NB] = { - [AVCOL_SPC_FCC] = { 0.30, 0.59, 0.11 }, - [AVCOL_SPC_BT470BG] = { 0.299, 0.587, 0.114 }, - [AVCOL_SPC_SMPTE170M] = { 0.299, 0.587, 0.114 }, - [AVCOL_SPC_BT709] = { 0.2126, 0.7152, 0.0722 }, - [AVCOL_SPC_SMPTE240M] = { 0.212, 0.701, 0.087 }, - [AVCOL_SPC_YCOCG] = { 0.25, 0.5, 0.25 }, - [AVCOL_SPC_RGB] = { 1, 1, 1 }, - [AVCOL_SPC_BT2020_NCL] = { 0.2627, 0.6780, 0.0593 }, - [AVCOL_SPC_BT2020_CL] = { 0.2627, 0.6780, 0.0593 }, -}; - -static const struct LumaCoefficients *get_luma_coefficients(enum AVColorSpace csp) -{ - const struct LumaCoefficients *coeffs; - - if (csp >= AVCOL_SPC_NB) - return NULL; - coeffs = &luma_coefficients[csp]; - if (!coeffs->cr) - return NULL; - - return coeffs; -} - -static void fill_rgb2yuv_table(const struct LumaCoefficients *coeffs, - double rgb2yuv[3][3]) -{ - double bscale, rscale; - - // special ycgco matrix - if (coeffs->cr == 0.25 && coeffs->cg == 0.5 && coeffs->cb == 0.25) { - memcpy(rgb2yuv, ycgco_matrix, sizeof(double) * 9); - return; - } else if (coeffs->cr == 1 && coeffs->cg == 1 && coeffs->cb == 1) { - memcpy(rgb2yuv, gbr_matrix, sizeof(double) * 9); - return; - } - - rgb2yuv[0][0] = coeffs->cr; - rgb2yuv[0][1] = coeffs->cg; - rgb2yuv[0][2] = coeffs->cb; - bscale = 0.5 / (coeffs->cb - 1.0); - rscale = 0.5 / (coeffs->cr - 1.0); - rgb2yuv[1][0] = bscale * coeffs->cr; - rgb2yuv[1][1] = bscale * coeffs->cg; - rgb2yuv[1][2] = 0.5; - rgb2yuv[2][0] = 0.5; - rgb2yuv[2][1] = rscale * coeffs->cg; - rgb2yuv[2][2] = rscale * coeffs->cb; -} - // FIXME I'm pretty sure gamma22/28 also have a linear toe slope, but I can't // find any actual tables that document their real values... // See http://www.13thmonkey.org/~boris/gammacorrection/ first graph why it matters @@ -669,7 +597,7 @@ static int create_filtergraph(AVFilterContext *ctx, s->in_rng = in->color_range; if (s->user_irng != AVCOL_RANGE_UNSPECIFIED) s->in_rng = s->user_irng; - s->in_lumacoef = get_luma_coefficients(s->in_csp); + s->in_lumacoef = ff_get_luma_coefficients(s->in_csp); if (!s->in_lumacoef) { av_log(ctx, AV_LOG_ERROR, "Unsupported input colorspace %d (%s)\n", @@ -682,7 +610,7 @@ static int create_filtergraph(AVFilterContext *ctx, if (!s->out_lumacoef) { s->out_csp = out->colorspace; s->out_rng = out->color_range; - s->out_lumacoef = get_luma_coefficients(s->out_csp); + s->out_lumacoef = ff_get_luma_coefficients(s->out_csp); if (!s->out_lumacoef) { if (s->out_csp == AVCOL_SPC_UNSPECIFIED) { if (s->user_all == CS_UNSPECIFIED) { @@ -724,7 +652,7 @@ static int create_filtergraph(AVFilterContext *ctx, } for (n = 0; n < 8; n++) s->yuv_offset[0][n] = off; - fill_rgb2yuv_table(s->in_lumacoef, rgb2yuv); + ff_fill_rgb2yuv_table(s->in_lumacoef, rgb2yuv); ff_matrix_invert_3x3(rgb2yuv, yuv2rgb); bits = 1 << (in_desc->comp[0].depth - 1); for (n = 0; n < 3; n++) { @@ -757,7 +685,7 @@ static int create_filtergraph(AVFilterContext *ctx, } for (n = 0; n < 8; n++) s->yuv_offset[1][n] = off; - fill_rgb2yuv_table(s->out_lumacoef, rgb2yuv); + ff_fill_rgb2yuv_table(s->out_lumacoef, rgb2yuv); bits = 1 << (29 - out_desc->comp[0].depth); for (out_rng = s->out_y_rng, n = 0; n < 3; n++, out_rng = s->out_uv_rng) { for (m = 0; m < 3; m++) { diff --git a/libavfilter/vf_tonemap_opencl.c b/libavfilter/vf_tonemap_opencl.c index 88b3107..e085659 100644 --- a/libavfilter/vf_tonemap_opencl.c +++ b/libavfilter/vf_tonemap_opencl.c @@ -35,7 +35,6 @@ // TODO: // - separate peak-detection from tone-mapping kernel to solve // one-frame-delay issue. -// - import colorspace matrix generation from vf_colorspace.c // - more format support #define DETECTION_FRAMES 63 @@ -72,16 +71,6 @@ typedef struct TonemapOpenCLContext { cl_mem util_mem; } TonemapOpenCLContext; -static const char *yuv_coff[AVCOL_SPC_NB] = { - [AVCOL_SPC_BT709] = "rgb2yuv_bt709", - [AVCOL_SPC_BT2020_NCL] = "rgb2yuv_bt2020", -}; - -static const char *rgb_coff[AVCOL_SPC_NB] = { - [AVCOL_SPC_BT709] = "yuv2rgb_bt709", - [AVCOL_SPC_BT2020_NCL] = "yuv2rgb_bt2020", -}; - static const char *linearize_funcs[AVCOL_TRC_NB] = { [AVCOL_TRC_SMPTE2084] = "eotf_st2084", [AVCOL_TRC_ARIB_STD_B67] = "inverse_oetf_hlg", @@ -92,11 +81,6 @@ static const char *delinearize_funcs[AVCOL_TRC_NB] = { [AVCOL_TRC_BT2020_10] = "inverse_eotf_bt1886", }; -static const struct LumaCoefficients luma_coefficients[AVCOL_SPC_NB] = { - [AVCOL_SPC_BT709] = { 0.2126, 0.7152, 0.0722 }, - [AVCOL_SPC_BT2020_NCL] = { 0.2627, 0.6780, 0.0593 }, -}; - static struct PrimaryCoefficients primaries_table[AVCOL_PRI_NB] = { [AVCOL_PRI_BT709] = { 0.640, 0.330, 0.300, 0.600, 0.150, 0.060 }, [AVCOL_PRI_BT2020] = { 0.708, 0.292, 0.170, 0.797, 0.131, 0.046 }, @@ -131,13 +115,25 @@ static void get_rgb2rgb_matrix(enum AVColorPrimaries in, enum AVColorPrimaries o // Average light level for SDR signals. This is equal to a signal level of 0.5 // under a typical presentation gamma of about 2.0. static const float sdr_avg = 0.25f; +static void print_opencl_const_matrix(AVBPrint *buf, const char *name_str, + double mat[3][3]) +{ + int i, j; + av_bprintf(buf, "__constant float %s[9] = {\n", name_str); + for (i = 0; i < 3; i++) { + for (j = 0; j < 3; j++) + av_bprintf(buf, " %.5ff,", mat[i][j]); + av_bprintf(buf, "\n"); + } + av_bprintf(buf, "};\n"); +} static int tonemap_opencl_init(AVFilterContext *avctx) { TonemapOpenCLContext *ctx = avctx->priv; int rgb2rgb_passthrough = 1; - double rgb2rgb[3][3]; - struct LumaCoefficients luma_src, luma_dst; + double rgb2rgb[3][3], rgb2yuv[3][3], yuv2rgb[3][3]; + const struct LumaCoefficients *luma_src, *luma_dst; cl_int cle; int err; AVBPrint header; @@ -214,27 +210,23 @@ static int tonemap_opencl_init(AVFilterContext *avctx) if (rgb2rgb_passthrough) av_bprintf(&header, "#define RGB2RGB_PASSTHROUGH\n"); - else { - av_bprintf(&header, "__constant float rgb2rgb[9] = {\n"); - av_bprintf(&header, " %.4ff, %.4ff, %.4ff,\n", - rgb2rgb[0][0], rgb2rgb[0][1], rgb2rgb[0][2]); - av_bprintf(&header, " %.4ff, %.4ff, %.4ff,\n", - rgb2rgb[1][0], rgb2rgb[1][1], rgb2rgb[1][2]); - av_bprintf(&header, " %.4ff, %.4ff, %.4ff};\n", - rgb2rgb[2][0], rgb2rgb[2][1], rgb2rgb[2][2]); - } + else + print_opencl_const_matrix(&header, "rgb2rgb", rgb2rgb); + + + luma_src = ff_get_luma_coefficients(ctx->colorspace_in); + luma_dst = ff_get_luma_coefficients(ctx->colorspace_out); + ff_fill_rgb2yuv_table(luma_dst, rgb2yuv); + print_opencl_const_matrix(&header, "yuv_matrix", rgb2yuv); - av_bprintf(&header, "#define rgb_matrix %s\n", - rgb_coff[ctx->colorspace_in]); - av_bprintf(&header, "#define yuv_matrix %s\n", - yuv_coff[ctx->colorspace_out]); + ff_fill_rgb2yuv_table(luma_src, rgb2yuv); + ff_matrix_invert_3x3(rgb2yuv, yuv2rgb); + print_opencl_const_matrix(&header, "rgb_matrix", yuv2rgb); - luma_src = luma_coefficients[ctx->colorspace_in]; - luma_dst = luma_coefficients[ctx->colorspace_out]; av_bprintf(&header, "constant float3 luma_src = {%.4ff, %.4ff, %.4ff};\n", - luma_src.cr, luma_src.cg, luma_src.cb); + luma_src->cr, luma_src->cg, luma_src->cb); av_bprintf(&header, "constant float3 luma_dst = {%.4ff, %.4ff, %.4ff};\n", - luma_dst.cr, luma_dst.cg, luma_dst.cb); + luma_dst->cr, luma_dst->cg, luma_dst->cb); av_bprintf(&header, "#define linearize %s\n", linearize_funcs[ctx->trc_in]); av_bprintf(&header, "#define delinearize %s\n",