From patchwork Tue Nov 14 19:47:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 6065 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.94 with SMTP id m30csp3722501jah; Tue, 14 Nov 2017 11:49:55 -0800 (PST) X-Google-Smtp-Source: AGs4zMY2Oje1Y9Yv6BLVG9kbE1dDfXM75ZNoTvNR79xHOHFdSuwQMMlgAqGNg1E/nY2ZgMb1ke5e X-Received: by 10.223.136.162 with SMTP id f31mr10128804wrf.130.1510688995142; Tue, 14 Nov 2017 11:49:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510688995; cv=none; d=google.com; s=arc-20160816; b=ybyr3bv2tJj7hGdbPRrFH2jnCUNwFcHFNmPggzIuR4VRSNoByz09hUmW5QEa+br1tZ OsWzsK926lxPpMM3FKng3vuEZuh2Mn+KtclQdJ1GW3XQuirmkUBcQhqSaO3OjdOUB+i9 m9FNfhhd3xJ8hzu1ZuoiH3XcO7opS1nuLnFrR1upwK5cEzLL5hbq0fToOafP4JFEp9A/ 8FBg6Fjq+hZ0H5mENrAD+JipJSBsi3Bz7/kNZtG5rZi7oxpWJ9pkXXuHlpejn4KnMhHX jaNVyW5feVbffdld0qB6h3wIL6fbpkQKbXP1hkl0/w6f6+QbVl7VTh4wMIT1sh9e6Pyq Vnmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=3W9mpG87o78lXdAiHeLob0oVIyntGCgkcbAcXbl9amw=; b=oQ5mmBD6ouChePogUR1IQjF0+3228vbx64uKVauM1O0MzSvD/InsT1guXqVgIYS449 LWBYFa9t0dpmjGjH9qIz0pQxj+myF97yiDX6jUhSqLaY9mR+RJUctm+B20iLgYsh9or3 XsgfwruZcfO0WOzF305iOG+cOH0BqbDEFgZ2tMlf9hzqf+HBBq1P9CRsh9MqyzNYBurz tu753imDMXge/sQxTPuuMpUCtsom9ynFmrN+0dLE7LkaNoxpAU29pPm6h0Bmxr9WITbI yxKpoCxLQg9a35F09drrhvVxvfaEAuVlop+pXXo9dtzEXjpPbI0UNHBhCFiI/qqXWgUw eSVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=QrMPrvCX; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 43si15111771wrz.329.2017.11.14.11.49.54; Tue, 14 Nov 2017 11:49:55 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=QrMPrvCX; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 93A5A68A1E3; Tue, 14 Nov 2017 21:47:38 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f171.google.com (mail-wr0-f171.google.com [209.85.128.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6373D68A18B for ; Tue, 14 Nov 2017 21:47:29 +0200 (EET) Received: by mail-wr0-f171.google.com with SMTP id 4so18532902wrt.0 for ; Tue, 14 Nov 2017 11:47:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ISVoD+jsR4Rz3nmsCIRvJ341s3Q+UEu8iFvAoTftLfE=; b=QrMPrvCXJPD8lrzCJh3hgmhkdO1FYqr0BYUswLGW4wz5YSI0OLnUB5SvuJbmRk75+W NH4/9vke7x3tWenu7a9XYHzbVKBN2MorDca0lKrE8DIVFhj7wQlbNto8xQUMuzP9+jtA RumUDIatjcY0ZUm3H0Q0um7dgawV5iLfW3t7NCHTkpD+3IJPYLAmrIFULVGKeIAFXXxd EqK81Zo45hWjhaTDfPa20JuiNgJ1GdnuvUwBbWMbdTgdv3X0eq2OuRsTa6bf0D28nwa3 97GmOflCYLItnrEUTNZnhsbsgCs0KwwMZ7FPvFMetpjhNPdsZaYduYNmzQm8vzctT7/K bfMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ISVoD+jsR4Rz3nmsCIRvJ341s3Q+UEu8iFvAoTftLfE=; b=OqwRqumd8zJt3BXppUwRAfrQJrJab+Ge/Wgl3TzMerx0fQ4sw3/DYGBZC5V1rQsQLe dR9tAAr0ezpmADMOUEsNXpX6Oa6kqjuCzpaaLwx/T29agV9Woy8J8kKGr5ilienyxx6i QlFv9BjT0BK7Quk1UrnDXubre7NpdSAufBrMDgVLbZ1J1L4SyI7+6IYv3hlaZ3wZ2CAh IKWl2zMwb6kl+oOs27/I5cpznCc1XtLhbXoayEgHYzscyfza/d1k1DDjkElV8hY3i5L/ 9bacyU3P//m5QSxdJ8y30ClYVspXAIooYoNe76fCjidRdCfYifMC8+hIyJRaeArGegRW Gttw== X-Gm-Message-State: AJaThX51pGowlVDh8jR9CDtx14YlGy5CvEdWG8NeUkaeOSFEATWm4uVM pghUM8uepk7QhInrv1oJo4mKwGAi X-Received: by 10.223.135.143 with SMTP id b15mr10145475wrb.278.1510688863810; Tue, 14 Nov 2017 11:47:43 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id v35sm38938226wrc.13.2017.11.14.11.47.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 14 Nov 2017 11:47:43 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Tue, 14 Nov 2017 19:47:24 +0000 Message-Id: <20171114194730.11052-10-sw@jkqxz.net> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171114194730.11052-1-sw@jkqxz.net> References: <20171114194730.11052-1-sw@jkqxz.net> Subject: [FFmpeg-devel] [PATCH 09/15] lavfi: Add filter to run an arbitrary OpenCL program on frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- configure | 1 + libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_program_opencl.c | 254 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 257 insertions(+) create mode 100644 libavfilter/vf_program_opencl.c diff --git a/configure b/configure index cefe4205e5..9557e7e1ba 100755 --- a/configure +++ b/configure @@ -3233,6 +3233,7 @@ perspective_filter_deps="gpl" phase_filter_deps="gpl" pp7_filter_deps="gpl" pp_filter_deps="gpl postproc" +program_opencl_filter_deps="opencl" pullup_filter_deps="gpl" removelogo_filter_deps="avcodec avformat swscale" repeatfields_filter_deps="gpl" diff --git a/libavfilter/Makefile b/libavfilter/Makefile index b7ddcd226d..6ebdc1f173 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -265,6 +265,7 @@ OBJS-$(CONFIG_PP_FILTER) += vf_pp.o OBJS-$(CONFIG_PP7_FILTER) += vf_pp7.o OBJS-$(CONFIG_PREMULTIPLY_FILTER) += vf_premultiply.o framesync.o OBJS-$(CONFIG_PREWITT_FILTER) += vf_convolution.o +OBJS-$(CONFIG_PROGRAM_OPENCL_FILTER) += vf_program_opencl.o opencl.o OBJS-$(CONFIG_PSEUDOCOLOR_FILTER) += vf_pseudocolor.o OBJS-$(CONFIG_PSNR_FILTER) += vf_psnr.o framesync.o OBJS-$(CONFIG_PULLUP_FILTER) += vf_pullup.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 3647a111ec..dfb92210a1 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -274,6 +274,7 @@ static void register_all(void) REGISTER_FILTER(PP7, pp7, vf); REGISTER_FILTER(PREMULTIPLY, premultiply, vf); REGISTER_FILTER(PREWITT, prewitt, vf); + REGISTER_FILTER(PROGRAM_OPENCL, program_opencl, vf); REGISTER_FILTER(PSEUDOCOLOR, pseudocolor, vf); REGISTER_FILTER(PSNR, psnr, vf); REGISTER_FILTER(PULLUP, pullup, vf); diff --git a/libavfilter/vf_program_opencl.c b/libavfilter/vf_program_opencl.c new file mode 100644 index 0000000000..ed99d827ae --- /dev/null +++ b/libavfilter/vf_program_opencl.c @@ -0,0 +1,254 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/buffer.h" +#include "libavutil/common.h" +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_opencl.h" +#include "libavutil/log.h" +#include "libavutil/mem.h" +#include "libavutil/pixdesc.h" +#include "libavutil/opt.h" + +#include "avfilter.h" +#include "internal.h" +#include "opencl.h" +#include "video.h" + +typedef struct ProgramOpenCLContext { + OpenCLFilterContext ocf; + + int initialised; + cl_uint index; + cl_kernel kernel; + cl_command_queue command_queue; + + const char *source_file; + const char *kernel_name; +} ProgramOpenCLContext; + +static int program_opencl_init(AVFilterContext *avctx) +{ + ProgramOpenCLContext *ctx = avctx->priv; + cl_int cle; + int err; + + err = ff_opencl_filter_load_program_from_file(avctx, ctx->source_file); + if (err < 0) + goto fail; + + ctx->command_queue = clCreateCommandQueue(ctx->ocf.hwctx->context, + ctx->ocf.hwctx->device_id, + 0, &cle); + if (!ctx->command_queue) { + av_log(avctx, AV_LOG_ERROR, "Failed to create OpenCL " + "command queue: %d.\n", cle); + err = AVERROR(EIO); + goto fail; + } + + ctx->kernel = clCreateKernel(ctx->ocf.program, ctx->kernel_name, &cle); + if (!ctx->kernel) { + av_log(avctx, AV_LOG_ERROR, "Failed to create kernel: %d.\n", cle); + err = AVERROR(EIO); + goto fail; + } + + ctx->initialised = 1; + return 0; + +fail: + if (ctx->command_queue) + clReleaseCommandQueue(ctx->command_queue); + if (ctx->kernel) + clReleaseKernel(ctx->kernel); + return err; +} + +static int program_opencl_filter_frame(AVFilterLink *inlink, AVFrame *input) +{ + AVFilterContext *avctx = inlink->dst; + AVFilterLink *outlink = avctx->outputs[0]; + ProgramOpenCLContext *ctx = avctx->priv; + AVFrame *output = NULL; + cl_int cle; + size_t global_work[2]; + cl_mem src, dst; + int err, plane; + + av_log(ctx, AV_LOG_DEBUG, "Filter input: %s, %ux%u (%"PRId64").\n", + av_get_pix_fmt_name(input->format), + input->width, input->height, input->pts); + + if (!input->hw_frames_ctx) + return AVERROR(EINVAL); + + if (!ctx->initialised) { + err = program_opencl_init(avctx); + if (err < 0) + goto fail; + } + + output = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!output) { + err = AVERROR(ENOMEM); + goto fail; + } + + for (plane = 0; plane < FF_ARRAY_ELEMS(output->data); plane++) { + src = (cl_mem) input->data[plane]; + dst = (cl_mem)output->data[plane]; + + if (!dst) + break; + + cle = clSetKernelArg(ctx->kernel, 0, sizeof(cl_mem), &dst); + if (cle != CL_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to set kernel " + "destination image argument: %d.\n", cle); + goto fail; + } + cle = clSetKernelArg(ctx->kernel, 1, sizeof(cl_mem), &src); + if (cle != CL_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to set kernel " + "source image argument: %d.\n", cle); + goto fail; + } + cle = clSetKernelArg(ctx->kernel, 2, sizeof(cl_uint), &ctx->index); + if (cle != CL_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to set kernel " + "index argument: %d.\n", cle); + goto fail; + } + + cle = clGetImageInfo(dst, CL_IMAGE_WIDTH, sizeof(size_t), + &global_work[0], NULL); + cle = clGetImageInfo(dst, CL_IMAGE_HEIGHT, sizeof(size_t), + &global_work[1], NULL); + + av_log(avctx, AV_LOG_DEBUG, "Run kernel on plane %d " + "(%zux%zu).\n", plane, global_work[0], global_work[1]); + + cle = clEnqueueNDRangeKernel(ctx->command_queue, ctx->kernel, 2, NULL, + global_work, NULL, 0, NULL, NULL); + if (cle != CL_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to enqueue kernel: %d.\n", + cle); + err = AVERROR(EIO); + goto fail; + } + } + + cle = clFinish(ctx->command_queue); + if (cle != CL_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to finish command queue: %d.\n", + cle); + err = AVERROR(EIO); + goto fail; + } + + err = av_frame_copy_props(output, input); + if (err < 0) + goto fail; + + av_frame_free(&input); + + ++ctx->index; + + av_log(ctx, AV_LOG_DEBUG, "Filter output: %s, %ux%u (%"PRId64").\n", + av_get_pix_fmt_name(output->format), + output->width, output->height, output->pts); + + return ff_filter_frame(outlink, output); + +fail: + clFinish(ctx->command_queue); + av_frame_free(&input); + av_frame_free(&output); + return err; +} + +static av_cold void program_opencl_uninit(AVFilterContext *avctx) +{ + ProgramOpenCLContext *ctx = avctx->priv; + cl_int cle; + + if (ctx->kernel) { + cle = clReleaseKernel(ctx->kernel); + if (cle != CL_SUCCESS) + av_log(avctx, AV_LOG_ERROR, "Failed to release " + "kernel: %d.\n", cle); + } + + if (ctx->command_queue) { + cle = clReleaseCommandQueue(ctx->command_queue); + if (cle != CL_SUCCESS) + av_log(avctx, AV_LOG_ERROR, "Failed to release " + "command queue: %d.\n", cle); + } + + ff_opencl_filter_uninit(avctx); +} + +#define OFFSET(x) offsetof(ProgramOpenCLContext, x) +#define FLAGS (AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM) +static const AVOption program_opencl_options[] = { + { "source", "OpenCL program source file", + OFFSET(source_file), AV_OPT_TYPE_STRING, { .str = 0 }, .flags = FLAGS }, + { "kernel", "Kernel name in program", + OFFSET(kernel_name), AV_OPT_TYPE_STRING, { .str = 0 }, .flags = FLAGS }, + { "w", "Output video width", + OFFSET(ocf.output_width), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, .flags = FLAGS }, + { "h", "Output video height", + OFFSET(ocf.output_height),AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, .flags = FLAGS }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(program_opencl); + +static const AVFilterPad program_opencl_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = &program_opencl_filter_frame, + .config_props = &ff_opencl_filter_config_input, + }, + { NULL } +}; + +static const AVFilterPad program_opencl_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = &ff_opencl_filter_config_output, + }, + { NULL } +}; + +AVFilter ff_vf_program_opencl = { + .name = "program_opencl", + .description = NULL_IF_CONFIG_SMALL("Filter using an OpenCL program"), + .priv_size = sizeof(ProgramOpenCLContext), + .priv_class = &program_opencl_class, + .init = &ff_opencl_filter_init, + .uninit = &program_opencl_uninit, + .query_formats = &ff_opencl_filter_query_formats, + .inputs = program_opencl_inputs, + .outputs = program_opencl_outputs, + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, +};