From patchwork Thu Sep 10 23:57:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex <3.14pi@ukr.net> X-Patchwork-Id: 22277 Delivered-To: andriy.gelman@gmail.com Received: by 2002:a25:5387:0:0:0:0:0 with SMTP id h129csp1634787ybb; Thu, 10 Sep 2020 16:57:26 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzLTyI4mL9+rkCWyutDi8utHLTuJT13KyOyGAeUd2k/QQ2vROsjiRK85jpUm2kifWTTNu3L X-Received: by 2002:a1c:2983:: with SMTP id p125mr2388537wmp.21.1599782245994; Thu, 10 Sep 2020 16:57:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1599782245; cv=none; d=google.com; s=arc-20160816; b=eSHs2lF9zxSMyCtqWDBFfifoyZ+hV4GJb4QxJM1zprndRt4UmE9SlhFLH+Uan0mLyQ i2AzxFXS9oyzXOMsSoTntUySlFeYFSLOHrxSSKJdLtQFi+94mqgaEoXzSXWIl85UalsS B5srOXN3Nq3h6aA/ZIqxeUkCOg0sEznMrpRY2LM+LVkQ4nfxASlbLPvSt03AyyRhPBsI yQJDenZyivF8hs+FqRCS51Ga3Araslsr+llSyrvkDgqo53MAjAGmvQba4ypnb6OeMYkd HT59BnR4vFFjk3hJkQqXHB0Jkob490bK7/AtdKsP3QMJYVi3PtdxUc4mwFC5zp2MxTgK ngkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:message-id:cc:to:from:date:dkim-signature:delivered-to; bh=2pUvDb0F3BIpxMPnIR+XLq8LLGZ6e2z2EYjHWI/f1Q4=; b=NKkmIP1bBv2syS6wkfnElbLejkaa2ZLwlNZ/8jL/Uz/8a28tt+ARoDxoYhEV+T6kyZ VmDH3TnW/9xJniXD+4aP4tjITRGx9LQsSOL+24+Xfev9GJ3f3PTEDb+C73r9MbA1QswH WLWuR5aQCePqKmXGHcSql373RzKPZa4Vin2tDh/+0RZrf2pHuBk4s8G1WW289Hlc51zJ 9E9ChbPpds/4eLVueikDVJCiDmbXiF6kDNXjp7krhDSwIudNXWOQzAap/74uTM44kWsp XJ5cEimcf4wOIbZMGrgMJZcbGJZxwG5tGP7Mi30sLSjYTPYTdaOW5alFER42hy2H3nEm LWmQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ukr.net header.s=ffe header.b=seiAZiTv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ukr.net Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v1si464217wru.542.2020.09.10.16.57.25; Thu, 10 Sep 2020 16:57:25 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ukr.net header.s=ffe header.b=seiAZiTv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ukr.net Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 450D068B9CB; Fri, 11 Sep 2020 02:57:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from frv191.fwdcdn.com (frv191.fwdcdn.com [212.42.77.191]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9353368065D for ; Fri, 11 Sep 2020 02:57:17 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Type:MIME-Version:Message-Id:Cc:To:Subject:From:Date:Sender: Reply-To:Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date :Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Id:List-Help:List-Unsubscribe:List-Subscribe:List-Post: List-Owner:List-Archive; bh=7++hVvPvHiwpXrxJanSPtk0nTA4M2SezVcEwbkMYjF8=; b=s eiAZiTvVHqLQxhl7r86NV0DMe+YidOApQPxc84UKXA49E7yzNUwPmOsSU4xYyavzTtBQlXCb0Gi/N 8jAaiZE7FmQnVoxyN2m/jzMWsGRkYww6PQW+iNLE/dBb7UAVJBMHHfUgVs/U366l++lwa2GBwckkw 34c8aOIfoYCWI2nQ=; Received: from [10.10.80.24] (helo=frv55.fwdcdn.com) by frv191.fwdcdn.com with smtp ID 1kGWRI-000PGC-Oc for ffmpeg-devel@ffmpeg.org; Fri, 11 Sep 2020 02:57:17 +0300 Date: Fri, 11 Sep 2020 02:57:16 +0300 From: Alex <3.14pi@ukr.net> To: FFmpeg development discussions and patches Cc: Received: from 3.14pi@ukr.net by frv55.fwdcdn.com; Fri, 11 Sep 2020 02:57:16 +0300 Message-Id: <1599782008.655533000.a5lxms1z@frv55.fwdcdn.com> X-Mailer: mail.ukr.net 5.0 MIME-Version: 1.0 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: [FFmpeg-devel] [PATCH] avfilter: add http video filter. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: wS+8cSic05kL Signed-off-by: alex_qt <3.14pi@ukr.net> --- Changelog | 1 + configure | 4 + doc/filters.texi | 28 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/version.h | 2 +- libavfilter/vf_http.c | 221 +++++++++++++++++++++++++++++++++++++++ 7 files changed, 257 insertions(+), 1 deletion(-) create mode 100644 libavfilter/vf_http.c diff --git a/Changelog b/Changelog index cd8be931ef..777cca679a 100644 --- a/Changelog +++ b/Changelog @@ -22,6 +22,7 @@ version : - MODS demuxer - PhotoCD decoder - MCA demuxer +- http video filter, send raw frames to remote url for postprocessing version 4.3: diff --git a/configure b/configure index ae8c6e61c8..f5131d4669 100755 --- a/configure +++ b/configure @@ -325,6 +325,7 @@ External library support: --enable-vulkan enable Vulkan code [no] --disable-xlib disable xlib [autodetect] --disable-zlib disable zlib [autodetect] + --enable-libcurl enable http filter that send raw frames to remote server The following libraries provide various hardware acceleration features: --disable-amf disable AMF video encoding code [autodetect] @@ -1827,6 +1828,7 @@ EXTERNAL_LIBRARY_LIST=" opengl pocketsphinx vapoursynth + libcurl " HWACCEL_AUTODETECT_LIBRARY_LIST=" @@ -3650,6 +3652,7 @@ vpp_qsv_filter_select="qsvvpp" xfade_opencl_filter_deps="opencl" yadif_cuda_filter_deps="ffnvcodec" yadif_cuda_filter_deps_any="cuda_nvcc cuda_llvm" +http_filter_deps="libcurl" # examples avio_list_dir_deps="avformat avutil" @@ -6367,6 +6370,7 @@ enabled libmysofa && { check_pkg_config libmysofa libmysofa mysofa.h mys enabled libnpp && { check_lib libnpp npp.h nppGetLibVersion -lnppig -lnppicc -lnppc -lnppidei || check_lib libnpp npp.h nppGetLibVersion -lnppi -lnppc -lnppidei || die "ERROR: libnpp not found"; } +enabled libcurl && require "libcurl >= 7.68.0" curl/curl.h curl_easy_init -lcurl enabled libopencore_amrnb && require libopencore_amrnb opencore-amrnb/interf_dec.h Decoder_Interface_init -lopencore-amrnb enabled libopencore_amrwb && require libopencore_amrwb opencore-amrwb/dec_if.h D_IF_init -lopencore-amrwb enabled libopencv && { check_headers opencv2/core/core_c.h && diff --git a/doc/filters.texi b/doc/filters.texi index cbb16f22b2..660ef8b4d9 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -12078,6 +12078,34 @@ For example, to horizontally flip the input video with @command{ffmpeg}: ffmpeg -i in.avi -vf "hflip" out.avi @end example +@anchor{http} +@section http + +Send raw frame data to the remote server for postprocessing and await response as new frame in same format and size. To enable filter configure ffmpeg with @code{./configure --enable-libcurl}. + +The filter accepts the following options: + +@table @option +@item url +Specify remote server url location. + +@item content_type +Specify content-type header in request header, default to "application/octet-stream". +@end table + +Simple demo http server for postprocessing frames can be found here: @url{https://github.com/devalexqt/simple_ffmpeg_http_filter_server} + +@subsection Examples +@itemize + +@item +Send raw frames to "http://localhost:3000/frame?param=abc" + +@example +ffmpeg -i input.mp4 -vf format=rgb24,http=url="http\\\://localhost\\\:3000/frame?param=abc":content_type=application/octet-stream -t 10 out.mp4 +@end example +@end itemize + @section histeq This filter applies a global color histogram equalization on a per-frame basis. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index e6d3c283da..38eb1f4204 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -467,6 +467,7 @@ OBJS-$(CONFIG_YAEPBLUR_FILTER) += vf_yaepblur.o OBJS-$(CONFIG_ZMQ_FILTER) += f_zmq.o OBJS-$(CONFIG_ZOOMPAN_FILTER) += vf_zoompan.o OBJS-$(CONFIG_ZSCALE_FILTER) += vf_zscale.o +OBJS-$(CONFIG_HTTP_FILTER) += vf_http.o OBJS-$(CONFIG_ALLRGB_FILTER) += vsrc_testsrc.o OBJS-$(CONFIG_ALLYUV_FILTER) += vsrc_testsrc.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index fa91e608e4..8626fbc331 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -445,6 +445,7 @@ extern AVFilter ff_vf_yaepblur; extern AVFilter ff_vf_zmq; extern AVFilter ff_vf_zoompan; extern AVFilter ff_vf_zscale; +extern AVFilter ff_vf_http; extern AVFilter ff_vsrc_allrgb; extern AVFilter ff_vsrc_allyuv; diff --git a/libavfilter/version.h b/libavfilter/version.h index 308fbe07c3..b8ba489da7 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -30,7 +30,7 @@ #include "libavutil/version.h" #define LIBAVFILTER_VERSION_MAJOR 7 -#define LIBAVFILTER_VERSION_MINOR 87 +#define LIBAVFILTER_VERSION_MINOR 88 #define LIBAVFILTER_VERSION_MICRO 100 diff --git a/libavfilter/vf_http.c b/libavfilter/vf_http.c new file mode 100644 index 0000000000..f4290ca254 --- /dev/null +++ b/libavfilter/vf_http.c @@ -0,0 +1,221 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * http video filter + */ + +#include "libavutil/avassert.h" +#include "libavutil/opt.h" +#include "libavutil/internal.h" +#include "libavutil/imgutils.h" +#include "libavutil/avstring.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" +#include "stdio.h" +#include "curl/curl.h" + +struct MemoryStruct { + uint8_t *memory; + size_t size; +}; + +typedef struct HttpContext { + const AVClass *class; + char *url; + char *content_type; + CURL *curl; + struct curl_slist *headers; +} HttpContext; + +#define OFFSET(x) offsetof(HttpContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM + +static const AVOption http_options[] = { + { "url", "set remote url address", OFFSET(url), AV_OPT_TYPE_STRING, {.str=NULL}, FLAGS }, + { "content_type", "set 'Content-Type' request header", OFFSET(content_type), AV_OPT_TYPE_STRING, {.str="application/octet-stream"}, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(http); + +static av_cold int init(AVFilterContext *ctx) +{ + HttpContext *http = ctx->priv; + http->curl = curl_easy_init(); + + /* check if remote server url is valid formated */ + CURLU *url= curl_url(); + CURLUcode result; + + /* parse a full URL */ + result = curl_url_set(url, CURLUPART_URL, http->url, 0); + if(result){ + av_log(NULL, AV_LOG_ERROR, "http filter failed: invalid input url!\n"); + return AVERROR(EINVAL); + } + + /* set request headers */ + http->headers=NULL; + http->headers = curl_slist_append(http->headers, av_asprintf("Content-Type: %s",http->content_type)); + http->headers = curl_slist_append(http->headers, "Expect:"); + curl_easy_setopt(http->curl, CURLOPT_HTTPHEADER, http->headers); + curl_easy_setopt(http->curl, CURLOPT_FOLLOWLOCATION, 1L); + + curl_url_cleanup(url); + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + HttpContext *http = ctx->priv; + + curl_easy_cleanup(http->curl); + curl_slist_free_all(http->headers); + curl_global_cleanup(); +} + +static int query_formats(AVFilterContext *ctx) +{ + const HttpContext *http = ctx->priv; + return ff_default_query_formats(ctx); +} + +static size_t WriteMemoryCallback(void *contents, size_t size, size_t nmemb, void *userp) +{ + size_t realsize = size * nmemb; + struct MemoryStruct *mem = (struct MemoryStruct *)userp; + + char *ptr = realloc(mem->memory, mem->size + realsize + 1); + if(!ptr) { + /* out of memory! */ + return AVERROR(ENOMEM); + } + + mem->memory = ptr; + memcpy(&(mem->memory[mem->size]), contents, realsize); + mem->size += realsize; + mem->memory[mem->size] = 0; + + return realsize; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + AVFilterLink *outlink = ctx->outputs[0]; + HttpContext *http = inlink->dst->priv; + AVFrame *out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + av_frame_copy_props(out, in); + + CURL *curl=http->curl; + CURLU *url= curl_url(); + CURLcode res; + struct MemoryStruct chunk; + chunk.memory = av_malloc(1); + chunk.size = 0; + + /* create buffer and copy input frame */ + int buff_size=av_image_get_buffer_size(in->format, in->width, in->height, 1); + uint8_t *buffer=av_malloc(buff_size); + av_image_copy_to_buffer(buffer, buff_size, in->data, in->linesize, in->format, in->width, in->height, 1); + + /* update url query */ + CURLUcode result; + const char *url_params=av_asprintf("width=%d&height=%d&format=%d&size=%d&pts=%ld",in->width, in->height, in->format, buff_size, in->pts); + curl_url_set(url, CURLUPART_URL, http->url, 0); + result = curl_url_set(url, CURLUPART_QUERY, url_params, CURLU_APPENDQUERY); + if(result){ + av_log(NULL, AV_LOG_ERROR, "http filter failed: failed to setup url query params!\n"); + return AVERROR(EINVAL); + } + + /* specify the POST data */ + curl_easy_setopt(http->curl, CURLOPT_CURLU,url); + curl_easy_setopt(curl, CURLOPT_POSTFIELDS, buffer); + curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, buff_size); + + /* send all data to this function */ + curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteMemoryCallback); + + /* we pass our 'chunk' struct to the callback function */ + curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void *)&chunk); + curl_easy_setopt(curl, CURLOPT_USERAGENT, "libcurl-agent/1.0"); + + /* Perform the request, res will get the return code */ + res = curl_easy_perform(curl); + + /* Check for errors */ + if(res != CURLE_OK){ + av_log(NULL, AV_LOG_ERROR, "http filter failed: %s\n", curl_easy_strerror(res)); + av_frame_free(&in); + av_frame_free(&out); + free(buffer); + return AVERROR(EINVAL); + } + else{ + if(chunk.size!=buff_size){ + av_log(NULL, AV_LOG_ERROR, "http filter failed: size of the input and received frames must be equal!\n"); + return AVERROR(EINVAL); + } + /* fill frame data from received buffer */ + av_image_fill_arrays(out->data, out->linesize, chunk.memory, out->format, out->width, out->height, 1); + } + + /* cleanup */ + curl_url_cleanup(url); + av_free(buffer); + av_frame_free(&in); + return ff_filter_frame(outlink, out); +} + +static const AVFilterPad avfilter_vf_http_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + }, + { NULL } +}; + +static const AVFilterPad avfilter_vf_http_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +AVFilter ff_vf_http = { + .name = "http", + .description = NULL_IF_CONFIG_SMALL("Send raw frame data to the remote server for postprocessing and await response as new frame in same format and size."), + .priv_size = sizeof(HttpContext), + .init = init, + .uninit = uninit, + .query_formats = query_formats, + .inputs = avfilter_vf_http_inputs, + .outputs = avfilter_vf_http_outputs, + .priv_class = &http_class, +};