From patchwork Wed May 31 11:36:49 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saverio Blasi X-Patchwork-Id: 3771 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.10.2 with SMTP id 2csp243020vsk; Wed, 31 May 2017 04:37:01 -0700 (PDT) X-Received: by 10.223.145.75 with SMTP id j69mr17152464wrj.23.1496230621219; Wed, 31 May 2017 04:37:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1496230621; cv=none; d=google.com; s=arc-20160816; b=ScBjX2QDWsS/zHt+H1RYHakS63AO+gv5UBbt0bwuOUkvjgJ0RL0Nh3NcBDRgoQ1jqY Ghs5uc9DBaBmi2PG4IHE6lrOPtmROZ0tIiZfx6zAzovg75xYJaJx3eWWXKFRWE/zln6s FTsoWShPc9VJs0c6/V7HA1NpujH/e9Fik44PQjs2ZALcdnbSWKaR+h6fLnpyjXR1VDW8 2Vji/u1/EYVR3sLVHS1ss8Y5L/+ht9K2+ScNfV4IiXSlcBoLH7JnNQcHq2rsB5dqwnoG MEGAedF6UZ8nf9w4x1Hu5oU6/ZpJHcMWT2EWzycOqQIndjiZDgBBBOXO7yQKO1E42l5o 08Vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:content-language:accept-language :in-reply-to:references:message-id:date:thread-index:thread-topic:to :from:delivered-to:arc-authentication-results; bh=ugZo2sm8x36RexPVlLIa4IokChOg/5gYVCTjJ5wsVAw=; b=UyqxEP1t6RDNtlXQLVgNFWVbfC/5vi/YdjzMjZdOIxKHFdmb40W6n8gBjNr4uL8e1+ H53qDuVIu6LpLRFkVblTuF0gx0CjzP45slsjeVz7I2/hOzzgLs93vXlUZOqA2iwt2hbP PpH/2C2b3R044eWmOYTKSb8kJUkmEo+1/oFcBHvxuqLympa/+1aZPNd2wC66t/vDx1Rx OMvWVXVl44WJPFxDX8q2X3vxUp/9JVZgzXw23XlhMzJID9GfBmWY3BCy9+DZcMHR9oq6 UPVGIhWm4Ewyiwd6aaTQCMOqkL0GBpS1fdKsMCBk8DI2bsoesyLEB67F0nCnW0jeNESd w/LA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o64si19222226wmd.102.2017.05.31.04.37.00; Wed, 31 May 2017 04:37:01 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C4BBC6891F9; Wed, 31 May 2017 14:36:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mailout0.telhc.bbc.co.uk (mailout0.telhc.bbc.co.uk [132.185.161.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0F4BC6806B9 for ; Wed, 31 May 2017 14:36:45 +0300 (EEST) Received: from BGB01XI1010.national.core.bbc.co.uk (bgb01xi1010.national.core.bbc.co.uk [10.161.14.14]) by mailout0.telhc.bbc.co.uk (8.15.2/8.15.2) with ESMTP id v4VBaogx013313 for ; Wed, 31 May 2017 12:36:50 +0100 (BST) Received: from BGB01XUD1009.national.core.bbc.co.uk ([10.161.14.7]) by BGB01XI1010.national.core.bbc.co.uk ([10.161.14.14]) with mapi id 14.03.0319.002; Wed, 31 May 2017 12:36:50 +0100 From: Saverio Blasi To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v10] - Added Turing codec interface for ffmpeg Thread-Index: AQHSyKw0O2okB1/XfEO6pj2F8gLt6aIOctPw Date: Wed, 31 May 2017 11:36:49 +0000 Message-ID: <59139428E3A17E4B867549526134D166126C6D6B@bgb01xud1009> References: <1493040555-7057-1-git-send-email-saverio.blasi@bbc.co.uk> <1494324497-21363-1-git-send-email-saverio.blasi@bbc.co.uk> In-Reply-To: <1494324497-21363-1-git-send-email-saverio.blasi@bbc.co.uk> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.19.161.212] x-exclaimer-md-config: c91d45b2-6e10-4209-9543-d9970fac71b7 x-tm-as-product-ver: SMEX-11.0.0.4255-8.100.1062-23102.006 x-tm-as-result: No--22.386600-0.000000-31 x-tm-as-user-approved-sender: Yes x-tm-as-user-blocked-sender: No MIME-Version: 1.0 Subject: Re: [FFmpeg-devel] [PATCH v10] - Added Turing codec interface for ffmpeg X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hello, Could you please provide us with some feedback on this patch? Thanks, Saverio -----Original Message----- From: Saverio Blasi [mailto:saverio.blasi@bbc.co.uk] Sent: 09 May 2017 11:08 To: ffmpeg-devel@ffmpeg.org Cc: Saverio Blasi Subject: [PATCH v10] - Added Turing codec interface for ffmpeg - This patch contains the changes to interface the Turing codec (http://turingcodec.org/) with ffmpeg. The patch was modified to address the comments in the review as follows: - Added a pkg-config file to list all dependencies required by libturing. This should address the issue pointed out by Hendrik Leppkes on Fri 18/11/2016 - As per suggestions of wm4, two functions (add_option and finalise_options) have been created. The former appends new options while the latter sets up the argv array of pointers to char* accordingly. add_option re-allocates the buffer for options using av_realloc - Additionally, both these functions handle the errors in case the memory wasn't allocated correctly - malloc|free|realloc have been substituted with their corresponding av_{malloc|free|realloc} version - Check on bit-depth has been removed since the ffmpeg already casts the right pix_fmt and bit depth - pix_fmts is now set in ff_libturing_encoder as in h264dec.c. - Changed usage of av_free with av_freep and fixed calls to free arrays - Added brackets to all if and for statements - Avoid repetition of code to free arrays in case of failure to initialise the libturing encoder - Some fixes to address the review from wm4 and Mark Thompson received on Wed 08/02/2017 - Fixed indentation --- LICENSE.md | 1 + configure | 6 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libturing.c | 313 +++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 322 insertions(+) create mode 100755 libavcodec/libturing.c diff --git a/LICENSE.md b/LICENSE.md index ba65b05..03787c0 100644 --- a/LICENSE.md +++ b/LICENSE.md @@ -84,6 +84,7 @@ The following libraries are under GPL: - frei0r - libcdio - librubberband +- libturing - libvidstab - libx264 - libx265 diff --git a/configure b/configure index 2e1786a..0adc4da 100755 --- a/configure +++ b/configure @@ -256,6 +256,7 @@ External library support: --enable-libssh enable SFTP protocol via libssh [no] --enable-libtesseract enable Tesseract, needed for ocr filter [no] --enable-libtheora enable Theora encoding via libtheora [no] + --enable-libturing enable H.265/HEVC encoding via libturing [no] --enable-libtwolame enable MP2 encoding via libtwolame [no] --enable-libv4l2 enable libv4l2/v4l-utils [no] --enable-libvidstab enable video stabilization using vid.stab [no] @@ -1497,6 +1498,7 @@ EXTERNAL_LIBRARY_GPL_LIST=" frei0r libcdio librubberband + libturing libvidstab libx264 libx265 @@ -2875,6 +2877,7 @@ libspeex_decoder_deps="libspeex" libspeex_encoder_deps="libspeex" libspeex_encoder_select="audio_frame_queue" libtheora_encoder_deps="libtheora" +libturing_encoder_deps="libturing" libtwolame_encoder_deps="libtwolame" libvo_amrwbenc_encoder_deps="libvo_amrwbenc" libvorbis_decoder_deps="libvorbis" @@ -5831,6 +5834,9 @@ enabled libssh && require_pkg_config libssh libssh/sftp.h sftp_init enabled libspeex && require_pkg_config speex speex/speex.h speex_decoder_init -lspeex enabled libtesseract && require_pkg_config tesseract tesseract/capi.h TessBaseAPICreate enabled libtheora && require libtheora theora/theoraenc.h th_info_init -ltheoraenc -ltheoradec -logg +enabled libturing && require_pkg_config libturing turing.h turing_version && + { check_cpp_condition turing.h "TURING_API_VERSION > 1" || + die "ERROR: libturing requires turing api +version 2 or greater."; } enabled libtwolame && require libtwolame twolame.h twolame_init -ltwolame && { check_lib libtwolame twolame.h twolame_encode_buffer_float32_interleaved -ltwolame || die "ERROR: libtwolame must be installed and version must be >= 0.3.10"; } diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 44acc95..0a11a6b 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -909,6 +909,7 @@ OBJS-$(CONFIG_LIBSHINE_ENCODER) += libshine.o OBJS-$(CONFIG_LIBSPEEX_DECODER) += libspeexdec.o OBJS-$(CONFIG_LIBSPEEX_ENCODER) += libspeexenc.o OBJS-$(CONFIG_LIBTHEORA_ENCODER) += libtheoraenc.o +OBJS-$(CONFIG_LIBTURING_ENCODER) += libturing.o OBJS-$(CONFIG_LIBTWOLAME_ENCODER) += libtwolame.o OBJS-$(CONFIG_LIBVO_AMRWBENC_ENCODER) += libvo-amrwbenc.o OBJS-$(CONFIG_LIBVORBIS_DECODER) += libvorbisdec.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 7fcc26f..c729b8d 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -620,6 +620,7 @@ static void register_all(void) REGISTER_ENCODER(LIBSHINE, libshine); REGISTER_ENCDEC (LIBSPEEX, libspeex); REGISTER_ENCODER(LIBTHEORA, libtheora); + REGISTER_ENCODER(LIBTURING, libturing); REGISTER_ENCODER(LIBTWOLAME, libtwolame); REGISTER_ENCODER(LIBVO_AMRWBENC, libvo_amrwbenc); REGISTER_ENCDEC (LIBVORBIS, libvorbis); diff --git a/libavcodec/libturing.c b/libavcodec/libturing.c new file mode 100755 index 0000000..c7b2311 --- /dev/null +++ b/libavcodec/libturing.c @@ -0,0 +1,313 @@ +/* + * libturing encoder + * + * Copyright (c) 2017 Turing Codec contributors + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, + * MA 02110-1301 USA + */ + +#include + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "avcodec.h" +#include "internal.h" + +#define MAX_OPTION_LENGTH 256 + +typedef struct libturingEncodeContext { + const AVClass *class; + turing_encoder *encoder; + const char *options; +} libturingEncodeContext; + +typedef struct optionContext { + char **argv; + char *options; + char *s; + int options_buffer_size; + int buffer_filled; + int options_added; +} optionContext; + +static av_cold int libturing_encode_close(AVCodecContext *avctx) { + libturingEncodeContext *ctx = avctx->priv_data; + turing_destroy_encoder(ctx->encoder); + return 0; +} + +static av_cold int add_option(const char *current_option, optionContext +*option_ctx) { + int option_length = strlen(current_option); + char *temp_ptr; + + + if (option_ctx->buffer_filled + option_length + 1 > option_ctx->options_buffer_size) { + if (!(option_ctx->options)) { + option_ctx->options = av_malloc(option_length + 1); + if (!(option_ctx->options)) { + return AVERROR(ENOMEM); + } + } else { + temp_ptr = av_realloc(option_ctx->options, option_ctx->options_buffer_size + option_length + 1); + if (!(temp_ptr)) { + return AVERROR(ENOMEM); + } + option_ctx->options = temp_ptr; + } + option_ctx->options_buffer_size += option_length + 1; + option_ctx->s = option_ctx->options + option_ctx->buffer_filled; + } + strcpy(option_ctx->s, current_option); + option_ctx->s += 1 + option_length; + option_ctx->options_added++; + option_ctx->buffer_filled += option_length + 1; + return 0; +} + +static av_cold int finalise_options(optionContext *option_ctx) { + int option_idx = 0; + if (option_ctx->options_added) { + char *p; + option_ctx->argv = av_malloc(option_ctx->options_added * sizeof(char*)); + if (!(option_ctx->argv)) { + return AVERROR(ENOMEM); + } + p = option_ctx->options; + for (option_idx=0; option_idxoptions_added; option_idx++) { + option_ctx->argv[option_idx] = p; + p += strlen(p) + 1; + } + } + return 0; +} + +static av_cold int libturing_encode_init(AVCodecContext *avctx) { + libturingEncodeContext *ctx = avctx->priv_data; + const int bit_depth = av_pix_fmt_desc_get(avctx->pix_fmt)->comp[0].depth; + int error_code = 0; + int i = 0; + + optionContext encoder_options = {0}; + turing_encoder_settings settings; + char option_string[MAX_OPTION_LENGTH]; + double frame_rate; + + frame_rate = (double)avctx->time_base.den / (avctx->time_base.num * + avctx->ticks_per_frame); + + encoder_options.buffer_filled = 0; + encoder_options.options_added = 0; + encoder_options.options_buffer_size = 0; + encoder_options.options = NULL; + encoder_options.s = encoder_options.options; + encoder_options.argv = NULL; + + if (error_code = add_option("turing", &encoder_options)) { + goto fail; + } + + if (error_code = add_option("--frames=0", &encoder_options)) { + goto fail; + } + + snprintf(option_string, MAX_OPTION_LENGTH, "--input-res=%dx%d", avctx->width, avctx->height); + if (error_code = add_option(option_string, &encoder_options)) { + goto fail; + } + + snprintf(option_string, MAX_OPTION_LENGTH, "--frame-rate=%f", frame_rate); + if (error_code = add_option(option_string, &encoder_options)) { + goto fail; + } + + snprintf(option_string, MAX_OPTION_LENGTH, "--bit-depth=%d", bit_depth); + if (error_code = add_option(option_string, &encoder_options)) { + goto fail; + } + + if (avctx->sample_aspect_ratio.num > 0 && avctx->sample_aspect_ratio.den > 0) { + int sar_num, sar_den; + + av_reduce(&sar_num, &sar_den, + avctx->sample_aspect_ratio.num, + avctx->sample_aspect_ratio.den, 65535); + snprintf(option_string, MAX_OPTION_LENGTH, "--sar=%d:%d", sar_num, sar_den); + if (error_code = add_option(option_string, &encoder_options)) { + goto fail; + } + } + + if (ctx->options) { + AVDictionary *dict = NULL; + AVDictionaryEntry *en = NULL; + + if (!av_dict_parse_string(&dict, ctx->options, "=", ":", 0)) { + while ((en = av_dict_get(dict, "", en, AV_DICT_IGNORE_SUFFIX))) { + int const illegal_option = av_match_name(en->key, "input-res,frame-rate,f,frames,sar,bit-depth,internal-bit-depth"); + if (illegal_option) { + av_log(avctx, AV_LOG_WARNING, "%s=%s ignored - this parameter is inferred from ffmpeg.\n", en->key, en->value); + } else { + if (turing_check_binary_option(en->key)) { + snprintf(option_string, MAX_OPTION_LENGTH, "--%s", en->key); + } else { + snprintf(option_string, MAX_OPTION_LENGTH, "--%s=%s", en->key, en->value); + } + if (error_code = add_option(option_string, &encoder_options)) { + goto fail; + } + } + } + av_dict_free(&dict); + } + } + + if (error_code = add_option("dummy-input-filename", &encoder_options)) { + goto fail; + } + + if (error_code = finalise_options(&encoder_options)) { + goto fail; + } + + settings.argv = (char const**)encoder_options.argv; + settings.argc = encoder_options.options_added; + + for (i = 0; i < settings.argc; i++) { + av_log(avctx, AV_LOG_VERBOSE, "arg %d: %s\n", i, settings.argv[i]); + } + + ctx->encoder = turing_create_encoder(settings); + + if (!ctx->encoder) { + av_log(avctx, AV_LOG_ERROR, "Failed to create libturing encoder.\n"); + error_code = AVERROR_INVALIDDATA; + goto fail; + } + + if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { + turing_bitstream const *bitstream; + bitstream = turing_encode_headers(ctx->encoder); + if (bitstream->size <= 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to encode headers.\n"); + turing_destroy_encoder(ctx->encoder); + error_code = AVERROR_INVALIDDATA; + goto fail; + } + + avctx->extradata_size = bitstream->size; + + avctx->extradata = av_mallocz(avctx->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE); + if (!avctx->extradata) { + av_log(avctx, AV_LOG_ERROR, "Failed to allocate HEVC extradata %d bytes\n", avctx->extradata_size); + turing_destroy_encoder(ctx->encoder); + error_code = AVERROR(ENOMEM); + goto fail; + } + + memcpy(avctx->extradata, bitstream->p, bitstream->size); + } + + av_freep(&encoder_options.argv); + av_freep(&encoder_options.options); + return 0; + +fail: + av_log(avctx, AV_LOG_ERROR, "Error while initialising the Turing codec.\n"); + av_freep(&encoder_options.argv); + av_freep(&encoder_options.options); + return error_code; +} + +static int libturing_encode_frame(AVCodecContext *avctx, AVPacket *pkt, +const AVFrame *pic, int *got_packet) { + libturingEncodeContext *ctx = avctx->priv_data; + turing_encoder_output const *output; + int ret = 0; + + if (pic) { + turing_picture picture; + + picture.image[0].p = pic->data[0]; + picture.image[1].p = pic->data[1]; + picture.image[2].p = pic->data[2]; + picture.image[0].stride = pic->linesize[0]; + picture.image[1].stride = pic->linesize[1]; + picture.image[2].stride = pic->linesize[2]; + picture.pts = pic->pts; + output = turing_encode_picture(ctx->encoder, &picture); + } else { + output = turing_encode_picture(ctx->encoder, 0); + } + + if (output->bitstream.size < 0) { + return AVERROR_EXTERNAL; + } + + if (!(output->bitstream.size)) { + return 0; + } + + ret = ff_alloc_packet2(avctx, pkt, output->bitstream.size, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n"); + return ret; + } + + memcpy(pkt->data, output->bitstream.p, output->bitstream.size); + + pkt->pts = output->pts; + pkt->dts = output->dts; + if (output->keyframe) { + pkt->flags |= AV_PKT_FLAG_KEY; + } + + *got_packet = 1; + return 0; +} + +static const AVOption options[] = { + { "turing-params", "configure additional turing encoder parameters", offsetof(libturingEncodeContext, options), AV_OPT_TYPE_STRING,{ .str = NULL }, 0, 0, AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM }, + { NULL } +}; + +static const AVClass class = { + .class_name = "libturing", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + +AVCodec ff_libturing_encoder = { + .name = "libturing", + .long_name = NULL_IF_CONFIG_SMALL("libturing HEVC"), + .type = AVMEDIA_TYPE_VIDEO, + .id = AV_CODEC_ID_HEVC, + .init = libturing_encode_init, + .encode2 = libturing_encode_frame, + .close = libturing_encode_close, + .priv_data_size = sizeof(libturingEncodeContext), + .priv_class = &class, + .capabilities = AV_CODEC_CAP_DELAY, + .pix_fmts = (const enum AVPixelFormat[]){AV_PIX_FMT_YUV420P10, AV_PIX_FMT_YUV420P, AV_PIX_FMT_NONE}, +}; -- 1.8.5.3