diff mbox

[FFmpeg-devel,v9] - Added Turing codec interface for ffmpeg

Message ID 59139428E3A17E4B867549526134D16612690C36@bgb01xud1009
State Superseded
Headers show

Commit Message

Saverio Blasi April 26, 2017, 7:55 a.m. UTC
Hello,

We have recently circulated this new version (see below) of our patch for integrating our HEVC Turing codec with FFmpeg. Assuming that there are no more requests for changes, we would like to understand what is the timeline for integration within the project.

Thanks a lot,

All the best,
----------------------------------------------------
Saverio Blasi, PhD
Senior Research Engineer 
BBC Research & Development
Centre House|56 Wood Lane|London|W12 7SB




-----Original Message-----
From: Saverio Blasi [mailto:saverio.blasi@bbc.co.uk] 
Sent: 24 April 2017 14:29
To: ffmpeg-devel@ffmpeg.org
Cc: Saverio Blasi <Saverio.Blasi@bbc.co.uk>
Subject: [PATCH v9] - Added Turing codec interface for ffmpeg

- This patch contains the changes to interface the Turing codec (http://turingcodec.org/) with ffmpeg. The patch was modified to address the comments in the review as follows:
  - Added a pkg-config file to list all dependencies required by libturing. This should address the issue pointed out by Hendrik Leppkes on Fri 18/11/2016
  - As per suggestions of wm4, two functions (add_option and finalise_options) have been created. The former appends new options while the latter sets up the argv array of pointers to char* accordingly. add_option re-allocates the buffer for options using av_realloc
  - Additionally, both these functions handle the errors in case the memory wasn't allocated correctly
  - malloc|free|realloc have been substituted with their corresponding av_{malloc|free|realloc} version
  - Check on bit-depth has been removed since the ffmpeg already casts the right pix_fmt and bit depth
  - pix_fmts is now set in ff_libturing_encoder as in h264dec.c.
  - Changed usage of av_free with av_freep and fixed calls to free arrays
  - Added brackets to all if and for statements
  - Avoid repetition of code to free arrays in case of failure to initialise the libturing encoder
  - Some fixes to address the review from wm4 and Mark Thompson received on Wed 08/02/2017
  - Fixed indentation
---
 LICENSE.md             |   1 +
 configure              |   6 +
 libavcodec/Makefile    |   1 +
 libavcodec/allcodecs.c |   1 +
 libavcodec/libturing.c | 314 +++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 323 insertions(+)
 create mode 100755 libavcodec/libturing.c

Comments

Michael Niedermayer April 26, 2017, 1:01 p.m. UTC | #1
On Wed, Apr 26, 2017 at 07:55:59AM +0000, Saverio Blasi wrote:
> Hello,
> 
> We have recently circulated this new version (see below) of our patch for integrating our HEVC Turing codec with FFmpeg. Assuming that there are no more requests for changes, we would like to understand what is the timeline for integration within the project.
> 
> Thanks a lot,

seems i cant automaticaly apply this locally

Applying: - Added Turing codec interface for ffmpeg
.git/rebase-apply/patch:56: trailing whitespace.
                             die "ERROR: libturing requires turing api
error: corrupt patch at line 63
error: could not build fake ancestor
Patch failed at 0001 - Added Turing codec interface for ffmpeg
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".


[...]
Saverio Blasi April 28, 2017, 3:18 p.m. UTC | #2
Hi Michael,

The patch applies nicely to my local ffmpeg repository. It is based on commit hash dd49eff93095110d2e878bbcc81b0062590d865f from the 23rd of April, so maybe something changed in the Configure file in the past few days?

Thanks,
Saverio

-----Original Message-----
From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf Of Michael Niedermayer
Sent: 26 April 2017 14:02
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Subject: Re: [FFmpeg-devel] [PATCH v9] - Added Turing codec interface for ffmpeg

On Wed, Apr 26, 2017 at 07:55:59AM +0000, Saverio Blasi wrote:
> Hello,
>
> We have recently circulated this new version (see below) of our patch for integrating our HEVC Turing codec with FFmpeg. Assuming that there are no more requests for changes, we would like to understand what is the timeline for integration within the project.
>
> Thanks a lot,

seems i cant automaticaly apply this locally

Applying: - Added Turing codec interface for ffmpeg
.git/rebase-apply/patch:56: trailing whitespace.
                             die "ERROR: libturing requires turing api
error: corrupt patch at line 63
error: could not build fake ancestor
Patch failed at 0001 - Added Turing codec interface for ffmpeg The copy of the patch that failed is found in: .git/rebase-apply/patch When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".


[...]
--
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Those who are best at talking, realize last or never when they are wrong.


-----------------------------
http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and
may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in
error, please delete it from your system.
Do not use, copy or disclose the
information in any way nor act in reliance on it and notify the sender
immediately.
Please note that the BBC monitors e-mails
sent or received.
Further communication will signify your consent to
this.
-----------------------------
Carl Eugen Hoyos April 30, 2017, 11:21 a.m. UTC | #3
2017-04-28 17:18 GMT+02:00 Saverio Blasi <Saverio.Blasi@bbc.co.uk>:

> The patch applies nicely to my local ffmpeg repository.

Please confirm that tools/patcheck does not report any issues
(trailing whitespace cannot be committed to our repository).

> This e-mail (and any attachments) is confidential

Please do not top-post here and please remove this
from your email footer, you cannot send a confidential
email to a public mailing list.

Carl Eugen
diff mbox

Patch

diff --git a/LICENSE.md b/LICENSE.md
index ba65b05..03787c0 100644
--- a/LICENSE.md
+++ b/LICENSE.md
@@ -84,6 +84,7 @@  The following libraries are under GPL:
 - frei0r
 - libcdio
 - librubberband
+- libturing
 - libvidstab
 - libx264
 - libx265
diff --git a/configure b/configure
index 487a4d7..6e5fab4 100755
--- a/configure
+++ b/configure
@@ -255,6 +255,7 @@  External library support:
   --enable-libssh          enable SFTP protocol via libssh [no]
   --enable-libtesseract    enable Tesseract, needed for ocr filter [no]
   --enable-libtheora       enable Theora encoding via libtheora [no]
+  --enable-libturing       enable H.265/HEVC encoding via libturing [no]
   --enable-libtwolame      enable MP2 encoding via libtwolame [no]
   --enable-libv4l2         enable libv4l2/v4l-utils [no]
   --enable-libvidstab      enable video stabilization using vid.stab [no]
@@ -1497,6 +1498,7 @@  EXTERNAL_LIBRARY_GPL_LIST="
     frei0r
     libcdio
     librubberband
+    libturing
     libvidstab
     libx264
     libx265
@@ -2866,6 +2868,7 @@  libspeex_decoder_deps="libspeex"
 libspeex_encoder_deps="libspeex"
 libspeex_encoder_select="audio_frame_queue"
 libtheora_encoder_deps="libtheora"
+libturing_encoder_deps="libturing"
 libtwolame_encoder_deps="libtwolame"
 libvo_amrwbenc_encoder_deps="libvo_amrwbenc"
 libvorbis_decoder_deps="libvorbis"
@@ -5810,6 +5813,9 @@  enabled libssh            && require_pkg_config libssh libssh/sftp.h sftp_init
 enabled libspeex          && require_pkg_config speex speex/speex.h speex_decoder_init -lspeex
 enabled libtesseract      && require_pkg_config tesseract tesseract/capi.h TessBaseAPICreate
 enabled libtheora         && require libtheora theora/theoraenc.h th_info_init -ltheoraenc -ltheoradec -logg
+enabled libturing         && require_pkg_config libturing turing.h turing_version &&
+                             { check_cpp_condition turing.h "TURING_API_VERSION > 1" ||
+                             die "ERROR: libturing requires turing api 
+version 2 or greater."; }
 enabled libtwolame        && require libtwolame twolame.h twolame_init -ltwolame &&
                              { check_lib twolame.h twolame_encode_buffer_float32_interleaved -ltwolame ||
                                die "ERROR: libtwolame must be installed and version must be >= 0.3.10"; } diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 3889ced..9230ac5 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -896,6 +896,7 @@  OBJS-$(CONFIG_LIBSHINE_ENCODER)           += libshine.o
 OBJS-$(CONFIG_LIBSPEEX_DECODER)           += libspeexdec.o
 OBJS-$(CONFIG_LIBSPEEX_ENCODER)           += libspeexenc.o
 OBJS-$(CONFIG_LIBTHEORA_ENCODER)          += libtheoraenc.o
+OBJS-$(CONFIG_LIBTURING_ENCODER)          += libturing.o
 OBJS-$(CONFIG_LIBTWOLAME_ENCODER)         += libtwolame.o
 OBJS-$(CONFIG_LIBVO_AMRWBENC_ENCODER)     += libvo-amrwbenc.o
 OBJS-$(CONFIG_LIBVORBIS_DECODER)          += libvorbisdec.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index b7d03ad..d3745d3 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -616,6 +616,7 @@  static void register_all(void)
     REGISTER_ENCODER(LIBSHINE,          libshine);
     REGISTER_ENCDEC (LIBSPEEX,          libspeex);
     REGISTER_ENCODER(LIBTHEORA,         libtheora);
+    REGISTER_ENCODER(LIBTURING,         libturing);
     REGISTER_ENCODER(LIBTWOLAME,        libtwolame);
     REGISTER_ENCODER(LIBVO_AMRWBENC,    libvo_amrwbenc);
     REGISTER_ENCDEC (LIBVORBIS,         libvorbis);
diff --git a/libavcodec/libturing.c b/libavcodec/libturing.c new file mode 100755 index 0000000..8073ddc
--- /dev/null
+++ b/libavcodec/libturing.c
@@ -0,0 +1,314 @@ 
+/*
+ * libturing encoder
+ *
+ * Copyright (c) 2017 Turing Codec contributors
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+ * MA 02110-1301 USA
+ */
+
+#include <turing.h>
+
+#include "libavutil/internal.h"
+#include "libavutil/common.h"
+#include "libavutil/avstring.h"
+#include "libavutil/opt.h"
+#include "libavutil/pixdesc.h"
+#include "avcodec.h"
+#include "internal.h"
+
+#define MAX_OPTION_LENGTH 256
+
+typedef struct libturingEncodeContext {
+    const AVClass *class;
+    turing_encoder *encoder;
+    const char *options;
+} libturingEncodeContext;
+
+typedef struct optionContext {
+    char **argv;
+    char *options;
+    char *s;
+    int options_buffer_size;
+    int buffer_filled;
+    int options_added;
+} optionContext;
+
+static av_cold int libturing_encode_close(AVCodecContext *avctx) {
+    libturingEncodeContext *ctx = avctx->priv_data;
+    turing_destroy_encoder(ctx->encoder);
+    return 0;
+}
+
+static av_cold int add_option(const char *current_option, optionContext 
+*option_ctx) {
+    int option_length = strlen(current_option);
+    char *temp_ptr;
+
+
+    if (option_ctx->buffer_filled + option_length + 1 > option_ctx->options_buffer_size) {
+        if(option_ctx->options == NULL) {
+            option_ctx->options = av_malloc(option_length + 1);
+            if(option_ctx->options == NULL) {
+                return AVERROR(ENOMEM);
+            }
+        } else {
+            temp_ptr = av_realloc(option_ctx->options, option_ctx->options_buffer_size + option_length + 1);
+            if (temp_ptr == NULL) {
+                return AVERROR(ENOMEM);
+            }
+            option_ctx->options = temp_ptr;
+        }
+        option_ctx->options_buffer_size += option_length + 1;
+        option_ctx->s = option_ctx->options + option_ctx->buffer_filled;
+    }
+    strcpy(option_ctx->s, current_option);
+    option_ctx->s += 1 + option_length;
+    option_ctx->options_added++;
+    option_ctx->buffer_filled += option_length + 1;
+    return 0;
+}
+
+static av_cold int finalise_options(optionContext *option_ctx) {
+    if (option_ctx->options_added) {
+        char *p;
+        option_ctx->argv = av_malloc(option_ctx->options_added * sizeof(char*));
+        if (option_ctx->argv == NULL) {
+            return AVERROR(ENOMEM);
+        }
+        p = option_ctx->options;
+        for(int option_idx=0; option_idx<option_ctx->options_added; option_idx++) {
+            option_ctx->argv[option_idx] = p;
+            p += strlen(p) + 1;
+        }
+    }
+    return 0;
+}
+
+static av_cold int libturing_encode_init(AVCodecContext *avctx) {
+    libturingEncodeContext *ctx = avctx->priv_data;
+    const int bit_depth = av_pix_fmt_desc_get(avctx->pix_fmt)->comp[0].depth;
+    int error_code = 0;
+    int i = 0;
+
+    optionContext encoder_options = {0};
+    turing_encoder_settings settings;
+    char option_string[MAX_OPTION_LENGTH];
+    double frame_rate;
+
+    frame_rate = (double)avctx->time_base.den / (avctx->time_base.num * 
+ avctx->ticks_per_frame);
+
+    encoder_options.buffer_filled = 0;
+    encoder_options.options_added = 0;
+    encoder_options.options_buffer_size = 0;
+    encoder_options.options = NULL;
+    encoder_options.s = encoder_options.options;
+    encoder_options.argv = NULL;
+
+    // Add baseline command line options
+    if ((error_code = add_option("turing", &encoder_options)) > 0) {
+        goto fail;
+    }
+
+    if ((error_code = add_option("--frames=0", &encoder_options)) > 0) {
+        goto fail;
+    }
+
+    snprintf(option_string, MAX_OPTION_LENGTH, "--input-res=%dx%d", avctx->width, avctx->height);
+    if ((error_code = add_option(option_string, &encoder_options)) > 0) {
+        goto fail;
+    }
+
+    snprintf(option_string, MAX_OPTION_LENGTH, "--frame-rate=%f", frame_rate);
+    if ((error_code = add_option(option_string, &encoder_options)) > 0) {
+        goto fail;
+    }
+
+    snprintf(option_string, MAX_OPTION_LENGTH, "--bit-depth=%d", bit_depth);
+    if ((error_code = add_option(option_string, &encoder_options)) > 0) {
+        goto fail;
+    }
+
+    if (avctx->sample_aspect_ratio.num > 0 && avctx->sample_aspect_ratio.den > 0) {
+        int sar_num, sar_den;
+
+        av_reduce(&sar_num, &sar_den,
+            avctx->sample_aspect_ratio.num,
+            avctx->sample_aspect_ratio.den, 65535);
+        snprintf(option_string, MAX_OPTION_LENGTH, "--sar=%d:%d", sar_num, sar_den);
+        if ((error_code = add_option(option_string, &encoder_options)) > 0) {
+            goto fail;
+        }
+    }
+
+    // Parse additional command line options
+    if (ctx->options) {
+        AVDictionary *dict = NULL;
+        AVDictionaryEntry *en = NULL;
+
+        if (!av_dict_parse_string(&dict, ctx->options, "=", ":", 0)) {
+            while ((en = av_dict_get(dict, "", en, AV_DICT_IGNORE_SUFFIX))) {
+                int const illegal_option = av_match_name(en->key, "input-res,frame-rate,f,frames,sar,bit-depth,internal-bit-depth");
+                if (illegal_option) {
+                    av_log(avctx, AV_LOG_WARNING, "%s=%s ignored - this parameter is inferred from ffmpeg.\n", en->key, en->value);
+                } else {
+                    if (turing_check_binary_option(en->key)) {
+                        snprintf(option_string, MAX_OPTION_LENGTH, "--%s", en->key);
+                    } else {
+                        snprintf(option_string, MAX_OPTION_LENGTH, "--%s=%s", en->key, en->value);
+                    }
+                    if ((error_code = add_option(option_string, &encoder_options)) > 0) {
+                        goto fail;
+                    }
+                }
+            }
+            av_dict_free(&dict);
+        }
+    }
+
+    if ((error_code = add_option("dummy-input-filename", &encoder_options)) > 0) {
+        goto fail;
+    }
+
+    if ((error_code = finalise_options(&encoder_options)) > 0) {
+        goto fail;
+    }
+
+    settings.argv = (char const**)encoder_options.argv;
+    settings.argc = encoder_options.options_added;
+
+    for (i = 0; i < settings.argc; i++) {
+        av_log(avctx, AV_LOG_VERBOSE, "arg %d: %s\n", i, settings.argv[i]);
+    }
+
+    ctx->encoder = turing_create_encoder(settings);
+
+    if (!ctx->encoder) {
+        av_log(avctx, AV_LOG_ERROR, "Failed to create libturing encoder.\n");
+        error_code = AVERROR_INVALIDDATA;
+        goto fail;
+    }
+
+    if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) {
+        turing_bitstream const *bitstream;
+        bitstream = turing_encode_headers(ctx->encoder);
+        if (bitstream->size <= 0) {
+            av_log(avctx, AV_LOG_ERROR, "Failed to encode headers.\n");
+            turing_destroy_encoder(ctx->encoder);
+            error_code = AVERROR_INVALIDDATA;
+            goto fail;
+        }
+
+        avctx->extradata_size = bitstream->size;
+
+        avctx->extradata = av_mallocz(avctx->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE);
+        if (!avctx->extradata) {
+            av_log(avctx, AV_LOG_ERROR, "Failed to allocate HEVC extradata %d bytes\n", avctx->extradata_size);
+            turing_destroy_encoder(ctx->encoder);
+            error_code =  AVERROR(ENOMEM);
+            goto fail;
+        }
+
+        memcpy(avctx->extradata, bitstream->p, bitstream->size);
+    }
+
+    av_freep(&encoder_options.argv);
+    av_freep(&encoder_options.options);
+    return 0;
+
+fail:
+    av_log(avctx, AV_LOG_ERROR, "Error while initialising the Turing codec.\n");
+    av_freep(&encoder_options.argv);
+    av_freep(&encoder_options.options);
+    return error_code;
+}
+
+static int libturing_encode_frame(AVCodecContext *avctx, AVPacket *pkt, 
+const AVFrame *pic, int *got_packet) {
+    libturingEncodeContext *ctx = avctx->priv_data;
+    turing_encoder_output const *output;
+    int ret = 0;
+
+    if (pic) {
+        turing_picture picture;
+
+        picture.image[0].p = pic->data[0];
+        picture.image[1].p = pic->data[1];
+        picture.image[2].p = pic->data[2];
+        picture.image[0].stride = pic->linesize[0];
+        picture.image[1].stride = pic->linesize[1];
+        picture.image[2].stride = pic->linesize[2];
+        picture.pts = pic->pts;
+        output = turing_encode_picture(ctx->encoder, &picture);
+    } else {
+        output = turing_encode_picture(ctx->encoder, 0);
+    }
+
+    if (output->bitstream.size < 0) {
+        return AVERROR_EXTERNAL;
+    }
+
+    if (output->bitstream.size ==0) {
+        return 0;
+    }
+
+    ret = ff_alloc_packet2(avctx, pkt, output->bitstream.size, 0);
+    if (ret < 0) {
+        av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n");
+        return ret;
+    }
+
+    memcpy(pkt->data, output->bitstream.p, output->bitstream.size);
+
+    pkt->pts = output->pts;
+    pkt->dts = output->dts;
+    if (output->keyframe) {
+        pkt->flags |= AV_PKT_FLAG_KEY;
+    }
+
+    *got_packet = 1;
+    return 0;
+}
+
+static const AVOption options[] = {
+    { "turing-params", "configure additional turing encoder parameters", offsetof(libturingEncodeContext, options), AV_OPT_TYPE_STRING,{ .str = NULL }, 0, 0, AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM },
+    { NULL }
+};
+
+static const AVClass class = {
+    .class_name = "libturing",
+    .item_name  = av_default_item_name,
+    .option     = options,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+AVCodec ff_libturing_encoder = {
+    .name           = "libturing",
+    .long_name      = NULL_IF_CONFIG_SMALL("libturing HEVC"),
+    .type           = AVMEDIA_TYPE_VIDEO,
+    .id             = AV_CODEC_ID_HEVC,
+    .init           = libturing_encode_init,
+    .encode2        = libturing_encode_frame,
+    .close          = libturing_encode_close,
+    .priv_data_size = sizeof(libturingEncodeContext),
+    .priv_class     = &class,
+    .capabilities   = AV_CODEC_CAP_DELAY,
+    .pix_fmts       = (const enum AVPixelFormat[]){AV_PIX_FMT_YUV420P10, AV_PIX_FMT_YUV420P, AV_PIX_FMT_NONE},
+};
--
1.8.5.3