[FFmpeg-devel] Added Turing codec to ffmpeg

Submitted by Matteo Naccari on Dec. 7, 2017, 11:08 a.m.

Details

Message ID 1512644927-28613-1-git-send-email-matteo.naccari@bbc.co.uk
State New
Headers show

Commit Message

Matteo Naccari Dec. 7, 2017, 11:08 a.m.
- This patch contains the changes required to interface the Turing codec
  (http://turingcodec.org/) to ffmpeg
---
 Changelog              |   1 +
 LICENSE.md             |   1 +
 configure              |   6 +
 libavcodec/Makefile    |   1 +
 libavcodec/allcodecs.c |   1 +
 libavcodec/libturing.c | 338 +++++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 348 insertions(+)
 create mode 100644 libavcodec/libturing.c

Comments

Josh de Kock Dec. 13, 2017, 10:39 p.m.
On Thu,  7 Dec 2017 11:08:47 +0000
Matteo Naccari <matteo.naccari@bbc.co.uk> wrote:

> - This patch contains the changes required to interface the Turing codec
> [...]

I still have an issue with this patch, while it would be nice to have another HEVC encoder available in FFmpeg for comparisons and whatnot, I feel that turingcodec isn't a mature enough encoder compared to what is already in the project.

Not to mention that development seems to have stalled (with only very minor fixes for several months). In FFmpeg it is very difficult to remove 'features', so if this patch were to be merged and then turingcodec to actually go stale we'd be left with dead code for longer than is ideal (forever).

My opinion is that you should just maintain this separately as an out-of-tree patch as it doesn't benefit us.
Jan Ekström Dec. 13, 2017, 11:46 p.m.
On Thu, Dec 14, 2017 at 12:39 AM, Josh de Kock <josh@itanimul.li> wrote:
> On Thu,  7 Dec 2017 11:08:47 +0000
> Matteo Naccari <matteo.naccari@bbc.co.uk> wrote:
>
>> - This patch contains the changes required to interface the Turing codec
>> [...]
>
> I still have an issue with this patch, while it would be nice to have another HEVC encoder available in FFmpeg for comparisons and whatnot, I feel that turingcodec isn't a mature enough encoder compared to what is already in the project.
>
> Not to mention that development seems to have stalled (with only very minor fixes for several months). In FFmpeg it is very difficult to remove 'features', so if this patch were to be merged and then turingcodec to actually go stale we'd be left with dead code for longer than is ideal (forever).
>
> My opinion is that you should just maintain this separately as an out-of-tree patch as it doesn't benefit us.
>

Hi,

After taking a look at the commit history of the last year+ or so,
issue tracker and pull request list over at
https://github.com/bbc/turingcodec I must say I find it somewhat hard
to disagree on a general level.

Creating an encoder is great work and I applaud people for that, but
unfortunately it - at the current point of time - looks like neither
from the licensing, performance or project activity point of view
merging a wrapper for this library is a good idea.


Best regards,
Jan
Matteo Naccari Dec. 14, 2017, 10:17 a.m.
> > I still have an issue with this patch, while it would be nice to have another

> HEVC encoder available in FFmpeg for comparisons and whatnot, I feel that

> turingcodec isn't a mature enough encoder compared to what is already in

> the project.

> >

> > Not to mention that development seems to have stalled (with only very

> minor fixes for several months). In FFmpeg it is very difficult to remove

> 'features', so if this patch were to be merged and then turingcodec to

> actually go stale we'd be left with dead code for longer than is ideal (forever).

> >

> > My opinion is that you should just maintain this separately as an out-of-

> tree patch as it doesn't benefit us.

> >

> 

> Hi,

> 

> After taking a look at the commit history of the last year+ or so, issue tracker

> and pull request list over at https://github.com/bbc/turingcodec I must say I

> find it somewhat hard to disagree on a general level.

> 

> Creating an encoder is great work and I applaud people for that, but

> unfortunately it - at the current point of time - looks like neither from the

> licensing, performance or project activity point of view merging a wrapper for

> this library is a good idea.


If stability of the project was your concern, you should have told us from the very beginning and save everyone's time. The project and its development is still active. We pushed additional fixes in November and we plan to add new features in the near future. We acknowledge that there is still a lot to do to get the encoder to the same level of maturity of  others both from the performance and the licensing/collaboration point of view. At the same time, we also trust you appreciate that being a small team of researchers (3) working part time on this project we are slow in replying to requests and progressing with the development. Indeed we thought having the codec interfaced to FFmpeg would have been a good opportunity to share our work and know-how on content distribution for broadcasting applications using HEVC and provide an alternative codec to ffmpeg. Clearly we were wrong.
We'll distribute the patch onto our website for those who are interested in having Turing in FFmpeg and stop pestering you with new requests.

---
Matteo Naccari, Ph.D.
Senior Technologist, BBC R&D
Block D, Centre House
56 Wood lane, W12 7SB, London (UK)
Phone: +44 (0)303 0409640

Patch hide | download patch | download mbox

diff --git a/Changelog b/Changelog
index a1a7557..9ed18b5 100644
--- a/Changelog
+++ b/Changelog
@@ -25,6 +25,7 @@  version <next>:
 - AMD AMF H.264 and HEVC encoders
 - video fillborders filter
 - video setrange filter
+- Added support to interface with the Turing codec
 
 
 version 3.4:
diff --git a/LICENSE.md b/LICENSE.md
index ba65b05..03787c0 100644
--- a/LICENSE.md
+++ b/LICENSE.md
@@ -84,6 +84,7 @@  The following libraries are under GPL:
 - frei0r
 - libcdio
 - librubberband
+- libturing
 - libvidstab
 - libx264
 - libx265
diff --git a/configure b/configure
index d053886..9be1fcb 100755
--- a/configure
+++ b/configure
@@ -260,6 +260,7 @@  External library support:
   --enable-libssh          enable SFTP protocol via libssh [no]
   --enable-libtesseract    enable Tesseract, needed for ocr filter [no]
   --enable-libtheora       enable Theora encoding via libtheora [no]
+  --enable-libturing       enable H.265/HEVC encoding via libturing [no]
   --enable-libtwolame      enable MP2 encoding via libtwolame [no]
   --enable-libv4l2         enable libv4l2/v4l-utils [no]
   --enable-libvidstab      enable video stabilization using vid.stab [no]
@@ -1551,6 +1552,7 @@  EXTERNAL_LIBRARY_GPL_LIST="
     frei0r
     libcdio
     librubberband
+    libturing
     libvidstab
     libx264
     libx265
@@ -2963,6 +2965,7 @@  libspeex_decoder_deps="libspeex"
 libspeex_encoder_deps="libspeex"
 libspeex_encoder_select="audio_frame_queue"
 libtheora_encoder_deps="libtheora"
+libturing_encoder_deps="libturing"
 libtwolame_encoder_deps="libtwolame"
 libvo_amrwbenc_encoder_deps="libvo_amrwbenc"
 libvorbis_decoder_deps="libvorbis"
@@ -5889,6 +5892,9 @@  enabled libssh            && require_pkg_config libssh libssh libssh/sftp.h sftp
 enabled libspeex          && require_pkg_config libspeex speex speex/speex.h speex_decoder_init
 enabled libtesseract      && require_pkg_config libtesseract tesseract tesseract/capi.h TessBaseAPICreate
 enabled libtheora         && require libtheora theora/theoraenc.h th_info_init -ltheoraenc -ltheoradec -logg
+enabled libturing         && require_pkg_config libturing libturing turing.h turing_version &&
+                             { check_cpp_condition turing.h "TURING_API_VERSION > 1" ||
+                               die "ERROR: libturing requires turing api version 2 or greater."; }
 enabled libtwolame        && require libtwolame twolame.h twolame_init -ltwolame &&
                              { check_lib libtwolame twolame.h twolame_encode_buffer_float32_interleaved -ltwolame ||
                                die "ERROR: libtwolame must be installed and version must be >= 0.3.10"; }
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index ab7893f..1c8d945 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -952,6 +952,7 @@  OBJS-$(CONFIG_LIBSHINE_ENCODER)           += libshine.o
 OBJS-$(CONFIG_LIBSPEEX_DECODER)           += libspeexdec.o
 OBJS-$(CONFIG_LIBSPEEX_ENCODER)           += libspeexenc.o
 OBJS-$(CONFIG_LIBTHEORA_ENCODER)          += libtheoraenc.o
+OBJS-$(CONFIG_LIBTURING_ENCODER)          += libturing.o
 OBJS-$(CONFIG_LIBTWOLAME_ENCODER)         += libtwolame.o
 OBJS-$(CONFIG_LIBVO_AMRWBENC_ENCODER)     += libvo-amrwbenc.o
 OBJS-$(CONFIG_LIBVORBIS_DECODER)          += libvorbisdec.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index ed1e7ab..b42630d 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -551,6 +551,7 @@  static void register_all(void)
     REGISTER_ENCODER(LIBSHINE,          libshine);
     REGISTER_ENCDEC (LIBSPEEX,          libspeex);
     REGISTER_ENCODER(LIBTHEORA,         libtheora);
+    REGISTER_ENCODER(LIBTURING,         libturing);
     REGISTER_ENCODER(LIBTWOLAME,        libtwolame);
     REGISTER_ENCODER(LIBVO_AMRWBENC,    libvo_amrwbenc);
     REGISTER_ENCDEC (LIBVORBIS,         libvorbis);
diff --git a/libavcodec/libturing.c b/libavcodec/libturing.c
new file mode 100644
index 0000000..ac64797
--- /dev/null
+++ b/libavcodec/libturing.c
@@ -0,0 +1,338 @@ 
+/*
+ * libturing encoder
+ *
+ * Copyright (c) 2017 Turing Codec contributors
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+ * MA 02110-1301 USA
+ */
+
+#if defined(_MSC_VER)
+#define TURING_API_IMPORTS 1
+#endif
+
+
+#include <turing.h>
+
+#include "libavutil/internal.h"
+#include "libavutil/common.h"
+#include "libavutil/avstring.h"
+#include "libavutil/opt.h"
+#include "libavutil/pixdesc.h"
+#include "avcodec.h"
+#include "internal.h"
+
+typedef struct libturingEncodeContext {
+    const AVClass *class;
+    turing_encoder *encoder;
+    const char *options;
+} libturingEncodeContext;
+
+typedef struct optionContext {
+    char **argv;
+    char *options;
+    char *s;
+    int options_buffer_size;
+    int buffer_filled;
+    int options_added;
+} optionContext;
+
+static av_cold int libturing_encode_close(AVCodecContext *avctx)
+{
+    libturingEncodeContext *ctx = avctx->priv_data;
+    turing_destroy_encoder(ctx->encoder);
+    return 0;
+}
+
+static av_cold int add_option(const char *current_option, optionContext *option_ctx)
+{
+    int option_length = strlen(current_option);
+
+    if (option_ctx->buffer_filled + option_length + 1 > option_ctx->options_buffer_size) {
+        option_ctx->options = av_realloc(option_ctx->options, option_ctx->options_buffer_size + option_length + 1);
+        if (!(option_ctx->options)) {
+            return AVERROR(ENOMEM);
+        }
+        option_ctx->options_buffer_size += option_length + 1;
+        option_ctx->s = option_ctx->options + option_ctx->buffer_filled;
+    }
+    av_strlcpy(option_ctx->s, current_option, (option_length + 1));
+    option_ctx->s += option_length + 1;
+    option_ctx->options_added++;
+    option_ctx->buffer_filled += option_length + 1;
+    return 0;
+}
+
+static av_cold int finalise_options(optionContext *option_ctx)
+{
+    int option_idx = 0;
+    if (option_ctx->options_added) {
+        char *p;
+        option_ctx->argv = av_malloc(option_ctx->options_added * sizeof(char*));
+        if (!(option_ctx->argv)) {
+            return AVERROR(ENOMEM);
+        }
+        p = option_ctx->options;
+        for (option_idx=0; option_idx<option_ctx->options_added; option_idx++) {
+            option_ctx->argv[option_idx] = p;
+            p += strlen(p) + 1;
+        }
+    }
+    return 0;
+}
+
+static av_cold int libturing_encode_init(AVCodecContext *avctx)
+{
+    libturingEncodeContext *ctx = avctx->priv_data;
+    const int bit_depth = av_pix_fmt_desc_get(avctx->pix_fmt)->comp[0].depth;
+    int error_code;
+    int i;
+
+    optionContext encoder_options = {0};
+    turing_encoder_settings settings;
+    char *option_string = NULL;
+    int current_option_length;
+    double frame_rate;
+
+    frame_rate = (double)avctx->time_base.den / (avctx->time_base.num * avctx->ticks_per_frame);
+    ctx->encoder = NULL;
+
+    encoder_options.buffer_filled = 0;
+    encoder_options.options_added = 0;
+    encoder_options.options_buffer_size = 0;
+    encoder_options.options = NULL;
+    encoder_options.s = encoder_options.options;
+    encoder_options.argv = NULL;
+
+    error_code = add_option("turing", &encoder_options);
+    if (error_code) {
+        goto fail;
+    }
+
+    error_code = add_option("--frames=0", &encoder_options);
+    if (error_code) {
+        goto fail;
+    }
+
+    current_option_length = snprintf(NULL, 0, "--input-res=%dx%d", avctx->width, avctx->height) + 1;
+    option_string = av_realloc(option_string, current_option_length);
+    snprintf(option_string, current_option_length, "--input-res=%dx%d", avctx->width, avctx->height);
+    error_code = add_option(option_string, &encoder_options);
+    if (error_code) {
+        goto fail;
+    }
+
+    current_option_length = snprintf(NULL, 0, "--frame-rate=%f", frame_rate) + 1;
+    option_string = av_realloc(option_string, current_option_length);
+    snprintf(option_string, current_option_length, "--frame-rate=%f", frame_rate);
+    error_code = add_option(option_string, &encoder_options);
+    if (error_code) {
+        goto fail;
+    }
+
+    current_option_length = snprintf(NULL, 0, "--bit-depth=%d", bit_depth) + 1;
+    option_string = av_realloc(option_string, current_option_length);
+    snprintf(option_string, current_option_length, "--bit-depth=%d", bit_depth);
+    error_code = add_option(option_string, &encoder_options);
+    if (error_code) {
+        goto fail;
+    }
+
+    if (avctx->sample_aspect_ratio.num > 0 && avctx->sample_aspect_ratio.den > 0) {
+        int sar_num, sar_den;
+
+        av_reduce(&sar_num, &sar_den,
+            avctx->sample_aspect_ratio.num,
+            avctx->sample_aspect_ratio.den, 65535);
+        current_option_length = snprintf(NULL, 0, "--sar=%d:%d", sar_num, sar_den) + 1;
+        option_string = av_realloc(option_string, current_option_length);
+        snprintf(option_string, current_option_length, "--sar=%d:%d", sar_num, sar_den);
+        error_code = add_option(option_string, &encoder_options);
+        if (error_code) {
+            goto fail;
+        }
+    }
+
+    if (ctx->options) {
+        AVDictionary *dict = NULL;
+        AVDictionaryEntry *en = NULL;
+
+        if (!av_dict_parse_string(&dict, ctx->options, "=", ":", 0)) {
+            while ((en = av_dict_get(dict, "", en, AV_DICT_IGNORE_SUFFIX))) {
+                int const illegal_option = av_match_name(en->key, "input-res,frame-rate,f,frames,sar,bit-depth,internal-bit-depth");
+                if (illegal_option) {
+                    av_log(avctx, AV_LOG_WARNING, "%s=%s ignored - this parameter is inferred from ffmpeg.\n", en->key, en->value);
+                } else {
+                    if (turing_check_binary_option(en->key)) {
+                        current_option_length = snprintf(NULL, 0, "--%s", en->key) + 1;
+                        option_string = av_realloc(option_string, current_option_length);
+                        snprintf(option_string, current_option_length, "--%s", en->key);
+                    } else {
+                        current_option_length = snprintf(NULL, 0, "--%s=%s", en->key, en->value) + 1;
+                        option_string = av_realloc(option_string, current_option_length);
+                        snprintf(option_string, current_option_length, "--%s=%s", en->key, en->value);
+                    }
+                    error_code = add_option(option_string, &encoder_options);
+                    if (error_code) {
+                        goto fail;
+                    }
+                }
+            }
+            av_dict_free(&dict);
+        }
+    }
+
+    error_code = add_option("dummy-input-filename", &encoder_options);
+    if (error_code) {
+        goto fail;
+    }
+
+    error_code = finalise_options(&encoder_options);
+    if (error_code) {
+        goto fail;
+    }
+
+    settings.argv = (char const**)encoder_options.argv;
+    settings.argc = encoder_options.options_added;
+
+    for (i = 0; i < settings.argc; i++) {
+        av_log(avctx, AV_LOG_VERBOSE, "arg %d: %s\n", i, settings.argv[i]);
+    }
+
+    ctx->encoder = turing_create_encoder(settings);
+
+    if (!ctx->encoder) {
+        av_log(avctx, AV_LOG_ERROR, "Failed to create libturing encoder.\n");
+        error_code = AVERROR_INVALIDDATA;
+        goto fail;
+    }
+
+    if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) {
+        turing_bitstream const *bitstream;
+        bitstream = turing_encode_headers(ctx->encoder);
+        if (bitstream->size <= 0) {
+            av_log(avctx, AV_LOG_ERROR, "Failed to encode headers.\n");
+            error_code = AVERROR_INVALIDDATA;
+            goto fail;
+        }
+
+        avctx->extradata_size = bitstream->size;
+
+        avctx->extradata = av_mallocz(avctx->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE);
+        if (!avctx->extradata) {
+            av_log(avctx, AV_LOG_ERROR, "Failed to allocate HEVC extradata %d bytes\n", avctx->extradata_size);
+            error_code = AVERROR(ENOMEM);
+            goto fail;
+        }
+
+        memcpy(avctx->extradata, bitstream->p, bitstream->size);
+    }
+
+    if (!(avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER)) {
+        error_code = add_option("--repeat-headers", &encoder_options);
+        if (error_code) {
+            goto fail;
+        }
+    }
+
+    av_freep(&encoder_options.argv);
+    av_freep(&encoder_options.options);
+    av_freep(&option_string);
+    return 0;
+
+fail:
+    av_freep(&encoder_options.argv);
+    av_freep(&encoder_options.options);
+    av_freep(&option_string);
+    if (ctx->encoder) {
+        turing_destroy_encoder(ctx->encoder);
+    }
+    return error_code;
+}
+
+static int libturing_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFrame *pic, int *got_packet)
+{
+    libturingEncodeContext *ctx = avctx->priv_data;
+    turing_encoder_output const *output;
+    int ret;
+
+    if (pic) {
+        turing_picture picture;
+
+        picture.image[0].p = pic->data[0];
+        picture.image[1].p = pic->data[1];
+        picture.image[2].p = pic->data[2];
+        picture.image[0].stride = pic->linesize[0];
+        picture.image[1].stride = pic->linesize[1];
+        picture.image[2].stride = pic->linesize[2];
+        picture.pts = pic->pts;
+        output = turing_encode_picture(ctx->encoder, &picture);
+    } else {
+        output = turing_encode_picture(ctx->encoder, NULL);
+    }
+
+    if (output->bitstream.size < 0) {
+        return AVERROR_EXTERNAL;
+    }
+
+    if (!(output->bitstream.size)) {
+        return 0;
+    }
+
+    ret = ff_alloc_packet2(avctx, pkt, output->bitstream.size, 0);
+    if (ret < 0) {
+        av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n");
+        return ret;
+    }
+
+    memcpy(pkt->data, output->bitstream.p, output->bitstream.size);
+
+    pkt->pts = output->pts;
+    pkt->dts = output->dts;
+    if (output->keyframe) {
+        pkt->flags |= AV_PKT_FLAG_KEY;
+    }
+
+    *got_packet = 1;
+    return 0;
+}
+
+static const AVOption options[] = {
+    { "turing-params", "additional turing encoder parameters", offsetof(libturingEncodeContext, options), AV_OPT_TYPE_STRING,{ .str = NULL }, 0, 0, AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM },
+    { NULL }
+};
+
+static const AVClass class = {
+    .class_name = "libturing",
+    .item_name  = av_default_item_name,
+    .option     = options,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+AVCodec ff_libturing_encoder = {
+    .name           = "libturing",
+    .long_name      = NULL_IF_CONFIG_SMALL("libturing HEVC"),
+    .type           = AVMEDIA_TYPE_VIDEO,
+    .id             = AV_CODEC_ID_HEVC,
+    .init           = libturing_encode_init,
+    .encode2        = libturing_encode_frame,
+    .close          = libturing_encode_close,
+    .priv_data_size = sizeof(libturingEncodeContext),
+    .priv_class     = &class,
+    .capabilities   = AV_CODEC_CAP_DELAY,
+    .pix_fmts       = (const enum AVPixelFormat[]){AV_PIX_FMT_YUV420P10, AV_PIX_FMT_YUV420P, AV_PIX_FMT_NONE},
+};