From patchwork Fri Oct 6 16:56:15 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 5443 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.90 with SMTP id m26csp162142jah; Fri, 6 Oct 2017 09:56:37 -0700 (PDT) X-Google-Smtp-Source: AOwi7QCI/IBzanGJFazRJO9fkW8blJwuDC44RGAyM87MTkMZVDVmqblA3jqbjKGjzYdy2T1HaFnv X-Received: by 10.28.139.72 with SMTP id n69mr2538872wmd.29.1507308997333; Fri, 06 Oct 2017 09:56:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1507308997; cv=none; d=google.com; s=arc-20160816; b=uFUrJgIAsKfOZPYQeYtSjB/5VAW3vjKzxejyLC5wq4Fxer4AmlDHAC17KNYFNvF1Am D2B2fzmoicea6jqJKulX0CkSvROGNQy9SrpBTPfz73cCch/BXk30bRts2Nhb7I1aCaGV wjfKrwapdxnCG9y/DPenxIvZ1DWGRQU1GiLIlB+LVOzmFWT/GC2tzHGFU6sZEZAj/JvY 4q5BSSCHEROQiqmKrkTwZocf8IGpDDIN1LlgSDNSoiY2xY1vSR7g910+cCj6SxaZVpi0 Gj7q8pb3PCtOuSafBYsrj6wyMQEmh+qT8w8McQJQTVnX8aOHfOmAuMrGJ/VgJAIXPtUE BAzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to :arc-authentication-results; bh=n0JMMHBLeWImEgWpHpu1V1c+ehXu7Ry1ZV0caC74Wo4=; b=Ae6GzA1lhJeQLfCPKAg1rpD8jPMTPW3iQuqJnAMT5ylcA+iJ9RUDovSCBf/2awZdj0 rst6eQoJ4Xso/BoOjQgIY8aUPY3rokFnCMFc7JsCZTyGMg0dZEfrCj701rUZbhCCuLIN GnSbvSrEDKIRYJPWDCAukINznSJOrHAp2dSmKObzuZYNl1tSdXGy9700zIK9M5qdvw91 h9Zi7k9bPp6bZTUT0IYQMw4uQMvRAii55c3XjXnG9POWygoANpHtndzNspZjcekv/lYr /m9DFGm5Mn5GSbXQgj7nnvxjCiz18mV0nkx8NIkpxBarCg8i6Swj+RzbdSHuAoHApk4t +T0Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 74si1588020wrm.334.2017.10.06.09.56.36; Fri, 06 Oct 2017 09:56:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8CCE968A3C0; Fri, 6 Oct 2017 19:56:32 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from was-smtp1.livetimenet.net (50-206-97-56-static.hfc.comcastbusiness.net [50.206.97.56]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E45E268A2A3 for ; Fri, 6 Oct 2017 19:56:25 +0300 (EEST) Received: by was-smtp1.livetimenet.net with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.84_2) (envelope-from ) id 1e0VvD-0000LY-RF; Fri, 06 Oct 2017 12:56:25 -0400 From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 6 Oct 2017 12:56:15 -0400 Message-Id: <20171006165615.39580-1-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.13.2 X-Spam-Score: -1.9 (-) Subject: [FFmpeg-devel] [PATCH RFC] libavdevice/decklink: Add support for EIA-708 output over SDI X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: jgreen@ltnglobal.com, Devin Heitmueller , Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Devin Heitmueller Hook in libklvanc and use it for output of EIA-708 captions over SDI. The bulk of this patch is just general support for ancillary data for the Decklink SDI module - the real work for construction of the EIA-708 CDP and VANC line construction is done by libklvanc. Libklvanc can be found at: https://github.com/stoth68000/libklvanc Signed-off-by: Devin Heitmueller --- configure | 3 ++ libavcodec/v210enc.c | 8 +++ libavdevice/decklink_common.h | 1 + libavdevice/decklink_enc.cpp | 113 +++++++++++++++++++++++++++++++++++++++--- 4 files changed, 119 insertions(+), 6 deletions(-) diff --git a/configure b/configure index 391c141e7a..18647896b1 100755 --- a/configure +++ b/configure @@ -238,6 +238,7 @@ External library support: --enable-libgsm enable GSM de/encoding via libgsm [no] --enable-libiec61883 enable iec61883 via libiec61883 [no] --enable-libilbc enable iLBC de/encoding via libilbc [no] + --enable-libklvanc enable Kernel Labs VANC processing [no] --enable-libkvazaar enable HEVC encoding via libkvazaar [no] --enable-libmodplug enable ModPlug via libmodplug [no] --enable-libmp3lame enable MP3 encoding via libmp3lame [no] @@ -1603,6 +1604,7 @@ EXTERNAL_LIBRARY_LIST=" libgsm libiec61883 libilbc + libklvanc libkvazaar libmodplug libmp3lame @@ -6027,6 +6029,7 @@ enabled libx264 && { use_pkg_config libx264 x264 "stdint.h x264.h" x26 enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get && require_cpp_condition x265.h "X265_BUILD >= 68" enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode -lxavs +enabled libklvanc && require libklvanc libklvanc/vanc.h vanc_context_create -lklvanc enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore enabled libzimg && require_pkg_config libzimg "zimg >= 2.3.0" zimg.h zimg_get_api_version enabled libzmq && require_pkg_config libzmq libzmq zmq.h zmq_ctx_new diff --git a/libavcodec/v210enc.c b/libavcodec/v210enc.c index a6afbbfc41..44cc3c5c81 100644 --- a/libavcodec/v210enc.c +++ b/libavcodec/v210enc.c @@ -123,6 +123,7 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, int aligned_width = ((avctx->width + 47) / 48) * 48; int stride = aligned_width * 8 / 3; int line_padding = stride - ((avctx->width * 8 + 11) / 12) * 4; + AVFrameSideData *side_data = NULL; int h, w, ret; uint8_t *dst; @@ -233,6 +234,13 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, } } + side_data = av_frame_get_side_data(pic, AV_FRAME_DATA_A53_CC); + if (side_data && side_data->size) { + uint8_t* buf = av_packet_new_side_data(pkt, AV_PKT_DATA_A53_CC, side_data->size); + if (buf) + memcpy(buf, side_data->data, side_data->size); + } + pkt->flags |= AV_PKT_FLAG_KEY; *got_packet = 1; return 0; diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h index 6b2525fb53..285a244000 100644 --- a/libavdevice/decklink_common.h +++ b/libavdevice/decklink_common.h @@ -78,6 +78,7 @@ struct decklink_ctx { AVStream *audio_st; AVStream *video_st; AVStream *teletext_st; + uint16_t cdp_sequence_num; /* Options */ int list_devices; diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp index 81df563b3b..3049e936a9 100644 --- a/libavdevice/decklink_enc.cpp +++ b/libavdevice/decklink_enc.cpp @@ -38,16 +38,20 @@ extern "C" { #include "decklink_common.h" #include "decklink_enc.h" - +#if CONFIG_LIBKLVANC +#include "libklvanc/vanc.h" +#include "libklvanc/vanc-lines.h" +#include "libklvanc/pixels.h" +#endif /* DeckLink callback class declaration */ class decklink_frame : public IDeckLinkVideoFrame { public: decklink_frame(struct decklink_ctx *ctx, AVFrame *avframe, AVCodecID codec_id, int height, int width) : - _ctx(ctx), _avframe(avframe), _avpacket(NULL), _codec_id(codec_id), _height(height), _width(width), _refs(1) { } + _ctx(ctx), _avframe(avframe), _avpacket(NULL), _codec_id(codec_id), _ancillary(NULL), _height(height), _width(width), _refs(1) { } decklink_frame(struct decklink_ctx *ctx, AVPacket *avpacket, AVCodecID codec_id, int height, int width) : - _ctx(ctx), _avframe(NULL), _avpacket(avpacket), _codec_id(codec_id), _height(height), _width(width), _refs(1) { } + _ctx(ctx), _avframe(NULL), _avpacket(avpacket), _codec_id(codec_id), _ancillary(NULL), _height(height), _width(width), _refs(1) { } virtual long STDMETHODCALLTYPE GetWidth (void) { return _width; } virtual long STDMETHODCALLTYPE GetHeight (void) { return _height; } @@ -87,8 +91,13 @@ public: } virtual HRESULT STDMETHODCALLTYPE GetTimecode (BMDTimecodeFormat format, IDeckLinkTimecode **timecode) { return S_FALSE; } - virtual HRESULT STDMETHODCALLTYPE GetAncillaryData(IDeckLinkVideoFrameAncillary **ancillary) { return S_FALSE; } - + virtual HRESULT STDMETHODCALLTYPE GetAncillaryData(IDeckLinkVideoFrameAncillary **ancillary) + { + *ancillary = _ancillary; + return _ancillary ? S_OK : S_FALSE; + } + virtual HRESULT STDMETHODCALLTYPE SetAncillaryData(IDeckLinkVideoFrameAncillary + *ancillary) { _ancillary = ancillary; return S_OK; } virtual HRESULT STDMETHODCALLTYPE QueryInterface(REFIID iid, LPVOID *ppv) { return E_NOINTERFACE; } virtual ULONG STDMETHODCALLTYPE AddRef(void) { return ++_refs; } virtual ULONG STDMETHODCALLTYPE Release(void) @@ -106,6 +115,7 @@ public: AVFrame *_avframe; AVPacket *_avpacket; AVCodecID _codec_id; + IDeckLinkVideoFrameAncillary *_ancillary; int _height; int _width; @@ -169,7 +179,7 @@ static int decklink_setup_video(AVFormatContext *avctx, AVStream *st) return -1; } if (ctx->dlo->EnableVideoOutput(ctx->bmd_mode, - bmdVideoOutputFlagDefault) != S_OK) { + bmdVideoOutputVANC) != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not enable video output!\n"); return -1; } @@ -265,6 +275,93 @@ av_cold int ff_decklink_write_trailer(AVFormatContext *avctx) return 0; } +#if CONFIG_LIBKLVANC +static int decklink_construct_vanc(struct decklink_ctx *ctx, AVPacket *pkt, + decklink_frame *frame) +{ + struct vanc_line_set_s vanc_lines; + memset(&vanc_lines, 0, sizeof(vanc_lines)); + + int size; + const uint8_t *data = av_packet_get_side_data(pkt, AV_PKT_DATA_A53_CC, &size); + if (data) { + struct packet_eia_708b_s *pkt; + uint16_t *cdp; + uint16_t len; + uint8_t cc_count = size / 3; + + klvanc_create_eia708_cdp(&pkt); + klvanc_set_framerate_EIA_708B(pkt, ctx->bmd_tb_num, ctx->bmd_tb_den); + + /* CC data */ + pkt->header.ccdata_present = 1; + pkt->ccdata.cc_count = cc_count; + for (size_t i = 0; i < cc_count; i++) { + if (data [3*i] & 0x40) + pkt->ccdata.cc[i].cc_valid = 1; + pkt->ccdata.cc[i].cc_type = data[3*i] & 0x03; + pkt->ccdata.cc[i].cc_data[0] = data[3*i+1]; + pkt->ccdata.cc[i].cc_data[1] = data[3*i+2]; + } + + klvanc_finalize_EIA_708B(pkt, ctx->cdp_sequence_num++); + convert_EIA_708B_to_words(pkt, &cdp, &len); + klvanc_destroy_eia708_cdp(pkt); + + vanc_line_insert(&vanc_lines, cdp, len, 11, 0); + } + + IDeckLinkVideoFrameAncillary *vanc; + int result = ctx->dlo->CreateAncillaryData(bmdFormat10BitYUV, &vanc); + if (result != S_OK) { + fprintf(stderr, "Failed to create vanc\n"); + return -1; + } + + /* Now that we've got all the VANC lines in a nice orderly manner, generate the + final VANC sections for the Decklink output */ + for (int i = 0; i < vanc_lines.num_lines; i++) { + struct vanc_line_s *line = vanc_lines.lines[i]; + uint16_t *out_line; + int real_line; + int out_len; + void *buf; + + if (line == NULL) + break; + + real_line = line->line_number; +#if 0 + /* FIXME: include hack for certain Decklink cards which mis-represent + line numbers for pSF frames */ + if (decklink_sys->b_psf_interlaced) + real_line = Calculate1080psfVancLine(line->line_number); +#endif + result = vanc->GetBufferForVerticalBlankingLine(real_line, &buf); + if (result != S_OK) { + fprintf(stderr, "Failed to get VANC line %d: %d", real_line, result); + vanc_line_free(line); + continue; + } + + /* Generate the full line taking into account all VANC packets on that line */ + generate_vanc_line(line, &out_line, &out_len, ctx->bmd_width); + + /* Repack the 16-bit ints into 10-bit, and push into final buffer */ + klvanc_y10_to_v210(out_line, (uint8_t *) buf, out_len); + free(out_line); + vanc_line_free(line); + } + + result = frame->SetAncillaryData(vanc); + if (result != S_OK) { + fprintf(stderr, "Failed to set vanc: %d", result); + return AVERROR(EIO); + } + return 0; +} +#endif + static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt) { struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; @@ -299,6 +396,10 @@ static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt) } frame = new decklink_frame(ctx, avpacket, st->codecpar->codec_id, ctx->bmd_height, ctx->bmd_width); + +#if CONFIG_LIBKLVANC + decklink_construct_vanc(ctx, pkt, frame); +#endif } if (!frame) {