From patchwork Wed Jun 6 20:50:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Marton Balint X-Patchwork-Id: 9275 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:11c:0:0:0:0:0 with SMTP id c28-v6csp1297544jad; Wed, 6 Jun 2018 13:50:41 -0700 (PDT) X-Google-Smtp-Source: ADUXVKJjHLWCV9HBnPiofSkBXppb0PAls3J5aqlQ2LZ15ZOCkg19Yzo0ok2rEFRMEvOcpHGAgwtZ X-Received: by 2002:a1c:40d4:: with SMTP id n203-v6mr2655058wma.101.1528318241329; Wed, 06 Jun 2018 13:50:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1528318241; cv=none; d=google.com; s=arc-20160816; b=LIrxwnoK7I4/AN+UYGgnxAkOc1NZZ5P46YQ3vuuxXfBc/F6dA1JN2yqhv1fIuoYhzt bAnHeo8sTVOEvzGOKS1iWwH8CJuTmneDEcU+Taab6zUBfwpTKQ96djEy+GolprSlf6GJ LQIE08XgdiLUJHow9N4gP/gvT9yFu+94EsQIZfuCEw2zg2wkDQtYk6WcBXbUBedYaw0Q jdFzBdS8zV0Bhfl6+v1TKzfkyI/g4m1q5eN2+GrIIaPRZmTpYkTBmj6XGQMszpGMsjJQ 1P8Zz4BrvdGgFUdMg4aWx9hII7WnUPYxauEC5aCbWKFcp88r/K3Da42CPRwx7TJzDwe7 0YyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:user-agent:references:message-id:in-reply-to:to:from :date:delivered-to:arc-authentication-results; bh=1xpKi9jQGjV+AXChMRTvdxHO7Z9qe8Ng87S7BN2gzhU=; b=JR9ykkus9XID2gmzL3yt8k5KQUDi2nxkeQkTpEyrZ/p551GpAbpqjtUGkXFGVCCbuP 6epvpzHeBbgTg2kF9KQPdtWSrdYxTpFxBt3FlBlcATDOaSh7Emh/byWXmhIFLm4GjeNW Ds4FVFwrFEV63TGVzZChR5qWvFyWTv0/uuIwaJ1q5iOCVLsxKYelO+vPHtWvzRF09hEd 0VV/3an2U7k/hmK8nI07yDbuNmT7vgh9rXN5yKz5ts5YsNxiSvFb/g2DrbRT5+bsAI7J 8P/Xs2vSWchW9MXwKiKUZd/JcY91DMWYGyVEm9hiIesd9QnMEUI9+LI1mFIiBRPMI5dg VJuA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a144-v6si3727948wmd.181.2018.06.06.13.50.40; Wed, 06 Jun 2018 13:50:41 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5553768A772; Wed, 6 Jun 2018 23:49:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from iq.passwd.hu (iq.passwd.hu [217.27.212.140]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 810E568A450 for ; Wed, 6 Jun 2018 23:49:45 +0300 (EEST) Received: from localhost (localhost [127.0.0.1]) by iq.passwd.hu (Postfix) with ESMTP id A8361E10F3 for ; Wed, 6 Jun 2018 22:50:32 +0200 (CEST) X-Virus-Scanned: amavisd-new at passwd.hu Received: from iq.passwd.hu ([127.0.0.1]) by localhost (iq.passwd.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XX3zuU4zcXAM for ; Wed, 6 Jun 2018 22:50:31 +0200 (CEST) Received: from iq (iq [217.27.212.140]) by iq.passwd.hu (Postfix) with ESMTPS id 49D1CE0F8E for ; Wed, 6 Jun 2018 22:50:31 +0200 (CEST) Date: Wed, 6 Jun 2018 22:50:31 +0200 (CEST) From: Marton Balint X-X-Sender: cus@iq To: FFmpeg development discussions and patches In-Reply-To: <6ACCC5D4-051F-4929-BA76-8554D695F663@dericed.com> Message-ID: References: <8B97BCF4-76E4-4068-AB64-605925FAAA6A@pixsystem.com> <11234E62-8C40-41D3-95BA-5A6ABF2D537C@pixsystem.com> <75A0DB8F-E005-4413-95B9-8206670AFEE3@dericed.com> <6578833D-01ED-44E7-BE38-2374A2E9A724@dericed.com> <0E7AB8E8-C969-4445-94E7-6A7692B8A615@dericed.com> <39F13CE0-F7B1-4093-B70B-3289B5FE0F9D@pixsystem.com> <502C00B5-F9C9-43BD-9D9A-FAD146FB8CBD@dericed.com> <6ACCC5D4-051F-4929-BA76-8554D695F663@dericed.com> User-Agent: Alpine 2.20 (LSU 67 2015-01-07) MIME-Version: 1.0 Subject: Re: [FFmpeg-devel] [PATCH] Limited timecode support for lavd/decklink X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On Mon, 4 Jun 2018, Dave Rice wrote: > >>> >>> In my testing the timecode value set here has corrected been >>> associated with the first video frame (maintaining the >>> timecode-to-first-frame relationship as found on the source video >>> stream). Although only having first timecode value known is limiting, >>> I think this is still quite useful. This function also mirrors how >>> BlackMagic Media Express and Adobe Premiere handle capturing >>> video+timecode where only the first value is documented and all >>> subsequent values are presumed. >> >> Could you give me an example? (e.g. ffmpeg command line?) > > ./ffmpeg -timecode_format vitc2 -f decklink -draw_bars 0 -audio_input embedded -video_input sdi -format_code ntsc -channels 8 -raw_format yuv422p10 -i "UltraStudio 3D" -c:v v210 -c:a aac output.mov > > This worked for me to embed a QuickTime timecode track based upon the > timecode value of the first frame. If the input contained non-sequential > timecode values then the timecode track would not be accurate from that > point onward, but creating a timecode track based only upon the initial > value is what BlackMagic Media Express and Adobe Premiere are doing > anyhow. > Hmm, either the decklink drivers became better in hinding the first few NoSignal frames, or maybe that issue only affected to old models? (e.g. DeckLink SDI or DeckLink Duo 1). I did some test with a Mini Recorder, and even the first frame was useful, in this case the timecode was indeed correct. >>>> I'd rather see a new AVPacketSideData type which will contain the timecode as a string, so you can set it frame-by-frame. >>> >>> Using side data for timecode would be preferable, but the possibility that a patch for that may someday arrive shouldn’t completely block this more limited patch. >> >> I would like to make sure the code works reliably even for the limited use case and no race conditions are affectig the way it works. > > Feel welcome to suggest any testing. I’ll have access for testing again tomorrow. I reworked the patch a bit (see attached), and added per-frame timcode support into the PKT_STRINGS_METADATA packet side data, this way the drawtext filter can also be used to blend the timecode into the frames, which seems like a useful feature. Regards, Marton From bb69bdab51d46df6ad9ffe934e395dc959904051 Mon Sep 17 00:00:00 2001 From: Jon Morley Date: Thu, 31 May 2018 02:45:07 -0700 Subject: [PATCH] avdevice/decklink_dec: capture timecode to metadata when requested If the user provides a valid timecode_format look for timecode of that format in the capture and if found store it on the video avstream's metadata. Slightly modified by Marton Balint to capture per-frame timecode as well. Signed-off-by: Marton Balint --- doc/indevs.texi | 6 ++++++ libavdevice/decklink_common.h | 12 ++++++++++++ libavdevice/decklink_common_c.h | 1 + libavdevice/decklink_dec.cpp | 41 +++++++++++++++++++++++++++++++++++++++++ libavdevice/decklink_dec_c.c | 9 +++++++++ 5 files changed, 69 insertions(+) diff --git a/doc/indevs.texi b/doc/indevs.texi index 6951940a93..632d1e4743 100644 --- a/doc/indevs.texi +++ b/doc/indevs.texi @@ -326,6 +326,12 @@ Defaults to @samp{2}. Sets the decklink device duplex mode. Must be @samp{unset}, @samp{half} or @samp{full}. Defaults to @samp{unset}. +@item timecode_format +Timecode type to include in the frame and video stream metadata. Must be +@samp{none}, @samp{rp188vitc}, @samp{rp188vitc2}, @samp{rp188ltc}, +@samp{rp188any}, @samp{vitc}, @samp{vitc2}, or @samp{serial}. Defaults to +@samp{none} (not included). + @item video_input Sets the video input source. Must be @samp{unset}, @samp{sdi}, @samp{hdmi}, @samp{optical_sdi}, @samp{component}, @samp{composite} or @samp{s_video}. diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h index 8064abdcb9..96b001c2d8 100644 --- a/libavdevice/decklink_common.h +++ b/libavdevice/decklink_common.h @@ -93,6 +93,7 @@ struct decklink_ctx { BMDDisplayMode bmd_mode; BMDVideoConnection video_input; BMDAudioConnection audio_input; + BMDTimecodeFormat tc_format; int bmd_width; int bmd_height; int bmd_field_dominance; @@ -169,6 +170,17 @@ static const BMDVideoConnection decklink_video_connection_map[] = { bmdVideoConnectionSVideo, }; +static const BMDTimecodeFormat decklink_timecode_format_map[] = { + (BMDTimecodeFormat)0, + bmdTimecodeRP188VITC1, + bmdTimecodeRP188VITC2, + bmdTimecodeRP188LTC, + bmdTimecodeRP188Any, + bmdTimecodeVITC, + bmdTimecodeVITCField2, + bmdTimecodeSerial, +}; + HRESULT ff_decklink_get_display_name(IDeckLink *This, const char **displayName); int ff_decklink_set_configs(AVFormatContext *avctx, decklink_direction_t direction); int ff_decklink_set_format(AVFormatContext *avctx, int width, int height, int tb_num, int tb_den, enum AVFieldOrder field_order, decklink_direction_t direction = DIRECTION_OUT, int num = 0); diff --git a/libavdevice/decklink_common_c.h b/libavdevice/decklink_common_c.h index 08e9f9bbd5..32a5d70ee1 100644 --- a/libavdevice/decklink_common_c.h +++ b/libavdevice/decklink_common_c.h @@ -50,6 +50,7 @@ struct decklink_cctx { DecklinkPtsSource video_pts_source; int audio_input; int video_input; + int tc_format; int draw_bars; char *format_code; int raw_format; diff --git a/libavdevice/decklink_dec.cpp b/libavdevice/decklink_dec.cpp index 974ee1d94c..f32b1b44af 100644 --- a/libavdevice/decklink_dec.cpp +++ b/libavdevice/decklink_dec.cpp @@ -752,6 +752,36 @@ HRESULT decklink_input_callback::VideoInputFrameArrived( "- Frames dropped %u\n", ctx->frameCount, ++ctx->dropped); } no_video = 0; + + // Handle Timecode (if requested) + if (ctx->tc_format) + { + IDeckLinkTimecode *timecode; + if (videoFrame->GetTimecode(ctx->tc_format, &timecode) == S_OK) { + const char *tc = NULL; + DECKLINK_STR decklink_tc; + if (timecode->GetString(&decklink_tc) == S_OK) { + tc = DECKLINK_STRDUP(decklink_tc); + DECKLINK_FREE(decklink_tc); + } + timecode->Release(); + if (tc) { + AVDictionary* metadata_dict = NULL; + int metadata_len; + uint8_t* packed_metadata; + if (av_dict_set(&metadata_dict, "timecode", tc, AV_DICT_DONT_STRDUP_VAL) >= 0) { + packed_metadata = av_packet_pack_dictionary(metadata_dict, &metadata_len); + av_dict_free(&metadata_dict); + if (packed_metadata) { + if (av_packet_add_side_data(&pkt, AV_PKT_DATA_STRINGS_METADATA, packed_metadata, metadata_len) < 0) + av_freep(&packed_metadata); + } + } + } + } else { + av_log(avctx, AV_LOG_DEBUG, "Unable to find timecode.\n"); + } + } } pkt.pts = get_pkt_pts(videoFrame, audioFrame, wallclock, abs_wallclock, ctx->video_pts_source, ctx->video_st->time_base, &initial_video_pts, cctx->copyts); @@ -969,6 +999,8 @@ av_cold int ff_decklink_read_header(AVFormatContext *avctx) ctx->teletext_lines = cctx->teletext_lines; ctx->preroll = cctx->preroll; ctx->duplex_mode = cctx->duplex_mode; + if (cctx->tc_format > 0 && (unsigned int)cctx->tc_format < FF_ARRAY_ELEMS(decklink_timecode_format_map)) + ctx->tc_format = decklink_timecode_format_map[cctx->tc_format]; if (cctx->video_input > 0 && (unsigned int)cctx->video_input < FF_ARRAY_ELEMS(decklink_video_connection_map)) ctx->video_input = decklink_video_connection_map[cctx->video_input]; if (cctx->audio_input > 0 && (unsigned int)cctx->audio_input < FF_ARRAY_ELEMS(decklink_audio_connection_map)) @@ -1222,6 +1254,15 @@ int ff_decklink_read_packet(AVFormatContext *avctx, AVPacket *pkt) avpacket_queue_get(&ctx->queue, pkt, 1); + if (ctx->tc_format && !(av_dict_get(ctx->video_st->metadata, "timecode", NULL, 0))) { + int size; + const uint8_t *side_metadata = av_packet_get_side_data(pkt, AV_PKT_DATA_STRINGS_METADATA, &size); + if (side_metadata) { + if (av_packet_unpack_dictionary(side_metadata, size, &ctx->video_st->metadata) < 0) + av_log(avctx, AV_LOG_ERROR, "Unable to set timecode\n"); + } + } + return 0; } diff --git a/libavdevice/decklink_dec_c.c b/libavdevice/decklink_dec_c.c index 47018dc681..6ab3819375 100644 --- a/libavdevice/decklink_dec_c.c +++ b/libavdevice/decklink_dec_c.c @@ -48,6 +48,15 @@ static const AVOption options[] = { { "unset", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0}, 0, 0, DEC, "duplex_mode"}, { "half", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1}, 0, 0, DEC, "duplex_mode"}, { "full", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 2}, 0, 0, DEC, "duplex_mode"}, + { "timecode_format", "timecode format", OFFSET(tc_format), AV_OPT_TYPE_INT, { .i64 = 0}, 0, 7, DEC, "tc_format"}, + { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0}, 0, 0, DEC, "tc_format"}, + { "rp188vitc", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1}, 0, 0, DEC, "tc_format"}, + { "rp188vitc2", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 2}, 0, 0, DEC, "tc_format"}, + { "rp188ltc", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 3}, 0, 0, DEC, "tc_format"}, + { "rp188any", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 4}, 0, 0, DEC, "tc_format"}, + { "vitc", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 5}, 0, 0, DEC, "tc_format"}, + { "vitc2", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 6}, 0, 0, DEC, "tc_format"}, + { "serial", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 7}, 0, 0, DEC, "tc_format"}, { "video_input", "video input", OFFSET(video_input), AV_OPT_TYPE_INT, { .i64 = 0}, 0, 6, DEC, "video_input"}, { "unset", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0}, 0, 0, DEC, "video_input"}, { "sdi", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1}, 0, 0, DEC, "video_input"}, -- 2.16.3