From patchwork Wed Jul 26 07:39:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maksym Veremeyenko X-Patchwork-Id: 4462 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.1.76 with SMTP id 73csp469063vsb; Wed, 26 Jul 2017 00:39:30 -0700 (PDT) X-Received: by 10.28.226.137 with SMTP id z131mr41819wmg.151.1501054770080; Wed, 26 Jul 2017 00:39:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501054770; cv=none; d=google.com; s=arc-20160816; b=KykumCx9LxApownb5BnJsHvNt6ZetBfXUmMQvrjirCon1t/boOI8YGVg8YL8l8vEGq KhGsiU1lkArdOT5PogUY0REIJq904cn5DZKsKWSXac6u1M2u9iMzHzq9pQSw4KSsGmUl f4FRlE3fV21jpE4cJTmB7Ytmby/nlPWfL0GotDeeJpZ6HCWcMBNTSRUscR6DnCmTNl0Q 1oWDtNOmyYJ7yKDQYetrsoZOKvzJwBOfEPHjIvaw28OBnpBhEy3EGz99mKKu8xRa873X 8Ju40O9siI9jQ+MXADouPyv/q1VeZ/LZLI6bLQW2wfyYPxCcNsWAJQ4aNwvieRhmEd2P UlVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :in-reply-to:mime-version:user-agent:date:message-id:from:references :to:delivered-to:arc-authentication-results; bh=pFGXopYnNp9hzSupbR4+nXPFIWlG+GuJhIzbNoBYxE4=; b=zYZog7bUa0tkeib5Qp9yo5MPKMhsXSsE9plXoGJvAhB+w5B54HcX8tLQL2OkJtwk5t Rue/4BKddqJwknZ080Ho2ru/pqb8eVMscJz8XzrkpVIix5rSu27gTDIhlkDUc/lShWF8 +ieA6yzckKg7zEd7fjIAXk9FOkwvmECUtNJp9aOuxrbnYIsE/1UVumfm4nARk1T4XAg4 P9vf0C2dDcL3RP/7C6i6L0Byl458EfZ6EQpiVXLosHq9ZS9exYIIa7TzMoBkm1q7ArNY ZH3QJJqf71EKJRPh6uZohsgalx1a4sBhTGI/isiEA+8UO1JuTiPgFPu4eio72lmNHTUa drYA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g42si17040583wrg.168.2017.07.26.00.39.29; Wed, 26 Jul 2017 00:39:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 51B27689958; Wed, 26 Jul 2017 10:39:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from kazbek.m1stereo.tv (mail.m1stereo.tv [91.244.124.37]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 30E97680D6D for ; Wed, 26 Jul 2017 10:39:19 +0300 (EEST) Received: from [10.1.5.65] (dev-3.internal.m1stereo.tv [10.1.5.65]) (authenticated bits=0) by kazbek.m1stereo.tv (8.14.4/8.14.4) with ESMTP id v6Q7dI32005899 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 26 Jul 2017 10:39:18 +0300 To: ffmpeg-devel@ffmpeg.org, Nicolas George References: <20170723102045.GA1244296@phare.normalesup.org> <20170724171110.GB1866050@phare.normalesup.org> From: Maksym Veremeyenko Message-ID: <1055478c-ad21-2372-1ebd-1d3d286f6822@m1stereo.tv> Date: Wed, 26 Jul 2017 10:39:18 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20170724171110.GB1866050@phare.normalesup.org> Subject: Re: [FFmpeg-devel] [PATCH] Implement NewTek NDI support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On 24.07.2017 20:11, Nicolas George wrote: > Le quintidi 5 thermidor, an CCXXV, Maksym Veremeyenko a écrit : >> would it be ok: >> >> Subject: [PATCH] lavf: implement NewTek NDI support > > Exactly. Except I mistyped: that would be lavd, not lavf; my bad. fixed > >> would you *newtek_ndi* or *libndi_newtek" > > I prefer libndi_newtek, but that is only my advice. > fixed >>> Is -vcodec wrapped_avframe really necessary? It should be automatic. >> yes, i can remove that if you prefer. > > I think it is better if examples are as simple as possible. > fixed >> as mentioned above, it used in a thread > > Yes, but you can give avctx to the thread and get the private context > from it instead of giving the private context and getting avctx from it. > I have not yet looked at what you did in the new version. > i dropped that >>>> + if (NDIlib_frame_type_video == t) >>> The test looks strange in that direction. >> why? that approach could save from mistypes like > > I know it is the usual justification for this idiom. But at the same > time, it is anti-idiomatic for most people. We say "Alice is in her > office", not "Alice's office contains her", and the same goes for > comparisons. Furthermore, all decent compilers know to print a warning > in these cases since quite some time. > > As above, if you intent to maintain this code on the long run, then > decision is yours. Otherwise, better follow the usual coding style. > fixed >>>> + ndi_enqeue_video(ctx, &v); >>>> + else if (NDIlib_frame_type_audio == t) >>>> + ndi_enqeue_audio(ctx, &a); >>>> + else if (NDIlib_frame_type_metadata == t) >>>> + NDIlib_recv_free_metadata(ctx->recv, &m); >>> Is there a risk of leak if a new type of packet is invented? Looks like >>> a bad API design by NewTek. >> API in a developing stage and not final - we can expect everythign > > You seem to be in contact with them, then you may report this issue: a > function capable of freeing any kind of packet is absolutely necessary. > Otherwise, if a new kind of packet is introduced, older applications > will not be able to free them and leak. With a generic function, > applications can just free any unknown packet. > i added unknow frame type warning >> could you give me a hint/examples? > > I think it has been addressed in other mails. > i found it throw code and reworked completely >> currently alpha channel is ignored > > Can you clarify who is ignoring the alpha channel? > alpha been ignored by *me* :-) there is no appropriate pixel format for handling it. >> i blindly copying further code from decklink module that provides setting >> interlaced flag > > It is entirely possible that existing components have code that follow > what is no longer considered best practice, especially when these > components depend on proprietary libraries that few people have. > > On the other hand, new code is expected to follow the best practices. In > particular, new code is expected to produce zero warnings with common > compilers (gcc, clang). > in current implementation: + st->codecpar->field_order = v->frame_format_type == NDIlib_frame_format_type_progressive + ? AV_FIELD_PROGRESSIVE : AV_FIELD_TT; >> thread is reading, main app is writing only... > > This is a common misconception. The fact that the modification are > unidirectional avoids race conditions, but they are not the only issue. > The compiler is perfectly allowed to decide that this store is useless > and optimize it away. Even if the compiler is prevented to do so using > the "volatile" keyword, the processor can keep the value inside a cache > that is not shared between SMP instances. > > For these reasons, a lock is not needed but a memory barrier is. A > memory barrier is exactly the instruction to flush the data completely > to/from shared memory. > i dropped thread for reading packet, so it is no more needed... >>>> + ctx->video->line_stride_in_bytes = avframe->linesize[0] < 0 >>>> + ? -avframe->linesize[0] >>>> + : avframe->linesize[0]; >>> abs()? >> copied from decklink code > > Could still be simplified in new code. > >>>> + ctx->video->p_data = (avframe->linesize[0] < 0) >>>> + ? (void *)(avframe->data[0] + avframe->linesize[0] * (avframe->height - 1)) >>>> + : (void *)(avframe->data[0]); >>> Did you test with negative linesize? It looks like it will flip the >>> video. >> i did not test, but i leaved it for further processing > > I find this block quite suspicious. It really needs to be tested. But I > do not know how to easily generate frames with negative strides. > i think it could work in synthetic tests or some TGA files has it or some real cases with OpenGL surface downloading... >>> Check channel layout too. >> i will drop this check at all > > Unfortunately, the documentation does not seem to be available without > registration. Can you check what they tell exactly? > > If the library rejects channels configurations it does not support, then > there is no need to check before. > > But if the library only checks the number of channels and makes > assumptions about the layout (very common behaviour), then the calling > code must check the layout. Otherwise, the auto-rematrixing code will > not be triggered and the output will be wrong. > > Actually, did you check that they use the same channel order as FFmpeg ? > Debugging problems with channel order is a real nightmare for users (I > have spent hours myself on it when I purchased 5.1 speakers), so > developers ought to be very careful about it. documentation says: [...] supporting 4K (and beyond) along with 16 channels (and more) of floating-point audio. [...] so actually channels numbers not limited. moreover, there is no channel layout - nor SDI, nor simple ADC/DAC i attached latest version of patch, please review From f40b87b4c7e08ccf23ee896923459e5901ae1086 Mon Sep 17 00:00:00 2001 From: Maksym Veremeyenko Date: Wed, 26 Jul 2017 03:22:20 -0400 Subject: [PATCH] lavd: implement NewTek NDI muxer/demuxer support --- Changelog | 1 + configure | 9 + doc/indevs.texi | 57 +++++++ doc/outdevs.texi | 37 +++++ libavdevice/Makefile | 4 + libavdevice/alldevices.c | 1 + libavdevice/libndi_newtek_common.h | 31 ++++ libavdevice/libndi_newtek_dec.c | 331 +++++++++++++++++++++++++++++++++++++ libavdevice/libndi_newtek_enc.c | 279 +++++++++++++++++++++++++++++++ libavdevice/version.h | 2 +- 10 files changed, 751 insertions(+), 1 deletion(-) create mode 100644 libavdevice/libndi_newtek_common.h create mode 100644 libavdevice/libndi_newtek_dec.c create mode 100644 libavdevice/libndi_newtek_enc.c diff --git a/Changelog b/Changelog index 187ae79..568b481 100644 --- a/Changelog +++ b/Changelog @@ -29,6 +29,7 @@ version : - limiter video filter - libvmaf video filter - Dolby E decoder and SMPTE 337M demuxer +- NewTek NDI muxer/demuxer version 3.3: - CrystalHD decoder moved to new decode API diff --git a/configure b/configure index 5811ee1..05d9e5f 100755 --- a/configure +++ b/configure @@ -277,6 +277,7 @@ External library support: --enable-libzvbi enable teletext support via libzvbi [no] --disable-lzma disable lzma [autodetect] --enable-decklink enable Blackmagic DeckLink I/O support [no] + --enable-libndi_newtek enable Newteck NDI I/O support [no] --enable-mediacodec enable Android MediaCodec support [no] --enable-libmysofa enable libmysofa, needed for sofalizer filter [no] --enable-openal enable OpenAL 1.1 capture support [no] @@ -1507,6 +1508,7 @@ EXTERNAL_LIBRARY_GPL_LIST=" EXTERNAL_LIBRARY_NONFREE_LIST=" decklink + libndi_newtek libfdk_aac openssl " @@ -3012,6 +3014,10 @@ decklink_indev_deps="decklink threads" decklink_indev_extralibs="-lstdc++" decklink_outdev_deps="decklink threads" decklink_outdev_extralibs="-lstdc++" +libndi_newtek_indev_deps="libndi_newtek" +libndi_newtek_indev_extralibs="-lndi" +libndi_newtek_outdev_deps="libndi_newtek" +libndi_newtek_outdev_extralibs="-lndi" dshow_indev_deps="IBaseFilter" dshow_indev_extralibs="-lpsapi -lole32 -lstrmiids -luuid -loleaut32 -lshlwapi" dv1394_indev_deps="dv1394" @@ -5564,6 +5570,8 @@ avisynth_demuxer_extralibs='$ldl' cuda_extralibs='$ldl' decklink_outdev_extralibs="$decklink_outdev_extralibs $ldl" decklink_indev_extralibs="$decklink_indev_extralibs $ldl" +libndi_newtek_outdev_extralibs="$libndi_newtek_outdev_extralibs $ldl" +libndi_newtek_indev_extralibs="$libndi_newtek_indev_extralibs $ldl" frei0r_filter_extralibs='$ldl' frei0r_src_filter_extralibs='$ldl' ladspa_filter_extralibs='$ldl' @@ -5822,6 +5830,7 @@ enabled coreimage_filter && { check_header_objcc QuartzCore/CoreImage.h || disa enabled coreimagesrc_filter && { check_header_objcc QuartzCore/CoreImage.h || disable coreimagesrc_filter; } enabled decklink && { { check_header DeckLinkAPI.h || die "ERROR: DeckLinkAPI.h header not found"; } && { check_cpp_condition DeckLinkAPIVersion.h "BLACKMAGIC_DECKLINK_API_VERSION >= 0x0a060100" || die "ERROR: Decklink API version must be >= 10.6.1."; } } +enabled libndi_newtek && { check_header Processing.NDI.Lib.h || die "ERROR: Processing.NDI.Lib.h header not found"; } enabled frei0r && { check_header frei0r.h || die "ERROR: frei0r.h header not found"; } enabled gmp && require gmp gmp.h mpz_export -lgmp enabled gnutls && require_pkg_config gnutls gnutls/gnutls.h gnutls_global_init diff --git a/doc/indevs.texi b/doc/indevs.texi index 09e3321..e8220a3 100644 --- a/doc/indevs.texi +++ b/doc/indevs.texi @@ -327,6 +327,63 @@ ffmpeg -channels 16 -format_code Hi50 -f decklink -i 'UltraStudio Mini Recorder' @end itemize +@section libndi_newtek + +The libndi_newtek input device provides capture capabilities for using NDI (Network +Device Interface, standard created by NewTek). + +Input filename is a NDI source name that could be found by sending -find_sources 1 +to command line - it has no specific syntax but human-readable formatted. + +To enable this input device, you need the NDI SDK and you +need to configure with the appropriate @code{--extra-cflags} +and @code{--extra-ldflags}. + +@subsection Options + +@table @option + +@item find_sources +If set to @option{true}, print a list of found/available NDI sources and exit. +Defaults to @option{false}. + +@item wait_sources +Override time to wait until the number of online sources have changed. +Defaults to @option{500000}. + +@item allow_video_fields +When this flag is @option{false}, all video that you receive will be progressive. +Defaults to @option{true}. + +@item max_frames_probe +Maximum frames for probe input format. +Defaults to @option{25}. + +@item reference_level +The audio reference level in dB. This specifies how many dB above the +reference level (+4dBU) is the full range of 16 bit audio. +Defaults to @option{0}. + +@end table + +@subsection Examples + +@itemize + +@item +List input devices: +@example +ffmpeg -f libndi_newtek -find_sources 1 -i dummy +@end example + +@item +Restream to NDI: +@example +ffmpeg -f libndi_newtek -i "DEV-5.INTERNAL.M1STEREO.TV (NDI_SOURCE_NAME_1)" -f libndi_newtek -y NDI_SOURCE_NAME_2 +@end example + +@end itemize + @section dshow Windows DirectShow input device. diff --git a/doc/outdevs.texi b/doc/outdevs.texi index df41cc8..8d7c068 100644 --- a/doc/outdevs.texi +++ b/doc/outdevs.texi @@ -182,6 +182,43 @@ ffmpeg -i test.avi -f decklink -pix_fmt uyvy422 -s 720x486 -r 24000/1001 'DeckLi @end itemize +@section libndi_newtek + +The libndi_newtek output device provides playback capabilities for using NDI (Network +Device Interface, standard created by NewTek). + +Output filename is a NDI name. + +To enable this output device, you need the NDI SDK and you +need to configure with the appropriate @code{--extra-cflags} +and @code{--extra-ldflags}. + +NDI is very picky about the formats it supports. Pixel format is always +uyvy422. Audio sample rate is always 48 kHz. + +@subsection Options + +@table @option + +@item reference_level +The audio reference level in dB. This specifies how many dB above the +reference level (+4dBU) is the full range of 16 bit audio. +Defaults to @option{0}. + +@end table + +@subsection Examples + +@itemize + +@item +Play video clip: +@example +ffmpeg -i "udp://@239.1.1.1:10480?fifo_size=1000000&overrun_nonfatal=1" -vf "scale=720:576,fps=fps=25,setdar=dar=16/9,format=pix_fmts=uyvy422" -f libndi_newtek NEW_NDI1 +@end example + +@end itemize + @section fbdev Linux framebuffer output device. diff --git a/libavdevice/Makefile b/libavdevice/Makefile index 1d4e9e6..80afa61 100644 --- a/libavdevice/Makefile +++ b/libavdevice/Makefile @@ -19,6 +19,8 @@ OBJS-$(CONFIG_BKTR_INDEV) += bktr.o OBJS-$(CONFIG_CACA_OUTDEV) += caca.o OBJS-$(CONFIG_DECKLINK_OUTDEV) += decklink_enc.o decklink_enc_c.o decklink_common.o OBJS-$(CONFIG_DECKLINK_INDEV) += decklink_dec.o decklink_dec_c.o decklink_common.o +OBJS-$(CONFIG_LIBNDI_NEWTEK_OUTDEV) += libndi_newtek_enc.o +OBJS-$(CONFIG_LIBNDI_NEWTEK_INDEV) += libndi_newtek_dec.o OBJS-$(CONFIG_DSHOW_INDEV) += dshow_crossbar.o dshow.o dshow_enummediatypes.o \ dshow_enumpins.o dshow_filter.o \ dshow_pin.o dshow_common.o @@ -60,6 +62,8 @@ SLIBOBJS-$(HAVE_GNU_WINDRES) += avdeviceres.o SKIPHEADERS += decklink_common.h SKIPHEADERS-$(CONFIG_DECKLINK) += decklink_enc.h decklink_dec.h \ decklink_common_c.h +SKIPHEADERS-$(CONFIG_LIBNDI_NEWTEK_INDEV) += libndi_newtek_common.h +SKIPHEADERS-$(CONFIG_LIBNDI_NEWTEK_OUTDEV) += libndi_newtek_common.h SKIPHEADERS-$(CONFIG_DSHOW_INDEV) += dshow_capture.h SKIPHEADERS-$(CONFIG_FBDEV_INDEV) += fbdev_common.h SKIPHEADERS-$(CONFIG_FBDEV_OUTDEV) += fbdev_common.h diff --git a/libavdevice/alldevices.c b/libavdevice/alldevices.c index a8ed53a..8d1cb86 100644 --- a/libavdevice/alldevices.c +++ b/libavdevice/alldevices.c @@ -46,6 +46,7 @@ static void register_all(void) REGISTER_INDEV (BKTR, bktr); REGISTER_OUTDEV (CACA, caca); REGISTER_INOUTDEV(DECKLINK, decklink); + REGISTER_INOUTDEV(LIBNDI_NEWTEK, libndi_newtek); REGISTER_INDEV (DSHOW, dshow); REGISTER_INDEV (DV1394, dv1394); REGISTER_INOUTDEV(FBDEV, fbdev); diff --git a/libavdevice/libndi_newtek_common.h b/libavdevice/libndi_newtek_common.h new file mode 100644 index 0000000..43be677 --- /dev/null +++ b/libavdevice/libndi_newtek_common.h @@ -0,0 +1,31 @@ +/* + * NewTek NDI common code + * Copyright (c) 2017 Maksym Veremeyenko + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVDEVICE_LIBNDI_NEWTEK_COMMON_H +#define AVDEVICE_LIBNDI_NEWTEK_COMMON_H + +#include + +#define NDI_TIME_BASE 10000000 +#define NDI_TIME_BASE_Q (AVRational){1, NDI_TIME_BASE} + +#endif + diff --git a/libavdevice/libndi_newtek_dec.c b/libavdevice/libndi_newtek_dec.c new file mode 100644 index 0000000..f0b1220 --- /dev/null +++ b/libavdevice/libndi_newtek_dec.c @@ -0,0 +1,331 @@ +/* + * Newtek NDI input + * Copyright (c) 2017 Maksym Veremeyenko + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavformat/avformat.h" +#include "libavformat/internal.h" +#include "libavutil/opt.h" +#include "libavutil/imgutils.h" + +#include "libndi_newtek_common.h" + +struct NDIContext { + const AVClass *cclass; + + /* Options */ + int find_sources; + int wait_sources; + int allow_video_fields; + int max_frames_probe; + int reference_level; + + /* Runtime */ + NDIlib_recv_create_t *recv; + + /* Streams */ + AVStream *video_st, *audio_st; +}; + +static int ndi_set_video_packet(AVFormatContext *avctx, NDIlib_video_frame_t *v, AVPacket *pkt) +{ + int ret; + struct NDIContext *ctx = avctx->priv_data; + + ret = av_new_packet(pkt, v->yres * v->line_stride_in_bytes); + if (ret < 0) + return ret; + + pkt->dts = pkt->pts = av_rescale_q(v->timecode, NDI_TIME_BASE_Q, ctx->video_st->time_base); + pkt->duration = av_rescale_q(1, (AVRational){v->frame_rate_D, v->frame_rate_N}, ctx->video_st->time_base); + + av_log(avctx, AV_LOG_DEBUG, "%s: pkt->dts = pkt->pts = %"PRId64", duration=%"PRId64", timecode=%"PRId64"\n", + __func__, pkt->dts, pkt->duration, v->timecode); + + pkt->flags |= AV_PKT_FLAG_KEY; + pkt->stream_index = ctx->video_st->index; + + memcpy(pkt->data, v->p_data, pkt->size); + + NDIlib_recv_free_video(ctx->recv, v); + + return 0; +} + +static int ndi_set_audio_packet(AVFormatContext *avctx, NDIlib_audio_frame_t *a, AVPacket *pkt) +{ + int ret; + struct NDIContext *ctx = avctx->priv_data; + + NDIlib_audio_frame_interleaved_16s_t dst; + + ret = av_new_packet(pkt, 2 * a->no_samples * a->no_channels); + if (ret < 0) + return ret; + + pkt->dts = pkt->pts = av_rescale_q(a->timecode, NDI_TIME_BASE_Q, ctx->audio_st->time_base); + pkt->duration = av_rescale_q(1, (AVRational){a->no_samples, a->sample_rate}, ctx->audio_st->time_base); + + av_log(avctx, AV_LOG_DEBUG, "%s: pkt->dts = pkt->pts = %"PRId64", duration=%"PRId64", timecode=%"PRId64"\n", + __func__, pkt->dts, pkt->duration, a->timecode); + + pkt->flags |= AV_PKT_FLAG_KEY; + pkt->stream_index = ctx->audio_st->index; + + dst.reference_level = 0; + dst.p_data = (short *)pkt->data; + NDIlib_util_audio_to_interleaved_16s(a, &dst); + + NDIlib_recv_free_audio(ctx->recv, a); + + return 0; +} + +static int ndi_find_sources(AVFormatContext *avctx, const char *name, NDIlib_source_t *source_to_connect_to) +{ + int j = AVERROR(ENODEV); + unsigned int n, i; + struct NDIContext *ctx = avctx->priv_data; + const NDIlib_source_t *ndi_srcs = NULL; + const NDIlib_find_create_t find_create_desc = { true, NULL }; + NDIlib_find_instance_t ndi_find; + + ndi_find = NDIlib_find_create2(&find_create_desc); + if (!ndi_find) { + av_log(avctx, AV_LOG_ERROR, "NDIlib_find_create failed.\n"); + return AVERROR(EIO); + } + + NDIlib_find_wait_for_sources(ndi_find, ctx->wait_sources / 1000); + ndi_srcs = NDIlib_find_get_current_sources(ndi_find, &n); + + if (ctx->find_sources) + av_log(avctx, AV_LOG_INFO, "Found %d NDI sources:\n", n); + + for (i = 0; i < n; i++) { + if (ctx->find_sources) + av_log(avctx, AV_LOG_INFO, "\t'%s'\t'%s'\n", ndi_srcs[i].p_ndi_name, ndi_srcs[i].p_ip_address); + + if (!strcmp(name, ndi_srcs[i].p_ndi_name)) { + *source_to_connect_to = ndi_srcs[i]; + j = i; + } + } + + NDIlib_find_destroy(ndi_find); + + return j; +} + +static int ndi_read_header(AVFormatContext *avctx) +{ + int ret; + NDIlib_recv_create_t recv_create_desc; + const NDIlib_tally_t tally_state = { true, false }; + struct NDIContext *ctx = avctx->priv_data; + + if (!NDIlib_initialize()) { + av_log(avctx, AV_LOG_ERROR, "NDIlib_initialize failed.\n"); + return AVERROR_EXTERNAL; + } + + /* Find available sources. */ + ret = ndi_find_sources(avctx, avctx->filename, &recv_create_desc.source_to_connect_to); + if (ctx->find_sources) { + return AVERROR_EXIT; + } + if (ret < 0) + return ret; + + /* Create receiver description */ + recv_create_desc.color_format = NDIlib_recv_color_format_e_UYVY_RGBA; + recv_create_desc.bandwidth = NDIlib_recv_bandwidth_highest; + recv_create_desc.allow_video_fields = ctx->allow_video_fields; + + /* Create the receiver */ + ctx->recv = NDIlib_recv_create2(&recv_create_desc); + if (!ctx->recv) { + av_log(avctx, AV_LOG_ERROR, "NDIlib_recv_create2 failed.\n"); + return AVERROR(EIO); + } + + /* Set tally */ + NDIlib_recv_set_tally(ctx->recv, &tally_state); + + avctx->ctx_flags |= AVFMTCTX_NOHEADER; + + return 0; +} + +static int ndi_create_video_stream(AVFormatContext *avctx, NDIlib_video_frame_t *v) +{ + AVStream *st; + struct NDIContext *ctx = avctx->priv_data; + + st = avformat_new_stream(avctx, NULL); + if (!st) { + av_log(avctx, AV_LOG_ERROR, "Cannot add video stream\n"); + return AVERROR(ENOMEM); + } + + st->time_base = NDI_TIME_BASE_Q; + av_stream_set_r_frame_rate(st, av_make_q(v->frame_rate_N, v->frame_rate_D)); + + st->sample_aspect_ratio = st->codecpar->sample_aspect_ratio = + av_mul_q(av_d2q(v->picture_aspect_ratio, INT_MAX), (AVRational){v->yres, v->xres}); + + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->width = v->xres; + st->codecpar->height = v->yres; + st->codecpar->codec_id = AV_CODEC_ID_RAWVIDEO; + st->codecpar->bit_rate = av_rescale(v->xres * v->yres * 16, st->time_base.den, st->time_base.num); + st->codecpar->field_order = v->frame_format_type == NDIlib_frame_format_type_progressive + ? AV_FIELD_PROGRESSIVE : AV_FIELD_TT; + + if (v->FourCC == NDIlib_FourCC_type_UYVY || v->FourCC == NDIlib_FourCC_type_UYVA) { + st->codecpar->format = AV_PIX_FMT_UYVY422; + st->codecpar->codec_tag = MKTAG('U', 'Y', 'V', 'Y'); + } else if (v->FourCC == NDIlib_FourCC_type_BGRA) { + st->codecpar->format = AV_PIX_FMT_BGRA; + st->codecpar->codec_tag = MKTAG('B', 'G', 'R', 'A'); + } else if (v->FourCC == NDIlib_FourCC_type_BGRX) { + st->codecpar->format = AV_PIX_FMT_BGR0; + st->codecpar->codec_tag = MKTAG('B', 'G', 'R', '0'); + } else if (v->FourCC == NDIlib_FourCC_type_RGBA) { + st->codecpar->format = AV_PIX_FMT_RGBA; + st->codecpar->codec_tag = MKTAG('R', 'G', 'B', 'A'); + } else if (v->FourCC == NDIlib_FourCC_type_RGBX) { + st->codecpar->format = AV_PIX_FMT_RGB0; + st->codecpar->codec_tag = MKTAG('R', 'G', 'B', '0'); + } else { + av_log(avctx, AV_LOG_ERROR, "Unsupported video stream format, v->FourCC=%d\n", v->FourCC); + return AVERROR(EINVAL); + } + + avpriv_set_pts_info(st, 64, 1, NDI_TIME_BASE); + ctx->video_st = st; + + return 0; +} + +static int ndi_create_audio_stream(AVFormatContext *avctx, NDIlib_audio_frame_t *a) +{ + AVStream *st; + struct NDIContext *ctx = avctx->priv_data; + + st = avformat_new_stream(avctx, NULL); + if (!st) { + av_log(avctx, AV_LOG_ERROR, "Cannot add audio stream\n"); + return AVERROR(ENOMEM); + } + + st->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; + st->codecpar->codec_id = AV_CODEC_ID_PCM_S16LE; + st->codecpar->sample_rate = a->sample_rate; + st->codecpar->channels = a->no_channels; + +// avpriv_set_pts_info(st, 64, 1, a->sample_rate); + avpriv_set_pts_info(st, 64, 1, NDI_TIME_BASE); + + ctx->audio_st = st; + + return 0; +} + +static int ndi_read_packet(AVFormatContext *avctx, AVPacket *pkt) +{ + int ret = 0; + struct NDIContext *ctx = avctx->priv_data; + + while (1) { + NDIlib_video_frame_t v; + NDIlib_audio_frame_t a; + NDIlib_metadata_frame_t m; + NDIlib_frame_type_e t; + + t = NDIlib_recv_capture(ctx->recv, &v, &a, &m, 40); + + if (t == NDIlib_frame_type_video) { + if (!ctx->video_st) { + ret = ndi_create_video_stream(avctx, &v); + if (ret < 0) + return ret; + } + ret = ndi_set_video_packet(avctx, &v, pkt); + break; + } + else if (t == NDIlib_frame_type_audio) { + if (!ctx->audio_st) { + ret = ndi_create_audio_stream(avctx, &a); + if (ret < 0) + return ret; + } + ret = ndi_set_audio_packet(avctx, &a, pkt); + break; + } + else if (t == NDIlib_frame_type_metadata) + NDIlib_recv_free_metadata(ctx->recv, &m); + else if (t != NDIlib_frame_type_none) { + av_log(avctx, AV_LOG_ERROR, "NDIlib_recv_capture returns %d\n", t); + } + }; + + return ret; +} + +static int ndi_read_close(AVFormatContext *avctx) +{ + struct NDIContext *ctx = (struct NDIContext *)avctx->priv_data; + + if (ctx->recv) + NDIlib_recv_destroy(ctx->recv); + + return 0; +} + +#define OFFSET(x) offsetof(struct NDIContext, x) +#define DEC AV_OPT_FLAG_DECODING_PARAM + +static const AVOption options[] = { + { "find_sources", "Find available sources" , OFFSET(find_sources), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, DEC }, + { "wait_sources", "Time to wait until the number of online sources have changed" , OFFSET(wait_sources), AV_OPT_TYPE_DURATION, { .i64 = 500000 }, 100000, 10000000, DEC }, + { "allow_video_fields", "When this flag is FALSE, all video that you receive will be progressive" , OFFSET(allow_video_fields), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, DEC }, + { "max_frames_probe", "Maximum frames for probe" , OFFSET(max_frames_probe), AV_OPT_TYPE_INT, { .i64 = 25 }, 6, 100, DEC }, + { "reference_level", "The audio reference level in dB" , OFFSET(reference_level), AV_OPT_TYPE_INT, { .i64 = 0 }, -20, 20, DEC | AV_OPT_FLAG_AUDIO_PARAM}, + { NULL }, +}; + +static const AVClass libndi_newtek_demuxer_class = { + .class_name = "NDI demuxer", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, + .category = AV_CLASS_CATEGORY_DEVICE_VIDEO_INPUT, +}; + +AVInputFormat ff_libndi_newtek_demuxer = { + .name = "libndi_newtek", + .long_name = NULL_IF_CONFIG_SMALL("Network Device Interface (NDI) input using NewTek library"), + .flags = AVFMT_NOFILE, + .priv_class = &libndi_newtek_demuxer_class, + .priv_data_size = sizeof(struct NDIContext), + .read_header = ndi_read_header, + .read_packet = ndi_read_packet, + .read_close = ndi_read_close, +}; diff --git a/libavdevice/libndi_newtek_enc.c b/libavdevice/libndi_newtek_enc.c new file mode 100644 index 0000000..99041a4 --- /dev/null +++ b/libavdevice/libndi_newtek_enc.c @@ -0,0 +1,279 @@ +/* + * NewTek NDI output + * Copyright (c) 2017 Maksym Veremeyenko + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavformat/avformat.h" +#include "libavformat/internal.h" +#include "libavutil/opt.h" +#include "libavutil/imgutils.h" + +#include "libndi_newtek_common.h" + +struct NDIContext { + const AVClass *cclass; + + /* Options */ + int reference_level; + + NDIlib_video_frame_t *video; + NDIlib_audio_frame_interleaved_16s_t *audio; + NDIlib_send_instance_t ndi_send; + AVFrame *last_avframe; +}; + +static int ndi_write_trailer(AVFormatContext *avctx) +{ + struct NDIContext *ctx = avctx->priv_data; + + if (ctx->ndi_send) { + if (ctx->last_avframe) { + /* send NULL to flush equeued frame */ + NDIlib_send_send_video_async(ctx->ndi_send, NULL); + + av_frame_free(&ctx->last_avframe); + } + + NDIlib_send_destroy(ctx->ndi_send); + } + + if (ctx->video) + av_freep(&ctx->video); + + if (ctx->audio) + av_freep(&ctx->audio); + + return 0; +} + +static int ndi_write_video_packet(AVFormatContext *avctx, AVStream *st, AVPacket *pkt) +{ + struct NDIContext *ctx = avctx->priv_data; + AVFrame *avframe, *tmp = (AVFrame *)pkt->data; + + if (tmp->format != AV_PIX_FMT_UYVY422 || + tmp->width != ctx->video->xres || + tmp->height != ctx->video->yres) { + av_log(avctx, AV_LOG_ERROR, "Got a frame with invalid pixel format or dimension.\n"); + av_log(avctx, AV_LOG_ERROR, "tmp->width=%d, tmp->height=%d, ctx->video->xres=%d, ctx->video->yres=%d\n", tmp->width, tmp->height, ctx->video->xres, ctx->video->yres); + return AVERROR(EINVAL); + } + avframe = av_frame_clone(tmp); + if (!avframe) { + av_log(avctx, AV_LOG_ERROR, "Could not clone video frame.\n"); + return AVERROR(ENOMEM); + } + + ctx->video->timecode = av_rescale_q(pkt->pts, st->time_base, NDI_TIME_BASE_Q); + + ctx->video->line_stride_in_bytes = avframe->linesize[0] < 0 + ? -avframe->linesize[0] + : avframe->linesize[0]; + ctx->video->p_data = (avframe->linesize[0] < 0) + ? (void *)(avframe->data[0] + avframe->linesize[0] * (avframe->height - 1)) + : (void *)(avframe->data[0]); + + av_log(avctx, AV_LOG_DEBUG, "%s: pkt->pts=%"PRId64", timecode=%"PRId64", st->time_base=%d/%d\n", + __func__, pkt->pts, ctx->video->timecode, st->time_base.num, st->time_base.den); + + /* asynchronous for one frame, but will block if a second frame + is given before the first one has been sent */ + NDIlib_send_send_video_async(ctx->ndi_send, ctx->video); + + if (ctx->last_avframe) + av_frame_free(&ctx->last_avframe); + ctx->last_avframe = avframe; + + return 0; +} + +static int ndi_write_audio_packet(AVFormatContext *avctx, AVStream *st, AVPacket *pkt) +{ + struct NDIContext *ctx = avctx->priv_data; + + ctx->audio->p_data = (short *)pkt->data; + ctx->audio->timecode = av_rescale_q(pkt->pts, st->time_base, NDI_TIME_BASE_Q); + ctx->audio->no_samples = pkt->size / (ctx->audio->no_channels << 1); + + av_log(avctx, AV_LOG_DEBUG, "%s: pkt->pts=%"PRId64", timecode=%"PRId64", st->time_base=%d/%d\n", + __func__, pkt->pts, ctx->video->timecode, st->time_base.num, st->time_base.den); + + NDIlib_util_send_send_audio_interleaved_16s(ctx->ndi_send, ctx->audio); + + return 0; +} + +static int ndi_write_packet(AVFormatContext *avctx, AVPacket *pkt) +{ + AVStream *st = avctx->streams[pkt->stream_index]; + + if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) + return ndi_write_video_packet(avctx, st, pkt); + else if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) + return ndi_write_audio_packet(avctx, st, pkt); + + return AVERROR_BUG; +} + +static int ndi_setup_audio(AVFormatContext *avctx, AVStream *st) +{ + struct NDIContext *ctx = avctx->priv_data; + AVCodecParameters *c = st->codecpar; + + if (ctx->audio) { + av_log(avctx, AV_LOG_ERROR, "Only one audio stream is supported!\n"); + return AVERROR(EINVAL); + } + if (c->sample_rate != 48000) { + av_log(avctx, AV_LOG_ERROR, "Unsupported sample rate!" + " Only 48kHz is supported.\n"); + return AVERROR(EINVAL); + } + + ctx->audio = av_mallocz(sizeof(NDIlib_audio_frame_interleaved_16s_t)); + if (!ctx->audio) + return AVERROR(ENOMEM); + + ctx->audio->sample_rate = c->sample_rate; + ctx->audio->no_channels = c->channels; + ctx->audio->reference_level = ctx->reference_level; + + avpriv_set_pts_info(st, 64, 1, NDI_TIME_BASE/*c->sample_rate*/); + + return 0; +} + +static int ndi_setup_video(AVFormatContext *avctx, AVStream *st) +{ + struct NDIContext *ctx = avctx->priv_data; + AVCodecParameters *c = st->codecpar; + + if (ctx->video) { + av_log(avctx, AV_LOG_ERROR, "Only one video stream is supported!\n"); + return AVERROR(EINVAL); + } + + if (c->codec_id != AV_CODEC_ID_WRAPPED_AVFRAME) { + av_log(avctx, AV_LOG_ERROR, "Unsupported codec format!" + " Only AV_CODEC_ID_WRAPPED_AVFRAME is supported (-vcodec wrapped_avframe).\n"); + return AVERROR(EINVAL); + } + + if (c->format != AV_PIX_FMT_UYVY422) { + av_log(avctx, AV_LOG_ERROR, "Unsupported pixel format!" + " Only AV_PIX_FMT_UYVY422 is supported.\n"); + return AVERROR(EINVAL); + } + + ctx->video = av_mallocz(sizeof(NDIlib_video_frame_t)); + if (!ctx->video) + return AVERROR(ENOMEM); + + ctx->video->FourCC = NDIlib_FourCC_type_UYVY; + ctx->video->xres = c->width; + ctx->video->yres = c->height; + ctx->video->frame_rate_N = st->avg_frame_rate.num; + ctx->video->frame_rate_D = st->avg_frame_rate.den; + ctx->video->frame_format_type = c->field_order == AV_FIELD_PROGRESSIVE + ? NDIlib_frame_format_type_progressive + : NDIlib_frame_format_type_interleaved; + + if (st->sample_aspect_ratio.num) { + AVRational display_aspect_ratio; + av_reduce(&display_aspect_ratio.num, &display_aspect_ratio.den, + st->codecpar->width * (int64_t)st->sample_aspect_ratio.num, + st->codecpar->height * (int64_t)st->sample_aspect_ratio.den, + 1024 * 1024); + ctx->video->picture_aspect_ratio = av_q2d(display_aspect_ratio); + } + else + ctx->video->picture_aspect_ratio = (double)st->codecpar->width/st->codecpar->height; + +// avpriv_set_pts_info(st, 64, st->time_base.num, st->time_base.den); + avpriv_set_pts_info(st, 64, 1, NDI_TIME_BASE); + + return 0; +} + +static int ndi_write_header(AVFormatContext *avctx) +{ + int ret = 0; + unsigned int n; + struct NDIContext *ctx = avctx->priv_data; + const NDIlib_send_create_t ndi_send_desc = {avctx->filename, NULL, true, false}; + + if (!NDIlib_initialize()) { + av_log(avctx, AV_LOG_ERROR, "NDIlib_initialize failed.\n"); + return AVERROR_EXTERNAL; + } + + /* check if streams compatible */ + for (n = 0; n < avctx->nb_streams; n++) { + AVStream *st = avctx->streams[n]; + AVCodecParameters *c = st->codecpar; + if (c->codec_type == AVMEDIA_TYPE_AUDIO) { + if ((ret = ndi_setup_audio(avctx, st))) + goto error; + } else if (c->codec_type == AVMEDIA_TYPE_VIDEO) { + if ((ret = ndi_setup_video(avctx, st))) + goto error; + } else { + av_log(avctx, AV_LOG_ERROR, "Unsupported stream type.\n"); + ret = AVERROR(EINVAL); + goto error; + } + } + + ctx->ndi_send = NDIlib_send_create(&ndi_send_desc); + if (!ctx->ndi_send) { + av_log(avctx, AV_LOG_ERROR, "Failed to create NDI output %s\n", avctx->filename); + ret = AVERROR_EXTERNAL; + } + +error: + return ret; +} + +#define OFFSET(x) offsetof(struct NDIContext, x) +static const AVOption options[] = { + { "reference_level", "The audio reference level in dB" , OFFSET(reference_level), AV_OPT_TYPE_INT, { .i64 = 0 }, -20, 20, AV_OPT_FLAG_ENCODING_PARAM | AV_OPT_FLAG_AUDIO_PARAM}, + { NULL }, +}; + +static const AVClass libndi_newtek_muxer_class = { + .class_name = "NDI muxer", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, + .category = AV_CLASS_CATEGORY_DEVICE_VIDEO_OUTPUT, +}; + +AVOutputFormat ff_libndi_newtek_muxer = { + .name = "libndi_newtek", + .long_name = NULL_IF_CONFIG_SMALL("Network Device Interface (NDI) output using NewTek library"), + .audio_codec = AV_CODEC_ID_PCM_S16LE, + .video_codec = AV_CODEC_ID_WRAPPED_AVFRAME, + .subtitle_codec = AV_CODEC_ID_NONE, + .flags = AVFMT_NOFILE, + .priv_class = &libndi_newtek_muxer_class, + .priv_data_size = sizeof(struct NDIContext), + .write_header = ndi_write_header, + .write_packet = ndi_write_packet, + .write_trailer = ndi_write_trailer, +}; diff --git a/libavdevice/version.h b/libavdevice/version.h index 0c06602..948e4e1 100644 --- a/libavdevice/version.h +++ b/libavdevice/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVDEVICE_VERSION_MAJOR 57 -#define LIBAVDEVICE_VERSION_MINOR 7 +#define LIBAVDEVICE_VERSION_MINOR 8 #define LIBAVDEVICE_VERSION_MICRO 100 #define LIBAVDEVICE_VERSION_INT AV_VERSION_INT(LIBAVDEVICE_VERSION_MAJOR, \ -- 1.8.3.1