From patchwork Tue Sep 5 07:09:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maksym Veremeyenko X-Patchwork-Id: 4988 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.15.201 with SMTP id 70csp3827720jao; Tue, 5 Sep 2017 00:10:03 -0700 (PDT) X-Google-Smtp-Source: ADKCNb5lYEZE2zi4v2osXptLSALv4TgyDWsQ04UwS+HZoT60cbOJTYqoomWsKZclFuc5xQgX4jHV X-Received: by 10.223.129.76 with SMTP id 70mr1683578wrm.65.1504595403520; Tue, 05 Sep 2017 00:10:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1504595403; cv=none; d=google.com; s=arc-20160816; b=N9VddwpJp/DGEh8BaBs4BqK51pW3oeCa703BiundedKVZtboMfuCUL/hog0Ybj9T7n pCaeqKiveXlYBDGQKoyg6gy6+6mHZo09/8bpleqsUEGeDLEr3n8NFwk9VT/7KIEcocoG CITAwqQM9DYLZ2DyQ1T2Ri6Z59Woh+C1TPsDTfoLJftYJfgqDgErlG/PLTFPWBk1Xt6o bZdaPstebJZi/hw9Q0Yo+wQPjJtHfy1mCD1R4837EEQ8SDQP7xgbpIqGHqZjcHtFWhE8 88xpgVG8enMbHO5TKLCGNdi8/7O9tSCSspKlkl7suWeFzIChjBhDjWHXdS4ZubzMmYC1 fLZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:to:delivered-to :arc-authentication-results; bh=bMaWbxFuEMXXVOO3JbvwjBZ27l3at+vBaCYgrDWyuDc=; b=xLn4kf/C5VVsUdHLvIFEuLy8yueocP4VlKda+SLk92W6tjYqQScCw38SRuJzLN6GZZ rVBgN9/wTagkea63NdiH0YwskvBJpROjItRYuB0ykhBxOFSeqG7dtHq+M80yJpDA56PV s3KYnSkppa7o/NE4RYP6hHU3YgtdruaP2KlihyUxYdiZIvlimjFU03hKubPcd8In04bo Z1/K+b44lDw6s3EiFAsr9Mjq22Y+yuZZlGK3MpW3hYQtGxZKOkfsy5codcBZcQEkQr/F i+ZWgnzx4vrSwgVXhS32ZT0NqhOm6HPJ2TH4XmKk6h27l1N5EAlYZXIHYZnppqURhfB9 b5qg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l4si1467wml.66.2017.09.05.00.10.02; Tue, 05 Sep 2017 00:10:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BB3CD68A339; Tue, 5 Sep 2017 10:09:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from kazbek.m1stereo.tv (mail.m1stereo.tv [91.244.124.37]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E623C68A2AC for ; Tue, 5 Sep 2017 10:09:52 +0300 (EEST) Received: from [10.1.5.65] (dev-3.internal.m1stereo.tv [10.1.5.65]) by kazbek.m1stereo.tv (8.14.4/8.14.4) with ESMTP id v8579stQ001187 for ; Tue, 5 Sep 2017 10:09:55 +0300 To: ffmpeg-devel@ffmpeg.org References: From: Maksym Veremeyenko Message-ID: Date: Tue, 5 Sep 2017 10:09:54 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: Re: [FFmpeg-devel] [PATCH] lavd: implement threaded NewTek NDI output X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" 04.09.2017 17:10, Maksym Veremeyenko пише: > Hi, > > attached patch implemented threaded NDI output - separate output thread > for each stream. it makes audio preview in my case more smooth. updated patch allows running audio/video threads separately please review diff --git a/configure b/configure index d582705..7626901 100755 --- a/configure +++ b/configure @@ -3019,7 +3019,7 @@ decklink_outdev_deps="decklink threads" decklink_outdev_extralibs="-lstdc++" libndi_newtek_indev_deps="libndi_newtek" libndi_newtek_indev_extralibs="-lndi" -libndi_newtek_outdev_deps="libndi_newtek" +libndi_newtek_outdev_deps="libndi_newtek threads" libndi_newtek_outdev_extralibs="-lndi" dshow_indev_deps="IBaseFilter" dshow_indev_extralibs="-lpsapi -lole32 -lstrmiids -luuid -loleaut32 -lshlwapi" diff --git a/doc/outdevs.texi b/doc/outdevs.texi index 0012b0f..595864b 100644 --- a/doc/outdevs.texi +++ b/doc/outdevs.texi @@ -213,6 +213,14 @@ Defaults to @option{false}. These specify whether audio "clock" themselves. Defaults to @option{false}. +@item video_queue +Enable video packets output in separate thread. Specify video packets queue length. +Defaults to @option{0}. + +@item audio_queue +Enable audio packets output in separate thread. Specify audio packets queue length. +Defaults to @option{0}. + @end table @subsection Examples diff --git a/libavdevice/libndi_newtek_enc.c b/libavdevice/libndi_newtek_enc.c index 6ca6f41..f8af851 100644 --- a/libavdevice/libndi_newtek_enc.c +++ b/libavdevice/libndi_newtek_enc.c @@ -23,9 +23,16 @@ #include "libavformat/internal.h" #include "libavutil/opt.h" #include "libavutil/imgutils.h" +#include "libavutil/threadmessage.h" #include "libndi_newtek_common.h" +#include + +#define THREAD_VIDEO 0 +#define THREAD_AUDIO 1 +#define THREAD_LAST 2 + struct NDIContext { const AVClass *cclass; @@ -37,12 +44,104 @@ struct NDIContext { NDIlib_audio_frame_interleaved_16s_t *audio; NDIlib_send_instance_t ndi_send; AVFrame *last_avframe; + + /* threaded operations */ + AVFormatContext *avctx; + struct + { + int length; + AVThreadMessageQueue *queue; + pthread_t thread; + } threads[THREAD_LAST]; }; +static int ndi_write_video_packet(AVFormatContext *avctx, AVStream *st, AVPacket *pkt); +static int ndi_write_audio_packet(AVFormatContext *avctx, AVStream *st, AVPacket *pkt); + +static void* ndi_thread_audio(void* p) +{ + int ret; + AVPacket pkt; + struct NDIContext *ctx = p; + + av_log(ctx->avctx, AV_LOG_DEBUG, "%s: entering\n", __func__); + + while (1) { + ret = av_thread_message_queue_recv(ctx->threads[THREAD_AUDIO].queue, &pkt, 0); + if (ret < 0) { + if (ret != AVERROR_EOF) + av_log(ctx->avctx, AV_LOG_ERROR, "Failed av_thread_message_queue_recv of audio queue.\n"); + break; + } + + ret = ndi_write_audio_packet(ctx->avctx, ctx->avctx->streams[pkt.stream_index], &pkt); + av_packet_unref(&pkt); + if (ret) { + av_log(ctx->avctx, AV_LOG_ERROR, "Failed ndi_write_audio_packet.\n"); + break; + } + } + + av_log(ctx->avctx, AV_LOG_DEBUG, "%s: exiting, ret=%d\n", __func__, ret); + + return NULL; +} + +static void* ndi_thread_video(void* p) +{ + int ret; + AVPacket pkt; + struct NDIContext *ctx = p; + + av_log(ctx->avctx, AV_LOG_DEBUG, "%s: entering\n", __func__); + + while (1) { + ret = av_thread_message_queue_recv(ctx->threads[THREAD_VIDEO].queue, &pkt, 0); + if (ret < 0) { + if (ret != AVERROR_EOF) + av_log(ctx->avctx, AV_LOG_ERROR, "Failed av_thread_message_queue_recv of video queue.\n"); + break; + } + + ret = ndi_write_video_packet(ctx->avctx, ctx->avctx->streams[pkt.stream_index], &pkt); + av_packet_unref(&pkt); + if (ret) { + av_log(ctx->avctx, AV_LOG_ERROR, "Failed ndi_write_video_packet.\n"); + break; + } + } + + av_log(ctx->avctx, AV_LOG_DEBUG, "%s: exiting, ret=%d\n", __func__, ret); + + return NULL; +} + static int ndi_write_trailer(AVFormatContext *avctx) { + int i; struct NDIContext *ctx = avctx->priv_data; + for (i = 0; i < THREAD_LAST; i++) + { + AVPacket pkt; + + if (!ctx->threads[i].queue) + continue; + + av_log(ctx->avctx, AV_LOG_DEBUG, "%s: freeing queue %d\n", __func__, i); + + av_thread_message_queue_set_err_recv(ctx->threads[i].queue, AVERROR_EOF); + + pthread_join(ctx->threads[i].thread, NULL); + + while (av_thread_message_queue_recv(ctx->threads[i].queue, &pkt, 0) >= 0) { + av_log(ctx->avctx, AV_LOG_DEBUG, "%s: freeing packet queue %d\n", __func__, i); + av_packet_unref(&pkt); + } + + av_thread_message_queue_free(&ctx->threads[i].queue); + } + if (ctx->ndi_send) { NDIlib_send_destroy(ctx->ndi_send); av_frame_free(&ctx->last_avframe); @@ -119,18 +218,47 @@ static int ndi_write_audio_packet(AVFormatContext *avctx, AVStream *st, AVPacket static int ndi_write_packet(AVFormatContext *avctx, AVPacket *pkt) { + struct NDIContext *ctx = avctx->priv_data; AVStream *st = avctx->streams[pkt->stream_index]; + AVThreadMessageQueue *queue; if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) - return ndi_write_video_packet(avctx, st, pkt); + queue = ctx->threads[THREAD_VIDEO].queue; else if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) - return ndi_write_audio_packet(avctx, st, pkt); + queue = ctx->threads[THREAD_AUDIO].queue; + + if (queue) { + + int ret; + AVPacket enq; + + av_init_packet(&enq); + + ret = av_packet_ref(&enq, pkt); + if (ret) + return ret; + + ret = av_thread_message_queue_send(queue, &enq, 0); + if (ret) { + av_packet_unref(&enq); + return ret; + } + + return 0; + + } else { + if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) + return ndi_write_video_packet(avctx, st, pkt); + else if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) + return ndi_write_audio_packet(avctx, st, pkt); + } return AVERROR_BUG; } static int ndi_setup_audio(AVFormatContext *avctx, AVStream *st) { + int ret; struct NDIContext *ctx = avctx->priv_data; AVCodecParameters *c = st->codecpar; @@ -149,11 +277,30 @@ static int ndi_setup_audio(AVFormatContext *avctx, AVStream *st) avpriv_set_pts_info(st, 64, 1, NDI_TIME_BASE); + if (ctx->threads[THREAD_AUDIO].length) { + ret = av_thread_message_queue_alloc(&ctx->threads[THREAD_AUDIO].queue, + ctx->threads[THREAD_AUDIO].length, sizeof(AVPacket)); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "Failed to av_thread_message_queue_alloc!\n"); + return ret; + } + + ret = pthread_create(&ctx->threads[THREAD_AUDIO].thread, NULL, ndi_thread_audio, ctx); + if (ret) { + av_log(NULL, AV_LOG_ERROR, "Failed to pthread_create: %s\n", strerror(ret)); + av_thread_message_queue_free(&ctx->threads[THREAD_AUDIO].queue); + return AVERROR(ret); + } + + ctx->avctx = avctx; + } + return 0; } static int ndi_setup_video(AVFormatContext *avctx, AVStream *st) { + int ret; struct NDIContext *ctx = avctx->priv_data; AVCodecParameters *c = st->codecpar; @@ -225,6 +372,24 @@ static int ndi_setup_video(AVFormatContext *avctx, AVStream *st) avpriv_set_pts_info(st, 64, 1, NDI_TIME_BASE); + if (ctx->threads[THREAD_VIDEO].length) { + ret = av_thread_message_queue_alloc(&ctx->threads[THREAD_VIDEO].queue, + ctx->threads[THREAD_VIDEO].length, sizeof(AVPacket)); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "Failed to av_thread_message_queue_alloc!\n"); + return ret; + } + + ret = pthread_create(&ctx->threads[THREAD_VIDEO].thread, NULL, ndi_thread_video, ctx); + if (ret) { + av_log(NULL, AV_LOG_ERROR, "Failed to pthread_create: %s\n", strerror(ret)); + av_thread_message_queue_free(&ctx->threads[THREAD_VIDEO].queue); + return AVERROR(ret); + } + + ctx->avctx = avctx; + } + return 0; } @@ -273,6 +438,8 @@ static const AVOption options[] = { { "reference_level", "The audio reference level in dB" , OFFSET(reference_level), AV_OPT_TYPE_INT, { .i64 = 0 }, -20, 20, AV_OPT_FLAG_ENCODING_PARAM | AV_OPT_FLAG_AUDIO_PARAM}, { "clock_video", "These specify whether video 'clock' themselves" , OFFSET(clock_video), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, AV_OPT_FLAG_ENCODING_PARAM | AV_OPT_FLAG_VIDEO_PARAM }, { "clock_audio", "These specify whether audio 'clock' themselves" , OFFSET(clock_audio), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, AV_OPT_FLAG_ENCODING_PARAM | AV_OPT_FLAG_AUDIO_PARAM }, + { "video_queue", "Video queue length", OFFSET(threads[THREAD_VIDEO].length), AV_OPT_TYPE_INT, { .i64 = 0 }, 1, 128, AV_OPT_FLAG_ENCODING_PARAM | AV_OPT_FLAG_VIDEO_PARAM}, + { "audio_queue", "Audio queue length", OFFSET(threads[THREAD_AUDIO].length), AV_OPT_TYPE_INT, { .i64 = 0 }, 1, 128, AV_OPT_FLAG_ENCODING_PARAM | AV_OPT_FLAG_AUDIO_PARAM}, { NULL }, };