From patchwork Mon Mar 27 16:47:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40881 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a30:b0:df:834d:2c1a with SMTP id t48csp1485805pzh; Mon, 27 Mar 2023 09:48:16 -0700 (PDT) X-Google-Smtp-Source: AKy350YV0cN4O2dCB3avACdVQUSjlAgvSuid4k/fgtGCS4GvoqPfhfIO874QN5Jp+XDJuPVlYfIi X-Received: by 2002:a17:906:3449:b0:93b:b8f3:225d with SMTP id d9-20020a170906344900b0093bb8f3225dmr12597733ejb.15.1679935696396; Mon, 27 Mar 2023 09:48:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679935696; cv=none; d=google.com; s=arc-20160816; b=sNaB64sdECU+2lqoOM+A4y+Ux/IPYWmqBQVIOo8PImHY+51AAc3MswQHkQi2uApHT3 393IWF7eTWfjwfNwdwk+Kab25t8LjmKf2dEJcc9StfrXaM56Y68wAn91l3DQskX7cfa6 cKQRkouwph9mo8tbusWSnQrcM0ECw5mnr+wMR/6F+Pym5PfYhPt9JX/QiTQ7qIiSTI5a qmR/ddplQpS3M0B/r0eNBVXHOpgifi7U6Ly+geT0MZ8l8DHnURFtkrC5IhuncRMII6yz mF1H9pIy//cQfZgqGjgMX9zya44gFMUD+lo/HroBm8D9kAeFe2YfW+yJ2+2OSCcV5LYX 1OPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=O36W7aGN2kebisaKX5P64792liI5XuZ0fR5JmBsoWkc=; b=MoIhw7/GNPwSN9Wo0Jy3SNAZ02WvNEZ4K6amAzRcFfcty22cNK0uZhBAYjMs0c6LK3 bofTang/Fzwtwiqmp2kOyAXkhlEIksm0bk4X0BpjshT9DQOCG6j54DCpmEbNktq8gC8J Sl9V5FcLZL0cGyBxJ2E3qcXuvcHb/6OlJLb/ym83fi5EllvJ6ibRsIQa7QnjOd+ky0Cc BBRxc1SzcFQ94gLg4qCqMTwBB3UHLVUEucZrNHGJV5MrB2n/NyY2ZsAIAVC38LYQLOSM GVl2YYY6vIt7p7AbJtsB/BK99daA6Pz0bLBEd+sDUXURFPAVjnJo+dKuTeJk8C3n7G7o dzUQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=YcP91Ur2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id kx7-20020a170907774700b009326c8764d5si25268716ejc.424.2023.03.27.09.48.15; Mon, 27 Mar 2023 09:48:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=YcP91Ur2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3EE8468CB69; Mon, 27 Mar 2023 19:47:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9508D68CB61 for ; Mon, 27 Mar 2023 19:47:30 +0300 (EEST) Received: by mail-qt1-f173.google.com with SMTP id t19so9182286qta.12 for ; Mon, 27 Mar 2023 09:47:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679935649; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=3jw0yCUDmThibVFY40UQ8XaTX3svdH5eDT0aM32HHTY=; b=YcP91Ur2ZUnCA3URPO7MLEBj/aOH9sbSpRhgSV6FAUOTb2tT9zw3aa93GS/k4rKkb6 oOrgQSh8E8L5ByOfXdKrKHaKlAE8HerJPC6pOiP78ZshKV1zMvlSnIM9RSv6jDpuKtjr 8+1QYigwSfE13DrUYFZHNrv64HGjcSCzm8QvuGZ722s+AzoCtHnVhLqY3IFKz06/W2VV 8Y9Ye18ewN9t8g9CsnbytYOjJRGLHz1UUJNEd9rg+bN+PVoD27dnWBc7YP6SMVBQuVxO iETo3Q5DkqUYU3BMr14sZDFQmxhKNL7/jjQTJCAXMgxJoknl311U/QAEJjpd7kFtJoKr n2pA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679935649; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3jw0yCUDmThibVFY40UQ8XaTX3svdH5eDT0aM32HHTY=; b=5p/tLOWG7iDQEBrLLDMR17lKDDvKNIohOp3IL/T0KAnPsUApHWrEXjDXUVRNtKI8Js +TnV8hPItRWIC1fiPd2ewyJKwXI34ECx71N1wwqiz/DoNnjwxY19POahukFJZC4r0Dcf CQxV9+eeQq1Wd1wXs6yIwwfZvBiAEH55XmjOdIixFS1ZGwSfh8VZ+ZyC46VHydB1/T9G dmnfwvzdt7UkRIn2XdeyEMRNPg5tPsjrz0ZoU3L5s3aj4mMGd39KNLIkyg1+HJ7fRHB2 gcrDuJw6GfMOLW54Zj8OUWBiIaCER8bBnolLcVqsy9wTZBGe3bGgh1wUaBwt7yn4Erfk UnDQ== X-Gm-Message-State: AO0yUKUDucrbDKArB7hxmUEemJr35Hu0NrGZoFquWcVf+bQwNpF2wGUy 4EA9Ov7MXTey4vRRX7mYA3P6UawYkDHHat4dihk= X-Received: by 2002:a05:622a:1648:b0:3e2:b529:e7f9 with SMTP id y8-20020a05622a164800b003e2b529e7f9mr21222389qtj.4.1679935649105; Mon, 27 Mar 2023 09:47:29 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id f67-20020a37d246000000b007486d9fa1d7sm1015371qkj.95.2023.03.27.09.47.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Mar 2023 09:47:28 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 Mar 2023 12:47:27 -0400 Message-Id: <20230327164727.13001-1-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3] decklink: Add support for compressed AC-3 output over SDI X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: irSUjNupwaMC Extend the decklink output to include support for compressed AC-3, encapsulated using the SMPTE ST 377:2015 standard. This functionality can be exercised by using the "copy" codec when the input audio stream is AC-3. For example: ./ffmpeg -i ~/foo.ts -codec:a copy -f decklink 'UltraStudio Mini Monitor' Note that the default behavior continues to be to do PCM output, which means without specifying the copy codec a stream containing AC-3 will be decoded and downmixed to stereo audio before output. Thanks to Marton Balint for providing feedback. Signed-off-by: Devin Heitmueller --- libavdevice/decklink_enc.cpp | 94 ++++++++++++++++++++++++++++++------ 1 file changed, 79 insertions(+), 15 deletions(-) diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp index 8d423f6b6e..a63aeaa088 100644 --- a/libavdevice/decklink_enc.cpp +++ b/libavdevice/decklink_enc.cpp @@ -32,6 +32,7 @@ extern "C" { extern "C" { #include "libavformat/avformat.h" +#include "libavcodec/bytestream.h" #include "libavutil/internal.h" #include "libavutil/imgutils.h" #include "avdevice.h" @@ -243,19 +244,32 @@ static int decklink_setup_audio(AVFormatContext *avctx, AVStream *st) av_log(avctx, AV_LOG_ERROR, "Only one audio stream is supported!\n"); return -1; } - if (c->sample_rate != 48000) { - av_log(avctx, AV_LOG_ERROR, "Unsupported sample rate!" - " Only 48kHz is supported.\n"); - return -1; - } - if (c->ch_layout.nb_channels != 2 && c->ch_layout.nb_channels != 8 && c->ch_layout.nb_channels != 16) { - av_log(avctx, AV_LOG_ERROR, "Unsupported number of channels!" - " Only 2, 8 or 16 channels are supported.\n"); + + if (c->codec_id == AV_CODEC_ID_AC3) { + /* Regardless of the number of channels in the codec, we're only + using 2 SDI audio channels at 48000Hz */ + ctx->channels = 2; + } else if (c->codec_id == AV_CODEC_ID_PCM_S16LE) { + if (c->sample_rate != 48000) { + av_log(avctx, AV_LOG_ERROR, "Unsupported sample rate!" + " Only 48kHz is supported.\n"); + return -1; + } + if (c->ch_layout.nb_channels != 2 && c->ch_layout.nb_channels != 8 && c->ch_layout.nb_channels != 16) { + av_log(avctx, AV_LOG_ERROR, "Unsupported number of channels!" + " Only 2, 8 or 16 channels are supported.\n"); + return -1; + } + ctx->channels = c->ch_layout.nb_channels; + } else { + av_log(avctx, AV_LOG_ERROR, "Unsupported codec specified!" + " Only PCM_S16LE and AC-3 are supported.\n"); return -1; } + if (ctx->dlo->EnableAudioOutput(bmdAudioSampleRate48kHz, bmdAudioSampleType16bitInteger, - c->ch_layout.nb_channels, + ctx->channels, bmdAudioOutputStreamTimestamped) != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not enable audio output!\n"); return -1; @@ -266,14 +280,45 @@ static int decklink_setup_audio(AVFormatContext *avctx, AVStream *st) } /* The device expects the sample rate to be fixed. */ - avpriv_set_pts_info(st, 64, 1, c->sample_rate); - ctx->channels = c->ch_layout.nb_channels; + avpriv_set_pts_info(st, 64, 1, 48000); ctx->audio = 1; return 0; } +static int create_s337_payload(AVPacket *pkt, enum AVCodecID codec_id, uint8_t **outbuf, int *outsize) +{ + int payload_size = pkt->size + 8; + uint16_t bitcount = pkt->size * 8; + uint8_t *s337_payload; + PutByteContext pb; + + if (codec_id != AV_CODEC_ID_AC3) + return AVERROR(EINVAL); + + /* Sanity check: According to SMPTE ST 340:2015 Sec 4.1, the AC-3 sync frame will + exactly match the 1536 samples of baseband (PCM) audio that it represents. */ + if (pkt->size > 1536) + return AVERROR(EINVAL); + + /* Encapsulate AC3 syncframe into SMPTE 337 packet */ + s337_payload = (uint8_t *) av_malloc(payload_size); + if (s337_payload == NULL) + return AVERROR(ENOMEM); + bytestream2_init_writer(&pb, s337_payload, payload_size); + bytestream2_put_le16u(&pb, 0xf872); /* Sync word 1 */ + bytestream2_put_le16u(&pb, 0x4e1f); /* Sync word 1 */ + bytestream2_put_le16u(&pb, 0x0001); /* Burst Info, including data type (1=ac3) */ + bytestream2_put_le16u(&pb, bitcount); /* Length code */ + for (int i = 0; i < pkt->size; i += 2) + bytestream2_put_le16u(&pb, (pkt->data[i] << 8) | pkt->data[i+1]); + + *outsize = payload_size; + *outbuf = s337_payload; + return 0; +} + av_cold int ff_decklink_write_trailer(AVFormatContext *avctx) { struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; @@ -531,21 +576,40 @@ static int decklink_write_audio_packet(AVFormatContext *avctx, AVPacket *pkt) { struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx; - int sample_count = pkt->size / (ctx->channels << 1); + AVStream *st = avctx->streams[pkt->stream_index]; + int sample_count; uint32_t buffered; + uint8_t *outbuf = NULL; + int ret = 0; ctx->dlo->GetBufferedAudioSampleFrameCount(&buffered); if (pkt->pts > 1 && !buffered) av_log(avctx, AV_LOG_WARNING, "There's no buffered audio." " Audio will misbehave!\n"); - if (ctx->dlo->ScheduleAudioSamples(pkt->data, sample_count, pkt->pts, + if (st->codecpar->codec_id == AV_CODEC_ID_AC3) { + /* Encapsulate AC3 syncframe into SMPTE 337 packet */ + int outbuf_size; + ret = create_s337_payload(pkt, st->codecpar->codec_id, + &outbuf, &outbuf_size); + if (ret < 0) + return ret; + sample_count = outbuf_size / 4; + } else { + sample_count = pkt->size / (ctx->channels << 1); + outbuf = pkt->data; + } + + if (ctx->dlo->ScheduleAudioSamples(outbuf, sample_count, pkt->pts, bmdAudioSampleRate48kHz, NULL) != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not schedule audio samples.\n"); - return AVERROR(EIO); + ret = AVERROR(EIO); } - return 0; + if (st->codecpar->codec_id == AV_CODEC_ID_AC3) + av_freep(&outbuf); + + return ret; } extern "C" {