From patchwork Mon Mar 27 16:08:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40875 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a30:b0:df:834d:2c1a with SMTP id t48csp1455534pzh; Mon, 27 Mar 2023 09:09:27 -0700 (PDT) X-Google-Smtp-Source: AKy350ZQ9SLsqEHH/3YaOye4FKOsyWF9qy3FOGd5jDg1Iu9F2Sk1JP8waJijwzUwncxAZlx/3XBs X-Received: by 2002:a17:906:cecf:b0:932:2412:8abf with SMTP id si15-20020a170906cecf00b0093224128abfmr13649219ejb.62.1679933367236; Mon, 27 Mar 2023 09:09:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679933367; cv=none; d=google.com; s=arc-20160816; b=YlPza+eE7ROj0HTU5XS8r10/qRD2QXge8XZHoNA9AaSi1Is8SDklCaVS5s5m12kdLw Q0wQFcOzYGEupsWq6zWyYXJPdbbkXiFwv+18/Da31ELV0+ZGH8xMJgaZznsLCEHo5sDU EfdZPHM9ocmyN8ADouRGPS9KjAGeCvACRUiIUwdwAgeGUBdoGNuGGwx76JKzZ0RCKlOA G5HIfx975w46N59WS/Sc6P/ywszp/xGOST+n/3APWhNI5NnIZ05eWmnzh1u5Xc5gJjoV Ssxk7iPBSuRga0TeRdpnO4Ri7bEnYgwKqxdmQBCVehTo4wGwGOyh4rTqZKh2SqLW5eWE n0Ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=O36W7aGN2kebisaKX5P64792liI5XuZ0fR5JmBsoWkc=; b=PD9PzXa1vR7H0M3DBEac390RXx8/y1pjhv+ZmLEKf2YDmY3C8hI+SI3d7xC/em+x/q fNH7BEEJkmyiLZN0om9NyfIIv+/efsjkbbD+XQCZAUVz1hHMLmnj3NYKl6fOLHqEwoQU NdJZBuOmwxamAMq4tdnZNfa9VNaLoGJ7XgtxnOio3qCFkI3mFODJt5pOCY5qd6cD5f/y 7dcrKyTh5freUmkRYL9Ok3jSMozvtwjwvwj18BQeEmVGNoP4kFl22i6al8XRx83gfhMd lwWoV1FBgDDmbLx3X47Dn+ytuo/iMsr5lxGlqvuYtP862O10Ssb4rQOKFrBjDDep0pAt NMtw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=EvHSc4V7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id fi12-20020a170906da0c00b009307879aebfsi25746797ejb.415.2023.03.27.09.09.26; Mon, 27 Mar 2023 09:09:27 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=EvHSc4V7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 30FA568CAD6; Mon, 27 Mar 2023 19:09:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f52.google.com (mail-qv1-f52.google.com [209.85.219.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DAF8B68C4C4 for ; Mon, 27 Mar 2023 19:09:07 +0300 (EEST) Received: by mail-qv1-f52.google.com with SMTP id x8so7045322qvr.9 for ; Mon, 27 Mar 2023 09:09:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679933346; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=3jw0yCUDmThibVFY40UQ8XaTX3svdH5eDT0aM32HHTY=; b=EvHSc4V7S6J+WLZSePMCV1ez9NANmcWm/TpK9JHaccoxj6aL2RZXHu2m+G8Q3vg2oz utVeQoV22Lq+g/tcCHMV9Ri4Lo8Fo65uIX1Qy5NkHsouXX1m0ghDXuGbgW0XQTROC6ui tMwSqscEZnLQYarE2kzSPspFD6wjko3i9W6tyKwHg1Q/hRpgGvKZadjDlVZese6Se9Bo yANMag0WcaAIA+dRrD1iHTY3S2FpvAGReDgZN8pcIc2wqTydDNstdVuialNNosshRxEd 6kPzXbXvnMjtA3lx6M5thhZkR5q0E3Z/8ywB9jNxYR7Yy4i+OqcahPVgP7r+e81Wu3f+ FKrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679933346; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3jw0yCUDmThibVFY40UQ8XaTX3svdH5eDT0aM32HHTY=; b=tnLdwC4EAEWGQN1sRg1p+znygAhxlPk4dCmvH62IuWXtvVlgNwDOKSpKncxl94715V XqVraTbE26wugvD0fEHcDss74VfmgSTgaAGMPrBzIC0Z5p/RXNWfTgiMVfqUu9dhuIDb zDc52Ond4kLT85QaMY5VfGZF0WSu47KwusrRDXguXqbAsTjEjSqjMXWKIa0j2VfpdkbU 0EJNEWsCVPGbdLrQIZPcHNtmRANvSia01x9sHGgT9MHNKp7GwOdOuEP9zQof4MuEA6ui PQsFypTQB9QRH1P3UH4+6gLjR7Vr+5SSUIZil9oehlXhzbr9nSmP6FXjtqKw5juyaB/o 7HrQ== X-Gm-Message-State: AAQBX9dFCGuHwiSxN1IX8PLhAYeTCSpwMx6effM2+F371phTi3F08frg bCyphNcVy8b9of+Q0LAqSrSAEuYtJK3Kd1PxQpo= X-Received: by 2002:ad4:5be2:0:b0:56a:d94d:6deb with SMTP id k2-20020ad45be2000000b0056ad94d6debmr17179863qvc.25.1679933346200; Mon, 27 Mar 2023 09:09:06 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id s188-20020ae9dec5000000b00742a23cada8sm16653009qkf.131.2023.03.27.09.09.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Mar 2023 09:09:01 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 Mar 2023 12:08:57 -0400 Message-Id: <20230327160857.3653-1-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3] decklink: Add support for compressed AC-3 output over SDI X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IJWqRqUdxXs5 Extend the decklink output to include support for compressed AC-3, encapsulated using the SMPTE ST 377:2015 standard. This functionality can be exercised by using the "copy" codec when the input audio stream is AC-3. For example: ./ffmpeg -i ~/foo.ts -codec:a copy -f decklink 'UltraStudio Mini Monitor' Note that the default behavior continues to be to do PCM output, which means without specifying the copy codec a stream containing AC-3 will be decoded and downmixed to stereo audio before output. Thanks to Marton Balint for providing feedback. Signed-off-by: Devin Heitmueller --- libavdevice/decklink_enc.cpp | 94 ++++++++++++++++++++++++++++++------ 1 file changed, 79 insertions(+), 15 deletions(-) diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp index 8d423f6b6e..a63aeaa088 100644 --- a/libavdevice/decklink_enc.cpp +++ b/libavdevice/decklink_enc.cpp @@ -32,6 +32,7 @@ extern "C" { extern "C" { #include "libavformat/avformat.h" +#include "libavcodec/bytestream.h" #include "libavutil/internal.h" #include "libavutil/imgutils.h" #include "avdevice.h" @@ -243,19 +244,32 @@ static int decklink_setup_audio(AVFormatContext *avctx, AVStream *st) av_log(avctx, AV_LOG_ERROR, "Only one audio stream is supported!\n"); return -1; } - if (c->sample_rate != 48000) { - av_log(avctx, AV_LOG_ERROR, "Unsupported sample rate!" - " Only 48kHz is supported.\n"); - return -1; - } - if (c->ch_layout.nb_channels != 2 && c->ch_layout.nb_channels != 8 && c->ch_layout.nb_channels != 16) { - av_log(avctx, AV_LOG_ERROR, "Unsupported number of channels!" - " Only 2, 8 or 16 channels are supported.\n"); + + if (c->codec_id == AV_CODEC_ID_AC3) { + /* Regardless of the number of channels in the codec, we're only + using 2 SDI audio channels at 48000Hz */ + ctx->channels = 2; + } else if (c->codec_id == AV_CODEC_ID_PCM_S16LE) { + if (c->sample_rate != 48000) { + av_log(avctx, AV_LOG_ERROR, "Unsupported sample rate!" + " Only 48kHz is supported.\n"); + return -1; + } + if (c->ch_layout.nb_channels != 2 && c->ch_layout.nb_channels != 8 && c->ch_layout.nb_channels != 16) { + av_log(avctx, AV_LOG_ERROR, "Unsupported number of channels!" + " Only 2, 8 or 16 channels are supported.\n"); + return -1; + } + ctx->channels = c->ch_layout.nb_channels; + } else { + av_log(avctx, AV_LOG_ERROR, "Unsupported codec specified!" + " Only PCM_S16LE and AC-3 are supported.\n"); return -1; } + if (ctx->dlo->EnableAudioOutput(bmdAudioSampleRate48kHz, bmdAudioSampleType16bitInteger, - c->ch_layout.nb_channels, + ctx->channels, bmdAudioOutputStreamTimestamped) != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not enable audio output!\n"); return -1; @@ -266,14 +280,45 @@ static int decklink_setup_audio(AVFormatContext *avctx, AVStream *st) } /* The device expects the sample rate to be fixed. */ - avpriv_set_pts_info(st, 64, 1, c->sample_rate); - ctx->channels = c->ch_layout.nb_channels; + avpriv_set_pts_info(st, 64, 1, 48000); ctx->audio = 1; return 0; } +static int create_s337_payload(AVPacket *pkt, enum AVCodecID codec_id, uint8_t **outbuf, int *outsize) +{ + int payload_size = pkt->size + 8; + uint16_t bitcount = pkt->size * 8; + uint8_t *s337_payload; + PutByteContext pb; + + if (codec_id != AV_CODEC_ID_AC3) + return AVERROR(EINVAL); + + /* Sanity check: According to SMPTE ST 340:2015 Sec 4.1, the AC-3 sync frame will + exactly match the 1536 samples of baseband (PCM) audio that it represents. */ + if (pkt->size > 1536) + return AVERROR(EINVAL); + + /* Encapsulate AC3 syncframe into SMPTE 337 packet */ + s337_payload = (uint8_t *) av_malloc(payload_size); + if (s337_payload == NULL) + return AVERROR(ENOMEM); + bytestream2_init_writer(&pb, s337_payload, payload_size); + bytestream2_put_le16u(&pb, 0xf872); /* Sync word 1 */ + bytestream2_put_le16u(&pb, 0x4e1f); /* Sync word 1 */ + bytestream2_put_le16u(&pb, 0x0001); /* Burst Info, including data type (1=ac3) */ + bytestream2_put_le16u(&pb, bitcount); /* Length code */ + for (int i = 0; i < pkt->size; i += 2) + bytestream2_put_le16u(&pb, (pkt->data[i] << 8) | pkt->data[i+1]); + + *outsize = payload_size; + *outbuf = s337_payload; + return 0; +} + av_cold int ff_decklink_write_trailer(AVFormatContext *avctx) { struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; @@ -531,21 +576,40 @@ static int decklink_write_audio_packet(AVFormatContext *avctx, AVPacket *pkt) { struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx; - int sample_count = pkt->size / (ctx->channels << 1); + AVStream *st = avctx->streams[pkt->stream_index]; + int sample_count; uint32_t buffered; + uint8_t *outbuf = NULL; + int ret = 0; ctx->dlo->GetBufferedAudioSampleFrameCount(&buffered); if (pkt->pts > 1 && !buffered) av_log(avctx, AV_LOG_WARNING, "There's no buffered audio." " Audio will misbehave!\n"); - if (ctx->dlo->ScheduleAudioSamples(pkt->data, sample_count, pkt->pts, + if (st->codecpar->codec_id == AV_CODEC_ID_AC3) { + /* Encapsulate AC3 syncframe into SMPTE 337 packet */ + int outbuf_size; + ret = create_s337_payload(pkt, st->codecpar->codec_id, + &outbuf, &outbuf_size); + if (ret < 0) + return ret; + sample_count = outbuf_size / 4; + } else { + sample_count = pkt->size / (ctx->channels << 1); + outbuf = pkt->data; + } + + if (ctx->dlo->ScheduleAudioSamples(outbuf, sample_count, pkt->pts, bmdAudioSampleRate48kHz, NULL) != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not schedule audio samples.\n"); - return AVERROR(EIO); + ret = AVERROR(EIO); } - return 0; + if (st->codecpar->codec_id == AV_CODEC_ID_AC3) + av_freep(&outbuf); + + return ret; } extern "C" {