From patchwork Fri Apr 7 21:36:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41016 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp618071pzb; Fri, 7 Apr 2023 13:40:57 -0700 (PDT) X-Google-Smtp-Source: AKy350ZpjfRbsnKswXWtVtQ2u7I7nTC4g0znnb6Shzn0+P3b+Zd1+unJHgDms4qojmuXp9PAqvvh X-Received: by 2002:a17:907:3fa3:b0:947:46e0:9e4f with SMTP id hr35-20020a1709073fa300b0094746e09e4fmr854008ejc.3.1680900057487; Fri, 07 Apr 2023 13:40:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680900057; cv=none; d=google.com; s=arc-20160816; b=QCfS67Iz6+tgFZPGCM8Jm/AgVuu38RzDUJ2JZzs7OcM9/Fd18Suelqiuw6Aja5P7Z9 iLDHxoxORJrquVgHgYkBIWAcv+ISBGvjr04g5ECq8GdhYKcdavQN4OaeeWlGCb+vKo7r bmwTqg09p/0iGwqAb/iCcERTLmYm+wnGomCniNFn/ozYiVv21ej4ZaK4wHvyekx55nrD NnAiPCIk7GWtKwWuoXk20cpYgolI1c/m8e0e1/RCRwDnPaL/K0hxiLRS6JIzK3O3SUar OnFwYqEZxexpdY/kaW+5s4j0Zbuxe1bzuC4DzRyhonQfGAosC0GkqS0WN48azLzf2B54 Zx3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to; bh=eCatiR+iv/P9DY7RSK38S/2flr4h2J3Wp+v220rqwC4=; b=SgZZq2jDVG6SjXrvuLB92W8RS/TrdjMITwMefBfems1o/ocF0hoDURhvc+5dJdUoTv TXxqBMShn2jcibAkf366PX4lCmilqeWd1j6I9vQXkeQ/hiarsOxp7PzWb6oN5ObNMmCa o7x+fTsv5IYtfmt9PYz7os2ls6h1zeiiGvT1iil1oWaXZ6Z8yj018rQg63X9ShUXgYUk qxpApw5i0JStVsYCrrpoO7q+eP/r5E0HSryYaxEwXL0nWD5IYvRHb01a90YB35/mPg1m 6WNT+NdWZ4KkTaZ7xtfkeZDrEebVhqg1CuvpVEmncciOFDxKBG5uqI5dFWLs0PrVNAVM VoCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=2M623rIq; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p12-20020a170906838c00b0093f87005ba7si3530134ejx.95.2023.04.07.13.40.56; Fri, 07 Apr 2023 13:40:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=2M623rIq; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 385EB68B5E6; Fri, 7 Apr 2023 23:40:54 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f52.google.com (mail-qv1-f52.google.com [209.85.219.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1B3556806EF for ; Fri, 7 Apr 2023 23:40:48 +0300 (EEST) Received: by mail-qv1-f52.google.com with SMTP id x8so30871293qvr.9 for ; Fri, 07 Apr 2023 13:40:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1680900046; x=1683492046; h=message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Y/yuzvHnBEv3f9SkQ6fZCuOQMayVzM61IcHZuN0EFLw=; b=2M623rIqa5P0sL9HPGwMnTOEi4ZdIeSXkErtSdxihMFLvZdQ6i/nYhrDOoNw59vRva CGXz5CgIaJMFOXWOVf/f9HQqpU0oLnPZeg6Y8JrQUHtPrNIc3SBG6lj5qHzyklif5Oix t34GSqhhYUvCG59jLZAnYrmgq5Tj6oL9Lok7y8z6lZFVTtODL5/EKRCjyE/LafqtSpFA 8xtKhcNIXuZQTtQW6LEHaLtL7dF+fQxzfPvT33S3YImGvxlVJ6EotJg60AOdpC8sTBuE x4OGwkQTpUE52Ls2q3He7J7MW3BxjuohyoN/Pa0rNNgd7xuMy5w1dSmLv6kt6oj4EI6R pK+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680900046; x=1683492046; h=message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y/yuzvHnBEv3f9SkQ6fZCuOQMayVzM61IcHZuN0EFLw=; b=OkVhGH7bd4nby92cgzAudpyTXCz0VFbGusJNK7vFBmjvZze7R9SkzHa5ZnzboIIlaI HPHdhLaCgRqHIucQIHwapp5w4FMZJnawOiIV9eUB0wjc01ZkfW88CpEY0RLKBj0P1P+c cjcsgy+2RCG37FpQdUX6Vubf6CpAXQHQh7zkEA0xYlu2lmWyvZSC8EgB/Zmb7EJVK1OG PfDxBMTIhU4FNryM9d+hHxsxkTypwExB4z7svtUKv10gyvjVhIN3Ck4/wmyXjLe0matV hHugZ/crzUedsyqm+OcCbV87tfersINPOLwVZx8/MJsFzb4ti9vjFRBIZmKIOjqcnJpK O/gQ== X-Gm-Message-State: AAQBX9d+mE/8eqfRiojqA9/Ui7dW/qI9ErL3VUGwZKMeMxrCwd5jt0fh u5gTxuh0ZdyUG1CVkm4fejE0gfnuqzi6pEd2hSs= X-Received: by 2002:a05:6214:1c0c:b0:5ae:419a:8c22 with SMTP id u12-20020a0562141c0c00b005ae419a8c22mr6844941qvc.5.1680900046521; Fri, 07 Apr 2023 13:40:46 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id ny15-20020a056214398f00b005dd8b93459bsm1599836qvb.51.2023.04.07.13.40.45 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Apr 2023 13:40:46 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 7 Apr 2023 17:36:14 -0400 Message-Id: <1680903374-11537-1-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 Subject: [FFmpeg-devel] [PATCH v6] avdevice/decklink_enc: Add support for compressed AC-3 output over SDI X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: c075QH9pDWyP Extend the decklink output to include support for compressed AC-3, encapsulated using the SMPTE ST 377:2015 standard. This functionality can be exercised by using the "copy" codec when the input audio stream is AC-3. For example: ./ffmpeg -i ~/foo.ts -codec:a copy -f decklink 'UltraStudio Mini Monitor' Note that the default behavior continues to be to do PCM output, which means without specifying the copy codec a stream containing AC-3 will be decoded and downmixed to stereo audio before output. Thanks to Marton Balint for providing feedback. Signed-off-by: Devin Heitmueller --- libavdevice/decklink_enc.cpp | 100 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 85 insertions(+), 15 deletions(-) diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp index 62676ea..92bfdb2 100644 --- a/libavdevice/decklink_enc.cpp +++ b/libavdevice/decklink_enc.cpp @@ -32,6 +32,7 @@ extern "C" { extern "C" { #include "libavformat/avformat.h" +#include "libavcodec/bytestream.h" #include "libavutil/internal.h" #include "libavutil/imgutils.h" #include "avdevice.h" @@ -243,19 +244,32 @@ static int decklink_setup_audio(AVFormatContext *avctx, AVStream *st) av_log(avctx, AV_LOG_ERROR, "Only one audio stream is supported!\n"); return -1; } - if (c->sample_rate != 48000) { - av_log(avctx, AV_LOG_ERROR, "Unsupported sample rate!" - " Only 48kHz is supported.\n"); - return -1; - } - if (c->ch_layout.nb_channels != 2 && c->ch_layout.nb_channels != 8 && c->ch_layout.nb_channels != 16) { - av_log(avctx, AV_LOG_ERROR, "Unsupported number of channels!" - " Only 2, 8 or 16 channels are supported.\n"); + + if (c->codec_id == AV_CODEC_ID_AC3) { + /* Regardless of the number of channels in the codec, we're only + using 2 SDI audio channels at 48000Hz */ + ctx->channels = 2; + } else if (c->codec_id == AV_CODEC_ID_PCM_S16LE) { + if (c->sample_rate != 48000) { + av_log(avctx, AV_LOG_ERROR, "Unsupported sample rate!" + " Only 48kHz is supported.\n"); + return -1; + } + if (c->ch_layout.nb_channels != 2 && c->ch_layout.nb_channels != 8 && c->ch_layout.nb_channels != 16) { + av_log(avctx, AV_LOG_ERROR, "Unsupported number of channels!" + " Only 2, 8 or 16 channels are supported.\n"); + return -1; + } + ctx->channels = c->ch_layout.nb_channels; + } else { + av_log(avctx, AV_LOG_ERROR, "Unsupported codec specified!" + " Only PCM_S16LE and AC-3 are supported.\n"); return -1; } + if (ctx->dlo->EnableAudioOutput(bmdAudioSampleRate48kHz, bmdAudioSampleType16bitInteger, - c->ch_layout.nb_channels, + ctx->channels, bmdAudioOutputStreamTimestamped) != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not enable audio output!\n"); return -1; @@ -266,14 +280,52 @@ static int decklink_setup_audio(AVFormatContext *avctx, AVStream *st) } /* The device expects the sample rate to be fixed. */ - avpriv_set_pts_info(st, 64, 1, c->sample_rate); - ctx->channels = c->ch_layout.nb_channels; + avpriv_set_pts_info(st, 64, 1, 48000); ctx->audio = 1; return 0; } +/* Wrap the AC-3 packet into an S337 payload that is in S16LE format which can be easily + injected into the PCM stream. Note: despite the function name, only AC-3 is implemented */ +static int create_s337_payload(AVPacket *pkt, uint8_t **outbuf, int *outsize) +{ + /* Note: if the packet size is not divisible by four, we need to make the actual + payload larger to ensure it ends on an two channel S16LE boundary */ + int payload_size = FFALIGN(pkt->size, 4) + 8; + uint16_t bitcount = pkt->size * 8; + uint8_t *s337_payload; + PutByteContext pb; + + /* Sanity check: According to SMPTE ST 340:2015 Sec 4.1, the AC-3 sync frame will + exactly match the 1536 samples of baseband (PCM) audio that it represents. */ + if (pkt->size > 1536) + return AVERROR(EINVAL); + + /* Encapsulate AC3 syncframe into SMPTE 337 packet */ + s337_payload = (uint8_t *) av_malloc(payload_size); + if (s337_payload == NULL) + return AVERROR(ENOMEM); + bytestream2_init_writer(&pb, s337_payload, payload_size); + bytestream2_put_le16u(&pb, 0xf872); /* Sync word 1 */ + bytestream2_put_le16u(&pb, 0x4e1f); /* Sync word 1 */ + bytestream2_put_le16u(&pb, 0x0001); /* Burst Info, including data type (1=ac3) */ + bytestream2_put_le16u(&pb, bitcount); /* Length code */ + for (int i = 0; i < (pkt->size - 1); i += 2) + bytestream2_put_le16u(&pb, (pkt->data[i] << 8) | pkt->data[i+1]); + + /* Ensure final payload is aligned on 4-byte boundary */ + if (pkt->size & 1) + bytestream2_put_le16u(&pb, pkt->data[pkt->size - 1] << 8); + if ((pkt->size & 3 == 1) || (pkt->size & 3 == 2)) + bytestream2_put_le16u(&pb, 0); + + *outsize = payload_size; + *outbuf = s337_payload; + return 0; +} + av_cold int ff_decklink_write_trailer(AVFormatContext *avctx) { struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; @@ -617,21 +669,39 @@ static int decklink_write_audio_packet(AVFormatContext *avctx, AVPacket *pkt) { struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx; - int sample_count = pkt->size / (ctx->channels << 1); + AVStream *st = avctx->streams[pkt->stream_index]; + int sample_count; uint32_t buffered; + uint8_t *outbuf = NULL; + int ret = 0; ctx->dlo->GetBufferedAudioSampleFrameCount(&buffered); if (pkt->pts > 1 && !buffered) av_log(avctx, AV_LOG_WARNING, "There's no buffered audio." " Audio will misbehave!\n"); - if (ctx->dlo->ScheduleAudioSamples(pkt->data, sample_count, pkt->pts, + if (st->codecpar->codec_id == AV_CODEC_ID_AC3) { + /* Encapsulate AC3 syncframe into SMPTE 337 packet */ + int outbuf_size; + ret = create_s337_payload(pkt, &outbuf, &outbuf_size); + if (ret < 0) + return ret; + sample_count = outbuf_size / 4; + } else { + sample_count = pkt->size / (ctx->channels << 1); + outbuf = pkt->data; + } + + if (ctx->dlo->ScheduleAudioSamples(outbuf, sample_count, pkt->pts, bmdAudioSampleRate48kHz, NULL) != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not schedule audio samples.\n"); - return AVERROR(EIO); + ret = AVERROR(EIO); } - return 0; + if (st->codecpar->codec_id == AV_CODEC_ID_AC3) + av_freep(&outbuf); + + return ret; } extern "C" {