| Message ID | 20180109011658.72370-4-dheitmueller@ltnglobal.com |
|---|---|
| State | New |
| Headers | show |
On Mon, 8 Jan 2018, Devin Heitmueller wrote: > Add support for the ability to capture all audio pairs available > to the capture hardware. Each pair is exposed as a different audio > stream, which matches up with the most common use cases for the > broadcast space (i.e. where there is one stereo pair per audio > language). > > To support the existing use case where multi-channel audio can be > captured (i.e. 7.1), we introduced a new configuration option, which > defaults to the existing behavior. > > Updated to reflect comments from Carl Eugen Hoyos <ceffmpeg@gmail.com>, > Aaron Levinson <alevinsn@levland.net>, and Matthias Hunstock > <atze@fem.tu-ilmenau.de>. > > Signed-off-by: Devin Heitmueller <dheitmueller@ltnglobal.com> > --- > doc/indevs.texi | 8 ++- > libavdevice/decklink_common.cpp | 12 ++++ > libavdevice/decklink_common.h | 8 ++- > libavdevice/decklink_common_c.h | 6 ++ > libavdevice/decklink_dec.cpp | 136 +++++++++++++++++++++++++++++++--------- > libavdevice/decklink_dec_c.c | 3 + > 6 files changed, 142 insertions(+), 31 deletions(-) > > diff --git a/doc/indevs.texi b/doc/indevs.texi > index 56066bf..4760d70 100644 > --- a/doc/indevs.texi > +++ b/doc/indevs.texi > @@ -278,9 +278,15 @@ For SD sources, ffmpeg needs to be compiled with @code{--enable-libzvbi}. For > HD sources, on older (pre-4K) DeckLink card models you have to capture in 10 > bit mode. > > +@item audio_mode > +Defines whether to capture a bundle of audio channels (the number of which is determined > +by the channels argument), or whether to capture a number of audio pairs (the number of > +which is determined by the maximum number of pairs supported by the card). Must be > +@samp{bundled} or @samp{pairs}. Defaults to @samp{bundled}. IMHO it makes more sense to always respect the requested number of capture channels. If you want to capture all channels, you can define a special constant (e.g.: all) which maps to -1 as the number of channels, and modify it according to the determined number of available maximum channels. > + > @item channels > Defines number of audio channels to capture. Must be @samp{2}, @samp{8} or @samp{16}. > -Defaults to @samp{2}. > +Defaults to @samp{2}. This parameter is ignored if audio_mode is set to pairs. > > @item duplex_mode > Sets the decklink device duplex mode. Must be @samp{unset}, @samp{half} or @samp{full}. > diff --git a/libavdevice/decklink_common.cpp b/libavdevice/decklink_common.cpp > index c432189..e7daa63 100644 > --- a/libavdevice/decklink_common.cpp > +++ b/libavdevice/decklink_common.cpp > @@ -446,6 +446,7 @@ int ff_decklink_init_device(AVFormatContext *avctx, const char* name) > struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; > struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx; > IDeckLink *dl = NULL; > + int64_t maxAudioChannels; > IDeckLinkIterator *iter = CreateDeckLinkIteratorInstance(); > if (!iter) { > av_log(avctx, AV_LOG_ERROR, "Could not create DeckLink iterator\n"); > @@ -479,5 +480,16 @@ int ff_decklink_init_device(AVFormatContext *avctx, const char* name) > return AVERROR_EXTERNAL; > } > > + if (ctx->attr->GetInt(BMDDeckLinkMaximumAudioChannels, &maxAudioChannels) != S_OK) { > + av_log(avctx, AV_LOG_WARNING, "Could not determine number of audio channels\n"); > + ctx->max_audio_channels = 0; I think you can return failure here. > + } else { > + ctx->max_audio_channels = maxAudioChannels; > + } > + if (ctx->max_audio_channels > DECKLINK_MAX_AUDIO_CHANNELS) { > + av_log(avctx, AV_LOG_WARNING, "Decklink card reported support for more channels than ffmpeg supports\n"); > + ctx->max_audio_channels = DECKLINK_MAX_AUDIO_CHANNELS; > + } > + > return 0; > } > diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h > index 143bbb9..b262780 100644 > --- a/libavdevice/decklink_common.h > +++ b/libavdevice/decklink_common.h > @@ -37,6 +37,10 @@ > #define DECKLINK_BOOL bool > #endif > > +/* Maximum number of channels possible across variants of Blackmagic cards. > + Actual number for any particular model of card may be lower */ This is actually the maximum number of channels FFMPEG supports. > +#define DECKLINK_MAX_AUDIO_CHANNELS 32 > + > class decklink_output_callback; > class decklink_input_callback; > > @@ -71,6 +75,7 @@ struct decklink_ctx { > int bmd_height; > int bmd_field_dominance; > int supports_vanc; > + int max_audio_channels; > > /* Capture buffer queue */ > AVPacketQueue queue; > @@ -85,7 +90,8 @@ struct decklink_ctx { > int64_t last_pts; > unsigned long frameCount; > unsigned int dropped; > - AVStream *audio_st; > + AVStream *audio_st[DECKLINK_MAX_AUDIO_CHANNELS]; > + int num_audio_streams; > AVStream *video_st; > AVStream *teletext_st; > uint16_t cdp_sequence_num; > diff --git a/libavdevice/decklink_common_c.h b/libavdevice/decklink_common_c.h > index 368ac25..3a21bae 100644 > --- a/libavdevice/decklink_common_c.h > +++ b/libavdevice/decklink_common_c.h > @@ -30,6 +30,11 @@ typedef enum DecklinkPtsSource { > PTS_SRC_WALLCLOCK = 4, > } DecklinkPtsSource; > > +typedef enum DecklinkAudioMode { > + AUDIO_MODE_BUNDLED = 0, > + AUDIO_MODE_PAIRS = 1, > +} DecklinkAudioMode; > + > struct decklink_cctx { > const AVClass *cclass; > > @@ -42,6 +47,7 @@ struct decklink_cctx { > double preroll; > int v210; > int audio_channels; > + int audio_mode; > int audio_depth; > int duplex_mode; > DecklinkPtsSource audio_pts_source; > diff --git a/libavdevice/decklink_dec.cpp b/libavdevice/decklink_dec.cpp > index 94dae26..6c1ff82 100644 > --- a/libavdevice/decklink_dec.cpp > +++ b/libavdevice/decklink_dec.cpp > @@ -627,9 +627,56 @@ static int64_t get_pkt_pts(IDeckLinkVideoInputFrame *videoFrame, > return pts; > } > > +static int setup_audio(AVFormatContext *avctx) > +{ > + struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; > + struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx; > + AVStream *st; > + int ret = 0; > + > + if (cctx->audio_mode == AUDIO_MODE_BUNDLED) { > + st = avformat_new_stream(avctx, NULL); > + if (!st) { > + av_log(avctx, AV_LOG_ERROR, "Cannot add stream\n"); > + ret = AVERROR(ENOMEM); > + goto error; > + } > + st->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; > + st->codecpar->codec_id = ctx->audio_depth == 32 ? AV_CODEC_ID_PCM_S32LE : AV_CODEC_ID_PCM_S16LE; > + st->codecpar->sample_rate = bmdAudioSampleRate48kHz; > + st->codecpar->channels = cctx->audio_channels; > + st->codecpar->channel_layout = AV_CH_LAYOUT_STEREO; > + avpriv_set_pts_info(st, 64, 1, 1000000); /* 64 bits pts in us */ > + ctx->audio_st[0] = st; > + ctx->num_audio_streams++; > + } else { > + for (int i = 0; i < ctx->max_audio_channels / 2; i++) { > + st = avformat_new_stream(avctx, NULL); > + if (!st) { > + av_log(avctx, AV_LOG_ERROR, "Cannot add stream %d\n", i); > + ret = AVERROR(ENOMEM); > + goto error; > + } > + st->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; > + st->codecpar->codec_id = ctx->audio_depth == 32 ? AV_CODEC_ID_PCM_S32LE : AV_CODEC_ID_PCM_S16LE; > + st->codecpar->sample_rate = bmdAudioSampleRate48kHz; > + st->codecpar->channels = 2; > + st->codecpar->channel_layout = AV_CH_LAYOUT_STEREO; > + avpriv_set_pts_info(st, 64, 1, 1000000); /* 64 bits pts in us */ > + ctx->audio_st[i] = st; > + ctx->num_audio_streams++; > + } > + cctx->audio_channels = ctx->max_audio_channels; > + } > + > +error: > + return ret; > +} > + > HRESULT decklink_input_callback::VideoInputFrameArrived( > IDeckLinkVideoInputFrame *videoFrame, IDeckLinkAudioInputPacket *audioFrame) > { > + decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; > void *frameBytes; > void *audioFrameBytes; > BMDTimeValue frameTime; > @@ -777,24 +824,57 @@ HRESULT decklink_input_callback::VideoInputFrameArrived( > > // Handle Audio Frame > if (audioFrame) { > - AVPacket pkt; > - BMDTimeValue audio_pts; > - av_init_packet(&pkt); > - > - //hack among hacks > - pkt.size = audioFrame->GetSampleFrameCount() * ctx->audio_st->codecpar->channels * (ctx->audio_depth / 8); > audioFrame->GetBytes(&audioFrameBytes); > - audioFrame->GetPacketTime(&audio_pts, ctx->audio_st->time_base.den); > - pkt.pts = get_pkt_pts(videoFrame, audioFrame, wallclock, ctx->audio_pts_source, ctx->audio_st->time_base, &initial_audio_pts); > - pkt.dts = pkt.pts; > > - //fprintf(stderr,"Audio Frame size %d ts %d\n", pkt.size, pkt.pts); > - pkt.flags |= AV_PKT_FLAG_KEY; > - pkt.stream_index = ctx->audio_st->index; > - pkt.data = (uint8_t *)audioFrameBytes; > + if (cctx->audio_mode == AUDIO_MODE_BUNDLED) { > + AVPacket pkt; > + BMDTimeValue audio_pts; > + av_init_packet(&pkt); > > - if (avpacket_queue_put(&ctx->queue, &pkt) < 0) { > - ++ctx->dropped; > + //hack among hacks > + pkt.size = audioFrame->GetSampleFrameCount() * ctx->audio_st[0]->codecpar->channels * (ctx->audio_depth / 8); > + audioFrame->GetBytes(&audioFrameBytes); > + audioFrame->GetPacketTime(&audio_pts, ctx->audio_st[0]->time_base.den); > + pkt.pts = get_pkt_pts(videoFrame, audioFrame, wallclock, ctx->audio_pts_source, ctx->audio_st[0]->time_base, &initial_audio_pts); > + pkt.dts = pkt.pts; > + > + pkt.flags |= AV_PKT_FLAG_KEY; > + pkt.stream_index = ctx->audio_st[0]->index; > + pkt.data = (uint8_t *)audioFrameBytes; > + > + if (avpacket_queue_put(&ctx->queue, &pkt) < 0) { > + ++ctx->dropped; > + } > + } else { > + /* Need to deinterleave audio */ > + int audio_offset = 0; > + int audio_stride = cctx->audio_channels * ctx->audio_depth / 8; > + for (int i = 0; i < ctx->num_audio_streams; i++) { > + int sample_size = ctx->audio_st[i]->codecpar->channels * > + ctx->audio_st[i]->codecpar->bits_per_coded_sample / 8; > + AVPacket pkt; > + int ret = av_new_packet(&pkt, audioFrame->GetSampleFrameCount() * sample_size); > + if (ret != 0) > + continue; > + > + pkt.pts = get_pkt_pts(videoFrame, audioFrame, wallclock, ctx->audio_pts_source, > + ctx->audio_st[i]->time_base, &initial_audio_pts); > + pkt.dts = pkt.pts; > + pkt.flags |= AV_PKT_FLAG_KEY; > + pkt.stream_index = ctx->audio_st[i]->index; > + > + uint8_t *audio_in = ((uint8_t *) audioFrameBytes) + audio_offset; > + for (int x = 0; x < pkt.size; x += sample_size) { > + memcpy(&pkt.data[x], audio_in, sample_size); > + audio_in += audio_stride; > + } > + > + if (avpacket_queue_put(&ctx->queue, &pkt) < 0) > + ++ctx->dropped; > + > + av_packet_unref(&pkt); > + audio_offset += sample_size; > + } > } > } > > @@ -999,18 +1079,7 @@ av_cold int ff_decklink_read_header(AVFormatContext *avctx) > #endif > > /* Setup streams. */ > - st = avformat_new_stream(avctx, NULL); > - if (!st) { > - av_log(avctx, AV_LOG_ERROR, "Cannot add stream\n"); > - ret = AVERROR(ENOMEM); > - goto error; > - } > - st->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; > - st->codecpar->codec_id = cctx->audio_depth == 32 ? AV_CODEC_ID_PCM_S32LE : AV_CODEC_ID_PCM_S16LE; > - st->codecpar->sample_rate = bmdAudioSampleRate48kHz; > - st->codecpar->channels = cctx->audio_channels; > - avpriv_set_pts_info(st, 64, 1, 1000000); /* 64 bits pts in us */ > - ctx->audio_st=st; > + setup_audio(avctx); > > st = avformat_new_stream(avctx, NULL); > if (!st) { > @@ -1096,8 +1165,17 @@ av_cold int ff_decklink_read_header(AVFormatContext *avctx) > ctx->teletext_st = st; > } > > - av_log(avctx, AV_LOG_VERBOSE, "Using %d input audio channels\n", ctx->audio_st->codecpar->channels); > - result = ctx->dli->EnableAudioInput(bmdAudioSampleRate48kHz, cctx->audio_depth == 32 ? bmdAudioSampleType32bitInteger : bmdAudioSampleType16bitInteger, ctx->audio_st->codecpar->channels); > + if (cctx->audio_mode == AUDIO_MODE_BUNDLED) { > + av_log(avctx, AV_LOG_VERBOSE, "Using %d input audio channels\n", ctx->audio_st[0]->codecpar->channels); > + result = ctx->dli->EnableAudioInput(bmdAudioSampleRate48kHz, > + ctx->audio_depth == 32 ? bmdAudioSampleType32bitInteger : bmdAudioSampleType16bitInteger, > + ctx->audio_st[0]->codecpar->channels); > + } else { > + av_log(avctx, AV_LOG_VERBOSE, "Using %d input audio channels\n", ctx->max_audio_channels); > + result = ctx->dli->EnableAudioInput(bmdAudioSampleRate48kHz, > + ctx->audio_depth == 32 ? bmdAudioSampleType32bitInteger : bmdAudioSampleType16bitInteger, > + ctx->max_audio_channels); > + } > > if (result != S_OK) { > av_log(avctx, AV_LOG_ERROR, "Cannot enable audio input\n"); > diff --git a/libavdevice/decklink_dec_c.c b/libavdevice/decklink_dec_c.c > index 1c6d826..fe8e9fd 100644 > --- a/libavdevice/decklink_dec_c.c > +++ b/libavdevice/decklink_dec_c.c > @@ -44,6 +44,9 @@ static const AVOption options[] = { > { "standard", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0x7fff9fffeLL}, 0, 0, DEC, "teletext_lines"}, > { "all", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0x7ffffffffLL}, 0, 0, DEC, "teletext_lines"}, > { "channels", "number of audio channels", OFFSET(audio_channels), AV_OPT_TYPE_INT , { .i64 = 2 }, 2, 16, DEC }, > + { "audio_mode", "audio mode", OFFSET(audio_mode), AV_OPT_TYPE_INT, { .i64 = AUDIO_MODE_BUNDLED}, 0, 1, DEC, "audio_mode"}, > + { "bundled", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = AUDIO_MODE_BUNDLED}, 0, 0, DEC, "audio_mode"}, > + { "pairs", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = AUDIO_MODE_PAIRS}, 0, 0, DEC, "audio_mode"}, > { "duplex_mode", "duplex mode", OFFSET(duplex_mode), AV_OPT_TYPE_INT, { .i64 = 0}, 0, 2, DEC, "duplex_mode"}, > { "unset", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0}, 0, 0, DEC, "duplex_mode"}, > { "half", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1}, 0, 0, DEC, "duplex_mode"}, Regards, Marton
diff --git a/doc/indevs.texi b/doc/indevs.texi index 56066bf..4760d70 100644 --- a/doc/indevs.texi +++ b/doc/indevs.texi @@ -278,9 +278,15 @@ For SD sources, ffmpeg needs to be compiled with @code{--enable-libzvbi}. For HD sources, on older (pre-4K) DeckLink card models you have to capture in 10 bit mode. +@item audio_mode +Defines whether to capture a bundle of audio channels (the number of which is determined +by the channels argument), or whether to capture a number of audio pairs (the number of +which is determined by the maximum number of pairs supported by the card). Must be +@samp{bundled} or @samp{pairs}. Defaults to @samp{bundled}. + @item channels Defines number of audio channels to capture. Must be @samp{2}, @samp{8} or @samp{16}. -Defaults to @samp{2}. +Defaults to @samp{2}. This parameter is ignored if audio_mode is set to pairs. @item duplex_mode Sets the decklink device duplex mode. Must be @samp{unset}, @samp{half} or @samp{full}. diff --git a/libavdevice/decklink_common.cpp b/libavdevice/decklink_common.cpp index c432189..e7daa63 100644 --- a/libavdevice/decklink_common.cpp +++ b/libavdevice/decklink_common.cpp @@ -446,6 +446,7 @@ int ff_decklink_init_device(AVFormatContext *avctx, const char* name) struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx; IDeckLink *dl = NULL; + int64_t maxAudioChannels; IDeckLinkIterator *iter = CreateDeckLinkIteratorInstance(); if (!iter) { av_log(avctx, AV_LOG_ERROR, "Could not create DeckLink iterator\n"); @@ -479,5 +480,16 @@ int ff_decklink_init_device(AVFormatContext *avctx, const char* name) return AVERROR_EXTERNAL; } + if (ctx->attr->GetInt(BMDDeckLinkMaximumAudioChannels, &maxAudioChannels) != S_OK) { + av_log(avctx, AV_LOG_WARNING, "Could not determine number of audio channels\n"); + ctx->max_audio_channels = 0; + } else { + ctx->max_audio_channels = maxAudioChannels; + } + if (ctx->max_audio_channels > DECKLINK_MAX_AUDIO_CHANNELS) { + av_log(avctx, AV_LOG_WARNING, "Decklink card reported support for more channels than ffmpeg supports\n"); + ctx->max_audio_channels = DECKLINK_MAX_AUDIO_CHANNELS; + } + return 0; } diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h index 143bbb9..b262780 100644 --- a/libavdevice/decklink_common.h +++ b/libavdevice/decklink_common.h @@ -37,6 +37,10 @@ #define DECKLINK_BOOL bool #endif +/* Maximum number of channels possible across variants of Blackmagic cards. + Actual number for any particular model of card may be lower */ +#define DECKLINK_MAX_AUDIO_CHANNELS 32 + class decklink_output_callback; class decklink_input_callback; @@ -71,6 +75,7 @@ struct decklink_ctx { int bmd_height; int bmd_field_dominance; int supports_vanc; + int max_audio_channels; /* Capture buffer queue */ AVPacketQueue queue; @@ -85,7 +90,8 @@ struct decklink_ctx { int64_t last_pts; unsigned long frameCount; unsigned int dropped; - AVStream *audio_st; + AVStream *audio_st[DECKLINK_MAX_AUDIO_CHANNELS]; + int num_audio_streams; AVStream *video_st; AVStream *teletext_st; uint16_t cdp_sequence_num; diff --git a/libavdevice/decklink_common_c.h b/libavdevice/decklink_common_c.h index 368ac25..3a21bae 100644 --- a/libavdevice/decklink_common_c.h +++ b/libavdevice/decklink_common_c.h @@ -30,6 +30,11 @@ typedef enum DecklinkPtsSource { PTS_SRC_WALLCLOCK = 4, } DecklinkPtsSource; +typedef enum DecklinkAudioMode { + AUDIO_MODE_BUNDLED = 0, + AUDIO_MODE_PAIRS = 1, +} DecklinkAudioMode; + struct decklink_cctx { const AVClass *cclass; @@ -42,6 +47,7 @@ struct decklink_cctx { double preroll; int v210; int audio_channels; + int audio_mode; int audio_depth; int duplex_mode; DecklinkPtsSource audio_pts_source; diff --git a/libavdevice/decklink_dec.cpp b/libavdevice/decklink_dec.cpp index 94dae26..6c1ff82 100644 --- a/libavdevice/decklink_dec.cpp +++ b/libavdevice/decklink_dec.cpp @@ -627,9 +627,56 @@ static int64_t get_pkt_pts(IDeckLinkVideoInputFrame *videoFrame, return pts; } +static int setup_audio(AVFormatContext *avctx) +{ + struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; + struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx; + AVStream *st; + int ret = 0; + + if (cctx->audio_mode == AUDIO_MODE_BUNDLED) { + st = avformat_new_stream(avctx, NULL); + if (!st) { + av_log(avctx, AV_LOG_ERROR, "Cannot add stream\n"); + ret = AVERROR(ENOMEM); + goto error; + } + st->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; + st->codecpar->codec_id = ctx->audio_depth == 32 ? AV_CODEC_ID_PCM_S32LE : AV_CODEC_ID_PCM_S16LE; + st->codecpar->sample_rate = bmdAudioSampleRate48kHz; + st->codecpar->channels = cctx->audio_channels; + st->codecpar->channel_layout = AV_CH_LAYOUT_STEREO; + avpriv_set_pts_info(st, 64, 1, 1000000); /* 64 bits pts in us */ + ctx->audio_st[0] = st; + ctx->num_audio_streams++; + } else { + for (int i = 0; i < ctx->max_audio_channels / 2; i++) { + st = avformat_new_stream(avctx, NULL); + if (!st) { + av_log(avctx, AV_LOG_ERROR, "Cannot add stream %d\n", i); + ret = AVERROR(ENOMEM); + goto error; + } + st->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; + st->codecpar->codec_id = ctx->audio_depth == 32 ? AV_CODEC_ID_PCM_S32LE : AV_CODEC_ID_PCM_S16LE; + st->codecpar->sample_rate = bmdAudioSampleRate48kHz; + st->codecpar->channels = 2; + st->codecpar->channel_layout = AV_CH_LAYOUT_STEREO; + avpriv_set_pts_info(st, 64, 1, 1000000); /* 64 bits pts in us */ + ctx->audio_st[i] = st; + ctx->num_audio_streams++; + } + cctx->audio_channels = ctx->max_audio_channels; + } + +error: + return ret; +} + HRESULT decklink_input_callback::VideoInputFrameArrived( IDeckLinkVideoInputFrame *videoFrame, IDeckLinkAudioInputPacket *audioFrame) { + decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data; void *frameBytes; void *audioFrameBytes; BMDTimeValue frameTime; @@ -777,24 +824,57 @@ HRESULT decklink_input_callback::VideoInputFrameArrived( // Handle Audio Frame if (audioFrame) { - AVPacket pkt; - BMDTimeValue audio_pts; - av_init_packet(&pkt); - - //hack among hacks - pkt.size = audioFrame->GetSampleFrameCount() * ctx->audio_st->codecpar->channels * (ctx->audio_depth / 8); audioFrame->GetBytes(&audioFrameBytes); - audioFrame->GetPacketTime(&audio_pts, ctx->audio_st->time_base.den); - pkt.pts = get_pkt_pts(videoFrame, audioFrame, wallclock, ctx->audio_pts_source, ctx->audio_st->time_base, &initial_audio_pts); - pkt.dts = pkt.pts; - //fprintf(stderr,"Audio Frame size %d ts %d\n", pkt.size, pkt.pts); - pkt.flags |= AV_PKT_FLAG_KEY; - pkt.stream_index = ctx->audio_st->index; - pkt.data = (uint8_t *)audioFrameBytes; + if (cctx->audio_mode == AUDIO_MODE_BUNDLED) { + AVPacket pkt; + BMDTimeValue audio_pts; + av_init_packet(&pkt); - if (avpacket_queue_put(&ctx->queue, &pkt) < 0) { - ++ctx->dropped; + //hack among hacks + pkt.size = audioFrame->GetSampleFrameCount() * ctx->audio_st[0]->codecpar->channels * (ctx->audio_depth / 8); + audioFrame->GetBytes(&audioFrameBytes); + audioFrame->GetPacketTime(&audio_pts, ctx->audio_st[0]->time_base.den); + pkt.pts = get_pkt_pts(videoFrame, audioFrame, wallclock, ctx->audio_pts_source, ctx->audio_st[0]->time_base, &initial_audio_pts); + pkt.dts = pkt.pts; + + pkt.flags |= AV_PKT_FLAG_KEY; + pkt.stream_index = ctx->audio_st[0]->index; + pkt.data = (uint8_t *)audioFrameBytes; + + if (avpacket_queue_put(&ctx->queue, &pkt) < 0) { + ++ctx->dropped; + } + } else { + /* Need to deinterleave audio */ + int audio_offset = 0; + int audio_stride = cctx->audio_channels * ctx->audio_depth / 8; + for (int i = 0; i < ctx->num_audio_streams; i++) { + int sample_size = ctx->audio_st[i]->codecpar->channels * + ctx->audio_st[i]->codecpar->bits_per_coded_sample / 8; + AVPacket pkt; + int ret = av_new_packet(&pkt, audioFrame->GetSampleFrameCount() * sample_size); + if (ret != 0) + continue; + + pkt.pts = get_pkt_pts(videoFrame, audioFrame, wallclock, ctx->audio_pts_source, + ctx->audio_st[i]->time_base, &initial_audio_pts); + pkt.dts = pkt.pts; + pkt.flags |= AV_PKT_FLAG_KEY; + pkt.stream_index = ctx->audio_st[i]->index; + + uint8_t *audio_in = ((uint8_t *) audioFrameBytes) + audio_offset; + for (int x = 0; x < pkt.size; x += sample_size) { + memcpy(&pkt.data[x], audio_in, sample_size); + audio_in += audio_stride; + } + + if (avpacket_queue_put(&ctx->queue, &pkt) < 0) + ++ctx->dropped; + + av_packet_unref(&pkt); + audio_offset += sample_size; + } } } @@ -999,18 +1079,7 @@ av_cold int ff_decklink_read_header(AVFormatContext *avctx) #endif /* Setup streams. */ - st = avformat_new_stream(avctx, NULL); - if (!st) { - av_log(avctx, AV_LOG_ERROR, "Cannot add stream\n"); - ret = AVERROR(ENOMEM); - goto error; - } - st->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; - st->codecpar->codec_id = cctx->audio_depth == 32 ? AV_CODEC_ID_PCM_S32LE : AV_CODEC_ID_PCM_S16LE; - st->codecpar->sample_rate = bmdAudioSampleRate48kHz; - st->codecpar->channels = cctx->audio_channels; - avpriv_set_pts_info(st, 64, 1, 1000000); /* 64 bits pts in us */ - ctx->audio_st=st; + setup_audio(avctx); st = avformat_new_stream(avctx, NULL); if (!st) { @@ -1096,8 +1165,17 @@ av_cold int ff_decklink_read_header(AVFormatContext *avctx) ctx->teletext_st = st; } - av_log(avctx, AV_LOG_VERBOSE, "Using %d input audio channels\n", ctx->audio_st->codecpar->channels); - result = ctx->dli->EnableAudioInput(bmdAudioSampleRate48kHz, cctx->audio_depth == 32 ? bmdAudioSampleType32bitInteger : bmdAudioSampleType16bitInteger, ctx->audio_st->codecpar->channels); + if (cctx->audio_mode == AUDIO_MODE_BUNDLED) { + av_log(avctx, AV_LOG_VERBOSE, "Using %d input audio channels\n", ctx->audio_st[0]->codecpar->channels); + result = ctx->dli->EnableAudioInput(bmdAudioSampleRate48kHz, + ctx->audio_depth == 32 ? bmdAudioSampleType32bitInteger : bmdAudioSampleType16bitInteger, + ctx->audio_st[0]->codecpar->channels); + } else { + av_log(avctx, AV_LOG_VERBOSE, "Using %d input audio channels\n", ctx->max_audio_channels); + result = ctx->dli->EnableAudioInput(bmdAudioSampleRate48kHz, + ctx->audio_depth == 32 ? bmdAudioSampleType32bitInteger : bmdAudioSampleType16bitInteger, + ctx->max_audio_channels); + } if (result != S_OK) { av_log(avctx, AV_LOG_ERROR, "Cannot enable audio input\n"); diff --git a/libavdevice/decklink_dec_c.c b/libavdevice/decklink_dec_c.c index 1c6d826..fe8e9fd 100644 --- a/libavdevice/decklink_dec_c.c +++ b/libavdevice/decklink_dec_c.c @@ -44,6 +44,9 @@ static const AVOption options[] = { { "standard", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0x7fff9fffeLL}, 0, 0, DEC, "teletext_lines"}, { "all", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0x7ffffffffLL}, 0, 0, DEC, "teletext_lines"}, { "channels", "number of audio channels", OFFSET(audio_channels), AV_OPT_TYPE_INT , { .i64 = 2 }, 2, 16, DEC }, + { "audio_mode", "audio mode", OFFSET(audio_mode), AV_OPT_TYPE_INT, { .i64 = AUDIO_MODE_BUNDLED}, 0, 1, DEC, "audio_mode"}, + { "bundled", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = AUDIO_MODE_BUNDLED}, 0, 0, DEC, "audio_mode"}, + { "pairs", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = AUDIO_MODE_PAIRS}, 0, 0, DEC, "audio_mode"}, { "duplex_mode", "duplex mode", OFFSET(duplex_mode), AV_OPT_TYPE_INT, { .i64 = 0}, 0, 2, DEC, "duplex_mode"}, { "unset", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0}, 0, 0, DEC, "duplex_mode"}, { "half", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1}, 0, 0, DEC, "duplex_mode"},
Add support for the ability to capture all audio pairs available to the capture hardware. Each pair is exposed as a different audio stream, which matches up with the most common use cases for the broadcast space (i.e. where there is one stereo pair per audio language). To support the existing use case where multi-channel audio can be captured (i.e. 7.1), we introduced a new configuration option, which defaults to the existing behavior. Updated to reflect comments from Carl Eugen Hoyos <ceffmpeg@gmail.com>, Aaron Levinson <alevinsn@levland.net>, and Matthias Hunstock <atze@fem.tu-ilmenau.de>. Signed-off-by: Devin Heitmueller <dheitmueller@ltnglobal.com> --- doc/indevs.texi | 8 ++- libavdevice/decklink_common.cpp | 12 ++++ libavdevice/decklink_common.h | 8 ++- libavdevice/decklink_common_c.h | 6 ++ libavdevice/decklink_dec.cpp | 136 +++++++++++++++++++++++++++++++--------- libavdevice/decklink_dec_c.c | 3 + 6 files changed, 142 insertions(+), 31 deletions(-)