Message ID | d8520110-12ff-33f8-5580-7d5d340bc9d7@mail.de |
---|---|
State | New |
Headers | show |
Series | [FFmpeg-devel] lavdevice: Add VideoToolbox output device. | expand |
Context | Check | Description |
---|---|---|
andriy/default | pending | |
andriy/make | fail | Make failed |
Thilo Borgmann (12020-06-08):
> OSX accepts numerous formats, so there are several output devices.
The other audio output devices support all formats in a single device.
Why does Apple need to be different again?
Regards,
Am 08.06.20 um 00:28 schrieb Nicolas George: > Thilo Borgmann (12020-06-08): >> OSX accepts numerous formats, so there are several output devices. > > The other audio output devices support all formats in a single device. > Why does Apple need to be different again? Not sure if you don't mix it with Video.... screw me... For Audio, I cannot find another device handling more than one format in one device. I'd appreciate a better way to do it than having N-devices... echo $subject | sed -e 's/Video/Audio/g' | send_new_mail() #TooLate -Thilo
Thilo Borgmann (12020-06-08): > Not sure if you don't mix it with Video.... screw me... Oh, that's why you re-sent the patch. I could not spot the difference. > For Audio, I cannot find another device handling more than one format in one device. > I'd appreciate a better way to do it than having N-devices... alsa_enc.c, at least: Output #0, alsa, to 'default': Metadata: encoder : Lavf58.42.100 Stream #0:0: Audio: pcm_u8, 44100 Hz, mono, u8, 352 kb/s (default) Output #0, alsa, to 'default': Metadata: encoder : Lavf58.42.100 Stream #0:0: Audio: pcm_s16le, 44100 Hz, mono, s16, 705 kb/s (default) Output #0, alsa, to 'default': Metadata: encoder : Lavf58.42.100 Stream #0:0: Audio: pcm_f32le, 44100 Hz, mono, flt, 1411 kb/s (default) Regards,
Am 08.06.20 um 00:35 schrieb Nicolas George: > Thilo Borgmann (12020-06-08): >> Not sure if you don't mix it with Video.... screw me... > > Oh, that's why you re-sent the patch. I could not spot the difference. > >> For Audio, I cannot find another device handling more than one format in one device. >> I'd appreciate a better way to do it than having N-devices... > > alsa_enc.c, at least: > > Output #0, alsa, to 'default': > Metadata: > encoder : Lavf58.42.100 > Stream #0:0: Audio: pcm_u8, 44100 Hz, mono, u8, 352 kb/s (default) > > Output #0, alsa, to 'default': > Metadata: > encoder : Lavf58.42.100 > Stream #0:0: Audio: pcm_s16le, 44100 Hz, mono, s16, 705 kb/s (default) > > Output #0, alsa, to 'default': > Metadata: > encoder : Lavf58.42.100 > Stream #0:0: Audio: pcm_f32le, 44100 Hz, mono, flt, 1411 kb/s (default) I guess I see how it does that. How do you avoid the auto-scaler to be scaling to alsa's default format of pcm_s16 on the command line? -Thilo
On Mon, Jun 08, 2020 at 12:26:28AM +0200, Thilo Borgmann wrote: > Hi, > > $subject. > > Enables native audio output on OSX. > OSX accepts numerous formats, so there are several output devices. > > -Thilo > configure | 3 > libavdevice/Makefile | 1 > libavdevice/alldevices.c | 10 + > libavdevice/audiotoolbox.m | 308 +++++++++++++++++++++++++++++++++++++++++++++ > 4 files changed, 322 insertions(+) > 21f03d85483da43f5de38101bc3b6d7367525297 0001-lavdevice-Add-AudioToolbox-output-device.patch > From bdabbc6a919ccff20d6ae16acff1a5d0f8ca46b8 Mon Sep 17 00:00:00 2001 > From: Thilo Borgmann <thilo.borgmann@mail.de> > Date: Mon, 8 Jun 2020 00:20:25 +0200 > Subject: [PATCH] lavdevice: Add AudioToolbox output device. this breaks build on linux libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x88): undefined reference to `ff_audiotoolbox_f32_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x90): undefined reference to `ff_audiotoolbox_s32_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x98): undefined reference to `ff_audiotoolbox_s24_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa0): undefined reference to `ff_audiotoolbox_s16_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa8): undefined reference to `ff_audiotoolbox_s8_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb0): undefined reference to `ff_audiotoolbox_u32_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb8): undefined reference to `ff_audiotoolbox_u24_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc0): undefined reference to `ff_audiotoolbox_u16_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc8): undefined reference to `ff_audiotoolbox_u8_muxer' collect2: error: ld returned 1 exit status Makefile:114: recipe for target 'ffmpeg_g' failed make: *** [ffmpeg_g] Error 1 make: *** Waiting for unfinished jobs.... libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x88): undefined reference to `ff_audiotoolbox_f32_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x90): undefined reference to `ff_audiotoolbox_s32_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x98): undefined reference to `ff_audiotoolbox_s24_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa0): undefined reference to `ff_audiotoolbox_s16_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa8): undefined reference to `ff_audiotoolbox_s8_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb0): undefined reference to `ff_audiotoolbox_u32_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb8): undefined reference to `ff_audiotoolbox_u24_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc0): undefined reference to `ff_audiotoolbox_u16_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc8): undefined reference to `ff_audiotoolbox_u8_muxer' collect2: error: ld returned 1 exit status Makefile:114: recipe for target 'ffplay_g' failed make: *** [ffplay_g] Error 1 libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x88): undefined reference to `ff_audiotoolbox_f32_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x90): undefined reference to `ff_audiotoolbox_s32_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x98): undefined reference to `ff_audiotoolbox_s24_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa0): undefined reference to `ff_audiotoolbox_s16_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa8): undefined reference to `ff_audiotoolbox_s8_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb0): undefined reference to `ff_audiotoolbox_u32_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb8): undefined reference to `ff_audiotoolbox_u24_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc0): undefined reference to `ff_audiotoolbox_u16_muxer' libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc8): undefined reference to `ff_audiotoolbox_u8_muxer' collect2: error: ld returned 1 exit status Makefile:114: recipe for target 'ffprobe_g' failed make: *** [ffprobe_g] Error 1 [...]
Am 08.06.20 um 09:52 schrieb Michael Niedermayer: > On Mon, Jun 08, 2020 at 12:26:28AM +0200, Thilo Borgmann wrote: >> Hi, >> >> $subject. >> >> Enables native audio output on OSX. >> OSX accepts numerous formats, so there are several output devices. >> >> -Thilo > >> configure | 3 >> libavdevice/Makefile | 1 >> libavdevice/alldevices.c | 10 + >> libavdevice/audiotoolbox.m | 308 +++++++++++++++++++++++++++++++++++++++++++++ >> 4 files changed, 322 insertions(+) >> 21f03d85483da43f5de38101bc3b6d7367525297 0001-lavdevice-Add-AudioToolbox-output-device.patch >> From bdabbc6a919ccff20d6ae16acff1a5d0f8ca46b8 Mon Sep 17 00:00:00 2001 >> From: Thilo Borgmann <thilo.borgmann@mail.de> >> Date: Mon, 8 Jun 2020 00:20:25 +0200 >> Subject: [PATCH] lavdevice: Add AudioToolbox output device. > > this breaks build on linux > > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x88): undefined reference to `ff_audiotoolbox_f32_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x90): undefined reference to `ff_audiotoolbox_s32_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x98): undefined reference to `ff_audiotoolbox_s24_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa0): undefined reference to `ff_audiotoolbox_s16_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa8): undefined reference to `ff_audiotoolbox_s8_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb0): undefined reference to `ff_audiotoolbox_u32_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb8): undefined reference to `ff_audiotoolbox_u24_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc0): undefined reference to `ff_audiotoolbox_u16_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc8): undefined reference to `ff_audiotoolbox_u8_muxer' > collect2: error: ld returned 1 exit status > Makefile:114: recipe for target 'ffmpeg_g' failed > make: *** [ffmpeg_g] Error 1 > make: *** Waiting for unfinished jobs.... > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x88): undefined reference to `ff_audiotoolbox_f32_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x90): undefined reference to `ff_audiotoolbox_s32_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x98): undefined reference to `ff_audiotoolbox_s24_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa0): undefined reference to `ff_audiotoolbox_s16_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa8): undefined reference to `ff_audiotoolbox_s8_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb0): undefined reference to `ff_audiotoolbox_u32_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb8): undefined reference to `ff_audiotoolbox_u24_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc0): undefined reference to `ff_audiotoolbox_u16_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc8): undefined reference to `ff_audiotoolbox_u8_muxer' > collect2: error: ld returned 1 exit status > Makefile:114: recipe for target 'ffplay_g' failed > make: *** [ffplay_g] Error 1 > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x88): undefined reference to `ff_audiotoolbox_f32_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x90): undefined reference to `ff_audiotoolbox_s32_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0x98): undefined reference to `ff_audiotoolbox_s24_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa0): undefined reference to `ff_audiotoolbox_s16_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xa8): undefined reference to `ff_audiotoolbox_s8_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb0): undefined reference to `ff_audiotoolbox_u32_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xb8): undefined reference to `ff_audiotoolbox_u24_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc0): undefined reference to `ff_audiotoolbox_u16_muxer' > libavdevice/libavdevice.a(alldevices.o):(.data.rel.ro+0xc8): undefined reference to `ff_audiotoolbox_u8_muxer' > collect2: error: ld returned 1 exit status > Makefile:114: recipe for target 'ffprobe_g' failed > make: *** [ffprobe_g] Error 1 > Seen that, will fix, thx! -Thilo
Thilo Borgmann (12020-06-08): > How do you avoid the auto-scaler to be scaling to alsa's default > format of pcm_s16 on the command line? We cannot, not with your original version, not with how alsa_enc does and the new version. Because they are not sample formats, they are codecs, and converting from a codec to another is more complex than just a resampler. And even if in these particular cases they are trivial codecs, they could easily not be: ALSA supports S/PDIF, so it could be AC3 or something. The solution to have format negotiation would be to make sure all output devices have a matching lavfi sink. Unfortunately, for now, I am not sure lavfi's scheduling work with non-buffer sinks. And it has other drawbacks, including preventing S/PDIF. Regards,
On Sun, Jun 7, 2020 at 6:31 PM Thilo Borgmann <thilo.borgmann@mail.de> wrote: > > > Not sure if you don't mix it with Video.... screw me... > For Audio, I cannot find another device handling more than one format in one device. > I'd appreciate a better way to do it than having N-devices... While not yet upstream, my version of the decklink output module provides a single avdevice that supports multiple formats, including AC-3 for passthrough. https://github.com/LTNGlobal-opensource/FFmpeg-ltn/blob/lted1/libavdevice/decklink_enc.cpp Devin
Hi, Am 08.06.20 um 16:27 schrieb Nicolas George: > Thilo Borgmann (12020-06-08): >> How do you avoid the auto-scaler to be scaling to alsa's default >> format of pcm_s16 on the command line? > > We cannot, not with your original version, not with how alsa_enc does > and the new version. > > Because they are not sample formats, they are codecs, and converting > from a codec to another is more complex than just a resampler. And even > if in these particular cases they are trivial codecs, they could easily > not be: ALSA supports S/PDIF, so it could be AC3 or something. yes I figured that codec thing out after querying on IRC. So with providing -a:c XY I can do all the formats (or pcm codecs so to speak) in one muxer where I had seperate muxers for before. So v2 works like the examples you gave for alsa. Might even work for compressed codecs depending on what OSX accepts. However you say we cannot avoid the auto scaler? I'm not sure what you mean we cannot do... And do you think v2 is still object for improvements regarding this? > The solution to have format negotiation would be to make sure all output > devices have a matching lavfi sink. Unfortunately, for now, I am not > sure lavfi's scheduling work with non-buffer sinks. And it has other > drawbacks, including preventing S/PDIF. This let's me think you're referring to some more general design shortcomings. Not sure if you relate that to the patch or something in general we should do prior to do the audiotoolbox output device. Thanks, Thilo
Hi, Am 08.06.20 um 16:36 schrieb Devin Heitmueller: > On Sun, Jun 7, 2020 at 6:31 PM Thilo Borgmann <thilo.borgmann@mail.de> wrote: >> >> >> Not sure if you don't mix it with Video.... screw me... >> For Audio, I cannot find another device handling more than one format in one device. >> I'd appreciate a better way to do it than having N-devices... > > While not yet upstream, my version of the decklink output module > provides a single avdevice that supports multiple formats, including > AC-3 for passthrough. > > https://github.com/LTNGlobal-opensource/FFmpeg-ltn/blob/lted1/libavdevice/decklink_enc.cpp thanks! Patch v2 supports several codecs now :) -Thilo
diff --git a/configure b/configure index f97cad0298..1aab998d8e 100755 --- a/configure +++ b/configure @@ -3366,6 +3366,8 @@ alsa_outdev_deps="alsa" avfoundation_indev_deps="avfoundation corevideo coremedia pthreads" avfoundation_indev_suggest="coregraphics applicationservices" avfoundation_indev_extralibs="-framework Foundation" +audiotoolbox_outdev_deps="audiotoolbox pthreads" +audiotoolbox_outdev_extralibs="-framework AudioToolbox -framework CoreAudio" bktr_indev_deps_any="dev_bktr_ioctl_bt848_h machine_ioctl_bt848_h dev_video_bktr_ioctl_bt848_h dev_ic_bt8xx_h" caca_outdev_deps="libcaca" decklink_deps_any="libdl LoadLibrary" @@ -6151,6 +6153,7 @@ enabled videotoolbox && check_apple_framework VideoToolbox check_apple_framework CoreFoundation check_apple_framework CoreMedia check_apple_framework CoreVideo +check_apple_framework CoreAudio enabled avfoundation && { disable coregraphics applicationservices diff --git a/libavdevice/Makefile b/libavdevice/Makefile index 6ea62b914e..0dfe47a1f4 100644 --- a/libavdevice/Makefile +++ b/libavdevice/Makefile @@ -15,6 +15,7 @@ OBJS-$(CONFIG_SHARED) += reverse.o OBJS-$(CONFIG_ALSA_INDEV) += alsa_dec.o alsa.o timefilter.o OBJS-$(CONFIG_ALSA_OUTDEV) += alsa_enc.o alsa.o OBJS-$(CONFIG_ANDROID_CAMERA_INDEV) += android_camera.o +OBJS-$(CONFIG_AUDIOTOOLBOX_OUTDEV) += audiotoolbox.o OBJS-$(CONFIG_AVFOUNDATION_INDEV) += avfoundation.o OBJS-$(CONFIG_BKTR_INDEV) += bktr.o OBJS-$(CONFIG_CACA_OUTDEV) += caca.o diff --git a/libavdevice/alldevices.c b/libavdevice/alldevices.c index 8633433254..4246474536 100644 --- a/libavdevice/alldevices.c +++ b/libavdevice/alldevices.c @@ -27,6 +27,16 @@ extern AVInputFormat ff_alsa_demuxer; extern AVOutputFormat ff_alsa_muxer; extern AVInputFormat ff_android_camera_demuxer; +extern AVOutputFormat ff_audiotoolbox_muxer; +extern AVOutputFormat ff_audiotoolbox_f32_muxer; +extern AVOutputFormat ff_audiotoolbox_s32_muxer; +extern AVOutputFormat ff_audiotoolbox_s24_muxer; +extern AVOutputFormat ff_audiotoolbox_s16_muxer; +extern AVOutputFormat ff_audiotoolbox_s8_muxer; +extern AVOutputFormat ff_audiotoolbox_u32_muxer; +extern AVOutputFormat ff_audiotoolbox_u24_muxer; +extern AVOutputFormat ff_audiotoolbox_u16_muxer; +extern AVOutputFormat ff_audiotoolbox_u8_muxer; extern AVInputFormat ff_avfoundation_demuxer; extern AVInputFormat ff_bktr_demuxer; extern AVOutputFormat ff_caca_muxer; diff --git a/libavdevice/audiotoolbox.m b/libavdevice/audiotoolbox.m new file mode 100644 index 0000000000..9950e1fadd --- /dev/null +++ b/libavdevice/audiotoolbox.m @@ -0,0 +1,308 @@ +/* + * AudioToolbox output device + * Copyright (c) 2020 Thilo Borgmann <thilo.borgmann@mail.de> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * AudioToolbox output device + * @author Thilo Borgmann <thilo.borgmann@mail.de> + */ + +#import <AudioToolbox/AudioToolbox.h> +#include <pthread.h> + +#include "libavutil/opt.h" +#include "libavformat/internal.h" +#include "libavutil/internal.h" +#include "avdevice.h" + +typedef struct +{ + AVClass* class; + + AudioDeviceID* devices; + int num_devices; + + AudioQueueBufferRef buffer[2]; + pthread_mutex_t buffer_lock[2]; + int cur_buf; + AudioQueueRef queue; + + int list_devices; + int audio_device_index; + +} ATContext; + +static int check_status(ATContext *ctx, OSStatus *status, const char *msg) +{ + if (*status != noErr) { + av_log(ctx, AV_LOG_ERROR, "Error: %s (%i)\n", msg, *status); + return 1; + } else { + av_log(ctx, AV_LOG_DEBUG, " OK : %s\n", msg); + return 0; + } +} + +static void queue_callback(void* atctx, AudioQueueRef inAQ, + AudioQueueBufferRef inBuffer) +{ + // unlock the buffer that has just been consumed + ATContext *ctx = (ATContext*)atctx; + for (int i = 0; i < 2; i++) { + if (inBuffer == ctx->buffer[i]) { + pthread_mutex_unlock(&ctx->buffer_lock[i]); + } + } +} + +static av_cold int at_write_header(AVFormatContext *s) +{ + ATContext *ctx = (ATContext*)s->priv_data; + OSStatus err = noErr; + CFStringRef device_UID = NULL; + + // get devices + UInt32 data_size = 0; + AudioObjectPropertyAddress prop; + prop.mSelector = kAudioHardwarePropertyDevices; + prop.mScope = kAudioObjectPropertyScopeGlobal; + prop.mElement = kAudioObjectPropertyElementMaster; + err = AudioObjectGetPropertyDataSize(kAudioObjectSystemObject, &prop, 0, NULL, &data_size); + if (check_status(ctx, &err, "AudioObjectGetPropertyDataSize devices")) + return AVERROR(EINVAL); + + ctx->num_devices = data_size / sizeof(AudioDeviceID); + + ctx->devices = (AudioDeviceID*)(av_malloc(data_size)); + err = AudioObjectGetPropertyData(kAudioObjectSystemObject, &prop, 0, NULL, &data_size, ctx->devices); + if (check_status(ctx, &err, "AudioObjectGetPropertyData devices")) + return AVERROR(EINVAL); + + // list devices + if (ctx->list_devices) { + CFStringRef device_name = NULL; + prop.mScope = kAudioDevicePropertyScopeInput; + + av_log(ctx, AV_LOG_INFO, "CoreAudio devices:\n"); + for(UInt32 i = 0; i < ctx->num_devices; ++i) { + // UID + data_size = sizeof(device_UID); + prop.mSelector = kAudioDevicePropertyDeviceUID; + err = AudioObjectGetPropertyData(ctx->devices[i], &prop, 0, NULL, &data_size, &device_UID); + if (check_status(ctx, &err, "AudioObjectGetPropertyData UID")) + continue; + + // name + data_size = sizeof(device_name); + prop.mSelector = kAudioDevicePropertyDeviceNameCFString; + err = AudioObjectGetPropertyData(ctx->devices[i], &prop, 0, NULL, &data_size, &device_name); + if (check_status(ctx, &err, "AudioObjecTGetPropertyData name")) + continue; + + av_log(ctx, AV_LOG_INFO, "[%d] %30s, %s\n", i, + CFStringGetCStringPtr(device_name, kCFStringEncodingMacRoman), + CFStringGetCStringPtr(device_UID, kCFStringEncodingMacRoman)); + } + } + + // get user-defined device UID or use default device + // -audio_device_index overrides any URL given + const char *stream_name = s->url; + if (stream_name && ctx->audio_device_index == -1) { + sscanf(stream_name, "%d", &ctx->audio_device_index); + } + + if (ctx->audio_device_index >= 0) { + // get UID of selected device + data_size = sizeof(device_UID); + prop.mSelector = kAudioDevicePropertyDeviceUID; + err = AudioObjectGetPropertyData(ctx->devices[ctx->audio_device_index], &prop, 0, NULL, &data_size, &device_UID); + if (check_status(ctx, &err, "AudioObjecTGetPropertyData UID")) + return AVERROR(EINVAL); + } else { + // use default device + device_UID = NULL; + } + + av_log(ctx, AV_LOG_DEBUG, "stream_name: %s\n", stream_name); + av_log(ctx, AV_LOG_DEBUG, "audio_device_idnex: %i\n", ctx->audio_device_index); + av_log(ctx, AV_LOG_DEBUG, "UID: %s\n", CFStringGetCStringPtr(device_UID, kCFStringEncodingMacRoman)); + + // check input stream + if (s->nb_streams != 1 || s->streams[0]->codecpar->codec_type != AVMEDIA_TYPE_AUDIO) { + av_log(ctx, AV_LOG_ERROR, "Only a single audio stream is supported.\n"); + return AVERROR(EINVAL); + } + + AVCodecParameters *codecpar = s->streams[0]->codecpar; + + // audio format + AudioStreamBasicDescription device_format = {0}; + device_format.mSampleRate = codecpar->sample_rate; + device_format.mFormatID = kAudioFormatLinearPCM; + device_format.mFormatFlags |= (codecpar->format == AV_SAMPLE_FMT_FLT) ? kLinearPCMFormatFlagIsFloat : 0; + device_format.mFormatFlags |= (codecpar->codec_id == AV_CODEC_ID_PCM_S8) ? kLinearPCMFormatFlagIsSignedInteger : 0; + device_format.mFormatFlags |= (codecpar->codec_id == AV_NE(AV_CODEC_ID_PCM_S16BE, AV_CODEC_ID_PCM_S16LE)) ? kLinearPCMFormatFlagIsSignedInteger : 0; + device_format.mFormatFlags |= (codecpar->codec_id == AV_NE(AV_CODEC_ID_PCM_S24BE, AV_CODEC_ID_PCM_S24LE)) ? kLinearPCMFormatFlagIsSignedInteger : 0; + device_format.mFormatFlags |= (codecpar->codec_id == AV_NE(AV_CODEC_ID_PCM_S32BE, AV_CODEC_ID_PCM_S32LE)) ? kLinearPCMFormatFlagIsSignedInteger : 0; + device_format.mFormatFlags |= (av_sample_fmt_is_planar(codecpar->format)) ? kAudioFormatFlagIsNonInterleaved : 0; + device_format.mFormatFlags |= AV_NE(kAudioFormatFlagIsBigEndian, 0); + device_format.mChannelsPerFrame = codecpar->channels; + device_format.mBitsPerChannel = (codecpar->codec_id == AV_NE(AV_CODEC_ID_PCM_S24BE, AV_CODEC_ID_PCM_S24LE)) ? 24 : (av_get_bytes_per_sample(codecpar->format) << 3); + device_format.mBytesPerFrame = (device_format.mBitsPerChannel >> 3) * device_format.mChannelsPerFrame; + device_format.mFramesPerPacket = 1; + device_format.mBytesPerPacket = device_format.mBytesPerFrame * device_format.mFramesPerPacket; + device_format.mReserved = 0; + + av_log(ctx, AV_LOG_DEBUG, "device_format.mSampleRate = %i\n", codecpar->sample_rate); + av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatID = %s\n", "kAudioFormatLinearPCM"); + av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatFlags |= %s\n", (codecpar->format == AV_SAMPLE_FMT_FLT) ? "kLinearPCMFormatFlagIsFloat" : "0"); + av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatFlags |= %s\n", (codecpar->codec_id == AV_CODEC_ID_PCM_S8) ? "kLinearPCMFormatFlagIsSignedInteger" : "0"); + av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatFlags |= %s\n", (codecpar->codec_id == AV_NE(AV_CODEC_ID_PCM_S32BE, AV_CODEC_ID_PCM_S32LE)) ? "kLinearPCMFormatFlagIsSignedInteger" : "0"); + av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatFlags |= %s\n", (codecpar->codec_id == AV_NE(AV_CODEC_ID_PCM_S16BE, AV_CODEC_ID_PCM_S16LE)) ? "kLinearPCMFormatFlagIsSignedInteger" : "0"); + av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatFlags |= %s\n", (codecpar->codec_id == AV_NE(AV_CODEC_ID_PCM_S24BE, AV_CODEC_ID_PCM_S24LE)) ? "kLinearPCMFormatFlagIsSignedInteger" : "0"); + av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatFlags |= %s\n", (av_sample_fmt_is_planar(codecpar->format)) ? "kAudioFormatFlagIsNonInterleaved" : "0"); + av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatFlags |= %s\n", AV_NE("kAudioFormatFlagIsBigEndian", "0")); + av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatFlags == %i\n", device_format.mFormatFlags); + av_log(ctx, AV_LOG_DEBUG, "device_format.mChannelsPerFrame = %i\n", codecpar->channels); + av_log(ctx, AV_LOG_DEBUG, "device_format.mBitsPerChannel = %i\n", av_get_bytes_per_sample(codecpar->format) << 3); + av_log(ctx, AV_LOG_DEBUG, "device_format.mBytesPerFrame = %i\n", (device_format.mBitsPerChannel >> 3) * codecpar->channels); + av_log(ctx, AV_LOG_DEBUG, "device_format.mBytesPerPacket = %i\n", device_format.mBytesPerFrame); + av_log(ctx, AV_LOG_DEBUG, "device_format.mFramesPerPacket = %i\n", 1); + av_log(ctx, AV_LOG_DEBUG, "device_format.mReserved = %i\n", 0); + + // create new output queue for the device + err = AudioQueueNewOutput(&device_format, queue_callback, ctx, + NULL, kCFRunLoopCommonModes, + 0, &ctx->queue); + if (check_status(ctx, &err, "AudioQueueNewOutput")) + return AVERROR(EINVAL); + + // set user-defined device or leave untouched for default + if (device_UID != NULL) { + err = AudioQueueSetProperty(ctx->queue, kAudioQueueProperty_CurrentDevice, &device_UID, sizeof(device_UID)); + if (check_status(ctx, &err, "AudioQueueSetProperty output UID")) + return AVERROR(EINVAL); + } + + // start the queue + err = AudioQueueStart(ctx->queue, NULL); + if (check_status(ctx, &err, "AudioQueueStart")) + return AVERROR(EINVAL); + + // init the mutexes for double-buffering + pthread_mutex_init(&ctx->buffer_lock[0], NULL); + pthread_mutex_init(&ctx->buffer_lock[1], NULL); + + return 0; +} + +static int at_write_packet(AVFormatContext *s, AVPacket *pkt) +{ + ATContext *ctx = (ATContext*)s->priv_data; + OSStatus err = noErr; + + // use the other buffer + ctx->cur_buf = !ctx->cur_buf; + + // lock for writing or wait for the buffer to be available + // will be unlocked by queue callback + pthread_mutex_lock(&ctx->buffer_lock[ctx->cur_buf]); + + // (re-)allocate the buffer if not existant or of different size + if (!ctx->buffer[ctx->cur_buf] || ctx->buffer[ctx->cur_buf]->mAudioDataBytesCapacity != pkt->size) { + err = AudioQueueAllocateBuffer(ctx->queue, pkt->size, &ctx->buffer[ctx->cur_buf]); + if (check_status(ctx, &err, "AudioQueueAllocateBuffer")) { + pthread_mutex_unlock(&ctx->buffer_lock[ctx->cur_buf]); + return AVERROR(ENOMEM); + } + } + + AudioQueueBufferRef buf = ctx->buffer[ctx->cur_buf]; + + // copy audio data into buffer and enqueue the buffer + memcpy(buf->mAudioData, pkt->data, buf->mAudioDataBytesCapacity); + buf->mAudioDataByteSize = buf->mAudioDataBytesCapacity; + err = AudioQueueEnqueueBuffer(ctx->queue, buf, 0, NULL); + if (check_status(ctx, &err, "AudioQueueEnqueueBuffer")) { + pthread_mutex_unlock(&ctx->buffer_lock[ctx->cur_buf]); + return AVERROR(EINVAL); + } + + return 0; +} + +static av_cold int at_write_trailer(AVFormatContext *s) +{ + ATContext *ctx = (ATContext*)s->priv_data; + OSStatus err = noErr; + + av_freep(&ctx->devices); + pthread_mutex_destroy(&ctx->buffer_lock[0]); + pthread_mutex_destroy(&ctx->buffer_lock[1]); + + err = AudioQueueFlush(ctx->queue); + check_status(ctx, &err, "AudioQueueFlush"); + err = AudioQueueDispose(ctx->queue, true); + check_status(ctx, &err, "AudioQueueDispose"); + + return 0; +} + +static const AVOption options[] = { + { "list_devices", "list available audio devices", offsetof(ATContext, list_devices), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AV_OPT_FLAG_ENCODING_PARAM }, + { "audio_device_index", "select audio device by index (starts at 0)", offsetof(ATContext, audio_device_index), AV_OPT_TYPE_INT, {.i64 = -1}, -1, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM }, + { NULL }, +}; + +static const AVClass at_class = { + .class_name = "AudioToolbox outdev", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, + .category = AV_CLASS_CATEGORY_DEVICE_AUDIO_OUTPUT, +}; + +#define ATDEF(name_, long_name_, codec) \ +AVOutputFormat ff_ ## name_ ## _muxer = { \ + .name = #name_, \ + .long_name = NULL_IF_CONFIG_SMALL(long_name_), \ + .priv_data_size = sizeof(ATContext), \ + .audio_codec = codec, \ + .video_codec = AV_CODEC_ID_NONE, \ + .write_header = at_write_header, \ + .write_packet = at_write_packet, \ + .write_trailer = at_write_trailer, \ + .flags = AVFMT_NOFILE, \ + .priv_class = &at_class, \ +}; + +ATDEF(audiotoolbox, "AudioToolbox (S16) output device", AV_NE(AV_CODEC_ID_PCM_S16BE, AV_CODEC_ID_PCM_S16LE)); +ATDEF(audiotoolbox_f32, "AudioToolbox (F32) output device", AV_NE(AV_CODEC_ID_PCM_F32BE, AV_CODEC_ID_PCM_F32LE)); +ATDEF(audiotoolbox_s32, "AudioToolbox (S32) output device", AV_NE(AV_CODEC_ID_PCM_S32BE, AV_CODEC_ID_PCM_S32LE)); +ATDEF(audiotoolbox_s24, "AudioToolbox (S24) output device", AV_NE(AV_CODEC_ID_PCM_S24BE, AV_CODEC_ID_PCM_S24LE)); +ATDEF(audiotoolbox_s16, "AudioToolbox (S16) output device", AV_NE(AV_CODEC_ID_PCM_S16BE, AV_CODEC_ID_PCM_S16LE)); +ATDEF(audiotoolbox_s8, "AudioToolbox (S8) output device", AV_CODEC_ID_PCM_S8); +ATDEF(audiotoolbox_u32, "AudioToolbox (U32) output device", AV_NE(AV_CODEC_ID_PCM_U32BE, AV_CODEC_ID_PCM_U32LE)); +ATDEF(audiotoolbox_u24, "AudioToolbox (U24) output device", AV_NE(AV_CODEC_ID_PCM_U24BE, AV_CODEC_ID_PCM_U24LE)); +ATDEF(audiotoolbox_u16, "AudioToolbox (U16) output device", AV_NE(AV_CODEC_ID_PCM_U16BE, AV_CODEC_ID_PCM_U16LE)); +ATDEF(audiotoolbox_u8, "AudioToolbox (U8) output device", AV_CODEC_ID_PCM_U8); +
Hi, $subject. Enables native audio output on OSX. OSX accepts numerous formats, so there are several output devices. -Thilo From bdabbc6a919ccff20d6ae16acff1a5d0f8ca46b8 Mon Sep 17 00:00:00 2001 From: Thilo Borgmann <thilo.borgmann@mail.de> Date: Mon, 8 Jun 2020 00:20:25 +0200 Subject: [PATCH] lavdevice: Add AudioToolbox output device. --- configure | 3 + libavdevice/Makefile | 1 + libavdevice/alldevices.c | 10 ++ libavdevice/audiotoolbox.m | 308 +++++++++++++++++++++++++++++++++++++ 4 files changed, 322 insertions(+) create mode 100644 libavdevice/audiotoolbox.m