From patchwork Tue Mar 22 13:39:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Romain Beauxis X-Patchwork-Id: 34897 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:ab0:5fda:0:0:0:0:0 with SMTP id g26csp680207uaj; Tue, 22 Mar 2022 06:41:26 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxPwfwNLpq/MwZmk11aTpMqt3SJU9bibt9sg9loc3twhkQBQ35siwfjZQ+PBuf2+skintkV X-Received: by 2002:a17:906:5d09:b0:6df:d52f:58f6 with SMTP id g9-20020a1709065d0900b006dfd52f58f6mr16402519ejt.721.1647956486662; Tue, 22 Mar 2022 06:41:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1647956486; cv=none; d=google.com; s=arc-20160816; b=l6Ot28lgHn5yheSKg/v2q7IzhxS55U0WRr2IHh2vs6VjCqLnK0RuAo3HxjH5zoVyMc 5Tvft7Aa4ER9U/6yjmmHuFm37ozX38KOBuMF3ptHD+IePJflqx9nwqBOxC+WcCAhama2 SL+Zp+hBRY8bAtq9Ecpv43aJbNKSBj2Faeu7VVxjgP2jQnRkTFcgSenxr0zBYqhALydz N8q8wgOzm8muBurj6asdXxYY1n3sIRV6jmU062cmO5XAJwXMXYsBAg6LSN9dtXoyZ8Th y96yITce2n9wyEwH64F2O/i/MCHvFB6PqJcXWfsZsCwfgTpd6ZiNF6SFcCfQSHrPsiUb qnAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=OaD8aNgWLxDbfKhUZq8OE+nG+6dx/hLJF0XTgjoVWvs=; b=dxzx1NyGefudFGLX0Efd1pBcat5Qfm9Qf4qCYGtY3TUoBiEVV7sXchbpO3vbWkobWB GIvRS0HZRsklxabUQdU4wpRPOHahR1IJSCdW8o4U45jh5PW4Ga3yyYrXgJoCmb6iugG+ zhoTN8HViSbhJ+CA/OYuRARjrMYOexTN+mp2238RXDhrkHtBGGyZT7d4Y3p1xUu4k14C mDiM/XMGMWL3i2ELuo7UMzZ7t2CEe1oxdbd3k4w3gcbvmW+sOPeAUnEnObuxxUS7F5Zu 5HIODjGl+w3UjzX5FU4qgkEPpOLT4ZEMZ0261Hd+ZkeL+55IKUrVLpKhkoYE9ErFluIu EBUQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id fy2-20020a170906b7c200b006df76385eebsi9651656ejb.907.2022.03.22.06.41.22; Tue, 22 Mar 2022 06:41:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 14F7468B1C5; Tue, 22 Mar 2022 15:40:36 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C05EF68B177 for ; Tue, 22 Mar 2022 15:40:28 +0200 (EET) Received: by mail-ej1-f52.google.com with SMTP id bg10so36265809ejb.4 for ; Tue, 22 Mar 2022 06:40:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=S8v3rejqqFiXOR3auaJPl9qtuBQ2YQ4lqfcBMiPjzZI=; b=1Xf2PVi0eOX4Fn3OZdx3DEZ06HyH4VXAwsVIpEmj4AHIp8PZUEHYT838+B752pap6G 32frLEGIO/SDMg0XojdEgFDVogtnkOg2XP0Tc5rEHmSfmqJUyasrqMifVaYz6ySpTdpF m5U9UoRNqTSctD14vYuQS1rAq/2RnWe9Emhu+iPQQ/zYQLeWddWMPjR79t6TQ9V5MSay EPeQ5o8FNDOgrxTPUn6Xcfc5+HmuoFtS+BU80GGSsCDg2KvxAQbS/2A0TvPGyLdVUEqE EkN17sI44N69gpLlmYODlqXJ7eh+aNx3cB9oWYXxXcEYIuiTDiOgP2NseGWuh4DKszPG 9+Rw== X-Gm-Message-State: AOAM533JDrFNp8UZqRT0+MLm9gR7NA+8qeGmExaWEvnOaTi1Z9FQuZ9J Emoab0bJ49Y1t2awwAp7ZkGnOxJfxXGX5hFW X-Received: by 2002:a17:906:2695:b0:6ce:f9c:b476 with SMTP id t21-20020a170906269500b006ce0f9cb476mr25942308ejc.235.1647956427468; Tue, 22 Mar 2022 06:40:27 -0700 (PDT) Received: from localhost.localdomain (82-171-134-41.fixed.kpn.net. [82.171.134.41]) by smtp.gmail.com with ESMTPSA id z6-20020a056402274600b004194fc1b7casm2125829edd.48.2022.03.22.06.40.26 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 22 Mar 2022 06:40:26 -0700 (PDT) From: toots@rastageeks.org To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Mar 2022 14:39:57 +0100 Message-Id: <20220322133957.81743-7-toots@rastageeks.org> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220322133957.81743-1-toots@rastageeks.org> References: <20220322133957.81743-1-toots@rastageeks.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 6/6] Add AudioToolbox audio input device. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Romain Beauxis Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 0dIOJ5WwufoA From: Romain Beauxis diff --git a/configure b/configure index a7953ffc16..37f9f7b80a 100755 --- a/configure +++ b/configure @@ -203,6 +203,7 @@ External library support: --disable-avfoundation disable Apple AVFoundation framework [autodetect] --enable-avisynth enable reading of AviSynth script files [no] --disable-bzlib disable bzlib [autodetect] + --disable-coremedia disable Apple CoreMedia framework [autodetect] --disable-coreimage disable Apple CoreImage framework [autodetect] --enable-chromaprint enable audio fingerprinting with chromaprint [no] --enable-frei0r enable frei0r video filtering [no] @@ -1751,6 +1752,7 @@ EXTERNAL_AUTODETECT_LIBRARY_LIST=" appkit avfoundation bzlib + coremedia coreimage iconv libxcb @@ -3488,6 +3490,8 @@ alsa_outdev_deps="alsa" avfoundation_indev_deps="avfoundation corevideo coremedia pthreads" avfoundation_indev_suggest="coregraphics applicationservices" avfoundation_indev_extralibs="-framework Foundation" +audiotoolbox_indev_deps="coremedia audiotoolbox" +audiotoolbox_indev_extralibs="-framework CoreMedia -framework AudioToolbox" audiotoolbox_outdev_deps="audiotoolbox pthreads" audiotoolbox_outdev_extralibs="-framework AudioToolbox -framework CoreAudio" bktr_indev_deps_any="dev_bktr_ioctl_bt848_h machine_ioctl_bt848_h dev_video_bktr_ioctl_bt848_h dev_ic_bt8xx_h" @@ -6330,6 +6334,7 @@ check_lib camera2ndk "stdbool.h stdint.h camera/NdkCameraManager.h" ACameraManag enabled appkit && check_apple_framework AppKit enabled audiotoolbox && check_apple_framework AudioToolbox enabled avfoundation && check_apple_framework AVFoundation +enabled coremedia && check_apple_framework CoreMedia enabled coreimage && check_apple_framework CoreImage enabled metal && check_apple_framework Metal enabled videotoolbox && check_apple_framework VideoToolbox diff --git a/doc/indevs.texi b/doc/indevs.texi index 858c0fa4e4..8d57a26a5f 100644 --- a/doc/indevs.texi +++ b/doc/indevs.texi @@ -103,6 +103,41 @@ Set the maximum number of frames to buffer. Default is 5. @end table +@section AudioToolbox + +AudioToolbox input device. + +Allows native input from CoreAudio devices on OSX. + +@subsection Options + +AudioToolbox supports the following options: + +@table @option + +@item channels +Set the number of channels. Default is device's default. + +@item frames_queue_length +Maximum of buffers in the input queue + +@item buffer_frame_size +Buffer frame size, gouverning internal latency + +@item big_endian +Return big endian samples + +@item sample_format +Sample format + +@end table + +@subsection Examples + +@itemize + +@end itemize + @section avfoundation AVFoundation input device. diff --git a/libavdevice/Makefile b/libavdevice/Makefile index 99fea7133a..78d4168521 100644 --- a/libavdevice/Makefile +++ b/libavdevice/Makefile @@ -15,6 +15,7 @@ OBJS-$(HAVE_LIBC_MSVCRT) += file_open.o OBJS-$(CONFIG_ALSA_INDEV) += alsa_dec.o alsa.o timefilter.o OBJS-$(CONFIG_ALSA_OUTDEV) += alsa_enc.o alsa.o OBJS-$(CONFIG_ANDROID_CAMERA_INDEV) += android_camera.o +OBJS-$(CONFIG_AUDIOTOOLBOX_INDEV) += audiotoolbox_dec.o OBJS-$(CONFIG_AUDIOTOOLBOX_OUTDEV) += audiotoolbox.o OBJS-$(CONFIG_AVFOUNDATION_INDEV) += avfoundation.o OBJS-$(CONFIG_BKTR_INDEV) += bktr.o diff --git a/libavdevice/alldevices.c b/libavdevice/alldevices.c index 22323a0a44..fbecdbb0b2 100644 --- a/libavdevice/alldevices.c +++ b/libavdevice/alldevices.c @@ -26,6 +26,7 @@ extern const AVInputFormat ff_alsa_demuxer; extern const AVOutputFormat ff_alsa_muxer; extern const AVInputFormat ff_android_camera_demuxer; extern const AVOutputFormat ff_audiotoolbox_muxer; +extern const AVInputFormat ff_audiotoolbox_demuxer; extern const AVInputFormat ff_avfoundation_demuxer; extern const AVInputFormat ff_bktr_demuxer; extern const AVOutputFormat ff_caca_muxer; diff --git a/libavdevice/audiotoolbox.m b/libavdevice/audiotoolbox.m index 0cb97b5e46..3216f14607 100644 --- a/libavdevice/audiotoolbox.m +++ b/libavdevice/audiotoolbox.m @@ -84,7 +84,7 @@ static av_cold int at_write_header(AVFormatContext *avctx) AudioObjectPropertyAddress prop; prop.mSelector = kAudioHardwarePropertyDevices; prop.mScope = kAudioObjectPropertyScopeGlobal; - prop.mElement = kAudioObjectPropertyElementMaster; + prop.mElement = 0; err = AudioObjectGetPropertyDataSize(kAudioObjectSystemObject, &prop, 0, NULL, &data_size); if (check_status(avctx, &err, "AudioObjectGetPropertyDataSize devices")) return AVERROR(EINVAL); @@ -173,7 +173,7 @@ static av_cold int at_write_header(AVFormatContext *avctx) device_format.mFormatFlags |= (codecpar->codec_id == AV_CODEC_ID_PCM_S16BE) ? kAudioFormatFlagIsBigEndian : 0; device_format.mFormatFlags |= (codecpar->codec_id == AV_CODEC_ID_PCM_S24BE) ? kAudioFormatFlagIsBigEndian : 0; device_format.mFormatFlags |= (codecpar->codec_id == AV_CODEC_ID_PCM_S32BE) ? kAudioFormatFlagIsBigEndian : 0; - device_format.mChannelsPerFrame = codecpar->channels; + device_format.mChannelsPerFrame = codecpar->ch_layout.nb_channels; device_format.mBitsPerChannel = (codecpar->codec_id == AV_NE(AV_CODEC_ID_PCM_S24BE, AV_CODEC_ID_PCM_S24LE)) ? 24 : (av_get_bytes_per_sample(codecpar->format) << 3); device_format.mBytesPerFrame = (device_format.mBitsPerChannel >> 3) * device_format.mChannelsPerFrame; device_format.mFramesPerPacket = 1; @@ -193,9 +193,9 @@ static av_cold int at_write_header(AVFormatContext *avctx) av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatFlags |= %s\n", (codecpar->codec_id == AV_CODEC_ID_PCM_S24BE) ? "kAudioFormatFlagIsBigEndian" : "0"); av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatFlags |= %s\n", (codecpar->codec_id == AV_CODEC_ID_PCM_S32BE) ? "kAudioFormatFlagIsBigEndian" : "0"); av_log(ctx, AV_LOG_DEBUG, "device_format.mFormatFlags == %i\n", device_format.mFormatFlags); - av_log(ctx, AV_LOG_DEBUG, "device_format.mChannelsPerFrame = %i\n", codecpar->channels); + av_log(ctx, AV_LOG_DEBUG, "device_format.mChannelsPerFrame = %i\n", codecpar->ch_layout.nb_channels); av_log(ctx, AV_LOG_DEBUG, "device_format.mBitsPerChannel = %i\n", av_get_bytes_per_sample(codecpar->format) << 3); - av_log(ctx, AV_LOG_DEBUG, "device_format.mBytesPerFrame = %i\n", (device_format.mBitsPerChannel >> 3) * codecpar->channels); + av_log(ctx, AV_LOG_DEBUG, "device_format.mBytesPerFrame = %i\n", (device_format.mBitsPerChannel >> 3) * codecpar->ch_layout.nb_channels); av_log(ctx, AV_LOG_DEBUG, "device_format.mBytesPerPacket = %i\n", device_format.mBytesPerFrame); av_log(ctx, AV_LOG_DEBUG, "device_format.mFramesPerPacket = %i\n", 1); av_log(ctx, AV_LOG_DEBUG, "device_format.mReserved = %i\n", 0); diff --git a/libavdevice/audiotoolbox_dec.m b/libavdevice/audiotoolbox_dec.m new file mode 100644 index 0000000000..ae80c0e54a --- /dev/null +++ b/libavdevice/audiotoolbox_dec.m @@ -0,0 +1,530 @@ +/* + * AudioToolbox input device + * Copyright (c) 2022 Romain Beauxis + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * AudioToolbox input device + * @author Romain Beauxis + */ + +#import +#import + +#include "libavutil/channel_layout.h" +#include "libavformat/internal.h" +#include "avdevice.h" + +typedef struct { + void *data; + int size; +} buffer_t; + +typedef struct { + AVClass *class; + CMSimpleQueueRef frames_queue; + AudioUnit audio_unit; + AudioStreamBasicDescription record_format; + uint64_t position; + int frames_queue_length; + int buffer_frame_size; + int stream_index; + int big_endian; + enum AVSampleFormat sample_format; + int channels; +} ATContext; + +static int check_status(void *ctx, OSStatus status, const char *msg) { + if (status == noErr) { + av_log(ctx, AV_LOG_DEBUG, "OK: %s\n", msg); + return 0; + } + + NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; + NSError *error = [NSError errorWithDomain:NSOSStatusErrorDomain code:status userInfo:nil]; + av_log(ctx, AV_LOG_ERROR, "Error: %s (%s)\n", msg, [[error localizedDescription] UTF8String]); + [pool release]; + return 1; +} + +static OSStatus input_callback(void *priv, + AudioUnitRenderActionFlags *ioActionFlags, + const AudioTimeStamp *inTimeStamp, + UInt32 inBusNumber, + UInt32 inNumberFrames, + AudioBufferList *ioData) { + ATContext *ctx = (ATContext *)priv; + OSStatus err; + + AudioBuffer audio_buffer; + + audio_buffer.mNumberChannels = ctx->channels; + audio_buffer.mDataByteSize = inNumberFrames * ctx->record_format.mBytesPerFrame; + + audio_buffer.mData = av_malloc(audio_buffer.mDataByteSize); + memset(audio_buffer.mData, 0, audio_buffer.mDataByteSize); + + AudioBufferList bufferList; + bufferList.mNumberBuffers = 1; + bufferList.mBuffers[0] = audio_buffer; + + err = AudioUnitRender(ctx->audio_unit, + ioActionFlags, + inTimeStamp, + inBusNumber, + inNumberFrames, + &bufferList); + if (check_status(ctx, err, "AudioUnitRender")) { + av_freep(&audio_buffer.mData); + return err; + } + + buffer_t *buffer = av_malloc(sizeof(buffer_t)); + buffer->data = audio_buffer.mData; + buffer->size = audio_buffer.mDataByteSize; + err = CMSimpleQueueEnqueue(ctx->frames_queue, buffer); + + if (err != noErr) { + av_log(ctx, AV_LOG_DEBUG, "Could not enqueue audio frame!\n"); + return err; + } + + return noErr; +} + +static av_cold int audiotoolbox_read_header(AVFormatContext *avctx) { + ATContext *ctx = (ATContext*)avctx->priv_data; + OSStatus err = noErr; + AudioChannelLayout *channel_layout = NULL; + AudioDeviceID device = kAudioObjectUnknown; + AudioObjectPropertyAddress prop; + UInt32 data_size; + + enum AVCodecID codec_id = av_get_pcm_codec(ctx->sample_format, ctx->big_endian); + + if (codec_id == AV_CODEC_ID_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error: invalid sample format!\n"); + return AVERROR(EINVAL); + } + + prop.mScope = kAudioObjectPropertyScopeGlobal; + prop.mElement = 0; + + if (!strcmp(avctx->url, "default")) { + prop.mSelector = kAudioHardwarePropertyDefaultInputDevice; + data_size = sizeof(AudioDeviceID); + err = AudioObjectGetPropertyData(kAudioObjectSystemObject, &prop, 0, NULL, &data_size, &device); + if (check_status(avctx, err, "AudioObjectGetPropertyData DefaultInputDevice")) + goto fail; + } else { + prop.mSelector = kAudioHardwarePropertyDeviceForUID; + CFStringRef deviceUID = CFStringCreateWithCStringNoCopy(NULL, avctx->url, kCFStringEncodingUTF8, kCFAllocatorNull); + AudioValueTranslation value; + value.mInputData = &deviceUID; + value.mInputDataSize = sizeof(deviceUID);; + value.mOutputData = &device; + value.mOutputDataSize = sizeof(device); + data_size = sizeof(value); + + err = AudioObjectGetPropertyData(kAudioObjectSystemObject, &prop, 0, NULL, &data_size, &value); + CFRelease(deviceUID); + + if (check_status(avctx, err, "AudioObjectGetPropertyData DeviceForUID")) + goto fail; + } + + Float64 sample_rate; + prop.mSelector = kAudioDevicePropertyNominalSampleRate; + prop.mScope = kAudioObjectPropertyScopeInput; + data_size = sizeof(sample_rate); + err = AudioObjectGetPropertyData(device, &prop, 0, NULL, &data_size, &sample_rate); + if (check_status(avctx, err, "AudioObjectGetPropertyData SampleRate")) + goto fail; + + if (!ctx->channels) { + prop.mSelector = kAudioDevicePropertyPreferredChannelLayout; + prop.mScope = kAudioObjectPropertyScopeInput; + UInt32 channel_layout_size; + + err = AudioObjectGetPropertyDataSize(device, &prop, 0, NULL, &channel_layout_size); + if (check_status(avctx, err, "AudioObjectGetPropertyDataSize PreferredChannelLayout")) + goto fail; + + channel_layout = av_malloc(channel_layout_size); + err = AudioObjectGetPropertyData(device, &prop, 0, NULL, &channel_layout_size, channel_layout); + if (check_status(avctx, err, "AudioObjectGetPropertyData PreferredChannelLayout")) + goto fail; + + data_size = sizeof(ctx->channels); + err = AudioFormatGetProperty(kAudioFormatProperty_NumberOfChannelsForLayout, channel_layout_size, channel_layout, &data_size, &ctx->channels); + if (check_status(avctx, err, "AudioFormatGetProperty NumberOfChannelsForLayout")) + goto fail; + } + + ctx->record_format.mFormatID = kAudioFormatLinearPCM; + ctx->record_format.mChannelsPerFrame = ctx->channels; + ctx->record_format.mFormatFlags = kAudioFormatFlagIsPacked; + ctx->record_format.mBitsPerChannel = av_get_bytes_per_sample(ctx->sample_format) << 3; + + if (ctx->big_endian) + ctx->record_format.mFormatFlags |= kAudioFormatFlagIsBigEndian; + + switch (ctx->sample_format) { + case AV_SAMPLE_FMT_S16: + case AV_SAMPLE_FMT_S32: + ctx->record_format.mFormatFlags |= kAudioFormatFlagIsSignedInteger; + break; + case AV_SAMPLE_FMT_FLT: + ctx->record_format.mFormatFlags |= kAudioFormatFlagIsFloat; + break; + default: + av_log(ctx, AV_LOG_ERROR, "Error: invalid sample format!\n"); + goto fail; + } + + av_log(ctx, AV_LOG_DEBUG, "Audio Input: %s\n", avctx->url); + av_log(ctx, AV_LOG_DEBUG, "samplerate: %d\n", (int)sample_rate); + av_log(ctx, AV_LOG_DEBUG, "channels: %d\n", ctx->channels); + av_log(ctx, AV_LOG_DEBUG, "Input format: %s\n", avcodec_get_name(codec_id)); + + data_size = sizeof(ctx->record_format); + err = AudioFormatGetProperty(kAudioFormatProperty_FormatInfo, 0, NULL, &data_size, &ctx->record_format); + if (check_status(avctx, err, "AudioFormatGetProperty FormatInfo")) + goto fail; + + AudioComponentDescription desc; + AudioComponent comp; + + desc.componentType = kAudioUnitType_Output; + desc.componentSubType = kAudioUnitSubType_HALOutput; + desc.componentManufacturer = kAudioUnitManufacturer_Apple; + desc.componentFlags = 0; + desc.componentFlagsMask = 0; + + comp = AudioComponentFindNext(NULL, &desc); + if (comp == NULL) { + av_log(ctx, AV_LOG_ERROR, "Error: AudioComponentFindNext\n"); + goto fail; + } + + err = AudioComponentInstanceNew(comp, &ctx->audio_unit); + if (check_status(avctx, err, "AudioComponentInstanceNew")) + goto fail; + + UInt32 enableIO = 1; + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioOutputUnitProperty_EnableIO, + kAudioUnitScope_Input, + 1, + &enableIO, + sizeof(enableIO)); + if (check_status(avctx, err, "AudioUnitSetProperty EnableIO")) + goto fail; + + enableIO = 0; + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioOutputUnitProperty_EnableIO, + kAudioUnitScope_Output, + 0, + &enableIO, + sizeof(enableIO)); + if (check_status(avctx, err, "AudioUnitSetProperty EnableIO")) + goto fail; + + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioOutputUnitProperty_CurrentDevice, + kAudioUnitScope_Global, + 0, + &device, + sizeof(device)); + if (check_status(avctx, err, "AudioUnitSetProperty CurrentDevice")) + goto fail; + + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioUnitProperty_StreamFormat, + kAudioUnitScope_Input, + 0, + &ctx->record_format, + sizeof(ctx->record_format)); + if (check_status(avctx, err, "AudioUnitSetProperty StreamFormat")) + goto fail; + + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioUnitProperty_StreamFormat, + kAudioUnitScope_Output, + 1, + &ctx->record_format, + sizeof(ctx->record_format)); + if (check_status(avctx, err, "AudioUnitSetProperty StreamFormat")) + goto fail; + + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioDevicePropertyBufferFrameSize, + kAudioUnitScope_Global, + 0, + &ctx->buffer_frame_size, + sizeof(ctx->buffer_frame_size)); + if (check_status(avctx, err, "AudioUnitSetProperty BufferFrameSize")) + goto fail; + + AURenderCallbackStruct callback = {0}; + callback.inputProc = input_callback; + callback.inputProcRefCon = ctx; + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioOutputUnitProperty_SetInputCallback, + kAudioUnitScope_Global, + 0, + &callback, + sizeof(callback)); + if (check_status(avctx, err, "AudioUnitSetProperty SetInputCallback")) + goto fail; + + err = AudioUnitInitialize(ctx->audio_unit); + if (check_status(avctx, err, "AudioUnitInitialize")) + goto fail; + + err = CMSimpleQueueCreate(kCFAllocatorDefault, ctx->frames_queue_length, &ctx->frames_queue); + if (check_status(avctx, err, "CMSimpleQueueCreate")) + goto fail; + + CFRetain(ctx->frames_queue); + + err = AudioUnitInitialize(ctx->audio_unit); + if (check_status(avctx, err, "AudioUnitInitialize")) + goto fail; + + err = AudioOutputUnitStart(ctx->audio_unit); + if (check_status(avctx, err, "AudioOutputUnitStart")) + goto fail; + + AVStream* stream = avformat_new_stream(avctx, NULL); + stream->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; + stream->codecpar->sample_rate = sample_rate; + stream->codecpar->channels = ctx->channels; + stream->codecpar->channel_layout = av_get_default_channel_layout(stream->codecpar->channels); + stream->codecpar->codec_id = codec_id; + + avpriv_set_pts_info(stream, 64, 1, sample_rate); + + ctx->stream_index = stream->index; + ctx->position = 0; + + av_freep(&channel_layout); + return 0; + +fail: + av_freep(&channel_layout); + return AVERROR(EINVAL); +} + +static int audiotoolbox_read_packet(AVFormatContext *avctx, AVPacket *pkt) { + ATContext *ctx = (ATContext*)avctx->priv_data; + + if (CMSimpleQueueGetCount(ctx->frames_queue) < 1) + return AVERROR(EAGAIN); + + buffer_t *buffer = (buffer_t *)CMSimpleQueueDequeue(ctx->frames_queue); + + int status = av_packet_from_data(pkt, buffer->data, buffer->size); + if (status < 0) { + av_freep(&buffer->data); + av_freep(&buffer); + return status; + } + + pkt->stream_index = ctx->stream_index; + pkt->flags |= AV_PKT_FLAG_KEY; + pkt->pts = pkt->dts = ctx->position; + + ctx->position += pkt->size / (ctx->channels * av_get_bytes_per_sample(ctx->sample_format)); + + av_freep(&buffer); + return 0; +} + +static av_cold int audiotoolbox_close(AVFormatContext *avctx) { + ATContext *ctx = (ATContext*)avctx->priv_data; + + if (ctx->audio_unit) { + AudioOutputUnitStop(ctx->audio_unit); + AudioComponentInstanceDispose(ctx->audio_unit); + ctx->audio_unit = NULL; + } + + if (ctx->frames_queue) { + buffer_t *buffer = (buffer_t *)CMSimpleQueueDequeue(ctx->frames_queue); + + while (buffer) { + av_freep(&buffer->data); + av_freep(&buffer); + buffer = (buffer_t *)CMSimpleQueueDequeue(ctx->frames_queue); + } + + CFRelease(ctx->frames_queue); + ctx->frames_queue = NULL; + } + + return 0; +} + +static int audiotoolbox_get_device_list(AVFormatContext *avctx, AVDeviceInfoList *device_list) +{ + OSStatus err = noErr; + CFStringRef device_UID = NULL; + CFStringRef device_name = NULL; + AudioDeviceID *devices = NULL; + AVDeviceInfo *avdevice_info = NULL; + int num_devices, ret; + UInt32 i; + + avdevice_info = av_mallocz(sizeof(AVDeviceInfo)); + + if (!avdevice_info) { + ret = AVERROR(ENOMEM); + goto fail; + } + + avdevice_info->device_name = av_strdup("default"); + avdevice_info->device_description = av_strdup("Default audio input device"); + if (!avdevice_info->device_name || !avdevice_info->device_description) { + ret = AVERROR(ENOMEM); + goto fail; + } + avdevice_info->media_types = av_malloc_array(1, sizeof(enum AVMediaType)); + if (avdevice_info->media_types) { + avdevice_info->nb_media_types = 1; + avdevice_info->media_types[0] = AVMEDIA_TYPE_AUDIO; + } + + if ((ret = av_dynarray_add_nofree(&device_list->devices, + &device_list->nb_devices, avdevice_info)) < 0) + goto fail; + + // get devices + UInt32 data_size = 0; + AudioObjectPropertyAddress prop; + prop.mSelector = kAudioHardwarePropertyDevices; + prop.mScope = kAudioObjectPropertyScopeGlobal; + prop.mElement = 0; + err = AudioObjectGetPropertyDataSize(kAudioObjectSystemObject, &prop, 0, NULL, &data_size); + if (check_status(avctx, err, "AudioObjectGetPropertyDataSize devices")) + return AVERROR(EINVAL); + + num_devices = data_size / sizeof(AudioDeviceID); + devices = av_malloc(data_size); + + err = AudioObjectGetPropertyData(kAudioObjectSystemObject, &prop, 0, NULL, &data_size, devices); + if (check_status(avctx, err, "AudioObjectGetPropertyData devices")) { + ret = AVERROR(EINVAL); + goto fail; + } + + for(i = 0; i < num_devices; ++i) { + prop.mSelector = kAudioDevicePropertyStreams; + prop.mScope = kAudioDevicePropertyScopeInput; + data_size = 0; + + err = AudioObjectGetPropertyDataSize(devices[i], &prop, 0, NULL, &data_size); + if (check_status(avctx, err, "AudioObjectGetPropertyData Streams")) + continue; + + UInt32 streamCount = data_size / sizeof(AudioStreamID); + + if (streamCount <= 0) + continue; + + // UID + data_size = sizeof(device_UID); + prop.mSelector = kAudioDevicePropertyDeviceUID; + err = AudioObjectGetPropertyData(devices[i], &prop, 0, NULL, &data_size, &device_UID); + if (check_status(avctx, err, "AudioObjectGetPropertyData UID")) + continue; + + // name + data_size = sizeof(device_name); + prop.mSelector = kAudioDevicePropertyDeviceNameCFString; + err = AudioObjectGetPropertyData(devices[i], &prop, 0, NULL, &data_size, &device_name); + if (check_status(avctx, err, "AudioObjectGetPropertyData name")) + continue; + + avdevice_info = av_mallocz(sizeof(AVDeviceInfo)); + if (!avdevice_info) { + ret = AVERROR(ENOMEM); + goto fail; + } + avdevice_info->device_name = av_strdup(CFStringGetCStringPtr(device_UID, kCFStringEncodingUTF8)); + avdevice_info->device_description = av_strdup(CFStringGetCStringPtr(device_name, kCFStringEncodingUTF8)); + if (!avdevice_info->device_name || !avdevice_info->device_description) { + ret = AVERROR(ENOMEM); + goto fail; + } + avdevice_info->media_types = av_malloc_array(1, sizeof(enum AVMediaType)); + if (avdevice_info->media_types) { + avdevice_info->nb_media_types = 1; + avdevice_info->media_types[0] = AVMEDIA_TYPE_AUDIO; + } + + if ((ret = av_dynarray_add_nofree(&device_list->devices, + &device_list->nb_devices, avdevice_info)) < 0) + goto fail; + } + + av_freep(&devices); + return 0; + +fail: + av_freep(&devices); + if (avdevice_info) { + av_freep(&avdevice_info->device_name); + av_freep(&avdevice_info->device_description); + av_freep(&avdevice_info); + } + + return ret; +} + +static const AVOption options[] = { + { "channels", "number of audio channels", offsetof(ATContext, channels), AV_OPT_TYPE_INT, {.i64=0}, 0, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM }, + { "frames_queue_length", "maximum of buffers in the input queue", offsetof(ATContext, frames_queue_length), AV_OPT_TYPE_INT, {.i64=10}, 0, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM }, + { "buffer_frame_size", "buffer frame size, gouverning internal latency", offsetof(ATContext, buffer_frame_size), AV_OPT_TYPE_INT, {.i64=1024}, 0, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM }, + { "big_endian", "return big endian samples", offsetof(ATContext, big_endian), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AV_OPT_FLAG_ENCODING_PARAM }, + { "sample_format", "sample format", offsetof(ATContext, sample_format), AV_OPT_TYPE_INT, {.i64=AV_SAMPLE_FMT_S16}, 0, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM }, + { NULL }, +}; + +static const AVClass audiotoolbox_class = { + .class_name = "AudioToolbox", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, + .category = AV_CLASS_CATEGORY_DEVICE_AUDIO_INPUT, +}; + +const AVInputFormat ff_audiotoolbox_demuxer = { + .name = "audiotoolbox", + .long_name = NULL_IF_CONFIG_SMALL("AudioToolbox input device"), + .priv_data_size = sizeof(ATContext), + .read_header = audiotoolbox_read_header, + .read_packet = audiotoolbox_read_packet, + .read_close = audiotoolbox_close, + .get_device_list = audiotoolbox_get_device_list, + .flags = AVFMT_NOFILE, + .priv_class = &audiotoolbox_class, +};