From patchwork Sun Jan 30 17:22:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Romain Beauxis X-Patchwork-Id: 33942 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2c4e:0:0:0:0 with SMTP id x14csp2117838iov; Sun, 30 Jan 2022 09:23:27 -0800 (PST) X-Google-Smtp-Source: ABdhPJz94LW4Fcm19Y20O7MdcNnOCB2D6dNXG85v4L7+QNaE9fPxqOIygBj7nDSGXuAT6Ijax1GI X-Received: by 2002:a17:907:6e09:: with SMTP id sd9mr10715291ejc.259.1643563407752; Sun, 30 Jan 2022 09:23:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643563407; cv=none; d=google.com; s=arc-20160816; b=RJOCehxu13s6qrEUiKW+fnnjbMx2I/1Es7148q/PHAIO7qS3a7ErdQuphSGX0gU5Kz G7wq2Ci0J+dDkQfrhspq/jzd7ReIQYR6Rv3hgCAnco3+1vOMUyTMHL+if8GdG+0yf2Te 65+chJTxsDKLuQz8+f1ueYvUNBMIjEjvFArRRnio/DygwBt7kbYubyFTPei9qV9c0uBA P5YOp6sttUcj9lGT9zSECvgL/zFdvT/VDAUEyUeLRlGYE1HveyjGML1jMNmVommVTOYv FInVU00CTsQhLMcZoKeeWjytD+X/s54GiznjRyzO+PFHtRN+qAiap56kC4DNa6g5UuqH EBkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=zbulAM3SeI71TdUetePu5F0JtYzRTxIlW1bGNOjRWYA=; b=Vxcb6Bj2765gSaV9SOO8DD3pJmU3DDoD3PlzIw4+4R28hk7Eddd5YgUoeut1hIuF0r 3YNgxpI4/xEAEuRVJHpBlofkrtukOTItqMjyB5k4MAvtUifV+Jhu5KukboM0xqqzDRuU s1XOSe6vLo1eDzgX9rovQ9gE8zawS0say5QREwlPjF+VZi2jlRNmschhGo53NMgJJYtY v5HrQL5VAETAxf18ziCt/3Lnqqny4eDH6ZGeHZ2m3zevJj6vvrX78ccAPMTVe7mIfwBb I9jYiQyLsDTthCFEAJO1viiO23OaRvj2MCnG6gTR/Fy1/aIE22scybYaNaV9vrv5Mcaf gY9w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c10si6847715ejf.519.2022.01.30.09.23.26; Sun, 30 Jan 2022 09:23:27 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7F52768B21A; Sun, 30 Jan 2022 19:23:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 522E368B1E9 for ; Sun, 30 Jan 2022 19:23:17 +0200 (EET) Received: by mail-oi1-f176.google.com with SMTP id s127so22445166oig.2 for ; Sun, 30 Jan 2022 09:23:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=aXzTGyeOC3bIfDmPfzpKV9CMt52l1ph+gYuPcCio7XQ=; b=YImuX6G8w2Zxlqla+NSGcqBFJXKvqKGNi8OpDUAEDtw/hcyPqadkIlhAsyBnFLJdtH /nO3NdVStM2Pa7bOnpUoSR1O+QrutNeReaNW4bFmQ0W4nhz7yL/Yuy2sEto9FVqyEwvj EKsipUhPasIOxIsXaV0XhtxTW9DJyzwh6YqpKaR/Ayhfbg11l2hOVS+GVJdlcwbZ2nH5 UlMUzvlRp+CnZcfHrjsBTkL/0Oh/nWuAbmf4MinjP1IsTIN2Fa8fQ1QeeEd8yJq1sHSZ IamB0JtnDGUhfM8GV9PawumyWHfTbA2qWabTv8GjuYeJMRkBAOCkirAdQ+OPLpcgmZ75 Y2Qw== X-Gm-Message-State: AOAM531lJ5vGLsoeBbeaMaccGsPMps1BuftSA8M+MGS4XSpdTxiVxubA l1impUobK39uTiqNWYsk30vmF1EDIogN9M2n X-Received: by 2002:a05:6808:1a0f:: with SMTP id bk15mr16079203oib.48.1643563394459; Sun, 30 Jan 2022 09:23:14 -0800 (PST) Received: from localhost.localdomain (wsip-98-173-234-196.no.no.cox.net. [98.173.234.196]) by smtp.gmail.com with ESMTPSA id e7sm93844otl.50.2022.01.30.09.23.12 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 30 Jan 2022 09:23:13 -0800 (PST) From: toots@rastageeks.org To: ffmpeg-devel@ffmpeg.org Date: Sun, 30 Jan 2022 11:22:37 -0600 Message-Id: <20220130172237.31759-1-toots@rastageeks.org> X-Mailer: git-send-email 2.32.0 (Apple Git-132) MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] Add AudioToolbox audio input device. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: thilo.borgmann@mail.de, Romain Beauxis , epirat07@gmail.com Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Z0bh/txwRIFF From: Romain Beauxis --- configure | 5 + doc/indevs.texi | 35 +++ libavdevice/Makefile | 1 + libavdevice/alldevices.c | 1 + libavdevice/audiotoolbox_dec.m | 520 +++++++++++++++++++++++++++++++++ 5 files changed, 562 insertions(+) create mode 100644 libavdevice/audiotoolbox_dec.m diff --git a/configure b/configure index 5b19a35f59..ead6df7d8f 100755 --- a/configure +++ b/configure @@ -204,6 +204,7 @@ External library support: --disable-avfoundation disable Apple AVFoundation framework [autodetect] --enable-avisynth enable reading of AviSynth script files [no] --disable-bzlib disable bzlib [autodetect] + --disable-coremedia disable Apple CoreMedia framework [autodetect] --disable-coreimage disable Apple CoreImage framework [autodetect] --enable-chromaprint enable audio fingerprinting with chromaprint [no] --enable-frei0r enable frei0r video filtering [no] @@ -1751,6 +1752,7 @@ EXTERNAL_AUTODETECT_LIBRARY_LIST=" appkit avfoundation bzlib + coremedia coreimage iconv libxcb @@ -3494,6 +3496,8 @@ alsa_outdev_deps="alsa" avfoundation_indev_deps="avfoundation corevideo coremedia pthreads" avfoundation_indev_suggest="coregraphics applicationservices" avfoundation_indev_extralibs="-framework Foundation" +audiotoolbox_indev_deps="coremedia audiotoolbox" +audiotoolbox_indev_extralibs="-framework CoreMedia -framework AudioToolbox" audiotoolbox_outdev_deps="audiotoolbox pthreads" audiotoolbox_outdev_extralibs="-framework AudioToolbox -framework CoreAudio" bktr_indev_deps_any="dev_bktr_ioctl_bt848_h machine_ioctl_bt848_h dev_video_bktr_ioctl_bt848_h dev_ic_bt8xx_h" @@ -6349,6 +6353,7 @@ check_lib camera2ndk "stdbool.h stdint.h camera/NdkCameraManager.h" ACameraManag enabled appkit && check_apple_framework AppKit enabled audiotoolbox && check_apple_framework AudioToolbox enabled avfoundation && check_apple_framework AVFoundation +enabled coremedia && check_apple_framework CoreMedia enabled coreimage && check_apple_framework CoreImage enabled metal && check_apple_framework Metal enabled videotoolbox && check_apple_framework VideoToolbox diff --git a/doc/indevs.texi b/doc/indevs.texi index 9d8020311a..35b5a875f6 100644 --- a/doc/indevs.texi +++ b/doc/indevs.texi @@ -103,6 +103,41 @@ Set the maximum number of frames to buffer. Default is 5. @end table +@section AudioToolbox + +AudioToolbox input device. + +Allows native input from CoreAudio devices on OSX. + +@subsection Options + +AudioToolbox supports the following options: + +@table @option + +@item channels +Set the number of channels. Default is device's default. + +@item frames_queue_length +Maximum of buffers in the input queue + +@item buffer_frame_size +Buffer frame size, gouverning internal latency + +@item big_endian +Return big endian samples + +@item sample_format +Sample format + +@end table + +@subsection Examples + +@itemize + +@end itemize + @section avfoundation AVFoundation input device. diff --git a/libavdevice/Makefile b/libavdevice/Makefile index 53efda0514..0c73255a21 100644 --- a/libavdevice/Makefile +++ b/libavdevice/Makefile @@ -14,6 +14,7 @@ OBJS-$(HAVE_LIBC_MSVCRT) += file_open.o OBJS-$(CONFIG_ALSA_INDEV) += alsa_dec.o alsa.o timefilter.o OBJS-$(CONFIG_ALSA_OUTDEV) += alsa_enc.o alsa.o OBJS-$(CONFIG_ANDROID_CAMERA_INDEV) += android_camera.o +OBJS-$(CONFIG_AUDIOTOOLBOX_INDEV) += audiotoolbox_dec.o OBJS-$(CONFIG_AUDIOTOOLBOX_OUTDEV) += audiotoolbox.o OBJS-$(CONFIG_AVFOUNDATION_INDEV) += avfoundation.o OBJS-$(CONFIG_BKTR_INDEV) += bktr.o diff --git a/libavdevice/alldevices.c b/libavdevice/alldevices.c index 22323a0a44..fbecdbb0b2 100644 --- a/libavdevice/alldevices.c +++ b/libavdevice/alldevices.c @@ -26,6 +26,7 @@ extern const AVInputFormat ff_alsa_demuxer; extern const AVOutputFormat ff_alsa_muxer; extern const AVInputFormat ff_android_camera_demuxer; extern const AVOutputFormat ff_audiotoolbox_muxer; +extern const AVInputFormat ff_audiotoolbox_demuxer; extern const AVInputFormat ff_avfoundation_demuxer; extern const AVInputFormat ff_bktr_demuxer; extern const AVOutputFormat ff_caca_muxer; diff --git a/libavdevice/audiotoolbox_dec.m b/libavdevice/audiotoolbox_dec.m new file mode 100644 index 0000000000..c2f38399df --- /dev/null +++ b/libavdevice/audiotoolbox_dec.m @@ -0,0 +1,520 @@ +/* + * AudioToolbox input device + * Copyright (c) 2022 Romain Beauxis + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * AudioToolbox input device + * @author Romain Beauxis + */ + +#import +#import + +#include "libavutil/channel_layout.h" +#include "libavformat/internal.h" +#include "avdevice.h" + +typedef struct { + void *data; + int size; +} buffer_t; + +typedef struct { + AVClass *class; + CMSimpleQueueRef frames_queue; + AudioUnit audio_unit; + AudioStreamBasicDescription record_format; + uint64_t position; + int frames_queue_length; + int buffer_frame_size; + int stream_index; + int big_endian; + enum AVSampleFormat sample_format; + int channels; +} ATContext; + +static int check_status(void *ctx, OSStatus status, const char *msg) { + if (status == noErr) { + av_log(ctx, AV_LOG_DEBUG, "OK: %s\n", msg); + return 0; + } + + NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; + NSError *error = [NSError errorWithDomain:NSOSStatusErrorDomain code:status userInfo:nil]; + av_log(ctx, AV_LOG_ERROR, "Error: %s (%s)\n", msg, [[error localizedDescription] UTF8String]); + [pool release]; + return 1; +} + +static OSStatus input_callback(void *priv, + AudioUnitRenderActionFlags *ioActionFlags, + const AudioTimeStamp *inTimeStamp, + UInt32 inBusNumber, + UInt32 inNumberFrames, + AudioBufferList *ioData) { + ATContext *ctx = (ATContext *)priv; + OSStatus err; + + AudioBuffer audio_buffer; + + audio_buffer.mNumberChannels = ctx->channels; + audio_buffer.mDataByteSize = inNumberFrames * ctx->record_format.mBytesPerFrame; + + audio_buffer.mData = av_malloc(audio_buffer.mDataByteSize); + memset(audio_buffer.mData, 0, audio_buffer.mDataByteSize); + + AudioBufferList bufferList; + bufferList.mNumberBuffers = 1; + bufferList.mBuffers[0] = audio_buffer; + + err = AudioUnitRender(ctx->audio_unit, + ioActionFlags, + inTimeStamp, + inBusNumber, + inNumberFrames, + &bufferList); + if (check_status(ctx, err, "AudioUnitRender")) { + av_freep(&audio_buffer.mData); + return err; + } + + buffer_t *buffer = av_malloc(sizeof(buffer_t)); + buffer->data = audio_buffer.mData; + buffer->size = audio_buffer.mDataByteSize; + err = CMSimpleQueueEnqueue(ctx->frames_queue, buffer); + + if (err != noErr) { + av_log(ctx, AV_LOG_DEBUG, "Could not enqueue audio frame!\n"); + return err; + } + + return noErr; +} + +static av_cold int audiotoolbox_read_header(AVFormatContext *avctx) { + ATContext *ctx = (ATContext*)avctx->priv_data; + OSStatus err = noErr; + AudioChannelLayout *channel_layout = NULL; + AudioDeviceID device = kAudioObjectUnknown; + AudioObjectPropertyAddress prop; + UInt32 data_size; + + enum AVCodecID codec_id = av_get_pcm_codec(ctx->sample_format, ctx->big_endian); + + if (codec_id == AV_CODEC_ID_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error: invalid sample format!\n"); + return AVERROR(EINVAL); + } + + prop.mScope = kAudioObjectPropertyScopeGlobal; + prop.mElement = 0; + + if (!strcmp(avctx->url, "default")) { + prop.mSelector = kAudioHardwarePropertyDefaultInputDevice; + data_size = sizeof(AudioDeviceID); + err = AudioObjectGetPropertyData(kAudioObjectSystemObject, &prop, 0, NULL, &data_size, &device); + if (check_status(avctx, err, "AudioObjectGetPropertyData DefaultInputDevice")) + goto fail; + } else { + prop.mSelector = kAudioHardwarePropertyDeviceForUID; + CFStringRef deviceUID = CFStringCreateWithCStringNoCopy(NULL, avctx->url, kCFStringEncodingUTF8, kCFAllocatorNull); + AudioValueTranslation value; + value.mInputData = &deviceUID; + value.mInputDataSize = sizeof(deviceUID);; + value.mOutputData = &device; + value.mOutputDataSize = sizeof(device); + data_size = sizeof(value); + + err = AudioObjectGetPropertyData(kAudioObjectSystemObject, &prop, 0, NULL, &data_size, &value); + CFRelease(deviceUID); + + if (check_status(avctx, err, "AudioObjectGetPropertyData DeviceForUID")) + goto fail; + } + + Float64 sample_rate; + prop.mSelector = kAudioDevicePropertyNominalSampleRate; + prop.mScope = kAudioObjectPropertyScopeInput; + data_size = sizeof(sample_rate); + err = AudioObjectGetPropertyData(device, &prop, 0, NULL, &data_size, &sample_rate); + if (check_status(avctx, err, "AudioObjectGetPropertyData SampleRate")) + goto fail; + + if (!ctx->channels) { + prop.mSelector = kAudioDevicePropertyPreferredChannelLayout; + prop.mScope = kAudioObjectPropertyScopeInput; + UInt32 channel_layout_size; + + err = AudioObjectGetPropertyDataSize(device, &prop, 0, NULL, &channel_layout_size); + if (check_status(avctx, err, "AudioObjectGetPropertyDataSize PreferredChannelLayout")) + goto fail; + + channel_layout = av_malloc(channel_layout_size); + err = AudioObjectGetPropertyData(device, &prop, 0, NULL, &channel_layout_size, channel_layout); + if (check_status(avctx, err, "AudioObjectGetPropertyData PreferredChannelLayout")) + goto fail; + + data_size = sizeof(ctx->channels); + err = AudioFormatGetProperty(kAudioFormatProperty_NumberOfChannelsForLayout, channel_layout_size, channel_layout, &data_size, &ctx->channels); + if (check_status(avctx, err, "AudioFormatGetProperty NumberOfChannelsForLayout")) + goto fail; + } + + ctx->record_format.mFormatID = kAudioFormatLinearPCM; + ctx->record_format.mChannelsPerFrame = ctx->channels; + ctx->record_format.mFormatFlags = kAudioFormatFlagIsPacked; + ctx->record_format.mBitsPerChannel = av_get_bytes_per_sample(ctx->sample_format) << 3; + + if (ctx->big_endian) + ctx->record_format.mFormatFlags |= kAudioFormatFlagIsBigEndian; + + switch (ctx->sample_format) { + case AV_SAMPLE_FMT_S16: + case AV_SAMPLE_FMT_S32: + ctx->record_format.mFormatFlags |= kAudioFormatFlagIsSignedInteger; + break; + case AV_SAMPLE_FMT_FLT: + ctx->record_format.mFormatFlags |= kAudioFormatFlagIsFloat; + break; + default: + av_log(ctx, AV_LOG_ERROR, "Error: invalid sample format!\n"); + goto fail; + } + + av_log(ctx, AV_LOG_DEBUG, "Audio Input: %s\n", avctx->url); + av_log(ctx, AV_LOG_DEBUG, "samplerate: %d\n", (int)sample_rate); + av_log(ctx, AV_LOG_DEBUG, "channels: %d\n", ctx->channels); + av_log(ctx, AV_LOG_DEBUG, "Input format: %s\n", avcodec_get_name(codec_id)); + + data_size = sizeof(ctx->record_format); + err = AudioFormatGetProperty(kAudioFormatProperty_FormatInfo, 0, NULL, &data_size, &ctx->record_format); + if (check_status(avctx, err, "AudioFormatGetProperty FormatInfo")) + goto fail; + + AudioComponentDescription desc; + AudioComponent comp; + + desc.componentType = kAudioUnitType_Output; + desc.componentSubType = kAudioUnitSubType_HALOutput; + desc.componentManufacturer = kAudioUnitManufacturer_Apple; + desc.componentFlags = 0; + desc.componentFlagsMask = 0; + + comp = AudioComponentFindNext(NULL, &desc); + if (comp == NULL) { + av_log(ctx, AV_LOG_ERROR, "Error: AudioComponentFindNext\n"); + goto fail; + } + + err = AudioComponentInstanceNew(comp, &ctx->audio_unit); + if (check_status(avctx, err, "AudioComponentInstanceNew")) + goto fail; + + UInt32 enableIO = 1; + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioOutputUnitProperty_EnableIO, + kAudioUnitScope_Input, + 1, + &enableIO, + sizeof(enableIO)); + if (check_status(avctx, err, "AudioUnitSetProperty EnableIO")) + goto fail; + + enableIO = 0; + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioOutputUnitProperty_EnableIO, + kAudioUnitScope_Output, + 0, + &enableIO, + sizeof(enableIO)); + if (check_status(avctx, err, "AudioUnitSetProperty EnableIO")) + goto fail; + + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioOutputUnitProperty_CurrentDevice, + kAudioUnitScope_Global, + 0, + &device, + sizeof(device)); + if (check_status(avctx, err, "AudioUnitSetProperty CurrentDevice")) + goto fail; + + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioUnitProperty_StreamFormat, + kAudioUnitScope_Input, + 0, + &ctx->record_format, + sizeof(ctx->record_format)); + if (check_status(avctx, err, "AudioUnitSetProperty StreamFormat")) + goto fail; + + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioUnitProperty_StreamFormat, + kAudioUnitScope_Output, + 1, + &ctx->record_format, + sizeof(ctx->record_format)); + if (check_status(avctx, err, "AudioUnitSetProperty StreamFormat")) + goto fail; + + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioDevicePropertyBufferFrameSize, + kAudioUnitScope_Global, + 0, + &ctx->buffer_frame_size, + sizeof(ctx->buffer_frame_size)); + if (check_status(avctx, err, "AudioUnitSetProperty BufferFrameSize")) + goto fail; + + AURenderCallbackStruct callback = {0}; + callback.inputProc = input_callback; + callback.inputProcRefCon = ctx; + err = AudioUnitSetProperty(ctx->audio_unit, + kAudioOutputUnitProperty_SetInputCallback, + kAudioUnitScope_Global, + 0, + &callback, + sizeof(callback)); + if (check_status(avctx, err, "AudioUnitSetProperty SetInputCallback")) + goto fail; + + err = AudioUnitInitialize(ctx->audio_unit); + if (check_status(avctx, err, "AudioUnitInitialize")) + goto fail; + + err = CMSimpleQueueCreate(kCFAllocatorDefault, ctx->frames_queue_length, &ctx->frames_queue); + if (check_status(avctx, err, "CMSimpleQueueCreate")) + goto fail; + + CFRetain(ctx->frames_queue); + + err = AudioUnitInitialize(ctx->audio_unit); + if (check_status(avctx, err, "AudioUnitInitialize")) + goto fail; + + err = AudioOutputUnitStart(ctx->audio_unit); + if (check_status(avctx, err, "AudioOutputUnitStart")) + goto fail; + + AVStream* stream = avformat_new_stream(avctx, NULL); + stream->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; + stream->codecpar->sample_rate = sample_rate; + stream->codecpar->channels = ctx->channels; + stream->codecpar->channel_layout = av_get_default_channel_layout(stream->codecpar->channels); + stream->codecpar->codec_id = codec_id; + + avpriv_set_pts_info(stream, 64, 1, sample_rate); + + ctx->stream_index = stream->index; + ctx->position = 0; + + av_freep(&channel_layout); + return 0; + +fail: + av_freep(&channel_layout); + return AVERROR(EINVAL); +} + +static int audiotoolbox_read_packet(AVFormatContext *avctx, AVPacket *pkt) { + ATContext *ctx = (ATContext*)avctx->priv_data; + + if (CMSimpleQueueGetCount(ctx->frames_queue) < 1) + return AVERROR(EAGAIN); + + buffer_t *buffer = (buffer_t *)CMSimpleQueueDequeue(ctx->frames_queue); + + int status = av_packet_from_data(pkt, buffer->data, buffer->size); + if (status < 0) { + av_freep(&buffer->data); + av_freep(&buffer); + return status; + } + + pkt->stream_index = ctx->stream_index; + pkt->flags |= AV_PKT_FLAG_KEY; + pkt->pts = pkt->dts = ctx->position; + + ctx->position += pkt->size / (ctx->channels * av_get_bytes_per_sample(ctx->sample_format)); + + av_freep(&buffer); + return 0; +} + +static av_cold int audiotoolbox_close(AVFormatContext *avctx) { + ATContext *ctx = (ATContext*)avctx->priv_data; + + if (ctx->audio_unit) { + AudioOutputUnitStop(ctx->audio_unit); + AudioComponentInstanceDispose(ctx->audio_unit); + ctx->audio_unit = NULL; + } + + if (ctx->frames_queue) { + buffer_t *buffer = (buffer_t *)CMSimpleQueueDequeue(ctx->frames_queue); + + while (buffer) { + av_freep(&buffer->data); + av_freep(&buffer); + buffer = (buffer_t *)CMSimpleQueueDequeue(ctx->frames_queue); + } + + CFRelease(ctx->frames_queue); + ctx->frames_queue = NULL; + } + + return 0; +} + +static int audiotoolbox_get_device_list(AVFormatContext *avctx, AVDeviceInfoList *device_list) +{ + OSStatus err = noErr; + CFStringRef device_UID = NULL; + CFStringRef device_name = NULL; + AudioDeviceID *devices = NULL; + AVDeviceInfo *avdevice_info = NULL; + int num_devices, ret; + UInt32 i; + + avdevice_info = av_mallocz(sizeof(AVDeviceInfo)); + + if (!avdevice_info) { + ret = AVERROR(ENOMEM); + goto fail; + } + + avdevice_info->device_name = av_strdup("default"); + avdevice_info->device_description = av_strdup("Default audio input device"); + if (!avdevice_info->device_name || !avdevice_info->device_description) { + ret = AVERROR(ENOMEM); + goto fail; + } + + if ((ret = av_dynarray_add_nofree(&device_list->devices, + &device_list->nb_devices, avdevice_info)) < 0) + goto fail; + + // get devices + UInt32 data_size = 0; + AudioObjectPropertyAddress prop; + prop.mSelector = kAudioHardwarePropertyDevices; + prop.mScope = kAudioObjectPropertyScopeGlobal; + prop.mElement = 0; + err = AudioObjectGetPropertyDataSize(kAudioObjectSystemObject, &prop, 0, NULL, &data_size); + if (check_status(avctx, err, "AudioObjectGetPropertyDataSize devices")) + return AVERROR(EINVAL); + + num_devices = data_size / sizeof(AudioDeviceID); + devices = av_malloc(data_size); + + err = AudioObjectGetPropertyData(kAudioObjectSystemObject, &prop, 0, NULL, &data_size, devices); + if (check_status(avctx, err, "AudioObjectGetPropertyData devices")) { + ret = AVERROR(EINVAL); + goto fail; + } + + for(i = 0; i < num_devices; ++i) { + prop.mSelector = kAudioDevicePropertyStreams; + prop.mScope = kAudioDevicePropertyScopeInput; + data_size = 0; + + err = AudioObjectGetPropertyDataSize(devices[i], &prop, 0, NULL, &data_size); + if (check_status(avctx, err, "AudioObjectGetPropertyData Streams")) + continue; + + UInt32 streamCount = data_size / sizeof(AudioStreamID); + + if (streamCount <= 0) + continue; + + // UID + data_size = sizeof(device_UID); + prop.mSelector = kAudioDevicePropertyDeviceUID; + err = AudioObjectGetPropertyData(devices[i], &prop, 0, NULL, &data_size, &device_UID); + if (check_status(avctx, err, "AudioObjectGetPropertyData UID")) + continue; + + // name + data_size = sizeof(device_name); + prop.mSelector = kAudioDevicePropertyDeviceNameCFString; + err = AudioObjectGetPropertyData(devices[i], &prop, 0, NULL, &data_size, &device_name); + if (check_status(avctx, err, "AudioObjectGetPropertyData name")) + continue; + + avdevice_info = av_mallocz(sizeof(AVDeviceInfo)); + if (!avdevice_info) { + ret = AVERROR(ENOMEM); + goto fail; + } + avdevice_info->device_name = av_strdup(CFStringGetCStringPtr(device_UID, kCFStringEncodingUTF8)); + avdevice_info->device_description = av_strdup(CFStringGetCStringPtr(device_name, kCFStringEncodingUTF8)); + if (!avdevice_info->device_name || !avdevice_info->device_description) { + ret = AVERROR(ENOMEM); + goto fail; + } + + if ((ret = av_dynarray_add_nofree(&device_list->devices, + &device_list->nb_devices, avdevice_info)) < 0) + goto fail; + } + + av_freep(&devices); + return 0; + +fail: + av_freep(&devices); + if (avdevice_info) { + av_freep(&avdevice_info->device_name); + av_freep(&avdevice_info->device_description); + av_freep(&avdevice_info); + } + + return ret; +} + +static const AVOption options[] = { + { "channels", "number of audio channels", offsetof(ATContext, channels), AV_OPT_TYPE_INT, {.i64=0}, 0, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM }, + { "frames_queue_length", "maximum of buffers in the input queue", offsetof(ATContext, frames_queue_length), AV_OPT_TYPE_INT, {.i64=10}, 0, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM }, + { "buffer_frame_size", "buffer frame size, gouverning internal latency", offsetof(ATContext, buffer_frame_size), AV_OPT_TYPE_INT, {.i64=1024}, 0, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM }, + { "big_endian", "return big endian samples", offsetof(ATContext, big_endian), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AV_OPT_FLAG_ENCODING_PARAM }, + { "sample_format", "sample format", offsetof(ATContext, sample_format), AV_OPT_TYPE_INT, {.i64=AV_SAMPLE_FMT_S16}, 0, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM }, + { NULL }, +}; + +static const AVClass audiotoolbox_class = { + .class_name = "AudioToolbox", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, + .category = AV_CLASS_CATEGORY_DEVICE_VIDEO_INPUT, +}; + +const AVInputFormat ff_audiotoolbox_demuxer = { + .name = "audiotoolbox", + .long_name = NULL_IF_CONFIG_SMALL("AudioToolbox input device"), + .priv_data_size = sizeof(ATContext), + .read_header = audiotoolbox_read_header, + .read_packet = audiotoolbox_read_packet, + .read_close = audiotoolbox_close, + .get_device_list = audiotoolbox_get_device_list, + .flags = AVFMT_NOFILE, + .priv_class = &audiotoolbox_class, +};