From patchwork Thu Jan 6 14:31:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Romain Beauxis X-Patchwork-Id: 33126 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp1701906iog; Thu, 6 Jan 2022 06:31:56 -0800 (PST) X-Google-Smtp-Source: ABdhPJzfHEp/d+2g7MJnMqIZYh6IZ+DKo6cUDwVJcBjyI+c5EUHynlUv57VFk3VNWnIzwMhrRueK X-Received: by 2002:a17:906:bcd6:: with SMTP id lw22mr1587177ejb.114.1641479516279; Thu, 06 Jan 2022 06:31:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1641479516; cv=none; d=google.com; s=arc-20160816; b=pgqa7Lo3NOd5eO+T7goqHQtgt2LUuI4P12rodOdxY1E5Ua68Op/vaWKcBh5Sgy+CkK 4jOAIL6dkp4N8CUO4FoDIdZ5RsJQF52JermaA4j/KuVQe8yJLi/J0TI8SzdGUtI9dsNp JdhIt/huios5hbLVCdn52mPLxTypa+y0gOG7xyaODBGctZs2YBDp+eHnM7i/IF9pHAp1 K+qei+FkqL0WSiWtuRvuj9UNjF9MqFd41X4E6yFlpra8t7swBB3xIsUfDodPF7/FuFzq bW6mKbzdTa6KWZVK+p6OyvlFbQqv32CgpaAz5Smll/4zQzwGzRZty6p8NUU0yu6Uy+cW JBmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:date:message-id:mime-version:from :delivered-to; bh=txKzSfx4TQCaN9+oUF4LQOrHU3JhcZ9LiOWtvb75FcA=; b=brGNYq6pCUasdpeaRHrDN811RMK9N8uMGutbJQsBY7NW0GnoDz4HPekwVKapfWYben Zc6AkwILyVOw92IztQAkqonG5uQhrhpfWhmD4aWJe/gQN2nzqZvbWtGyM4FZHzMbVLFz nPzbQTVEs3gCtVPpmo5MFi6ILDqqrNkqDSx+N1/AR3AJldR758GLQKlq6FyxkD15+sH0 gWku4aqxGbP4TO9McX9XQ/t2I/hWbXCPy10UJVxgdaKN+ZUj/m2Za4sxt8UadRCG3Ti0 X09xz12oV1otS6I1eGiQ+Na2UL2lQ6gocP3MTKNjSczsooIbTBVnbdRYJ6PD3WPVvYoW Dvfw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d21si100254edr.60.2022.01.06.06.31.55; Thu, 06 Jan 2022 06:31:56 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B83EF68A8F5; Thu, 6 Jan 2022 16:31:53 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f52.google.com (mail-qv1-f52.google.com [209.85.219.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2DFDC68A7FD for ; Thu, 6 Jan 2022 16:31:47 +0200 (EET) Received: by mail-qv1-f52.google.com with SMTP id kk22so2519749qvb.0 for ; Thu, 06 Jan 2022 06:31:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:message-id:date:cc:to; bh=OoVBTQtBtKarobpm+NRx35CTx8FUeSrmvwba4r4qY0s=; b=2D94BTwmhp2GCrRkdiZEbv3yqCPOHHmliSFrGU0ixyl7TCAGLjSeZ+DiSllnVtvndH P/TZ/92MESfU3Awj1YLIUIGUgwN0xJ5ytY2Z9EMCjrOoxHBhr5+N7GPmWOEq/i2jHEEI puvXgJvpLa7e5gVflEthT5aUywRyLmCMhLuPjqIq9fidyv/YHLSMLBVCg+VoB8oceKgn 3qb6QrHg8ETvLJeSdWYu6ybuWY9wCF9qC+h1lk8nWiiANwZZZb6wcOXrcsHxaJafRDWe yRF3mSS2YFwzvRoFf/hnHE6YuVKlTjw2+1/t7iZChRXU/SOFWQkYVB3A1sr4AbS0KzeO a2NA== X-Gm-Message-State: AOAM5313Dh4ffNfkKlyKZmt2n4VwU89sgl2yczJJiMam7+xARh+Yz7FM 6hJkBTB264TqbndW7kFZy0atqIR0DhfkUg== X-Received: by 2002:a05:6214:29c1:: with SMTP id gh1mr53555270qvb.108.1641479505531; Thu, 06 Jan 2022 06:31:45 -0800 (PST) Received: from smtpclient.apple ([172.58.129.153]) by smtp.gmail.com with ESMTPSA id v5sm1673812qkp.126.2022.01.06.06.31.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Jan 2022 06:31:45 -0800 (PST) From: Romain Beauxis Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.40.0.1.81\)) Message-Id: Date: Thu, 6 Jan 2022 08:31:42 -0600 To: FFmpeg development discussions and patches X-Mailer: Apple Mail (2.3693.40.0.1.81) Subject: [FFmpeg-devel] [PATCH v9 3/3] libavdevice/avfoundation.m: Allow to select devices by unique ID X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: thilo.borgmann@mail.de, Aman Karmani , epirat07@gmail.com Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: vJzdsddi/LyF Signed-off-by: Romain Beauxis --- This is the third patch of a series of 3 that fix, cleanup and enhance the avfoundation implementation for libavdevice. These patches come from an actual user-facing application relying on libavdevice’s implementation of avfoundation audio input. Without them, Avfoundation is practically unusable as it will: * Refuse to process certain specific audio input format that are actually returned by the OS for some users (packed PCM audio) * Drop audio frames, resulting in corrupted audio input. This might have been unnoticed with video frames but this makes avfoundation essentially unusable for audio. The patches are now being included in our production build so they are tested and usable in production. Changes: v2: None v3: * Switched unique ID to use system-prodvided unique ID * Implemented unique IDs for screen capture v4: Cleanup v5: Fix indentation/wrapping v6: None v7: None V8: None v9: None doc/indevs.texi | 6 ++-- libavdevice/avfoundation.m | 72 +++++++++++++++++++++++++++++--------- 2 files changed, 60 insertions(+), 18 deletions(-) diff --git a/doc/indevs.texi b/doc/indevs.texi index 9d8020311a..858c0fa4e4 100644 --- a/doc/indevs.texi +++ b/doc/indevs.texi @@ -114,7 +114,7 @@ The input filename has to be given in the following syntax: -i "[[VIDEO]:[AUDIO]]" @end example The first entry selects the video input while the latter selects the audio input. -The stream has to be specified by the device name or the device index as shown by the device list. +The stream has to be specified by the device name, index or ID as shown by the device list. Alternatively, the video and/or audio input device can be chosen by index using the @option{ -video_device_index @@ -127,7 +127,9 @@ and/or device name or index given in the input filename. All available devices can be enumerated by using @option{-list_devices true}, listing -all device names and corresponding indices. +all device names, corresponding indices and IDs, when available. Device name can be +tricky to use when localized and device index can change when devices are plugged or unplugged. A device +hash, when available, uniquely identifies a device and should not change over time. There are two device name aliases: @table @code diff --git a/libavdevice/avfoundation.m b/libavdevice/avfoundation.m index 5ee19f4863..c4a4272965 100644 --- a/libavdevice/avfoundation.m +++ b/libavdevice/avfoundation.m @@ -46,6 +46,8 @@ static inline void av_log_avfoundation(void *s, int lvl, const char *str, OSStat [pool release]; } +#define CLEANUP_DEVICE_ID(s) [[s stringByReplacingOccurrencesOfString:@":" withString:@"."] UTF8String] + static const int avf_time_base = 1000000; static const AVRational avf_time_base_q = { @@ -817,21 +819,23 @@ static int avf_read_header(AVFormatContext *s) int index = 0; av_log(ctx, AV_LOG_INFO, "AVFoundation video devices:\n"); for (AVCaptureDevice *device in devices) { - const char *name = [[device localizedName] UTF8String]; - index = [devices indexOfObject:device]; - av_log(ctx, AV_LOG_INFO, "[%d] %s\n", index, name); + const char *name = [[device localizedName] UTF8String]; + const char *uniqueId = CLEANUP_DEVICE_ID([device uniqueID]); + index = [devices indexOfObject:device]; + av_log(ctx, AV_LOG_INFO, "[%d] %s (ID: %s)\n", index, name, uniqueId); } for (AVCaptureDevice *device in devices_muxed) { - const char *name = [[device localizedName] UTF8String]; - index = [devices count] + [devices_muxed indexOfObject:device]; - av_log(ctx, AV_LOG_INFO, "[%d] %s\n", index, name); + const char *name = [[device localizedName] UTF8String]; + const char *uniqueId = CLEANUP_DEVICE_ID([device uniqueID]); + index = [devices count] + [devices_muxed indexOfObject:device]; + av_log(ctx, AV_LOG_INFO, "[%d] %s (ID: %s)\n", index, name, uniqueId); } #if !TARGET_OS_IPHONE && __MAC_OS_X_VERSION_MIN_REQUIRED >= 1070 if (num_screens > 0) { CGDirectDisplayID screens[num_screens]; CGGetActiveDisplayList(num_screens, screens, &num_screens); for (int i = 0; i < num_screens; i++) { - av_log(ctx, AV_LOG_INFO, "[%d] Capture screen %d\n", ctx->num_video_devices + i, i); + av_log(ctx, AV_LOG_INFO, "[%d] Capture screen %d (ID: AvfilterAvfoundationCaptureScreen%d)\n", ctx->num_video_devices + i, i, screens[i]); } } #endif @@ -839,9 +843,10 @@ static int avf_read_header(AVFormatContext *s) av_log(ctx, AV_LOG_INFO, "AVFoundation audio devices:\n"); devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeAudio]; for (AVCaptureDevice *device in devices) { - const char *name = [[device localizedName] UTF8String]; - int index = [devices indexOfObject:device]; - av_log(ctx, AV_LOG_INFO, "[%d] %s\n", index, name); + const char *name = [[device localizedName] UTF8String]; + const char *uniqueId = CLEANUP_DEVICE_ID([device uniqueID]); + int index = [devices indexOfObject:device]; + av_log(ctx, AV_LOG_INFO, "[%d] %s (ID: %s)\n", index, name, uniqueId); } goto fail; } @@ -903,14 +908,29 @@ static int avf_read_header(AVFormatContext *s) } else { // looking for video inputs for (AVCaptureDevice *device in devices) { - if (!strncmp(ctx->video_filename, [[device localizedName] UTF8String], strlen(ctx->video_filename))) { + const char *name = [[device localizedName] UTF8String]; + if (!strncmp(ctx->video_filename, name, strlen(ctx->video_filename))) { + video_device = device; + break; + } + + const char *uniqueId = CLEANUP_DEVICE_ID([device uniqueID]); + if (!strncmp(ctx->video_filename, uniqueId, strlen(ctx->video_filename))) { video_device = device; break; } } // looking for muxed inputs for (AVCaptureDevice *device in devices_muxed) { - if (!strncmp(ctx->video_filename, [[device localizedName] UTF8String], strlen(ctx->video_filename))) { + const char *name = [[device localizedName] UTF8String]; + if (!strncmp(ctx->video_filename, name, strlen(ctx->video_filename))) { + video_device = device; + ctx->video_is_muxed = 1; + break; + } + + const char *uniqueId = CLEANUP_DEVICE_ID([device uniqueID]); + if (!strncmp(ctx->video_filename, uniqueId, strlen(ctx->video_filename))) { video_device = device; ctx->video_is_muxed = 1; break; @@ -921,10 +941,23 @@ static int avf_read_header(AVFormatContext *s) // looking for screen inputs if (!video_device) { int idx; + CGDirectDisplayID screens[num_screens]; + CGGetActiveDisplayList(num_screens, screens, &num_screens); + AVCaptureScreenInput* capture_screen_input = NULL; + if(sscanf(ctx->video_filename, "Capture screen %d", &idx) && idx < num_screens) { - CGDirectDisplayID screens[num_screens]; - CGGetActiveDisplayList(num_screens, screens, &num_screens); - AVCaptureScreenInput* capture_screen_input = [[[AVCaptureScreenInput alloc] initWithDisplayID:screens[idx]] autorelease]; + capture_screen_input = [[[AVCaptureScreenInput alloc] initWithDisplayID:screens[idx]] autorelease]; + } + + if(sscanf(ctx->video_filename, "AvfilterAvfoundationCaptureScreen%d", &idx)) { + for (int i = 0; i < num_screens; i++) { + if (screens[i] == idx) { + capture_screen_input = [[[AVCaptureScreenInput alloc] initWithDisplayID:idx] autorelease]; + } + } + } + + if (capture_screen_input) { video_device = (AVCaptureDevice*) capture_screen_input; ctx->video_device_index = ctx->num_video_devices + idx; ctx->video_is_screen = 1; @@ -975,7 +1008,14 @@ static int avf_read_header(AVFormatContext *s) NSArray *devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeAudio]; for (AVCaptureDevice *device in devices) { - if (!strncmp(ctx->audio_filename, [[device localizedName] UTF8String], strlen(ctx->audio_filename))) { + const char *name = [[device localizedName] UTF8String]; + if (!strncmp(ctx->audio_filename, name, strlen(ctx->audio_filename))) { + audio_device = device; + break; + } + + const char *uniqueId = CLEANUP_DEVICE_ID([device uniqueID]); + if (!strncmp(ctx->audio_filename, uniqueId, strlen(ctx->audio_filename))) { audio_device = device; break; }