From patchwork Thu Feb 9 14:22:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aline Gondim Santos Gondim Santos X-Patchwork-Id: 40339 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5494:b0:bf:7b3a:fd32 with SMTP id i20csp503327pzk; Thu, 9 Feb 2023 06:23:34 -0800 (PST) X-Google-Smtp-Source: AK7set/juHhWR1/tpvUWReHAlDLMfoH7CECRB2IgGqlU4zJq4xocBMou33rNq9Z+33Ovq3VWqP40 X-Received: by 2002:a17:907:a804:b0:8a6:5720:9101 with SMTP id vo4-20020a170907a80400b008a657209101mr14014413ejc.4.1675952614488; Thu, 09 Feb 2023 06:23:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675952614; cv=none; d=google.com; s=arc-20160816; b=oQ0Ar5EQeMGvYhzszAs70RQm8si6FxeZOsSV6Uv4XpP1wZLR9o7d/GknILfpErw7uW sGD/4V9/ggjCuXeD6KBIBFG+Mt2Dn3CKojVaq7Gw5PnFtaXTTtNUPWiuKyh6OSrrQL5m rRI484p+SsxeZbO0eZQ3ZXsY4K0dYDPzqo9sI70VzyccCjDvUj5V6/mRmtkT9iJ3SGc2 Wl9pZZEa+FowO3H8fIpmVDTHu7nQE1LXEj81XC8RskGNSaRC26IIYY0G6cFXkr0DL8iD iBXxxJw1Wrc/CeOyhTh1OQiQlc9n90aeUVTfApiH1baD2peGifPOQYPoroJHBOmTceNk NKRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:dkim-filter:delivered-to; bh=MHouaofq/J3oi0Px/o/I3wn3Yx8hKuRVmeIHMhAAe94=; b=oO6xE4eK2Op+qumcRlLmw5R/NL+GebFl40xdz8tAPC2ZjuhT2xGbLGIEDUYq3FkLVW okXtdYyC3iSWoHAtOhumLbJ2cb058wbtT1/u9kSu2E+JPkXQwKjL6TSBwhMM/JRhc6Yu D7UI4IneVgsa7kgL3OODXa5rSzm+bkkvmgEVmOh46u1kY1x4I3RofswTdZK+85/AAJlR XQrbRkk9A3qKQIzvyUo6SPRnq6uarc+aCFDtvRxdFldVeP0YWYNXtPYqjwmHCyL1vop0 6nG2Y59qzSz2h2MVT1Zm9l42z9PgNUQfkbr2Da7eOeqUzS9bePiM6bz1TR3+ZZG+iUfu 5zoQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@savoirfairelinux.com header.s=DFC430D2-D198-11EC-948E-34200CB392D2 header.b="iu/hH4/E"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id mu15-20020a1709068a8f00b008716321a08fsi2092185ejc.708.2023.02.09.06.23.21; Thu, 09 Feb 2023 06:23:34 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@savoirfairelinux.com header.s=DFC430D2-D198-11EC-948E-34200CB392D2 header.b="iu/hH4/E"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B392568BE84; Thu, 9 Feb 2023 16:23:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.savoirfairelinux.com (mail.savoirfairelinux.com [208.88.110.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5808468B7DE for ; Thu, 9 Feb 2023 16:23:11 +0200 (EET) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id CEE459C1B62 for ; Thu, 9 Feb 2023 09:23:08 -0500 (EST) Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id j-ZMYEFGsu1t; Thu, 9 Feb 2023 09:23:03 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id 93B959C1CC5; Thu, 9 Feb 2023 09:23:03 -0500 (EST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.savoirfairelinux.com 93B959C1CC5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=savoirfairelinux.com; s=DFC430D2-D198-11EC-948E-34200CB392D2; t=1675952583; bh=/cpdqElCsGQQHcikWqMY1snw/oMyTf/+v6bJj8qXD9c=; h=From:To:Date:Message-Id:MIME-Version; b=iu/hH4/E+Cr5FzClmoOmLBqEuVF4W8SJ2onckalu9saJyk3Ojezf6cSVg5ar03beI PMuaGVr3tKdO38nrisFl7rK7Ig49I9Ny58EVLU/nxh+ee5fhDq5ZqpdbHdszA8zAe8 FjRVG8ZU/6kz3/8YieJ4zkjN2ukXINa/oQ/mYEdRgSjkCArr2AmueInIOpd1yK8oYk l45o5GQF6MGQrLS46DPaKQKoj5itL43lSX53i5SwNHQvGTu5S00Zj95sxXtucScxE9 JaTbpek1R5ahAUSbMRu7c2gImgmJW1DFOIus3obCmpPod1tBR0UZ22i2faWOdas4LY Izj4v5usvNARw== X-Virus-Scanned: amavisd-new at mail.savoirfairelinux.com Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id UtLh8egsSmeX; Thu, 9 Feb 2023 09:23:03 -0500 (EST) Received: from localhost.localdomain (unknown [181.221.107.93]) by mail.savoirfairelinux.com (Postfix) with ESMTPSA id BC3919C1B62; Thu, 9 Feb 2023 09:23:02 -0500 (EST) From: aline.gondimsantos@savoirfairelinux.com To: ffmpeg-devel@ffmpeg.org Date: Thu, 9 Feb 2023 11:22:55 -0300 Message-Id: <20230209142255.38657-1-aline.gondimsantos@savoirfairelinux.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avdevice: add dxgigrab X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Aline Gondim Santos Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: G6oGKo6UQt/X From: Aline Gondim Santos A dxgi grab device may be either a display or a window. Differently from the existing gdigrab, this new device does not make the mouse to flick and also allows proper grabbing of accelerated windows, such as chrome or visual studio code. Signed-off-by: Aline Gondim Santos --- configure | 1 + libavdevice/Makefile | 4 + libavdevice/alldevices.c | 1 + libavdevice/d3dHelpers.h | 59 ++++++++ libavdevice/direct3d11.interop.h | 51 +++++++ libavdevice/dxgigrab.cpp | 225 +++++++++++++++++++++++++++++++ libavdevice/dxgigrab.h | 83 ++++++++++++ libavdevice/dxgigrab_c.c | 59 ++++++++ libavdevice/dxgigrab_c.h | 98 ++++++++++++++ libavdevice/windows_capture.cpp | 184 +++++++++++++++++++++++++ libavdevice/windows_capture.h | 82 +++++++++++ 11 files changed, 847 insertions(+) create mode 100644 libavdevice/d3dHelpers.h create mode 100644 libavdevice/direct3d11.interop.h create mode 100644 libavdevice/dxgigrab.cpp create mode 100644 libavdevice/dxgigrab.h create mode 100644 libavdevice/dxgigrab_c.c create mode 100644 libavdevice/dxgigrab_c.h create mode 100644 libavdevice/windows_capture.cpp create mode 100644 libavdevice/windows_capture.h diff --git a/configure b/configure index 12184c7f26..c9cbee0c09 100755 --- a/configure +++ b/configure @@ -3529,6 +3529,7 @@ fbdev_outdev_deps="linux_fb_h" gdigrab_indev_deps="CreateDIBSection" gdigrab_indev_extralibs="-lgdi32" gdigrab_indev_select="bmp_decoder" +dxgigrab_indev_extralibs="-ldxgi -ld3d11" iec61883_indev_deps="libiec61883" iec61883_indev_select="dv_demuxer" jack_indev_deps="libjack" diff --git a/libavdevice/Makefile b/libavdevice/Makefile index 8a62822b69..6740012000 100644 --- a/libavdevice/Makefile +++ b/libavdevice/Makefile @@ -30,6 +30,7 @@ OBJS-$(CONFIG_FBDEV_INDEV) += fbdev_dec.o \ OBJS-$(CONFIG_FBDEV_OUTDEV) += fbdev_enc.o \ fbdev_common.o OBJS-$(CONFIG_GDIGRAB_INDEV) += gdigrab.o +OBJS-$(CONFIG_DXGIGRAB_INDEV) += windows_capture.o dxgigrab.o dxgigrab_c.o OBJS-$(CONFIG_IEC61883_INDEV) += iec61883.o OBJS-$(CONFIG_JACK_INDEV) += jack.o timefilter.o OBJS-$(CONFIG_KMSGRAB_INDEV) += kmsgrab.o @@ -72,5 +73,8 @@ SKIPHEADERS-$(CONFIG_V4L2_INDEV) += v4l2-common.h SKIPHEADERS-$(CONFIG_V4L2_OUTDEV) += v4l2-common.h SKIPHEADERS-$(CONFIG_ALSA) += alsa.h SKIPHEADERS-$(CONFIG_SNDIO) += sndio.h +SKIPHEADERS-$(CONFIG_DXGIGRAB_INDEV) += dxgigrab.h \ + windows_capture.h \ + dxgigrab_c.h TESTPROGS-$(CONFIG_JACK_INDEV) += timefilter diff --git a/libavdevice/alldevices.c b/libavdevice/alldevices.c index 22323a0a44..fb0a37513b 100644 --- a/libavdevice/alldevices.c +++ b/libavdevice/alldevices.c @@ -35,6 +35,7 @@ extern const AVInputFormat ff_dshow_demuxer; extern const AVInputFormat ff_fbdev_demuxer; extern const AVOutputFormat ff_fbdev_muxer; extern const AVInputFormat ff_gdigrab_demuxer; +extern const AVInputFormat ff_dxgigrab_demuxer; extern const AVInputFormat ff_iec61883_demuxer; extern const AVInputFormat ff_jack_demuxer; extern const AVInputFormat ff_kmsgrab_demuxer; diff --git a/libavdevice/d3dHelpers.h b/libavdevice/d3dHelpers.h new file mode 100644 index 0000000000..d8d2c003ec --- /dev/null +++ b/libavdevice/d3dHelpers.h @@ -0,0 +1,59 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 + * of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#pragma once + +#include +#include +#include + +inline auto +CreateD3DDevice( + D3D_DRIVER_TYPE const type, + winrt::com_ptr& device) +{ + WINRT_ASSERT(!device); + + UINT flags = D3D11_CREATE_DEVICE_BGRA_SUPPORT; + + return D3D11CreateDevice( + nullptr, + type, + nullptr, + flags, + nullptr, 0, + D3D11_SDK_VERSION, + device.put(), + nullptr, + nullptr); +} + +inline auto +CreateD3DDevice() +{ + winrt::com_ptr device; + HRESULT hr = CreateD3DDevice(D3D_DRIVER_TYPE_HARDWARE, device); + + if (DXGI_ERROR_UNSUPPORTED == hr) + { + hr = CreateD3DDevice(D3D_DRIVER_TYPE_WARP, device); + } + + winrt::check_hresult(hr); + return device; +} diff --git a/libavdevice/direct3d11.interop.h b/libavdevice/direct3d11.interop.h new file mode 100644 index 0000000000..62c9b0843e --- /dev/null +++ b/libavdevice/direct3d11.interop.h @@ -0,0 +1,51 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 + * of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#pragma once +#include + +extern "C" +{ + HRESULT __stdcall CreateDirect3D11DeviceFromDXGIDevice(::IDXGIDevice* dxgiDevice, + ::IInspectable** graphicsDevice); + + HRESULT __stdcall CreateDirect3D11SurfaceFromDXGISurface(::IDXGISurface* dgxiSurface, + ::IInspectable** graphicsSurface); +} + +struct __declspec(uuid("A9B3D012-3DF2-4EE3-B8D1-8695F457D3C1")) + IDirect3DDxgiInterfaceAccess : ::IUnknown +{ + virtual HRESULT __stdcall GetInterface(GUID const& id, void** object) = 0; +}; + +inline auto CreateDirect3DDevice(IDXGIDevice* dxgi_device) +{ + winrt::com_ptr<::IInspectable> d3d_device; + winrt::check_hresult(CreateDirect3D11DeviceFromDXGIDevice(dxgi_device, d3d_device.put())); + return d3d_device.as(); +} + +template +auto GetDXGIInterfaceFromObject(winrt::Windows::Foundation::IInspectable const& object) +{ + auto access = object.as(); + winrt::com_ptr result; + winrt::check_hresult(access->GetInterface(winrt::guid_of(), result.put_void())); + return result; +} diff --git a/libavdevice/dxgigrab.cpp b/libavdevice/dxgigrab.cpp new file mode 100644 index 0000000000..9774e72ab1 --- /dev/null +++ b/libavdevice/dxgigrab.cpp @@ -0,0 +1,225 @@ +/* + * DXGI video grab interface + * + * This file is part of FFmpeg. + * + * Copyright (C) 2022 Aline Gondim Santos + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 + * of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * DXGI frame device demuxer + * @author Aline Gondim Santos + */ + +#include "dxgigrab.h" + +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif +#include +#ifdef __cplusplus +} +#endif + +static BOOL CALLBACK enumMonitor(HMONITOR handle, HDC hdc, LPRECT rect, + LPARAM lParam) +{ + MonitorData* monitorData = reinterpret_cast(lParam); + + if (monitorData->curId == 0 || monitorData->desiredId == monitorData->curId) { + monitorData->rect = *rect; + monitorData->id = monitorData->curId; + + monitorData->hmnt = handle; + } + + return (monitorData->desiredId > monitorData->curId++); +} + +int dxgigrab_read_header(AVFormatContext *s1) +{ + struct dxgigrab *s = static_cast(s1->priv_data); + if (!s->internal_struct) + s->internal_struct = new dxgigrab_internal(); + struct dxgigrab_internal* x_internal = static_cast(s->internal_struct); + MonitorData monitorData; + monitorData.rect.top = 0; + monitorData.rect.left = 0; + monitorData.rect.bottom = 0; + monitorData.rect.right = 0; + std::string handle; + AVStream *st = NULL; + + if (!strncmp(s1->url, "hwnd=", 5)) { + if (!s->hwnd) { + handle = s1->url + 5; + try { + s->hwnd = reinterpret_cast(std::stoull(handle, nullptr, 16)); + } catch (...) {} + if (!s->hwnd) { + av_log(s1, AV_LOG_ERROR, + "Can't find window from handle '%s', aborting.\n", handle.c_str()); + return AVERROR_EXTERNAL; + } + } + } else { + s->hwnd = NULL; + char *display_number = av_strdup(s1->url); + if (!sscanf(s1->url, "%[^+]+%d,%d ", display_number, &s->offset_x, &s->offset_y)) { + av_log(s1, AV_LOG_ERROR, + "Please use \"+ \" or \"hwnd=\" to specify your target.\n"); + return AVERROR_EXTERNAL; + } + if (s->offset_x || s->offset_y) { + av_log(s1, + AV_LOG_ERROR, + "This device does not support partial screen sharing.\n"); + return AVERROR_PATCHWELCOME; + } + monitorData.desiredId = std::stoi(display_number); + av_freep(&display_number); + try { + LPARAM lParam = reinterpret_cast(&monitorData); + EnumDisplayMonitors(NULL, NULL, enumMonitor, lParam); + } catch (...) { + } + } + + s->hmnt = monitorData.hmnt; + s->rect = monitorData.rect; + + if (!s->hmnt && !s->hwnd) { + av_log(s1, AV_LOG_ERROR, + "Please use \"+ \" or \"hwnd=\" to " + "specify your target.\n"); + return AVERROR_EXTERNAL; + } + + bool openWindow = true; + if (!s1->nb_streams) { + st = avformat_new_stream(s1, NULL); + } else { + openWindow = false; + st = s1->streams[0]; + } + if (!st) { + return AVERROR(ENOMEM); + } + + std::lock_guard lk(x_internal->mtx); + if (openWindow) { + auto d3dDevice = CreateD3DDevice(); + auto dxgiDevice = d3dDevice.as(); + auto m_device = CreateDirect3DDevice(dxgiDevice.get()); + + auto activation_factory = winrt::get_activation_factory(); + auto interop_factory = activation_factory.as(); + winrt::Windows::Graphics::Capture::GraphicsCaptureItem windowItem = { nullptr }; + + try { + if (s->hwnd) { + winrt::check_hresult(interop_factory + ->CreateForWindow(s->hwnd, // we should have create for display too + winrt::guid_of(), + reinterpret_cast(winrt::put_abi(windowItem)))); + } else if (s->hmnt) { + winrt::check_hresult(interop_factory + ->CreateForMonitor(s->hmnt, // we should have create for display too + winrt::guid_of(), + reinterpret_cast(winrt::put_abi(windowItem)))); + } + } catch (winrt::hresult_error &err) { + av_log(s, AV_LOG_ERROR, "CreateForWindow failed: %s\n", err.message().c_str()); + return AVERROR_EXTERNAL; + } catch (...) { + av_log(s, AV_LOG_ERROR, "CreateForWindow failed\n"); + return AVERROR_EXTERNAL; + } + av_log(s, AV_LOG_ERROR, "CreateForWindow success\n"); + + x_internal->m_capture.reset(new WindowsCapture(m_device, windowItem, monitorData)); + x_internal->m_capture->StartCapture(); + x_internal->m_capture->checkNewFrameArrived(); + x_internal->m_capture->window = handle; + x_internal->windowHandle = handle; + } + + s->time_base = av_inv_q(s->framerate); + + avpriv_set_pts_info(st, 64, 1, 1000000); + + s->width = x_internal->m_capture->m_DeviceSize.Width; + s->height = x_internal->m_capture->m_DeviceSize.Height; + + x_internal->m_capture->screen.first = monitorData.hmnt; + x_internal->m_capture->screen.second = monitorData.rect; + + s->time_frame = av_gettime_relative(); + + st->avg_frame_rate = av_inv_q(s->time_base); + + auto frame_size_bits = (int64_t)s->width * s->height * 4 * 8; + if (frame_size_bits / 8 + AV_INPUT_BUFFER_PADDING_SIZE > INT_MAX) { + av_log(s, AV_LOG_ERROR, "Captured area is too large\n"); + return AVERROR_PATCHWELCOME; + } + s->frame_size = frame_size_bits / 8; + + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_RAWVIDEO; + st->codecpar->width = s->width; + st->codecpar->height = s->height; + st->codecpar->format = AV_PIX_FMT_RGBA; + st->codecpar->bit_rate = av_rescale(frame_size_bits, st->avg_frame_rate.num, st->avg_frame_rate.den); + + return 0; +} + +int dxgigrab_read_packet(AVFormatContext *s1, AVPacket *pkt) +{ + struct dxgigrab *s = static_cast(s1->priv_data); + struct dxgigrab_internal *x_internal = static_cast(s->internal_struct); + std::lock_guard lk(x_internal->mtx); + int ret = 0; + int64_t pts = av_gettime(); + + if (!x_internal->m_capture) + return 0; // If m_capture not available, return empty frame + ret = x_internal->m_capture->GetPkt(pkt); + if (ret < 0) + return ret; + pkt->dts = pkt->pts = pts; + + return ret; +} + +int dxgigrab_read_close(AVFormatContext *s1) +{ + struct dxgigrab* s = static_cast(s1->priv_data); + struct dxgigrab_internal* x_internal = static_cast(s->internal_struct); + std::lock_guard lk(x_internal->mtx); + + if (!x_internal->m_capture) + return 0; // If m_capture not available, no need to close it + x_internal->m_capture->Close(); + + return 0; +} diff --git a/libavdevice/dxgigrab.h b/libavdevice/dxgigrab.h new file mode 100644 index 0000000000..92cad7dab8 --- /dev/null +++ b/libavdevice/dxgigrab.h @@ -0,0 +1,83 @@ +/* + * DXGI video grab interface + * + * This file is part of FFmpeg. + * + * Copyright (C) 2022 Aline Gondim Santos + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 + * of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * DXGI frame device demuxer + * @author Aline Gondim Santos + */ + +#ifndef AVDEVICE_DXGI_H +#define AVDEVICE_DXGI_H + +#include + +#ifdef __cplusplus +extern "C" { +#endif +#include "dxgigrab_c.h" +#ifdef __cplusplus +} +#endif + +#include +#include + +// WinRT +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +// STL +#include +#include +#include + +// D3D +#include +#include +#include +#include + +// Internal +#include "d3dHelpers.h" +#include "direct3d11.interop.h" +#include "windows_capture.h" + +struct dxgigrab_internal { + std::unique_ptr m_capture{ nullptr }; + std::string windowHandle; + std::mutex mtx; +}; + +#endif /* AVDEVICE_DXGI_H */ diff --git a/libavdevice/dxgigrab_c.c b/libavdevice/dxgigrab_c.c new file mode 100644 index 0000000000..c53d757abd --- /dev/null +++ b/libavdevice/dxgigrab_c.c @@ -0,0 +1,59 @@ +/* + * DXGI video grab interface + * + * This file is part of FFmpeg. + * + * Copyright (C) 2022 Aline Gondim Santos + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 + * of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * DXGI frame device demuxer + * @author Aline Gondim Santos + */ + +#include "dxgigrab_c.h" + +#define OFFSET(x) offsetof(struct dxgigrab, x) +#define DEC AV_OPT_FLAG_DECODING_PARAM +static const AVOption options[] = { + { "framerate", "set video frame rate", OFFSET(framerate), AV_OPT_TYPE_VIDEO_RATE, {.str = "ntsc"}, 0, INT_MAX, DEC }, + { "video_size", "set video frame size", OFFSET(width), AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, DEC }, + { "offset_x", "capture area x offset", OFFSET(offset_x), AV_OPT_TYPE_INT, {.i64 = 0}, INT_MIN, INT_MAX, DEC }, + { "offset_y", "capture area y offset", OFFSET(offset_y), AV_OPT_TYPE_INT, {.i64 = 0}, INT_MIN, INT_MAX, DEC }, + { NULL }, +}; + +static const AVClass dxgigrab_class = { + .class_name = "DXGIgrab indev", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, + .category = AV_CLASS_CATEGORY_DEVICE_VIDEO_INPUT, +}; + +/** dxgi grabber device demuxer declaration */ +AVInputFormat ff_dxgigrab_demuxer = { + .name = "dxgigrab", + .long_name = NULL_IF_CONFIG_SMALL("DXGI API Windows frame grabber"), + .priv_data_size = sizeof(struct dxgigrab), + .read_header = dxgigrab_read_header, + .read_packet = dxgigrab_read_packet, + .read_close = dxgigrab_read_close, + .flags = AVFMT_NOFILE, + .priv_class = &dxgigrab_class, +}; diff --git a/libavdevice/dxgigrab_c.h b/libavdevice/dxgigrab_c.h new file mode 100644 index 0000000000..d624ca0683 --- /dev/null +++ b/libavdevice/dxgigrab_c.h @@ -0,0 +1,98 @@ +/* + * DXGI video grab interface + * + * This file is part of FFmpeg. + * + * Copyright (C) 2022 Aline Gondim Santos + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 + * of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * DXGI frame device demuxer + * @author Aline Gondim Santos + */ + +#ifndef AVDEVICE_DXGI_C_H +#define AVDEVICE_DXGI_C_H + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#include "libavformat/internal.h" +#include "config.h" +#include "libavutil/opt.h" +#include "libavutil/time.h" + +#include + +/** + * DXGI Device Demuxer context + */ +struct dxgigrab { + const AVClass *avclass; /**< Class for private options */ + + int frame_size; /**< Size in bytes of the frame pixel data */ + AVRational time_base; /**< Time base */ + int64_t time_frame; /**< Current time */ + + AVRational framerate; /**< Capture framerate (private option) */ + int width; /**< Width of the grab frame (private option) */ + int height; /**< Height of the grab frame (private option) */ + int offset_x; /**< Capture x offset (private option) */ + int offset_y; /**< Capture y offset (private option) */ + + HWND hwnd; /**< Handle of the Window for the grab */ + HMONITOR hmnt; /**< Handle of the Screen for the grab */ + RECT rect; /**< Rect of the Screen for the grab */ + + void* internal_struct; +}; + +/** + * Initializes the dxgi grab device demuxer (public device demuxer API). + * + * @param s1 Context from avformat core + * @return AVERROR_IO error, 0 success + */ +int dxgigrab_read_header(AVFormatContext *s1); + +/** + * Grabs a frame from dxgi (public device demuxer API). + * + * @param s1 Context from avformat core + * @param pkt Packet holding the grabbed frame + * @return frame size in bytes + */ +int dxgigrab_read_packet(AVFormatContext *s1, AVPacket *pkt); + + +/** + * Closes dxgi frame grabber (public device demuxer API). + * + * @param s1 Context from avformat core + * @return 0 success, !0 failure + */ +int dxgigrab_read_close(AVFormatContext *s1); + +#ifdef __cplusplus +} +#endif + +#endif /* AVDEVICE_DXGI_C_H */ diff --git a/libavdevice/windows_capture.cpp b/libavdevice/windows_capture.cpp new file mode 100644 index 0000000000..c6b29f1a1d --- /dev/null +++ b/libavdevice/windows_capture.cpp @@ -0,0 +1,184 @@ +/* + * This file is part of FFmpeg. + * + * Copyright (C) 2022 Aline Gondim Santos + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 + * of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @author Aline Gondim Santos + */ + +#include "dxgigrab.h" + +#include + +using namespace winrt; +using namespace Windows; +using namespace Windows::Foundation; +using namespace Windows::System; +using namespace Windows::Graphics; +using namespace Windows::Graphics::Capture; +using namespace Windows::Graphics::DirectX; +using namespace Windows::Graphics::DirectX::Direct3D11; +using namespace Windows::Foundation::Numerics; +using namespace Windows::UI; +using namespace Windows::UI::Composition; + +WindowsCapture::WindowsCapture( + IDirect3DDevice const& device, + GraphicsCaptureItem const& item, + MonitorData& monitorData) +{ + m_item = item; + m_device = device; + + m_d3dDevice = GetDXGIInterfaceFromObject(m_device); + m_d3dDevice->GetImmediateContext(m_d3dContext.put()); + screen.first = monitorData.hmnt; + screen.second = monitorData.rect; + + m_DeviceSize = m_item.Size(); + + // Create framepool, define pixel format (DXGI_FORMAT_B8G8R8A8_UNORM), and frame size. + m_framePool = Direct3D11CaptureFramePool::Create( + m_device, + DirectXPixelFormat::R8G8B8A8UIntNormalized, + 2, + m_DeviceSize); + m_session = m_framePool.CreateCaptureSession(m_item); + auto sup = m_session.IsSupported(); +} + +// Start sending capture frames +void +WindowsCapture::StartCapture() +{ + std::lock_guard lk(mtx_); + running_ = true; + m_session.StartCapture(); +} + +// Process captured frames +void +WindowsCapture::Close() +{ + std::lock_guard lk(mtx_); + running_ = false; + if (texture) { + texture->Release(); + } + m_framePool.Close(); + m_session.Close(); + m_d3dContext->ClearState(); + m_d3dContext->Flush(); + m_d3dDevice->Release(); + + texture = nullptr; + m_framePool = nullptr; + m_session = nullptr; + m_item = nullptr; +} + +bool +WindowsCapture::checkNewFrameArrived() +{ + std::lock_guard lk(mtx_); + if (!running_) + return false; + auto shouldResize = false; + + auto frame = m_framePool.TryGetNextFrame(); + if (!frame) + return false; + + auto frameSurface = GetDXGIInterfaceFromObject(frame.Surface()); + + D3D11_TEXTURE2D_DESC desc; + frameSurface->GetDesc(&desc); + auto frameContentSize = frame.ContentSize(); + if (desc.Width <= 0 || desc.Height <= 0) + return false; + + shouldResize = frameContentSize.Width != m_DeviceSize.Width || frameContentSize.Height != m_DeviceSize.Height; + + if (shouldResize) { + m_DeviceSize.Width = frameContentSize.Width; + m_DeviceSize.Height = frameContentSize.Height; + m_framePool.Recreate(m_device, DirectXPixelFormat::R8G8B8A8UIntNormalized, 2, m_DeviceSize); + return false; + } + + texDesc.Width = desc.Width; + texDesc.Height = desc.Height; + texDesc.MipLevels = desc.MipLevels; + texDesc.ArraySize = desc.ArraySize; + texDesc.Format = desc.Format; + texDesc.SampleDesc = desc.SampleDesc; + texDesc.Usage = D3D11_USAGE_STAGING; + texDesc.BindFlags = 0; + texDesc.CPUAccessFlags = D3D11_CPU_ACCESS_READ; + texDesc.MiscFlags = 0; + + m_d3dDevice->CreateTexture2D(&texDesc, nullptr, &texture); + + // copy the texture to a staging resource + m_d3dContext->CopyResource(texture, frameSurface.get()); + + return true; +} + +int +WindowsCapture::GetPkt(AVPacket *pkt) +{ + if (!checkNewFrameArrived() || !running_ || !texture) + return 0; + std::lock_guard lk(mtx_); + + // now, map the staging resource TO CPU + D3D11_MAPPED_SUBRESOURCE mapInfo; + m_d3dContext->Map( + texture, + 0, + D3D11_MAP_READ, + 0, + &mapInfo); + + m_d3dContext->Unmap(texture, 0); + + auto ret = av_new_packet(pkt, mapInfo.DepthPitch); + if (ret < 0) { + if (mapInfo.DepthPitch) { + av_packet_unref(pkt); + } + texture->Release(); + texture = nullptr; + return ret; + } + + auto idx = 0; + for (auto y = 0; y < m_DeviceSize.Height; y++) { + for (auto x = 0; x < m_DeviceSize.Width * 4; x++) { + pkt->data[idx] = static_cast(mapInfo.pData)[y * mapInfo.RowPitch + x]; + idx++; + } + } + + texture->Release(); + texture = nullptr; + + return 0; +} diff --git a/libavdevice/windows_capture.h b/libavdevice/windows_capture.h new file mode 100644 index 0000000000..3e9269ed58 --- /dev/null +++ b/libavdevice/windows_capture.h @@ -0,0 +1,82 @@ +/* + * This file is part of FFmpeg. + * + * Copyright (C) 2022 Aline Gondim Santos + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 + * of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @author Aline Gondim Santos + */ + +#pragma once + +#ifdef __cplusplus +extern "C" { +#endif +#include +#ifdef __cplusplus +} +#endif + +#include +#include + +#include + +struct MonitorData { + HMONITOR hmnt = NULL; + RECT rect; + int id = 0; + int curId = 0; + int desiredId = 0; +}; + +class WindowsCapture +{ +public: + WindowsCapture( + winrt::Windows::Graphics::DirectX::Direct3D11::IDirect3DDevice const& device, + winrt::Windows::Graphics::Capture::GraphicsCaptureItem const& item, + MonitorData& monitorData); + ~WindowsCapture() { Close(); } + + void StartCapture(); + + int GetPkt(AVPacket *pkt); + + void Close(); + std::string window; + + bool checkNewFrameArrived(); + winrt::Windows::Graphics::SizeInt32 m_DeviceSize; + std::pair screen; + +private: + winrt::Windows::Graphics::Capture::GraphicsCaptureItem m_item{ nullptr }; + winrt::Windows::Graphics::Capture::Direct3D11CaptureFramePool m_framePool{ nullptr }; + winrt::Windows::Graphics::Capture::GraphicsCaptureSession m_session{ nullptr }; + + winrt::Windows::Graphics::DirectX::Direct3D11::IDirect3DDevice m_device {nullptr}; + winrt::com_ptr m_d3dDevice{nullptr}; + winrt::com_ptr m_d3dContext {nullptr}; + + D3D11_TEXTURE2D_DESC texDesc; + ID3D11Texture2D *texture {nullptr}; + + std::mutex mtx_; + bool running_; +};