From patchwork Sat May 21 14:07:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 34720 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9992:b0:82:461d:f3b with SMTP id ve18csp1081642pzb; Sat, 21 May 2022 07:07:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw86tt4B5nh+rWA0aM5s2GhmxBBabPjIOuQFqkR3sRipvcdlV+p1GW3aw5jTLbJhCQVsgm/ X-Received: by 2002:a17:907:7248:b0:6fe:a121:d060 with SMTP id ds8-20020a170907724800b006fea121d060mr9534639ejc.9.1653142078000; Sat, 21 May 2022 07:07:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1653142077; cv=none; d=google.com; s=arc-20160816; b=exn94JPLcEFjh4K7HWqoD0AAXH0gTCDFPJ2XzmuHovY49ACjNWOQDMAJJ31scHBSMS SDZpC+Wrpc5jSLqy3IVFY2qDttuagMQFNb5DtWanErnSjaE9AaJ9MNzB6ojXr7WDF8OF vzo1GG8o0oThLCmZ+C5eP+jxAnK1bJurNh6EJlUAyz8ZD38QybkqAFBGdh5N7DLwFvOg M8AE5hlerqoEgMAT73wChY38I3biYC6wuXwK0bmTDyz102vPfxz8Yls5ZiKVSohyzSTg NHBBQ3O1bsVlz0HEbQ1D/eZkoRR5Q30qnB3XYVOcDtNYH+bzhi348l8gtfD2P7nZB3bT 4rIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:from:references :in-reply-to:message-id:dkim-signature:delivered-to; bh=zani2bQZRZSJ2u+XJagywwEDYt3u625wiJ07RdGQOWA=; b=g8XThrLgHMATqoHIgkvr6MyCemODaIxtvM/W6THWew2b8vedmNT4L3ldRZtvt+OfE4 95HtxwG0gwH5sSXd5MwkdpvgMGDVN2IM2LgnaKPTg33p0lJUfXi18mS4SDVs5W1ktdhI 4ZltybACZa78/Nk+0x8/bFQD6ZY8P9jTLi5j4NLBtvutONTvApUz1RLnsj/BM6lBHpju zmEMk9k0ToUHAANpGSTsLg2a+zPbu75M7tGJAfDMO0ybC/u+yBlWqq0IaeH0tCNg21Z8 d6tLPFz3ZwKGqOfCDy1JYiEWDp8AamXTg9jNA/QFxQDAL1k0RWSa6Xuxi0HvtsOBliMv LLlA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=TFcqTsPy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sd28-20020a1709076e1c00b006f456ae4b92si11026729ejc.638.2022.05.21.07.07.57; Sat, 21 May 2022 07:07:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=TFcqTsPy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 957B368B3E1; Sat, 21 May 2022 17:07:53 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 398D368A2F6 for ; Sat, 21 May 2022 17:07:46 +0300 (EEST) Received: by mail-pg1-f172.google.com with SMTP id 31so9972647pgp.8 for ; Sat, 21 May 2022 07:07:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=6SSB1xV/dkKxi4XtVsVMDlYSLdDd0o2YrrTbjgFBeFk=; b=TFcqTsPyQ6WCOK8pbUfhtmj7tbGDaMfXzLQwPo3Y3ZZJhBdMMG12frZsFgxm3+MGhf o+8V0PN8v5sy+Wv4DMECVyfoOV/tM917SK6CKOH6nFMfAgn+W7X7ENQE/OnUlWykG/0N OU3aup2jZfVjxNyVwwRS0xABkiI6oAYR5YBuKE+XL9Vl5Pv2iV4OTK1jTqF1ITKt6MJZ XQWmhXB6J3XFMOMvrxWvuLKePsRE7V+njnPMznAa2Euxfi+6vkiXD63iqQ1il8X7Fi1l FcEA+E/GZHcHjiHnSSoGoF9IYGz6FaLVf18CPkQ8hd/+vdZYjH8FtiXGt1YpbQ5U/qXs hsOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=6SSB1xV/dkKxi4XtVsVMDlYSLdDd0o2YrrTbjgFBeFk=; b=pjnXX7C7drbpQz5DIyZW1suFBNEtXdUnomnuV0D8iRrTy8dwjGByUVgIRTYHQqp26S SqdVVVM5kThMYWMSmcrVgA7hfEvDchxx1F4r0Q1IYg+mcgviDEli0POPmamnHe36ouB9 xeuOB3em09MuzNlWFOL5Z07hYQ+cEW6XBYNAqsvYJjoPaQl2j5yNkuJ3tq5zQGxnoFsN aWR8tNW9Fmjexuui03Xm5u64InNP7qJmhpxIVzpEzn5JFS8SD+2CH1ci98BtqlO4e3fr YyM+BHIPqh/0FEUsNlCz5mqSOJiJEkJxalFOR8OnDArEGzYxvO1e7V8KKrOD3ZIVxBvU ViKQ== X-Gm-Message-State: AOAM532FFv2+s4cn+21hHQ7j1Vx/hBBhOLpK7pwojeYWlJmyZwRgcFb2 l8damVeIrX8U7VtfXSAz++VjeYAM0knJnQ== X-Received: by 2002:a65:4989:0:b0:3c6:52a0:1347 with SMTP id r9-20020a654989000000b003c652a01347mr12731627pgs.251.1653142064004; Sat, 21 May 2022 07:07:44 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id h17-20020a656391000000b003db6f4a96c4sm1573421pgv.32.2022.05.21.07.07.42 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 May 2022 07:07:43 -0700 (PDT) Message-Id: In-Reply-To: References: From: ffmpegagent Date: Sat, 21 May 2022 14:07:38 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v2 0/4] Add derive-device function which searches for existing devices in both directions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mark Thompson , softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Xe/hBbYf7c3O This is an updated version of: [PATCH v4 1/1] avutils/hwcontext: When deriving a hwdevice, search for existing device in both directions There has been an objection that the earlier patchset would change API behavior, and that this change should be limited to ffmpeg cli. To achieve this, the API behavior is left unchanged now and a new function av_hwdevice_ctx_get_or_create_derived() is added and used by the hwupload and hwmap filters. v2: Implemented concept for "weak references" to avoid circular reference lockup. softworkz (4): avutil/buffer: add av_ref_from_buffer() function avutils/hwcontext: add derive-device function which searches for existing devices in both directions lavu: bump minor version and add doc/APIchanges entry for av_hwdevice_ctx_get_or_create_derived() avfilter/hwmap,hwupload: use new av_hwdevice_ctx_get_or_create_derived method doc/APIchanges | 6 ++ libavfilter/vf_hwmap.c | 4 +- libavfilter/vf_hwupload.c | 2 +- libavutil/buffer.c | 16 ++++ libavutil/buffer.h | 8 ++ libavutil/hwcontext.c | 167 +++++++++++++++++++++++++++++++-- libavutil/hwcontext.h | 20 ++++ libavutil/hwcontext_internal.h | 11 +++ libavutil/hwcontext_qsv.c | 11 ++- libavutil/version.h | 4 +- 10 files changed, 234 insertions(+), 15 deletions(-) base-commit: 9ab20b1614194280b862d98dfcdb7b1bcff03329 Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-25%2Fsoftworkz%2Fderive_devices-v2 Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-25/softworkz/derive_devices-v2 Pull-Request: https://github.com/ffstaging/FFmpeg/pull/25 Range-diff vs v1: -: ---------- > 1: e9c920c237 avutil/buffer: add av_ref_from_buffer() function 1: 85ff784b6f ! 2: bec8ccfcc2 avutils/hwcontext: add derive-device function which searches for existing devices in both directions @@ Commit message Signed-off-by: softworkz ## libavutil/hwcontext.c ## -@@ libavutil/hwcontext.c: static const AVClass hwdevice_ctx_class = { - static void hwdevice_ctx_free(void *opaque, uint8_t *data) +@@ + #include "buffer.h" + #include "common.h" + #include "hwcontext.h" ++ ++#include ++ ++#include "buffer_internal.h" + #include "hwcontext_internal.h" + #include "imgutils.h" + #include "log.h" + #include "mem.h" + #include "pixdesc.h" + #include "pixfmt.h" ++#include "thread.h" + + static const HWContextType * const hw_table[] = { + #if CONFIG_CUDA +@@ libavutil/hwcontext.c: static const char *const hw_type_names[] = { + [AV_HWDEVICE_TYPE_VULKAN] = "vulkan", + }; + ++#define DEVICE_REGISTRY_SIZE 1024 ++ ++static AVMutex mutex; ++static int is_mutex_initialized = 0; ++static int max_device_reg_id = 1; ++static AVBuffer *hw_device_registry[DEVICE_REGISTRY_SIZE]; ++ ++static int register_hw_device(const AVBufferRef *ref) ++{ ++ AVHWDeviceContext *ctx = (AVHWDeviceContext*)ref->data; ++ const int reg_id = max_device_reg_id; ++ ++ if (ctx == NULL) ++ return AVERROR(EINVAL); ++ ++ if (!is_mutex_initialized) { ++ int ret; ++ ret = ff_mutex_init(&mutex, NULL); ++ if (ret) { ++ av_log(ctx, AV_LOG_ERROR, "hwcontext: mutex initialization failed! Error code: %d\n", ret); ++ return AVERROR(EINVAL); ++ } ++ ++ is_mutex_initialized = 1; ++ } ++ ++ ff_mutex_lock(&mutex); ++ ++ for (int i = 0; i < max_device_reg_id; ++i) { ++ if (hw_device_registry[i] != NULL && hw_device_registry[i] == ref->buffer) { ++ ff_mutex_unlock(&mutex); ++ return i; ++ } ++ } ++ ++ if (max_device_reg_id >= DEVICE_REGISTRY_SIZE) { ++ ff_mutex_unlock(&mutex); ++ av_log(ctx, AV_LOG_ERROR, "Device registry limit (%d) reached. Please check for excessive device creation.", DEVICE_REGISTRY_SIZE); ++ return AVERROR(ENOMEM); ++ } ++ ++ hw_device_registry[reg_id] = ref->buffer; ++ max_device_reg_id++; ++ ++ ff_mutex_unlock(&mutex); ++ ++ return reg_id; ++} ++ ++static void unregister_hw_device(const AVHWDeviceContext *ctx) ++{ ++ if (ctx == NULL || !is_mutex_initialized) ++ return; ++ ++ ff_mutex_lock(&mutex); ++ ++ hw_device_registry[ctx->internal->registered_device_id] = NULL; ++ ++ ff_mutex_unlock(&mutex); ++} ++ ++static AVBufferRef *get_registered_hw_device(int registered_id) ++{ ++ if (registered_id <= 0 || registered_id >= max_device_reg_id) ++ return NULL; ++ ++ ff_mutex_lock(&mutex); ++ ++ if (hw_device_registry[registered_id] != NULL && hw_device_registry[registered_id]->data != NULL) { ++ AVBufferRef *ref = av_ref_from_buffer(hw_device_registry[registered_id]); ++ return ref; ++ } ++ ++ ff_mutex_unlock(&mutex); ++ ++ return NULL; ++} ++ + enum AVHWDeviceType av_hwdevice_find_type_by_name(const char *name) + { + int type; +@@ libavutil/hwcontext.c: static void hwdevice_ctx_free(void *opaque, uint8_t *data) { AVHWDeviceContext *ctx = (AVHWDeviceContext*)data; -+ int i; ++ unregister_hw_device(ctx); ++ /* uninit might still want access the hw context and the user * free() callback might destroy it, so uninit has to be called first */ -@@ libavutil/hwcontext.c: static void hwdevice_ctx_free(void *opaque, uint8_t *data) - ctx->free(ctx); + if (ctx->internal->hw_type->device_uninit) +@@ libavutil/hwcontext.c: int av_hwdevice_ctx_create(AVBufferRef **pdevice_ref, enum AVHWDeviceType type, + const char *device, AVDictionary *opts, int flags) + { + AVBufferRef *device_ref = NULL; +- AVHWDeviceContext *device_ctx; ++ AVHWDeviceContext *device_ctx = NULL; + int ret = 0; - av_buffer_unref(&ctx->internal->source_device); -+ for (i = 0; i < AV_HWDEVICE_TYPE_NB; i++) -+ av_buffer_unref(&ctx->internal->derived_devices[i]); + device_ref = av_hwdevice_ctx_alloc(type); +@@ libavutil/hwcontext.c: int av_hwdevice_ctx_create(AVBufferRef **pdevice_ref, enum AVHWDeviceType type, + if (ret < 0) + goto fail; - av_freep(&ctx->hwctx); - av_freep(&ctx->internal->priv); -@@ libavutil/hwcontext.c: fail: ++ ret = register_hw_device(device_ref); ++ if (ret < 0) ++ goto fail; ++ + ret = av_hwdevice_ctx_init(device_ref); + if (ret < 0) + goto fail; + ++ device_ctx->internal->registered_device_id = ret; ++ + *pdevice_ref = device_ref; + return 0; + fail: ++ unregister_hw_device(device_ctx); + av_buffer_unref(&device_ref); + *pdevice_ref = NULL; return ret; } @@ libavutil/hwcontext.c: fail: - AVDictionary *options, int flags) +static AVBufferRef* find_derived_hwdevice_ctx(AVBufferRef *src_ref, enum AVHWDeviceType type) +{ -+ AVBufferRef *tmp_ref; ++ AVBufferRef *derived_ref; + AVHWDeviceContext *src_ctx; + int i; + @@ libavutil/hwcontext.c: fail: + return src_ref; + + for (i = 0; i < AV_HWDEVICE_TYPE_NB; i++) -+ if (src_ctx->internal->derived_devices[i]) { -+ tmp_ref = find_derived_hwdevice_ctx(src_ctx->internal->derived_devices[i], type); -+ if (tmp_ref) -+ return tmp_ref; ++ if (src_ctx->internal->derived_device_ids[i]) { ++ AVBufferRef *tmp_ref = get_registered_hw_device(src_ctx->internal->derived_device_ids[i]); ++ ++ if (tmp_ref) { ++ derived_ref = find_derived_hwdevice_ctx(tmp_ref, type); ++ ++ if (tmp_ref != derived_ref) ++ av_buffer_unref(&tmp_ref); ++ ++ if (derived_ref) ++ return derived_ref; ++ } + } + + return NULL; @@ libavutil/hwcontext.c: int av_hwdevice_ctx_create_derived_opts(AVBufferRef **dst ret = AVERROR(ENOMEM); goto fail; } -+ if (!tmp_ctx->internal->derived_devices[type]) { -+ tmp_ctx->internal->derived_devices[type] = av_buffer_ref(dst_ref); -+ if (!tmp_ctx->internal->derived_devices[type]) { -+ ret = AVERROR(ENOMEM); -+ goto fail; -+ } -+ } ++ if (!tmp_ctx->internal->derived_device_ids[type]) ++ tmp_ctx->internal->derived_device_ids[type] = dst_ctx->internal->registered_device_id; ++ ret = av_hwdevice_ctx_init(dst_ref); if (ret < 0) goto fail; @@ libavutil/hwcontext_internal.h: struct AVHWDeviceInternal { AVBufferRef *source_device; + + /** -+ * An array of reference to device contexts which ++ * An array of device registration ids from device contexts which + * were derived from this device. + */ -+ AVBufferRef *derived_devices[AV_HWDEVICE_TYPE_NB]; ++ int derived_device_ids[AV_HWDEVICE_TYPE_NB]; ++ ++ /** ++ * ID under wich the hw context is registered internally. ++ */ ++ int registered_device_id; }; struct AVHWFramesInternal { @@ libavutil/hwcontext_qsv.c: static int qsv_device_create(AVHWDeviceContext *ctx, + ret = qsv_device_derive_from_child(ctx, impl, child_device, 0); + if (ret >= 0) { + ctx->internal->source_device = av_buffer_ref(priv->child_device_ctx); -+ child_device->internal->derived_devices[ctx->type] = av_buffer_create((uint8_t*)ctx, sizeof(*ctx), qsv_release_dummy, ctx, 0); -+ if (!child_device->internal->derived_devices[ctx->type]) -+ return AVERROR(ENOMEM); ++ child_device->internal->derived_device_ids[ctx->type] = ctx->internal->registered_device_id; + } - return qsv_device_derive_from_child(ctx, impl, child_device, 0); 2: 8c6a1a8b8c ! 3: a2c8e79e8a lavu: bump minor version and add doc/APIchanges entry for av_hwdevice_ctx_get_or_create_derived() @@ doc/APIchanges: libavutil: 2021-04-27 API changes, most recent first: -+2022-04-30 - xxxxxxxxxx - lavu 57.25.100 - hwcontext.h ++2022-05-21 - xxxxxxxxxx - lavu 57.25.100 - hwcontext.h + Add av_hwdevice_ctx_get_or_create_derived(). ++ ++2022-04-30 - xxxxxxxxxx - lavu 57.25.100 - buffer.h ++ Add av_ref_from_buffer(). + 2022-03-16 - xxxxxxxxxx - all libraries - version_major.h Add lib/version_major.h as new installed headers, which only 3: 023597a557 = 4: d0957afde6 avfilter/hwmap,hwupload: use new av_hwdevice_ctx_get_or_create_derived method