From patchwork Tue Nov 21 05:57:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rogozhkin, Dmitry V" X-Patchwork-Id: 44738 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8c2a:b0:181:818d:5e7f with SMTP id j42csp338269pzh; Mon, 20 Nov 2023 21:58:11 -0800 (PST) X-Google-Smtp-Source: AGHT+IEKHYs75m7kON9a5gXB63mmIzJ3cg3gFLi4M7BS5kSKeThUIS8rQfDqy9zIwZOQnF7xiy2F X-Received: by 2002:a17:906:ca:b0:a01:9985:30d9 with SMTP id 10-20020a17090600ca00b00a01998530d9mr717880eji.12.1700546291392; Mon, 20 Nov 2023 21:58:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700546291; cv=none; d=google.com; s=arc-20160816; b=CecXVp10F5UJIF4R1/tV/9c7I0oJhJgLrW0ZbdtY4VbalLwbXZnMOxskl3+ijZnWEe zvIl4PbxsLXpyJlWOqC2rM1Af9eO0YFxcQkZ93Lwma3W2SvQyUAJQ6zNOzZRYvXI3LaT qRr7dj3hvl25GtnEUr1eUjfmdfOs7V3bG6EwCEs37lCg5aNfoA8mlUEvyB8B62oS/rPL sYiWPEbCcIdFdPydxYqM6XwV0oy7eTHPoVpHk9DJ1tQ7q1UcVjzOZWtlTPhpXdxlBXlB UG63iWdYgpqaxYjJcCmbbyQP7NGkfHXlFczbj0exGB7fgux2ZGHJQZXaExLuzj5OEJL/ fhaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=Y0lCLoIP+Rz+jBexPkyZhVpvyAKR4J6UdE+hZJ4/RbA=; fh=z34nsZay6HvjYIRPk5BwyArNH+rsauucRUMCMjMxepw=; b=Ybcn8jTXn36CWaK6i5H43jHwNK85a5VMm6bL8f3owy4qE0JLU4xmbJsX9QkONsOyxX lIKDv/YupWtCB2QGmZiHpBGvuU6GSXnGZGVl4WG9bLgcmGVd4EZeLnAgr6eIk1nTg/bU oxNsnUHe6MBn0MRUlfk29wU3wUyUSXj0XqfP/VW1/xK/wWvdLMk8/7BjXWyj+kFu7YkK j1yyGKBG/z61tf47Swx/5/CkK+ZiqbWAzdlfv7PODOyhlsPQXf9XIU8OWbwsED74NAyh 9YAHK+YZpJGt1dUhXSohB+8DHHzX2YSrkhoR5YWqVFo5ww7uo1IRnwGmUOQVpChLdNQQ lQHw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Lu2b10hU; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ha3-20020a170906a88300b009fee213afa1si1759959ejb.389.2023.11.20.21.58.10; Mon, 20 Nov 2023 21:58:11 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Lu2b10hU; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 15E7968A9C0; Tue, 21 Nov 2023 07:58:07 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CDC3868C99F for ; Tue, 21 Nov 2023 07:57:59 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1700546285; x=1732082285; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=uwTjntibsBdMGkqTmPxPWPdcJr362vZM2ULvyTx5vPo=; b=Lu2b10hUuN6aDL5tJs4TBDVdnpSEbz1FS+JrR9Wa+f2mL/bzppc27X7z 4gcLw9WDNdCNADAyV7GGe/5n0+55zL6K/CIlRcEy4rdfg7Y9oOpjM+Su0 yf0CxI1kL4ZiqzopIo9RROWH8m58vNla1fGskdUS7ALjacmFNUAPbKAIG BMIPIdqHS4KzGT1x7omXERdnGhmnUJS+peiv2BWAeEMzFw1ZhQuGj2MBY p5XWJosUCIb+1hP7RrQkdQx0ME4aoAXydIn/GPshuksrdIUehz//DUTR+ 9MyzoFgL7W7Wtgwvh2Y55dW1CZQzOlkk4JG4T6mTGVbaAALbBE+IKDELG Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10900"; a="395700188" X-IronPort-AV: E=Sophos;i="6.04,215,1695711600"; d="scan'208";a="395700188" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2023 21:57:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,215,1695711600"; d="scan'208";a="14800815" Received: from dvrscl.jf.intel.com ([10.54.72.26]) by fmviesa001.fm.intel.com with ESMTP; 20 Nov 2023 21:57:56 -0800 From: Dmitry Rogozhkin To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 Nov 2023 21:57:32 -0800 Message-Id: <1700546252-17490-1-git-send-email-dmitry.v.rogozhkin@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1699986151-24741-1-git-send-email-dmitry.v.rogozhkin@intel.com> References: <1699986151-24741-1-git-send-email-dmitry.v.rogozhkin@intel.com> Subject: [FFmpeg-devel] [PATCH v5] avcodec/decode: validate hw_frames_ctx when AVHWAccel.free_frame_priv is used X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Christoph Reiter , Dmitry Rogozhkin , Lynne MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: aFbX1Omb4e6W Validate that a hw_frames_ctx is available before using it for the AVHWAccel.free_frame_priv callback, and don't require it to be present when the callback is not in use by the HWAccel. v2: check for free_frame_priv (Hendrik) v3: return EINVAL (Christoph Reiter) v4: better commit message (Hendrik) v5: fix typo with missed frames_ctx (Lynne) See[1]: https://github.com/msys2/MINGW-packages/pull/19050 Fixes: be07145109 ("avcodec: add AVHWAccel.free_frame_priv callback") CC: Lynne CC: Christoph Reiter Signed-off-by: Dmitry Rogozhkin --- libavcodec/decode.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/libavcodec/decode.c b/libavcodec/decode.c index ad39021..2cfb3fc 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -1838,17 +1838,26 @@ int ff_copy_palette(void *dst, const AVPacket *src, void *logctx) int ff_hwaccel_frame_priv_alloc(AVCodecContext *avctx, void **hwaccel_picture_private) { const FFHWAccel *hwaccel = ffhwaccel(avctx->hwaccel); - AVHWFramesContext *frames_ctx; if (!hwaccel || !hwaccel->frame_priv_data_size) return 0; av_assert0(!*hwaccel_picture_private); - frames_ctx = (AVHWFramesContext *)avctx->hw_frames_ctx->data; - *hwaccel_picture_private = ff_refstruct_alloc_ext(hwaccel->frame_priv_data_size, 0, - frames_ctx->device_ctx, - hwaccel->free_frame_priv); + if (hwaccel->free_frame_priv) { + AVHWFramesContext *frames_ctx; + + if (!avctx->hw_frames_ctx) + return AVERROR(EINVAL); + + frames_ctx = (AVHWFramesContext *) avctx->hw_frames_ctx->data; + *hwaccel_picture_private = ff_refstruct_alloc_ext(hwaccel->frame_priv_data_size, 0, + frames_ctx->device_ctx, + hwaccel->free_frame_priv); + } else { + *hwaccel_picture_private = ff_refstruct_allocz(hwaccel->frame_priv_data_size); + } + if (!*hwaccel_picture_private) return AVERROR(ENOMEM);