From patchwork Mon Sep 30 08:27:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Guillem X-Patchwork-Id: 51939 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:d8ca:0:b0:48e:c0f8:d0de with SMTP id dy10csp1789175vqb; Mon, 30 Sep 2024 01:28:00 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUlwMgP/LJhoRQhAPpxFA1yRkmmZldXfqRojMiWOO9glkk08y5DrDeYF98Hlc2OQlU41ASwi+HbNdxwUC8HlVYF@gmail.com X-Google-Smtp-Source: AGHT+IE/3kIlAfyjX89Z5+g96NjTKsCvNNZwQnpsnPqE8kI4SnRLX7OlAh9/JPFtD631OZWNNpyT X-Received: by 2002:a05:6402:40d2:b0:5c7:2209:dedb with SMTP id 4fb4d7f45d1cf-5c8777b59c1mr16282210a12.8.1727684880258; Mon, 30 Sep 2024 01:28:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1727684880; cv=none; d=google.com; s=arc-20240605; b=XXC6dnSniGCCaZE0OJHt0LFolXRsrYzpyzIrNls8FutQcMkIRbe2cr5Drm+eLd5A9Z ZAS0ea8fKW/AswJ0i1an4tsKFvHUr3Vc2tG4/1Jbc0+xXweM8IEVhg1RsNmYeWaYwhcj r1xjjJV1RrTmOu1GL5XWrZvq1t4qnZ4j24yPmobaD4RkdTOzxVJbV0vNa4ioBkI5BV9q asFCfFjZElNhWNsCVJKCb1JTw2LRfbp7P2b8ZHXtVcIlucqmwd+8pxJjfO5a1JT8JGAK OF0ig31bJTClGuQY2ryHikURauWrpT8xaXjeF2kYm38LGFzHFr7GvAVSQtQZPb2raXDY TJ2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to :feedback-id:delivered-to; bh=K99x1JxO+1mTdROgG6XNSVOOgfDAlofR/DA6pf57c/0=; fh=3LY024vSFS/PP4LTfcCy63MQoPE5JAibP23dzfT0gvE=; b=UeF7TRDfJxgGg+yX7fQyUg5pOg4E36AQY/YJMPqz7x7Rq0JD9PxWIM07wz5L6WcfTO ju523NPT+LCo8Wm1sXk4KYXF0yOE5faEvvEEAIt2K0f3b1mhlZaQwWYz20OHbOyFXhoD +M3p2G5utL62skVOLrrIRxrlxdwIMx51t1nD+v3BSksd8NR7/MPZcg65NbsqOi7pzKIa or9MUFxF1GVO3X/cWT6884kEOA7kHv/bxu5hJ97fjHFJomSa2nvcGA/M00gpOoPjBHYg gJKa5jJChFjBBMhPw3ixoeNqiB4uX+oZl2OK6UtPHGpxHp4Fph5wMbonjAcjTcHfAc6N ovLg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c88249bd76si5436198a12.461.2024.09.30.01.27.59; Mon, 30 Sep 2024 01:28:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7160968DD6F; Mon, 30 Sep 2024 11:27:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from fout-a2-smtp.messagingengine.com (fout-a2-smtp.messagingengine.com [103.168.172.145]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2065968DBA6 for ; Mon, 30 Sep 2024 11:27:50 +0300 (EEST) Received: from phl-compute-11.internal (phl-compute-11.phl.internal [10.202.2.51]) by mailfout.phl.internal (Postfix) with ESMTP id EA8CA138041A for ; Mon, 30 Sep 2024 04:27:48 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-11.internal (MEProxy); Mon, 30 Sep 2024 04:27:48 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrvdduhedgtdefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefhvffuff fkofgggfestdekredtredttdenucfhrhhomhepvfhhohhmrghsucfiuhhilhhlvghmuceo thhhohhmrghssehglhhlmhdrfhhrqeenucggtffrrghtthgvrhhnpefgtdelgfevtddtle dufeetgfefjeehgefhvdevffevgfdtvedtkeejieekvefgjeenucevlhhushhtvghrufhi iigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehthhhomhgrshesghhllhhmrdhfrh dpnhgspghrtghpthhtohepuddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepfhhf mhhpvghgqdguvghvvghlsehffhhmphgvghdrohhrgh X-ME-Proxy: Feedback-ID: i239440f1:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 30 Sep 2024 04:27:48 -0400 (EDT) To: ffmpeg-devel@ffmpeg.org Date: Mon, 30 Sep 2024 10:27:46 +0200 Message-ID: <20240930082746.187437-1-thomas@gllm.fr> X-Mailer: git-send-email 2.45.2 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/decode: clean-up if get_hw_frames_parameters fails X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Thomas Guillem via ffmpeg-devel From: Thomas Guillem Reply-To: FFmpeg development discussions and patches Cc: Thomas Guillem Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: q9TRycIoqSqa Fixes the following assert: [00007f1df83d17e0] vaapi generic error: avcodec_get_hw_frames_parameters failed: -22 Assertion p_dst->hwaccel_threadsafe || (!dst->hwaccel && !dst->internal->hwaccel_priv_data) failed at libavcodec/pthread_frame.c:349 Reproduced from VLC with VAAPI, when fallbacking from hw to sw. --- libavcodec/decode.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/libavcodec/decode.c b/libavcodec/decode.c index c331bb8596..e90a6630e1 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -19,6 +19,7 @@ */ #include +#include #include #include "config.h" @@ -1173,6 +1174,7 @@ int avcodec_get_hw_frames_parameters(AVCodecContext *avctx, const AVCodecHWConfigInternal *hw_config; const FFHWAccel *hwa; int i, ret; + bool clean_priv_data = false; for (i = 0;; i++) { hw_config = ffcodec(avctx->codec)->hw_configs[i]; @@ -1197,6 +1199,7 @@ int avcodec_get_hw_frames_parameters(AVCodecContext *avctx, av_buffer_unref(&frames_ref); return AVERROR(ENOMEM); } + clean_priv_data = true; } ret = hwa->frame_params(avctx, frames_ref); @@ -1217,6 +1220,10 @@ int avcodec_get_hw_frames_parameters(AVCodecContext *avctx, *out_frames_ref = frames_ref; } else { + if (clean_priv_data) { + av_freep(avctx->internal->hwaccel_priv_data); + avctx->internal->hwaccel_priv_data = NULL; + } av_buffer_unref(&frames_ref); } return ret;