From patchwork Mon Nov 14 01:16:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 39263 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp2159213pzb; Sun, 13 Nov 2022 17:16:24 -0800 (PST) X-Google-Smtp-Source: AA0mqf7y/pbd2o+dOSzsAFSUHX+REiUNuo6oaCD7lS+BSZUTLMdaZwwmuaxLq3bd2lfkb4hiLWmg X-Received: by 2002:a17:906:5f8b:b0:79a:101a:7e57 with SMTP id a11-20020a1709065f8b00b0079a101a7e57mr8965754eju.368.1668388584094; Sun, 13 Nov 2022 17:16:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668388584; cv=none; d=google.com; s=arc-20160816; b=yr3uwq6ISibKszpIslQAUV89jOwquOIDTN7QDMq2suPbpMyZoHHVVnUeImetXGSggp iabpxJvwTHIFkNveuAxdb4d5WQAXvnWbbcmub53rQVIQDqR4AErpTNTqxY+J5c+OWPDh DTb3uma8qRbn8GqaO9bxKRRZUJcEKlMPDrbUn5+i1nsseQuR7oBxOm6fN0ybmVyMxzv9 89hWh4WcV0Sh9n+Lx9XV/scI24r0CnCzi8rxJXsxHuzTnpK+2yW80AOwFFghGnSt7JgK D3YCcHSEy5Oei16PATb8dBHOP1DY5KnbJZ4oVbm99F+iEeqHJ/Bsnxmo+V812zFZECoc Qxag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=lIB3QqJHzx4tKoN9avwLxMk163O6vdmmQn6NjiGfB38=; b=Da0DyRtlZBwkweh9Yh0GI0ZIN9byfa9hAcVtaAEbnLY0oI4F6GrFHSp49zXq0oJHBf CQOVmijksfqTWjpuOGMPuD3uTw3OyFEyi/JarEbMe3N8Q/mvtWNSjK2X5jB5cWOjItT7 5UR1zrP3OPbclAE3SDaysqgEwr0ZhUcu0Jy5RKSfpKwYSyEsx8fYtdaKpLgvurZgPMl7 7nL9tb/HV7Z3gyAmN9gMAeXw5WbRflBodUJN4U2o6qlfpVy3zJU/2+uz03zZswNE0C7Q z9Sm+nKfbUr8/Hdg46Y5CkZn9wnou8u8zwbrZbGeXnIujSW8bncasvU2XvVGTZeso5Vn GgBg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Er65d0bz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v27-20020a50d59b000000b00461e95b2721si6772159edi.129.2022.11.13.17.16.23; Sun, 13 Nov 2022 17:16:24 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Er65d0bz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 544A268BCFA; Mon, 14 Nov 2022 03:16:19 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 66C1A68B797 for ; Mon, 14 Nov 2022 03:16:12 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668388577; x=1699924577; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=MSTwYKjFLqbSW7sRbwewsvMgR6/UibEZU4JwvRlgc/o=; b=Er65d0bzQxnXO6f1GlWRHUfkE+XS9eMEtX7QcU5DKmlh4Uao4RZgxxlV 1JrSTT4g0aTduoKkZsrmUJxAQECKFrLHSjzLb6rkLh01AxHkAynJXg7I0 CbPjdoaM9mPzjfHAGi1RmtX9kbf4yMC2x+P92hXp3UY1tDPK8ye9jYDnq EFN9NzGLyhI7nb2GI3FzrY7PYTgsjQvZYnYSux1k//joKgdNVqjxcs1bs /hiEWG/TVJV/dK6l6uX1eVDFxA3nBK/sZLN2TRfYRTL3srTV3ocxwV0Wk qgY3Wbzip+ht875c61ajYzCIM8AYFzuuKtMJs+C4VIqHamqyCjMH6A8mj w==; X-IronPort-AV: E=McAfee;i="6500,9779,10530"; a="338639933" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="338639933" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Nov 2022 17:16:09 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10530"; a="616100165" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616100165" Received: from t.sh.intel.com ([10.239.159.159]) by orsmga006.jf.intel.com with ESMTP; 13 Nov 2022 17:16:08 -0800 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Mon, 14 Nov 2022 09:16:04 +0800 Message-Id: <20221114011605.1157707-1-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v5 1/2] lavc: add HWACCEL_CAP_RESET_WITHOUT_UNINIT capacity for hwaccel X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: BVzI8wFu9LJZ The capacity can avoid hwaccel being uninited when do the reset. It provides the method for hwaccel if it still want to use the previous initialized configuration after reset. And the configuration can be updated in AVHWAccel.init() if needed. For example, when use vaapi vp9 decode dynamic resolution clips, need to avoid changing vaContext in avctx->internal->hwaccel_priv_data if current frame resolution change and it reference a pervious frame with different resolution. Otherwise reference frame's information bound in vaContext will be lost, then corrupt current frame. Signed-off-by: Fei Wang --- update: 1. consider the case of va_config/va_context equal to 0. libavcodec/decode.c | 10 ++++++---- libavcodec/hwconfig.h | 7 +++++++ 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 6be2d3d6ed..cfada048e8 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -1109,7 +1109,7 @@ static int hwaccel_init(AVCodecContext *avctx, return AVERROR_PATCHWELCOME; } - if (hwaccel->priv_data_size) { + if (hwaccel->priv_data_size && !avctx->internal->hwaccel_priv_data) { avctx->internal->hwaccel_priv_data = av_mallocz(hwaccel->priv_data_size); if (!avctx->internal->hwaccel_priv_data) @@ -1134,10 +1134,12 @@ static int hwaccel_init(AVCodecContext *avctx, static void hwaccel_uninit(AVCodecContext *avctx) { - if (avctx->hwaccel && avctx->hwaccel->uninit) - avctx->hwaccel->uninit(avctx); + if (avctx->hwaccel && !(avctx->hwaccel->caps_internal & HWACCEL_CAP_RESET_WITHOUT_UNINIT)) { + if (avctx->hwaccel->uninit) + avctx->hwaccel->uninit(avctx); - av_freep(&avctx->internal->hwaccel_priv_data); + av_freep(&avctx->internal->hwaccel_priv_data); + } avctx->hwaccel = NULL; diff --git a/libavcodec/hwconfig.h b/libavcodec/hwconfig.h index 721424912c..5fb4e06d5f 100644 --- a/libavcodec/hwconfig.h +++ b/libavcodec/hwconfig.h @@ -25,6 +25,13 @@ #define HWACCEL_CAP_ASYNC_SAFE (1 << 0) +/** + * The hwaccel supports reset without calling back AVHWAccel.uninit() + * and realloc avctx->internal->hwaccel_priv_data. + * + * New configuration can set up through AVHWAccel.init(). + */ +#define HWACCEL_CAP_RESET_WITHOUT_UNINIT (1 << 1) typedef struct AVCodecHWConfigInternal { /** From patchwork Mon Nov 14 01:16:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 39264 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp2159273pzb; Sun, 13 Nov 2022 17:16:33 -0800 (PST) X-Google-Smtp-Source: AA0mqf5il++//ncT6dXCeSBpXfOBW2JxCNcpSZGXRygsQ6gs+jfwrQs//JpozDebyy4+g82IHC/c X-Received: by 2002:a05:6402:1949:b0:467:9864:9463 with SMTP id f9-20020a056402194900b0046798649463mr5386383edz.360.1668388593319; Sun, 13 Nov 2022 17:16:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668388593; cv=none; d=google.com; s=arc-20160816; b=HrCKQ/0f4kBCBR1mJjdoK+f3o99E4yRbJzumgzp8w4RPSFp/jmnvp02+XIfyafJ0YY 4J3qZDrnMjPRb1OatYHe0cdh/nxyiPwdCOphdJZuXjcvX/g8wLU8hoE6GxUNr+KVPlR1 yc6mRJGr4S5YuvP9aOE0BZQhXozcDHX54hF4N/EqRIkqxWWSC75QONXCwGUrWlqU0aTi h0O/JH7iUl9WOBEjvvv57SjEP2ap2sxqzFm3dePX0oZBtnVxK/ce4Udbl4TGmG8bTW8n pqZZDrdLp2cXHxRp4mJdY/eHD9YE8KfKS1srtbkJddo531Vn7KjuLbmkuqyxC0W++R0r GyjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=6dtKI+ccmu0A1QwdV2Tu43wyueRg7SX4jx3zammxXPo=; b=kJ04KBokgqqdhRZzw32YpuzQMULvmqOoe9ZEWFu4i9XCTMLL+Vo1AfG6DBAz8cximM DbuctweZvGLmk6S/qouNY//D5dG1EytjyUnL2skDib68sGp/GjvkvXvAe8wtvhQcGnqq A8VyorzLrcYqjrokd2GtX35HjSAF2vyfGmXJKMmaic36RAqibQW+KzNcR272cTvV7MfP 6vf6rWRkT38VccuiKdWgODmGfJEZgleck9pda8bPGlIa1bFcUI8XwIL3Yc1yrbV9MtZ+ cwe/hgtln8ZoyG3aXWMT1QOZYHpmvRbc23fh6R2wNae7VR2ZBG8ANFCyqa3Y+OGWWFSE GqjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=mAGN17r5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o15-20020a170906600f00b00782e85ae302si6363607ejj.574.2022.11.13.17.16.33; Sun, 13 Nov 2022 17:16:33 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=mAGN17r5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 750E468BD1E; Mon, 14 Nov 2022 03:16:22 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3CB6168BBCF for ; Mon, 14 Nov 2022 03:16:15 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668388580; x=1699924580; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6n+Ww5ymTBQrevDj2fRUwCjS0umjGFouzPjW73h2Z0Y=; b=mAGN17r5sYv1ksYMo6LXo1X3UorNCxLB/+PG4mu0mfhf1TI2ORcqGACu e8rCqdJlgpLV+xM8WVOtoYWzSd0MmeFmONI2mkvX4Sq1RdYuxTg/W8N3u BEx80G8yTyAQDwgsEZIWbUFjHa0Uj/2nIow4vFzzEnNn86acWUFbZ4MHL ex5zoxwq7f3uu/XswzbizsreXvM/EO51nXI2x/JPhBJ9q9UEKL4vGEm0+ bpUXzLk5PcDy8kbhvyj3/1cS3SIKC9o2FggMK573LwUFySKMpxdlKxAZW G4dttSwVXKvvbie2KmF8Jgq0Rlam94Q6ZwFIjTkWItuEa9/0ltZwQyZxD w==; X-IronPort-AV: E=McAfee;i="6500,9779,10530"; a="338639938" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="338639938" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Nov 2022 17:16:13 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10530"; a="616100178" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616100178" Received: from t.sh.intel.com ([10.239.159.159]) by orsmga006.jf.intel.com with ESMTP; 13 Nov 2022 17:16:12 -0800 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Mon, 14 Nov 2022 09:16:05 +0800 Message-Id: <20221114011605.1157707-2-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221114011605.1157707-1-fei.w.wang@intel.com> References: <20221114011605.1157707-1-fei.w.wang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v5 2/2] lavc/vaapi_decode: add support for HWACCEL_CAP_RESET_WITHOUT_UNINIT X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: bGhF9R0dk0u2 This can fix vp9 decode image corruption when the frame size is change, but the pervious frames still be referenced. Surfaces don't need to be bound to vaContext only after VAAPI 1.0.0: https://github.com/intel/libva/commit/492b692005ccd0d8da190209d5b3ae7b7825f4b8 Signed-off-by: Fei Wang --- libavcodec/vaapi_decode.c | 11 ++++++++--- libavcodec/vaapi_decode.h | 1 + libavcodec/vaapi_vp9.c | 4 ++++ 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c index 134f10eca5..d950471b6d 100644 --- a/libavcodec/vaapi_decode.c +++ b/libavcodec/vaapi_decode.c @@ -658,9 +658,6 @@ int ff_vaapi_decode_init(AVCodecContext *avctx) VAStatus vas; int err; - ctx->va_config = VA_INVALID_ID; - ctx->va_context = VA_INVALID_ID; - err = ff_decode_get_hw_frames_ctx(avctx, AV_HWDEVICE_TYPE_VAAPI); if (err < 0) goto fail; @@ -670,6 +667,12 @@ int ff_vaapi_decode_init(AVCodecContext *avctx) ctx->device = ctx->frames->device_ctx; ctx->hwctx = ctx->device->hwctx; + if (ctx->inited) + return 0; + + ctx->va_config = VA_INVALID_ID; + ctx->va_context = VA_INVALID_ID; + err = vaapi_decode_make_config(avctx, ctx->frames->device_ref, &ctx->va_config, NULL); if (err) @@ -691,6 +694,8 @@ int ff_vaapi_decode_init(AVCodecContext *avctx) av_log(avctx, AV_LOG_DEBUG, "Decode context initialised: " "%#x/%#x.\n", ctx->va_config, ctx->va_context); + ctx->inited = 1; + return 0; fail: diff --git a/libavcodec/vaapi_decode.h b/libavcodec/vaapi_decode.h index 6beda14e52..62a4f37ed9 100644 --- a/libavcodec/vaapi_decode.h +++ b/libavcodec/vaapi_decode.h @@ -61,6 +61,7 @@ typedef struct VAAPIDecodeContext { int surface_count; VASurfaceAttrib pixel_format_attribute; + int inited; } VAAPIDecodeContext; diff --git a/libavcodec/vaapi_vp9.c b/libavcodec/vaapi_vp9.c index 776382f683..245b7a1b3a 100644 --- a/libavcodec/vaapi_vp9.c +++ b/libavcodec/vaapi_vp9.c @@ -181,5 +181,9 @@ const AVHWAccel ff_vp9_vaapi_hwaccel = { .uninit = ff_vaapi_decode_uninit, .frame_params = ff_vaapi_common_frame_params, .priv_data_size = sizeof(VAAPIDecodeContext), +#if VA_CHECK_VERSION(1, 0, 0) + .caps_internal = HWACCEL_CAP_ASYNC_SAFE | HWACCEL_CAP_RESET_WITHOUT_UNINIT, +#else .caps_internal = HWACCEL_CAP_ASYNC_SAFE, +#endif };