From patchwork Mon Oct 10 10:54:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 38657 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp299432pzb; Mon, 10 Oct 2022 03:55:30 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6ubUz2VsBrHNaimYB39IFO5WiYpzLFdZFCUDOI37IPKOx6P9yQsIapKmiaiPjS6FNo+iht X-Received: by 2002:a05:6402:2787:b0:459:5139:7e4c with SMTP id b7-20020a056402278700b0045951397e4cmr17594397ede.17.1665399330114; Mon, 10 Oct 2022 03:55:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1665399330; cv=none; d=google.com; s=arc-20160816; b=Q0pAxdoGxfbyb4536+SNnbTMtPAkcO611TjCQazeBkDgL7mB9tG0K7lOddAiX1IMmw tVAdiuY3qIVSROoMlZFKiGVQTjAVLanSiYFwUhMnZiBlClm8GDdDmt0e6N/500WfaPqP poPkKvsfvIKR1DIQ5NmEVa+VZSGhCYQ6A9kpmn5hW2SmEb1H0WI6/N7Zzol82bzg0f6u 0DnSVPNdexHz5Fzq0EqFRd7SyADXxEsduHWpnnz0L7uNJuukeYIbSXIwdc8ggTUtqnC9 r83fXSK/VSWMIa//aCevLuNiMCwQUDu0CtsnTGDWRh8oSTLcdre124osKJY5wKep9DJM OXzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=SiIaHJ0kbax8r06TeDjO1OeI/EQJQko5flIGhbF82KQ=; b=q5zt03SxsedXi6tIcL41rOGFnr1wwWIQIuVnUMYJ5Iq8FbYozW9zJVKG7ijbUK0XiB pze1HiSPqT0FkcTuY5vTvXT8RR3Jpe5/lsWw9C2eMoOJpPSlIPss1ChaaUMNgMOAe55Q Y0KYm9YohhI5NNI0g0E0J7ov4g0gUvXnqakvKdzoEmSbCz0nxBRLg0SLNRQdAp3DMYNx +7kYnplzic68gTlxu7esfi4n7REkQUIokyAT/x7yoShWB+ZnoLTvQBDW0HazKrgGoQ7q J4zOEPYzT4iKAlOmKQwlasWp8W9YRA6VAKvqdpKN7sudFoReKFDSaFERcA02FlmYvwtu oj0w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=T72NjCIl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qa15-20020a170907868f00b00779f8e7ec5bsi9808901ejc.42.2022.10.10.03.55.29; Mon, 10 Oct 2022 03:55:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=T72NjCIl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 72A1A68BCDD; Mon, 10 Oct 2022 13:54:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4680368BCDB for ; Mon, 10 Oct 2022 13:54:47 +0300 (EEST) Received: by mail-pj1-f53.google.com with SMTP id l1-20020a17090a72c100b0020a6949a66aso10059641pjk.1 for ; Mon, 10 Oct 2022 03:54:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=LJFrx32BhsR1JGS95uspSe/It+1V+hDKXLRZXKUA1Zs=; b=T72NjCIlpd0DRx4JGcDMEc/SoPD+jOv/37ZGnsbReW3rZ3p0rKxCDFyv9csVFkJaEr shQYg2IWi4bdx2kpWaHc//If/ViuPSVsCOZ3fxWrn56GT6xhCnN4za8g2wgWB+D8s+Ic WgcQ3kJioFqCAWYrrVDAj2Ng3G5vB+E5sciecMXP/Nce/9ZClxY1LFwzlswgkDx9BYcg PHV+4r2yOPgBrZnHNjgbjmQfCJ0Yhm0UV2moHek1WAXz/kFvujCv64qsRC2gSP2W2gU8 GVNArgNzxV00n0IgVYDRPsO0DY+TlDaemyIXay0aJ1eLNguD33oOuvH7Q8cFjZxjZ+Wn srmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=LJFrx32BhsR1JGS95uspSe/It+1V+hDKXLRZXKUA1Zs=; b=Zyv02xqkLJrkKEwgWHrNhKJk+PPazOxzMAGTrJsiMeYh2kw9mumDewxpT4fylAeGcp gE4U6Js1hYxTZtqmvX+MBuXaTMIM6kQienM9p+ZWlrIOHSM5+9z+F2Km8KpL1Lhwarvx aYt8ou9BgiW6gfcbQYm5fyJdiOwZttefGSi0g0YS5iDV34qQSuOXdcz1ZUbhAalmYeby TAQJS2UsKsZUimlM8ZVhmGgHN0BvYAPJ3xpdYwjPWfPNJpa791YUJpcswQjYAlRfVgW/ FHRCmMB0eUA1bs7RLV0GjG3nVN4CNawrPoCJ6962sN1bkIkZQ1XDgGqQThalWBC9og7u dq1Q== X-Gm-Message-State: ACrzQf3+gsiQ4QlwKZkx6ioERyhUPZQjkMRMOne16QCCeSgBGCBO2Z6t 2UzziGXaFWI4hoWWFg2PWrKS4pwuBGs= X-Received: by 2002:a17:90a:d48a:b0:20a:f65b:143c with SMTP id s10-20020a17090ad48a00b0020af65b143cmr19899335pju.199.1665399285581; Mon, 10 Oct 2022 03:54:45 -0700 (PDT) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id 63-20020a620642000000b0056109e15638sm6563678pfg.54.2022.10.10.03.54.45 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Oct 2022 03:54:45 -0700 (PDT) From: softworkz X-Google-Original-From: softworkz Message-Id: In-Reply-To: References: Date: Mon, 10 Oct 2022 10:54:34 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 04/11] avfilter/overlay_vaapi: handle secondary null input X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: softworkz Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 0j6/quJhor5/ From: softworkz Currently segfaults in this case. Signed-off-by: softworkz --- libavfilter/vf_overlay_vaapi.c | 94 ++++++++++++++++++---------------- 1 file changed, 49 insertions(+), 45 deletions(-) diff --git a/libavfilter/vf_overlay_vaapi.c b/libavfilter/vf_overlay_vaapi.c index 66e736cce4..1281038c36 100644 --- a/libavfilter/vf_overlay_vaapi.c +++ b/libavfilter/vf_overlay_vaapi.c @@ -106,18 +106,6 @@ static int overlay_vaapi_render_picture(AVFilterContext *avctx, params_id); - vas = vaCreateBuffer(ctx->hwctx->display, ctx->va_context, - VAProcPipelineParameterBufferType, - sizeof(*subpic_params), 1, subpic_params, &subpic_params_id); - if (vas != VA_STATUS_SUCCESS) { - av_log(avctx, AV_LOG_ERROR, "Failed to create parameter buffer: " - "%d (%s).\n", vas, vaErrorStr(vas)); - err = AVERROR(EIO); - goto fail_after_begin; - } - av_log(avctx, AV_LOG_DEBUG, "Pipeline subpic parameter buffer is %#x.\n", - subpic_params_id); - vas = vaRenderPicture(ctx->hwctx->display, ctx->va_context, ¶ms_id, 1); if (vas != VA_STATUS_SUCCESS) { @@ -127,13 +115,27 @@ static int overlay_vaapi_render_picture(AVFilterContext *avctx, goto fail_after_begin; } - vas = vaRenderPicture(ctx->hwctx->display, ctx->va_context, - &subpic_params_id, 1); - if (vas != VA_STATUS_SUCCESS) { - av_log(avctx, AV_LOG_ERROR, "Failed to render subpic parameter buffer: " - "%d (%s).\n", vas, vaErrorStr(vas)); - err = AVERROR(EIO); - goto fail_after_begin; + if (subpic_params) { + vas = vaCreateBuffer(ctx->hwctx->display, ctx->va_context, + VAProcPipelineParameterBufferType, + sizeof(*subpic_params), 1, subpic_params, &subpic_params_id); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to create parameter buffer: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail_after_begin; + } + av_log(avctx, AV_LOG_DEBUG, "Pipeline subpic parameter buffer is %#x.\n", + subpic_params_id); + + vas = vaRenderPicture(ctx->hwctx->display, ctx->va_context, + &subpic_params_id, 1); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to render subpic parameter buffer: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail_after_begin; + } } vas = vaEndPicture(ctx->hwctx->display, ctx->va_context); @@ -177,7 +179,7 @@ static int overlay_vaapi_blend(FFFrameSync *fs) AVFrame *input_main, *input_overlay; AVFrame *output; VAProcPipelineParameterBuffer params, subpic_params; - VABlendState blend_state; /**< Blend State */ + VABlendState blend_state = { 0 }; /**< Blend State */ VARectangle overlay_region, output_region; int err; @@ -192,10 +194,6 @@ static int overlay_vaapi_blend(FFFrameSync *fs) av_get_pix_fmt_name(input_main->format), input_main->width, input_main->height, input_main->pts); - av_log(avctx, AV_LOG_DEBUG, "Filter overlay: %s, %ux%u (%"PRId64").\n", - av_get_pix_fmt_name(input_overlay->format), - input_overlay->width, input_overlay->height, input_overlay->pts); - if (vpp_ctx->va_context == VA_INVALID_ID) return AVERROR(EINVAL); @@ -214,13 +212,6 @@ static int overlay_vaapi_blend(FFFrameSync *fs) if (err < 0) goto fail; - overlay_region = (VARectangle) { - .x = ctx->overlay_ox, - .y = ctx->overlay_oy, - .width = ctx->overlay_ow ? ctx->overlay_ow : input_overlay->width, - .height = ctx->overlay_oh ? ctx->overlay_oh : input_overlay->height, - }; - output_region = (VARectangle) { .x = 0, .y = 0, @@ -228,29 +219,42 @@ static int overlay_vaapi_blend(FFFrameSync *fs) .height = output->height, }; - if (overlay_region.x + overlay_region.width > input_main->width || - overlay_region.y + overlay_region.height > input_main->height) { - av_log(ctx, AV_LOG_WARNING, - "The overlay image exceeds the scope of the main image, " - "will crop the overlay image according based on the main image.\n"); - } - params.filters = &vpp_ctx->filter_buffers[0]; params.num_filters = vpp_ctx->nb_filter_buffers; params.output_region = &output_region; params.output_background_color = VAAPI_VPP_BACKGROUND_BLACK; - memcpy(&subpic_params, ¶ms, sizeof(subpic_params)); + if (input_overlay) { + av_log(avctx, AV_LOG_DEBUG, "Filter overlay: %s, %ux%u (%"PRId64").\n", + av_get_pix_fmt_name(input_overlay->format), + input_overlay->width, input_overlay->height, input_overlay->pts); + + overlay_region = (VARectangle) { + .x = ctx->overlay_ox, + .y = ctx->overlay_oy, + .width = ctx->overlay_ow ? ctx->overlay_ow : input_overlay->width, + .height = ctx->overlay_oh ? ctx->overlay_oh : input_overlay->height, + }; + + if (overlay_region.x + overlay_region.width > input_main->width || + overlay_region.y + overlay_region.height > input_main->height) { + av_log(ctx, AV_LOG_WARNING, + "The overlay image exceeds the scope of the main image, " + "will crop the overlay image according based on the main image.\n"); + } + + memcpy(&subpic_params, ¶ms, sizeof(subpic_params)); - blend_state.flags = VA_BLEND_GLOBAL_ALPHA; - blend_state.global_alpha = ctx->alpha; - subpic_params.blend_state = &blend_state; + blend_state.flags = VA_BLEND_GLOBAL_ALPHA; + blend_state.global_alpha = ctx->alpha; + subpic_params.blend_state = &blend_state; - subpic_params.surface = (VASurfaceID)(uintptr_t)input_overlay->data[3]; - subpic_params.output_region = &overlay_region; + subpic_params.surface = (VASurfaceID)(uintptr_t)input_overlay->data[3]; + subpic_params.output_region = &overlay_region; + } - err = overlay_vaapi_render_picture(avctx, ¶ms, &subpic_params, output); + err = overlay_vaapi_render_picture(avctx, ¶ms, input_overlay ? &subpic_params : NULL, output); if (err < 0) goto fail;