From patchwork Mon Jan 9 12:50:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39944 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bc95:b0:ad:ade2:bfd2 with SMTP id fx21csp3066037pzb; Mon, 9 Jan 2023 04:51:02 -0800 (PST) X-Google-Smtp-Source: AMrXdXu87edRJHeN/2//BE2Spn/HB4+FHaVGIo8xCv8bGzgoNLBwsDSl10xRhAmS3EQYMt+z5JxZ X-Received: by 2002:a17:906:c0c2:b0:836:3d22:5d73 with SMTP id bn2-20020a170906c0c200b008363d225d73mr54959279ejb.39.1673268662145; Mon, 09 Jan 2023 04:51:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673268662; cv=none; d=google.com; s=arc-20160816; b=HjOW/TQhG8nYLH129DJ1vLJfLkAd9jyqqUpcJrzyJgPRuj/edfAOkaWBB8zVXwqZTb bjkfixj6pise7gjpt4R3WhnUtYIe/hvzgr/mfqrLW1UdOhg7d51zMnAylwduYIu3kkXO M6LbnLuvMumKKorCl7wt0L2+hpWCQwJZHVtrCwKeTztpQF4+TkSEjl1WYMepnVWvxufM 3RjAJlWMgP3bRvWqg31fbTu9b7MsZCJpQFyF7IkEORndQB9UVNE0kCqr1xnbIKwbWJl2 C5JgKqfLcDQfFLMB+u4UNEXaQFWyWdWj1b5lgeiKzpYsXKGard1uQu+kz/byQmD6dDIM ue9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:date:to:from:message-id :dkim-signature:delivered-to; bh=vXXbt/+iloE7dnD+qSphE/ZkyFo/ZGjg84UN48dWDtE=; b=p0NKc2GgeNl1TCriw1yT78nmfe9lsvvWHAEbSU4VlWxGlP7Ip06EtRF8N5ALFXVOte M4LHZqNlM11mdv81/JtNvhZ0kOmkhFsBunp7+bPeRN8nwW6zJxBXo2sHLSAokrNa7GXQ HE0C638NJpl9b6shKALRsVmGmFkmwe51/x/0PxW111jL2Eya06PsnzSfEUJmE4gWQk0+ xEMlRMKCAmoD/wObDbvs5DJffeRa8ymjf2Poa9iS3OrIEiroKGANBvOJwZgl7mcgWjgK Mn9zcR/RR7vtr40qB48Xa3rMVcndWAtuWq6QeDF46FnHL13yCRCoQKJKjVHKqw6dcsG7 +NZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=KnK4BYVY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id xd2-20020a170907078200b0084d302afcc3si5366439ejb.79.2023.01.09.04.51.01; Mon, 09 Jan 2023 04:51:02 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=KnK4BYVY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 87DE168BB61; Mon, 9 Jan 2023 14:50:26 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-221-245.mail.qq.com (out203-205-221-245.mail.qq.com [203.205.221.245]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5B14B68BCCE for ; Mon, 9 Jan 2023 14:50:17 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1673268611; bh=eZ7dEWIyxmrWHPHF9IQR/0REjKWikG0Jlle6lPZEWZM=; h=From:To:Cc:Subject:Date; b=KnK4BYVY2WXkrrHZ8kUToS2aViH0v6+ltgP1IfLBZfTxI0wdLes2BQ4m3BBa6Y7W/ VB/O2W/sXmLZwKUZjltJvDQefHo3cLyhiEIf64u53CzkRaaF8xBiNErqqc1THtFtYx XgNF3lQWh2Ab3l7sd6Kpj+AO4RmG/adlvN5hfB9c= Received: from ZHILIZHAO-MB2.tencent.com ([119.147.10.192]) by newxmesmtplogicsvrszc5-0.qq.com (NewEsmtp) with SMTP id C898A4CD; Mon, 09 Jan 2023 20:50:09 +0800 X-QQ-mid: xmsmtpt1673268609t7a1p103l Message-ID: X-QQ-XMAILINFO: MiE+axgVDEQLkwG7svVIda68ZKEEA+58AXn6CiiaGaMw2esiXyKMsjT/OwREBB 1iprHtxWiahzY8XFuXG/5PRleJYZdpmRzJUy/9p6XexM3+4zKd6mivZ79vrCFF3YstJmtfZ5u3OF JCzfkFxBgIp9GkEZJIJeiihSMcFAMxKclJSm2PMUptOB0Z5jxsjc+vqPRFPe24DMedRl1zcSDoSq 9GZkc510VPQCD/gyVUsaOTvNsJ3534kLQh1nQofjuWQPkPB5r4bl1ftp2qL6F/LK+O2AZ63cbRv8 0/i0sEhVVbe+scZEaRS/VKlJH8QIGknu2jeNOb3BhyZYPjQV9bXqeHXHrnAe8t13T40d6IqwfBrM Tv/oqf4T3WpN9ntLaq6RV8wErdDuqrYUtOcgOcYWtduUIAUEkD3ax1hLLOmKUSfLHQ5PoYYIqC6T U8TWrFPqQrv53S+bX0jl1QcXb63wPFVZwRP34JSz+sgQVJRuA+UDpu8VRviY1YceIM2LJMajjpIz rNIrau/r+NFvA3xXxvdtO58ykSS5EjkxQ2ZnNue+Fbdl3CCLq21SwwIlrAuzDfsXHV8wxFFBvokB 1j2ZsOZs0Iu7OKooyIdKEL8xNqRaGQZ5T/an6Ibukh+7ELljEBHV33RxEkAkB/vpshCJtz8sKbmO cH+C4xHRJHlFPRdkpPBIjWOS/KKvjKCny7nzD73qcaBHKxf3xypeR/W2WyhO2QAh0oEOmisw70dd /3Jyi/O7v2KIyAihvJG2RN3xQj6gXdaQr7zkccVC8XR7wFFvZgVJ72+1U7aw0AUYMox+y+na1LVz nWmqEo+j9tsuWA1XMOC8kRgdaXP2CrRMNPDrQRQm6dq/c2XSgEmgJQvA7Djg5JDsurIP8KYGzONN ri6xu2NrF6JZbS9wA86YAIs7RefB8ko7MDOrLctYkWxkXiKHnvqYAGHqZlKKjyj86ODNiMU0fbxp eCJuySdfMYQdeZrEK/8dlbeohuuy0z From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 9 Jan 2023 20:50:03 +0800 X-OQ-MSGID: <20230109125008.13336-1-quinkblack@foxmail.com> X-Mailer: git-send-email 2.35.3 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/5] avcodec/videotoolbox: fix NULL pointer dereference X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: bL/MlHVnYm3g From: Zhao Zhili In the code path of av_videotoolbox_default_init/init2(), avctx->internal->hwaccel_priv_data is NULL and passed to decoder_cb.decompressionOutputRefCon. Then it will be dereferenced inside videotoolbox_decoder_callback(). Delay videotoolbox_star() until ff_videotoolbox_common_init() to fix the bug. --- libavcodec/videotoolbox.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c index 1b1be8ddb4..a18b49007d 100644 --- a/libavcodec/videotoolbox.c +++ b/libavcodec/videotoolbox.c @@ -1181,9 +1181,8 @@ int ff_videotoolbox_common_init(AVCodecContext *avctx) vtctx->logctx = avctx; - // Old API - do nothing. if (avctx->hwaccel_context) - return 0; + return videotoolbox_start(avctx); if (!avctx->hw_frames_ctx && !avctx->hw_device_ctx) { av_log(avctx, AV_LOG_ERROR, @@ -1404,7 +1403,7 @@ int av_videotoolbox_default_init2(AVCodecContext *avctx, AVVideotoolboxContext * avctx->hwaccel_context = vtctx ?: av_videotoolbox_alloc_context_with_pix_fmt(pix_fmt, full_range); if (!avctx->hwaccel_context) return AVERROR(ENOMEM); - return videotoolbox_start(avctx); + return 0; } void av_videotoolbox_default_free(AVCodecContext *avctx)