From patchwork Tue Aug 13 06:18:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: sharpbai X-Patchwork-Id: 14477 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 57837449C75 for ; Tue, 13 Aug 2019 09:47:01 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2AF0668A803; Tue, 13 Aug 2019 09:47:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 30F8968013E for ; Tue, 13 Aug 2019 09:46:54 +0300 (EEST) Received: by mail-pf1-f178.google.com with SMTP id q139so2733917pfc.13 for ; Mon, 12 Aug 2019 23:46:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=dFzc92aahjU8dl1c1Mt1rmvP6fZkK2y7SIRRniyP66A=; b=GstJL9PULbLUzGn5mOH6G9AkH9k0eTBsLo9XCTVCyuDOeUIUDYp+jUul+iTzPMQ3dV G6Y2Rh842PAfbyuF6SKkFl8ev52Mk1O2jz6Hsyb8eqc82tfSAPOPcNIpBJnLOGEKM23J EDkAvS1oUB6NHrz3DU/mJI49HlGn//gGF4Cce0D6CoH7vSB8ZHS3RUSwht8Yn9V/9IXH alzaTgh7plmXoCAu6Uy7X0e9AVkKY+pa0KWdTC3WTAC6+ooJEjggnhto4q3Y5mADlWjv jnIQkcUH563jeUVI8EjcjavhRnG1ned9TBLuDdO1iUzw7HSAfb5HeOyV99SMXdU9JSvL xbWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=dFzc92aahjU8dl1c1Mt1rmvP6fZkK2y7SIRRniyP66A=; b=Hsq5FXdAlmY4BFWrIBRMegrB0cVl1Twf7o03bSpWLJLz+b1w3DFy6h7/ZPOe1kbqVd dJT6dgRFdhuwvj01edFFYbkJW/RhfpE2ViFv63ZWt7dfpq+ePyla3sPaO36nB+U4kveB LoVUZkS6UkS/Epwbxw2os1mGrnkoNLDXuqsERiFxTllK1oajjykvfH2GDq3YhnJ/Znzf SHYwWfekmpCtL0mAKQtUtK/uDSU3woTBMbclqtMMpCHQz8wpbuRLX6pENNbPy/3CNAmR qU7oTIVvD+cgxp4VFzlYfLaztdY/snKr8XvsCt1MwThxul/UGG1AcbzLKhsYhO4lhEFL w6mQ== X-Gm-Message-State: APjAAAUwtPAPsV3EFuLvGKCP5Xah55e6Wjl7+otLbpMGMRnwuaTUbeOf WF/xJI27g4YLUhdYOenoSBtTWlz4g0w= X-Google-Smtp-Source: APXvYqzLcM8tlsLnGDZL0WNB89kmyH8j6WEcIGOogEM2/k+E92NbroUSXGnOjJi2G4YRxbyGlPh3fg== X-Received: by 2002:a17:90a:1b0c:: with SMTP id q12mr765823pjq.76.1565677188858; Mon, 12 Aug 2019 23:19:48 -0700 (PDT) Received: from localhost.localdomain (ec2-13-230-214-69.ap-northeast-1.compute.amazonaws.com. [13.230.214.69]) by smtp.gmail.com with ESMTPSA id f19sm160120844pfk.180.2019.08.12.23.19.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 12 Aug 2019 23:19:48 -0700 (PDT) From: sharpbai To: ffmpeg-devel@ffmpeg.org Date: Tue, 13 Aug 2019 14:18:21 +0800 Message-Id: <20190813061820.58120-1-sharpbai@gmail.com> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] avcodec/videotoolboxenc: fix encoding frame crash on iOS 11 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: sharpbai Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On iOS 11, encoding a frame may return error with log "Error encoding frame 0", which means vtenc_output_callback is called with status=0 and sample_buffer=NULL. Then the encoding session will be crashed on next callback wether or not closing the codec context. Let us look through the link below introducing VTCompressionOutputCallback, https://developer.apple.com/documentation/videotoolbox/vtcompressionoutputcallback?language=objc "status=0" (noErr) means compression was successful. "sampleBuffer=NULL" means the frame was dropped when compression was successful (status=0) or compression was not successful (status!=0). So we should not set AVERROR_EXTERNAL on "status=0" and "sample_buffer=NULL" as it is not a error. The fix is that we only set AVERROR_EXTERNAL with status value non zero. When sample_buffer is NULL and status value is zero, we simply return with no other operation. This crash often occurs on iOS 11 for example encoding 720p@25fps. Signed-off-by: sharpbai --- libavcodec/videotoolboxenc.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index d76bb7f646..8afdd125d2 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -569,12 +569,16 @@ static void vtenc_output_callback( return; } - if (status || !sample_buffer) { + if (status) { av_log(avctx, AV_LOG_ERROR, "Error encoding frame: %d\n", (int)status); set_async_error(vtctx, AVERROR_EXTERNAL); return; } + if (!sample_buffer) { + return; + } + if (!avctx->extradata && (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER)) { int set_status = set_extradata(avctx, sample_buffer); if (set_status) {