From patchwork Tue Nov 14 15:24:13 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philip Langdale X-Patchwork-Id: 6045 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.94 with SMTP id m30csp3453820jah; Tue, 14 Nov 2017 07:24:58 -0800 (PST) X-Google-Smtp-Source: AGs4zMZ5U74Y8aoX8MOnb4GkgBp015w/4Ba+I+CIo4dreTQmuCNIPRldCDIxuTUZebG9k6PoUqxc X-Received: by 10.28.15.5 with SMTP id 5mr10107099wmp.43.1510673098034; Tue, 14 Nov 2017 07:24:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510673097; cv=none; d=google.com; s=arc-20160816; b=nSrr3OBgVjIJOzR7T3hVWPJV4zJDVHkGWTPS1Ne+J4uhGZ3vouijSdCoLaLOtrWjQy 7NEsGe4wcbJMrKr+wzaDopEfOGQpTyBd1WAxZREUWq1SjblpfP5+8zMn+25kmRe9bmb9 S0FJQdSGVpV2jTCkix3UggxyCLcZOqZUjPE+iiCMY0n80EpRHOHQnrZNCvKm4ej98Qvj FnvIGSKDygk/po+dX2ocv3AUyERKch7iZI8Cgb4EoP3Qm+s5tFHVMV9PvuPPwoFhS6lb GGprdtt4s4Frgc5wSSZBU8vaTQCJqAYbVx98nU0OIfDF6gTwgJ/mXP8e0hDC2EWElpEp 3kDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=5w0u5orr5PIrxmI/M2KW0j4T0DvF8oKLREPpiA/x4KQ=; b=o84MJqgzZD4xvU9KR8QnJYAGPfQ0TWQ4wWYQ7yapzKe5o1fENAPOiX+/sEO0aWRd8R ki4ZZfDphf9GCX1HjvRcp3lLkS11ay88b6ny8ag72Td3HqpNlGMmMxnWa6KhL1ByThfD skulYR3oKuJG3mYzn0nwLxh5If1nwAFZyWHQC5YbnrXpqF2m6ZhcTWy1A+gqPZkF1BPS CBJUC9XT3itSRNl0qiAv9v85QMJb9ozFWyJGvew80yYunO/Z8weJsycKC5wzn9E+oVWh UtleXTBy8H6vxXfw7DcwVVQYwGvlK6XgUoMLxczYvRgVSsARSOyiae0IaSTbYFyEgydK OebQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=lz73q++v; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 188si7674523wmx.151.2017.11.14.07.24.57; Tue, 14 Nov 2017 07:24:57 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=lz73q++v; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 86FB4689F8B; Tue, 14 Nov 2017 17:24:34 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io0-f225.google.com (mail-io0-f225.google.com [209.85.223.225]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B1EBD68045F for ; Tue, 14 Nov 2017 17:24:26 +0200 (EET) Received: by mail-io0-f225.google.com with SMTP id g73so9167450ioj.8 for ; Tue, 14 Nov 2017 07:24:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=r4cZk8A4R2Fvl+RYqjc5DXRC7WrMD6C51THZV169TGw=; b=P+3XkX5sqzIV6IELddEr3yq34hVpM8yFMFlxUGPRUVLOw42TsCVnrJUwc4P70n19aI N/FUkUJQaP8b7ZBDf/OuvFOYMRNDxb78Dt2TrupTWdghknaOzhj1QmVLturRylHuwjlf BQ7ip7YFzBLIaVcWx1oGMHh8d11myTv8Zt/gsBhx2dtTBTDX4/RShx+nTQb9ARncXElH TflRDKUeog8sqYOIlp1+s6HiXE7zEdmBWtXcPtXK97M+y11t6xOz713S+sG2rYaThGpY VQkOGHG2zf4FvQSOotJAZA1NSwCDJFjAH1P5owRVfWctr3d4AyADxCDTJCUeS32zywrG z9dg== X-Gm-Message-State: AJaThX7vFjnvv8wpbyw/P2BUMm10uxElJnk6gNHUPKiToMo8tdn6SIhm 9q4qU+Bd1xGFKZfHhd31CFDPokhdO4i9XbrsM9B3Ck7h7y+CRg== X-Received: by 10.107.201.78 with SMTP id z75mr6402455iof.123.1510673080350; Tue, 14 Nov 2017 07:24:40 -0800 (PST) Received: from mail.overt.org (155.208.178.107.bc.googleusercontent.com. [107.178.208.155]) by smtp-relay.gmail.com with ESMTPS id m14sm3794351iti.1.2017.11.14.07.24.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 14 Nov 2017 07:24:40 -0800 (PST) X-Relaying-Domain: gapps.overt.org Received: from authenticated-user (mail.overt.org [107.178.208.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.overt.org (Postfix) with ESMTPSA id D379B627C8; Tue, 14 Nov 2017 15:24:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=overt.org; s=mail; t=1510673080; bh=rmP/PlIYpgBTs1NmqY4jnvZNUIqGYmmGJkUcFPmoMQU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lz73q++vjkmqMP231gQUvYGCX4CoXH6kqrpjrSPoHm+JtmKHHa6kiaZGi/qjZBzE5 yg3L7HsYhsSWCHKaVTK/WsizNqf22DJp7f4RZcQ8Bh7w7TE5DsmEBdKUiiSKZ8LObN Kl5i+0c771Kr3oLOKoD2ocz4z0UIUIVlQ7brAexRG1gDht4vYHo74QLJgq2PkMXyGL MUHFPfg8IyeohTG44Ixen5r0/k5n9We9Yf4F1ygG1RrBrWvaRukl63kirePTVjyecG b6c5p7IdLF/qwvE7oSE+Gafk2LDHLfjMdy4oAD48FNWQ1yTv9E2DQmlip0XQbZTJb/ 27cVZZvL0RS7Q== From: Philip Langdale To: ffmpeg-devel@ffmpeg.org Date: Tue, 14 Nov 2017 07:24:13 -0800 Message-Id: <20171114152414.18478-2-philipl@overt.org> In-Reply-To: <20171114152414.18478-1-philipl@overt.org> References: <20171114152414.18478-1-philipl@overt.org> Subject: [FFmpeg-devel] [PATCH 1/2] avcodec: Don't assume separate u and v planes in ff_alloc_picture X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Philip Langdale MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" I'm confused at how none of the previous hwaccels hit this, but alloc_frame_buffer in ff_alloc_picture asserts that the linesize of planes 1 and 2 are the same. If the pixfmt has a single uv plane, like NV12, this won't be true. So, let's only do this check if there are more than 2 planes. I didn't bother investigating why, but vdpau avoids this by having all linesizes set to zero. Cute. Signed-off-by: Philip Langdale --- libavcodec/mpegpicture.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/libavcodec/mpegpicture.c b/libavcodec/mpegpicture.c index 9811a778b7..2b72346fb2 100644 --- a/libavcodec/mpegpicture.c +++ b/libavcodec/mpegpicture.c @@ -22,6 +22,7 @@ #include "libavutil/avassert.h" #include "libavutil/common.h" +#include "libavutil/pixdesc.h" #include "avcodec.h" #include "motion_est.h" @@ -155,7 +156,8 @@ static int alloc_frame_buffer(AVCodecContext *avctx, Picture *pic, return -1; } - if (pic->f->linesize[1] != pic->f->linesize[2]) { + if (av_pix_fmt_count_planes(pic->f->format) > 2 && + pic->f->linesize[1] != pic->f->linesize[2]) { av_log(avctx, AV_LOG_ERROR, "get_buffer() failed (uv stride mismatch)\n"); ff_mpeg_unref_picture(avctx, pic);