From patchwork Mon Jul 2 23:08:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 9578 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:104:0:0:0:0:0 with SMTP id c4-v6csp485865jad; Mon, 2 Jul 2018 16:10:00 -0700 (PDT) X-Google-Smtp-Source: AAOMgpfm5+S9an3YXzKPXymX0hMcFtBxgV3UXPcggl9b5Y2igTt+TeYh4pJg+KtJ11jlQsQRZCdG X-Received: by 2002:a1c:6c03:: with SMTP id h3-v6mr8530510wmc.38.1530573000623; Mon, 02 Jul 2018 16:10:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1530573000; cv=none; d=google.com; s=arc-20160816; b=w7/lojTNKgHhMVW6heMlbjxnOX1C/LEnJRHHegEkjr2bwlFa5CpHcpGzyjC9apuoU0 fv+QekUlVn7KZpLQWSi2Ed8S/zLKjHh2T/BD9Bew63zRTS60cImB84wB7Ssf5pYvI190 8SUPPCpxEl6tBUVASXndH/Da7qmWVfb9u46YGQU+OLiZ7meHJD3++Rfn5pjrAm8dAEDB d1qD47gFVAuFsQXWaU1rtFEAwWCEzIFC+U2EmvJpUirpyVHnne1t3Q03B1BmV4bmTVjl vO7iFAnPf+v7y1JSv+vnpBghR5DQwN49oeGAikA87eQihYmSgigZFslSI/JGXm4VnBj8 BirA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to:arc-authentication-results; bh=B68xuYPCyQ54Aeie1oDa+zbofkBL6ezCVOTk1rrVm68=; b=V+Wf9tW6+9caRjBkAzZGKxyg30c1RTASsE/nRRZ1QaDfMfMgLzTCvnaNSOPDbd7x00 yANSuFYmeyRbjFOiVWu830tls5rzImTJO+JLlUeUc4rFxtdbgXyg1zNXGn3kznZFIPKz +L1S3r7H0LQbmbyMUu/q2eIl5SxexclEqX63Qh07NhhCBOR5iuAfIZ0cmt9KybwvJj/1 yuooiAvqptmQ1W/RMP6vD1WRmRd/mwoqndr8FMauYwkmJTCwTSinMCNKJf3vcvq4XIP2 uF+P9iewuRaWLHz8lU3JUQ9qNIT799ilfVt8j2pIUyupyOGKrBUc3D0NVxAdNNCXM7Df 8gwg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f12-v6si480042wrm.65.2018.07.02.16.09.58; Mon, 02 Jul 2018 16:10:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E8F2C689D30; Tue, 3 Jul 2018 02:09:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vie01a-dmta-pe06-3.mx.upcmail.net (vie01a-dmta-pe06-3.mx.upcmail.net [84.116.36.16]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E862A6804C1 for ; Tue, 3 Jul 2018 02:09:46 +0300 (EEST) Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe06.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1fa7x7-0002Uy-UZ for ffmpeg-devel@ffmpeg.org; Tue, 03 Jul 2018 01:09:49 +0200 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id 5z9n1y00p0S5wYM01z9oTP; Tue, 03 Jul 2018 01:09:48 +0200 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 3 Jul 2018 01:08:58 +0200 Message-Id: <20180702230858.17781-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.18.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/mpeg4videodec: Remove use of FF_PROFILE_MPEG4_SIMPLE_STUDIO as indicator of studio profile X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The profile field is changed by code inside and outside the decoder, its not a reliable indicator of the internal codec state. Maintaining it consistency with studio_profile is messy. Its easier to just avoid it and use only studio_profile Fixes: assertion failure Fixes: ffmpeg_crash_9.avi Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart Signed-off-by: Michael Niedermayer --- libavcodec/error_resilience.c | 3 +-- libavcodec/h263dec.c | 6 ++++-- libavcodec/mpeg4videodec.c | 1 - 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/libavcodec/error_resilience.c b/libavcodec/error_resilience.c index 339042e206..1abae53f41 100644 --- a/libavcodec/error_resilience.c +++ b/libavcodec/error_resilience.c @@ -814,8 +814,7 @@ static int er_supported(ERContext *s) { if(s->avctx->hwaccel && s->avctx->hwaccel->decode_slice || !s->cur_pic.f || - s->cur_pic.field_picture || - s->avctx->profile == FF_PROFILE_MPEG4_SIMPLE_STUDIO + s->cur_pic.field_picture ) return 0; return 1; diff --git a/libavcodec/h263dec.c b/libavcodec/h263dec.c index f8a38083f2..5967fca285 100644 --- a/libavcodec/h263dec.c +++ b/libavcodec/h263dec.c @@ -47,9 +47,10 @@ static enum AVPixelFormat h263_get_format(AVCodecContext *avctx) { + MpegEncContext *s = avctx->priv_data; /* MPEG-4 Studio Profile only, not supported by hardware */ if (avctx->bits_per_raw_sample > 8) { - av_assert1(avctx->profile == FF_PROFILE_MPEG4_SIMPLE_STUDIO); + av_assert1(s->studio_profile); return avctx->pix_fmt; } @@ -670,7 +671,8 @@ retry: av_assert1(s->bitstream_buffer_size == 0); frame_end: - ff_er_frame_end(&s->er); + if (!s->studio_profile) + ff_er_frame_end(&s->er); if (avctx->hwaccel) { ret = avctx->hwaccel->end_frame(avctx); diff --git a/libavcodec/mpeg4videodec.c b/libavcodec/mpeg4videodec.c index 54a8496244..2df525e03a 100644 --- a/libavcodec/mpeg4videodec.c +++ b/libavcodec/mpeg4videodec.c @@ -3244,7 +3244,6 @@ end: s->avctx->has_b_frames = !s->low_delay; if (s->studio_profile) { - av_assert0(s->avctx->profile == FF_PROFILE_MPEG4_SIMPLE_STUDIO); if (!s->avctx->bits_per_raw_sample) { av_log(s->avctx, AV_LOG_ERROR, "Missing VOL header\n"); return AVERROR_INVALIDDATA;