From patchwork Fri Aug 14 10:35:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linjie Fu X-Patchwork-Id: 21657 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 5A8B444A8BB for ; Fri, 14 Aug 2020 13:36:09 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3867768ABB4; Fri, 14 Aug 2020 13:36:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 97417687F4E for ; Fri, 14 Aug 2020 13:36:02 +0300 (EEST) Received: by mail-pj1-f48.google.com with SMTP id mw10so4195253pjb.2 for ; Fri, 14 Aug 2020 03:36:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=AtqX8s3+1KgW1G0lvRfOMcEJMhl4GK85aTmsIc+RxMg=; b=XYpX5TwWsW42cBh7e+XpYRdjNYRYS6HG5MiOg/gVD0q5x0nYH4tCuYjKs2LRxvO2Eo FjC7UIbxpe2ROHiO1izj2xHAPXaHEBBg15ORZSWlUi5y5tmlF2R/mHtIQ6hBBIpUUpkR TgYE8VVtj77o5khQPqWYmmh5s9HkmSB3lZ3U9jxw7K3gsfBm8kYaxMfB7Uz7+EDRDhW5 kMTEO7r0SXQtcccpcYn1aG18wmL+4tw+jKGGgum5Sf7BshSLN5BwNgEVd5ekyXP9QbIA bnnkCPezQkI1LEHdNKSRFhPlvR5nQbpewFOzt6Yv0h4SeGHGuA5kQcSzbuw+5fmWWRSZ ITYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=AtqX8s3+1KgW1G0lvRfOMcEJMhl4GK85aTmsIc+RxMg=; b=iK5K4PztEEerT/9kQ1e4QAKk8NYwOjUpPiuCfB4xh2GfzhI97p8fkUfve/dtm3Zhp6 XbljHLTDSCFfWVBIgyK5L0UsN1HGaOp63NS+jirsMNrHLvtILIewEexWOavUazDcUK/9 Q6eVxPzKdDFKxDts+mE+oGKSUeCF979RzB4jUf+GrkzaW7JRlClpbn40akf56AlV83tJ r3t43wB1DrZQAA3W8RpxPDnacxsZ2bOPV5J6FkVWtIPUPYGUkVxGHSzBU9Oe0atoy9vS EV1BEaarCjmaIc8N4lB83ARqnC0zgJLdr32PQcea+qxO764SF9OZfNe+kNQN3EX2AKsQ wjGQ== X-Gm-Message-State: AOAM531HQQmljllUttQRzTSb3QRwKu18s5UXqGzqzEHbnQHd8Lixs8BQ sR9LTzOFszWeJrEAckXIVhQwGMyh5UWfAw== X-Google-Smtp-Source: ABdhPJxffYEa8qnBnqYeTra7vGi+kbN0TCqz0uLkTXDhevA78nxB3BEfhpk5yNpxQCKkpaXhBGuB/Q== X-Received: by 2002:a17:90a:6e09:: with SMTP id b9mr1747167pjk.136.1597401360410; Fri, 14 Aug 2020 03:36:00 -0700 (PDT) Received: from localhost.localdomain ([47.89.83.88]) by smtp.gmail.com with ESMTPSA id 131sm8414032pgh.67.2020.08.14.03.35.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Aug 2020 03:35:59 -0700 (PDT) From: Linjie Fu To: ffmpeg-devel@ffmpeg.org Date: Fri, 14 Aug 2020 18:35:43 +0800 Message-Id: <20200814103543.23695-1-linjie.justin.fu@gmail.com> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH, RFC] lavc/hevcdec: add invalid for skip_frame to skip invalid nalus before IRAP X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Linjie Fu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Add "-skip_frame invalid" option to allow user to request decoder to skip invalid nalus before an IRAP. This would benefit decoding pipeline of bitstreams who didn't start from an IRAP frame. NULL pointer pointing to missing reference may lead to unexpected hang issues[1] in sub-level like hardware decoding. Fixing the hang in driver is the first correct thing. Besides, adding a check in nalu parsing and skip frames until we got the first IRAP would be another good choice to be more robust for error tolerance. Also this needs worker thread to update the skip_frames field in time. [1] https://github.com/intel/media-driver/issues/992 Signed-off-by: Linjie Fu --- Request for comments: The purpose is to allow decoder to skip frames until an IRAP has arrived, however not sure whether we already had this in ffmpeg, hence submit a patch and request for comments. Skip logic identical for AVDISCARD_NONKEY in decode_nal_unit(). libavcodec/avcodec.h | 1 + libavcodec/hevcdec.c | 5 ++++- libavcodec/options_table.h | 1 + libavcodec/pthread_frame.c | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index c91b2fd169..376d7f4d6d 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -233,6 +233,7 @@ enum AVDiscard{ AVDISCARD_BIDIR = 16, ///< discard all bidirectional frames AVDISCARD_NONINTRA= 24, ///< discard all non intra frames AVDISCARD_NONKEY = 32, ///< discard all frames except keyframes + AVDISCARD_INVALID = 33, ///< discard invalid packets like NALs before IRAP AVDISCARD_ALL = 48, ///< discard all }; diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index b77df8d89f..78658fd537 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -539,8 +539,11 @@ static int hls_slice_header(HEVCContext *s) ff_hevc_clear_refs(s); } sh->no_output_of_prior_pics_flag = 0; - if (IS_IRAP(s)) + if (IS_IRAP(s)) { sh->no_output_of_prior_pics_flag = get_bits1(gb); + if (s->avctx->skip_frame == AVDISCARD_INVALID) + s->avctx->skip_frame = AVDISCARD_DEFAULT; + } sh->pps_id = get_ue_golomb_long(gb); if (sh->pps_id >= HEVC_MAX_PPS_COUNT || !s->ps.pps_list[sh->pps_id]) { diff --git a/libavcodec/options_table.h b/libavcodec/options_table.h index 1d0db1b5a4..d52bce5908 100644 --- a/libavcodec/options_table.h +++ b/libavcodec/options_table.h @@ -310,6 +310,7 @@ static const AVOption avcodec_options[] = { {"bidir" , "discard all bidirectional frames", 0, AV_OPT_TYPE_CONST, {.i64 = AVDISCARD_BIDIR }, INT_MIN, INT_MAX, V|D, "avdiscard"}, {"nokey" , "discard all frames except keyframes", 0, AV_OPT_TYPE_CONST, {.i64 = AVDISCARD_NONKEY }, INT_MIN, INT_MAX, V|D, "avdiscard"}, {"nointra" , "discard all frames except I frames", 0, AV_OPT_TYPE_CONST, {.i64 = AVDISCARD_NONINTRA}, INT_MIN, INT_MAX, V|D, "avdiscard"}, +{"invalid" , "discard NALUs before IRAP", 0, AV_OPT_TYPE_CONST, {.i64 = AVDISCARD_INVALID }, INT_MIN, INT_MAX, V|D, "avdiscard"}, {"all" , "discard all frames", 0, AV_OPT_TYPE_CONST, {.i64 = AVDISCARD_ALL }, INT_MIN, INT_MAX, V|D, "avdiscard"}, {"bidir_refine", "refine the two motion vectors used in bidirectional macroblocks", OFFSET(bidir_refine), AV_OPT_TYPE_INT, {.i64 = 1 }, 0, 4, V|E}, #if FF_API_PRIVATE_OPT diff --git a/libavcodec/pthread_frame.c b/libavcodec/pthread_frame.c index 3255aa9337..302e6149ac 100644 --- a/libavcodec/pthread_frame.c +++ b/libavcodec/pthread_frame.c @@ -259,6 +259,7 @@ static int update_context_from_thread(AVCodecContext *dst, AVCodecContext *src, dst->has_b_frames = src->has_b_frames; dst->idct_algo = src->idct_algo; + dst->skip_frame = src->skip_frame; dst->bits_per_coded_sample = src->bits_per_coded_sample; dst->sample_aspect_ratio = src->sample_aspect_ratio;