From patchwork Sun Jul 9 13:39:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 42558 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:6da1:b0:131:a7d0:bc6d with SMTP id gl33csp120593pzb; Sun, 9 Jul 2023 06:39:41 -0700 (PDT) X-Google-Smtp-Source: APBJJlEh0K6FF7MdzWH7vxKic0oq0znzGLDJMhbapH7X1pXlhTVccRywTP3xIb6SgFwAEcOREuNo X-Received: by 2002:aa7:c486:0:b0:51d:88b2:872e with SMTP id m6-20020aa7c486000000b0051d88b2872emr6814265edq.42.1688909981276; Sun, 09 Jul 2023 06:39:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688909981; cv=none; d=google.com; s=arc-20160816; b=SBB47GmeTP4H7B+XdlIm4Xkmwnl7J7sz9Zkh/4tIk/Z0Ny9l+tS/iwMWZVjKKaiACl glwwACKqbLq1/iVrEEfYFhV2TVVverlW7uGPTM7gFfMGIKDlrfyiHXdd1K4hqd0Bw6nY Ex6YbNDL7wIe4skrNFv6pZ/AnGhJrYNpcHL9BfOyvnZjmlpcaKg66rxwNAXrQ7Z6ZLa/ h+/7hOAsEyP7SnIMAJDC87ZwfyxJpt4joo+OlcXBqOv8APTUfqe8SNsDORQEaroOm0H8 GXUA+q/WVTaRfP53Ogm/mBEQt/MrtIn/rNzPTBe9M9uMewYyHo0R13+dhhkotTflDMMt xTyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=zBqHfIdSubFKLwQb7hDdzbvmxS5eFkABJypYIFbQKIE=; fh=J3zlMo7rVW2t2IQYogliBcMNuBh6YQg7NRChcrschf4=; b=BzO07Yb0xqPCi0wdbQ08iFAbXDJCT9xx32crD+GAgq5zYXwNX84NDguv/cIvjGIcID 6O+Qv3OBRzcH6EREXszNIYWD3AmO13o01GA1GnWjnxnz7P680UxKyK1XI0P6X5ua6Un5 5jctcZp3abDSdo/hCcAg05NQvSS9U0YIV/VYYqSLZWO00XBU+NC0T0k9GBEi8nLebWnK jMWwXbpzTSM6CHkP+TCXIJLQUudVdJRYL4M6pL8GtkRBMOK0ahfz4oBZDcID5zJ0kE5F HLLpFHF9w079T1APaNRs1Z9xY7BKbjPIJYfJISbfguLHHAUqQGO1iK5f85uyLxuiOVuL 2SeQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b="Q3x/TDdv"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dy7-20020a05640231e700b0051be9317ccasi6045532edb.508.2023.07.09.06.39.40; Sun, 09 Jul 2023 06:39:41 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b="Q3x/TDdv"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3D82D68C654; Sun, 9 Jul 2023 16:39:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6EC1C68C38A for ; Sun, 9 Jul 2023 16:39:30 +0300 (EEST) Received: by mail-ot1-f51.google.com with SMTP id 46e09a7af769-6b5d1498670so3056776a34.1 for ; Sun, 09 Jul 2023 06:39:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1688909968; x=1691501968; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=+GmwnwG1LW5bnyICgZI8YeWIllHovSP84Oo5d2cMq10=; b=Q3x/TDdvzNwM9u4+1a8quhxQW6y0krEyPYbLebXhbVtqLVf0SorvwbifasWdNDIbbQ B0BzsA3Qwb1XAGzZi8vqI9HTmLdX5/hi5+o6wpHPGF13oVpC0G45nFl2H9Umd4CUaJkq gC4YbOq3S8sNSdIqx5EIr7u1C6CLVTJcH48CAEYRr8S5MvIrMaOIpngu77nGaGrH5eSt mSl00AA5UonVKl4zQdn/NozedvqRlMivNl+a7LDrvGSUVRmmLDeTOvpQ2Pix4fq4PmGC MUOxT+DK+BX6pjEmaL14XkQfoGdFVIi+wfTlABu5W9v68YJaa+ebHgp1a6srOZOHywS7 f/ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688909968; x=1691501968; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+GmwnwG1LW5bnyICgZI8YeWIllHovSP84Oo5d2cMq10=; b=gj8rtBTE5Sm6gI0/5CI/1TRZHpT7zXSmSR9yK6fW6Q5UkmZ3vFLN5fAzMQL4YSsME6 HXizZYLv+Xfvz/PVdlAUr8DYX1Seo+bu9Wb5cD4H6vuRJhUY9PQL/KGkoFV0AlNz9fEB 4v/vBkJtMefMC8t3Et33l92XyVugZnDXSeXgHLy0DbOItgdW0q1eD6I0o8kFhRQ+BuVr CLSB9kAszQZE/AGlUOUwHL4QfJY+vICKvvjcw2Jylg3Gh74bcqu6wU24YBHOrSNOXnbx xeGkIXDBK8bmwjTHc0GYUluvoShAHg03skDZ5DSRm7ssnh+eLi/D6RF6DN66YWwToUVR dgHw== X-Gm-Message-State: ABy/qLZqngAevHnYhFoFetdIuDsfBZel+0TWfdfOwAIn6IGsIRCz9Cg8 syUBCpDNeC5ElCY3vMSt0KtfSzVYeAE= X-Received: by 2002:a05:6830:4b1:b0:6b9:4216:c209 with SMTP id l17-20020a05683004b100b006b94216c209mr1665937otd.12.1688909968233; Sun, 09 Jul 2023 06:39:28 -0700 (PDT) Received: from localhost.localdomain (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id r18-20020a9d7512000000b006b756242c98sm3404413otk.19.2023.07.09.06.39.27 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 09 Jul 2023 06:39:27 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sun, 9 Jul 2023 10:39:15 -0300 Message-ID: <20230709133917.5403-1-jamrial@gmail.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/3] avcodec/decode: move processing discard samples to its own function X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 6gluXYZ80JLY As an intended side effect, avci->skip_samples is no longer being set when no frame is output. Signed-off-by: James Almer --- libavcodec/decode.c | 269 +++++++++++++++++++++++--------------------- 1 file changed, 138 insertions(+), 131 deletions(-) diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 269633ce10..a39af2d014 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -289,6 +289,120 @@ static int64_t guess_correct_pts(AVCodecContext *ctx, return pts; } +static int discard_samples(AVCodecContext *avctx, AVFrame *frame, int64_t *discarded_samples) +{ + AVCodecInternal *avci = avctx->internal; + int ret = 0; + uint8_t *side; + size_t side_size; + uint32_t discard_padding = 0; + uint8_t skip_reason = 0; + uint8_t discard_reason = 0; + + if (frame->format == AV_SAMPLE_FMT_NONE) + frame->format = avctx->sample_fmt; + if (!frame->ch_layout.nb_channels) { + int ret2 = av_channel_layout_copy(&frame->ch_layout, &avctx->ch_layout); + if (ret2 < 0) { + ret = ret2; + } + } +#if FF_API_OLD_CHANNEL_LAYOUT +FF_DISABLE_DEPRECATION_WARNINGS + if (!frame->channel_layout) + frame->channel_layout = avctx->ch_layout.order == AV_CHANNEL_ORDER_NATIVE ? + avctx->ch_layout.u.mask : 0; + if (!frame->channels) + frame->channels = avctx->ch_layout.nb_channels; +FF_ENABLE_DEPRECATION_WARNINGS +#endif + if (!frame->sample_rate) + frame->sample_rate = avctx->sample_rate; + + side = av_packet_get_side_data(avci->last_pkt_props, AV_PKT_DATA_SKIP_SAMPLES, &side_size); + if (side && side_size >= 10) { + avci->skip_samples = AV_RL32(side); + avci->skip_samples = FFMAX(0, avci->skip_samples); + discard_padding = AV_RL32(side + 4); + av_log(avctx, AV_LOG_DEBUG, "skip %d / discard %d samples due to side data\n", + avci->skip_samples, (int)discard_padding); + skip_reason = AV_RL8(side + 8); + discard_reason = AV_RL8(side + 9); + } + + if ((frame->flags & AV_FRAME_FLAG_DISCARD) && + !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + avci->skip_samples = FFMAX(0, avci->skip_samples - frame->nb_samples); + *discarded_samples += frame->nb_samples; + return AVERROR(EAGAIN); + } + + if (avci->skip_samples > 0 && + !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + if (frame->nb_samples <= avci->skip_samples){ + *discarded_samples += frame->nb_samples; + avci->skip_samples -= frame->nb_samples; + av_log(avctx, AV_LOG_DEBUG, "skip whole frame, skip left: %d\n", + avci->skip_samples); + return AVERROR(EAGAIN); + } else { + av_samples_copy(frame->extended_data, frame->extended_data, 0, avci->skip_samples, + frame->nb_samples - avci->skip_samples, avctx->ch_layout.nb_channels, frame->format); + if (avctx->pkt_timebase.num && avctx->sample_rate) { + int64_t diff_ts = av_rescale_q(avci->skip_samples, + (AVRational){1, avctx->sample_rate}, + avctx->pkt_timebase); + if (frame->pts != AV_NOPTS_VALUE) + frame->pts += diff_ts; + if (frame->pkt_dts != AV_NOPTS_VALUE) + frame->pkt_dts += diff_ts; + if (frame->duration >= diff_ts) + frame->duration -= diff_ts; + } else + av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for skipped samples.\n"); + + av_log(avctx, AV_LOG_DEBUG, "skip %d/%d samples\n", + avci->skip_samples, frame->nb_samples); + *discarded_samples += avci->skip_samples; + frame->nb_samples -= avci->skip_samples; + avci->skip_samples = 0; + } + } + + if (discard_padding > 0 && discard_padding <= frame->nb_samples && + !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + if (discard_padding == frame->nb_samples) { + *discarded_samples += frame->nb_samples; + return AVERROR(EAGAIN); + } else { + if (avctx->pkt_timebase.num && avctx->sample_rate) { + int64_t diff_ts = av_rescale_q(frame->nb_samples - discard_padding, + (AVRational){1, avctx->sample_rate}, + avctx->pkt_timebase); + frame->duration = diff_ts; + } else + av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for discarded samples.\n"); + + av_log(avctx, AV_LOG_DEBUG, "discard %d/%d samples\n", + (int)discard_padding, frame->nb_samples); + frame->nb_samples -= discard_padding; + } + } + + if ((avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + AVFrameSideData *fside = av_frame_new_side_data(frame, AV_FRAME_DATA_SKIP_SAMPLES, 10); + if (fside) { + AV_WL32(fside->data, avci->skip_samples); + AV_WL32(fside->data + 4, discard_padding); + AV_WL8(fside->data + 8, skip_reason); + AV_WL8(fside->data + 9, discard_reason); + avci->skip_samples = 0; + } + } + + return ret; +} + /* * The core of the receive_frame_wrapper for the decoders implementing * the simple API. Certain decoders might consume partial packets without @@ -300,7 +414,7 @@ static inline int decode_simple_internal(AVCodecContext *avctx, AVFrame *frame, AVCodecInternal *avci = avctx->internal; AVPacket *const pkt = avci->in_pkt; const FFCodec *const codec = ffcodec(avctx->codec); - int got_frame, actual_got_frame; + int got_frame, consumed; int ret; if (!pkt->data && !avci->draining) { @@ -323,9 +437,9 @@ static inline int decode_simple_internal(AVCodecContext *avctx, AVFrame *frame, got_frame = 0; if (HAVE_THREADS && avctx->active_thread_type & FF_THREAD_FRAME) { - ret = ff_thread_decode_frame(avctx, frame, &got_frame, pkt); + consumed = ff_thread_decode_frame(avctx, frame, &got_frame, pkt); } else { - ret = codec->cb.decode(avctx, frame, &got_frame, pkt); + consumed = codec->cb.decode(avctx, frame, &got_frame, pkt); if (!(codec->caps_internal & FF_CODEC_CAP_SETS_PKT_DTS)) frame->pkt_dts = pkt->dts; @@ -347,132 +461,30 @@ FF_ENABLE_DEPRECATION_WARNINGS } } emms_c(); - actual_got_frame = got_frame; - - if (avctx->codec->type == AVMEDIA_TYPE_VIDEO) { - if (frame->flags & AV_FRAME_FLAG_DISCARD) - got_frame = 0; - } else if (avctx->codec->type == AVMEDIA_TYPE_AUDIO) { - uint8_t *side; - size_t side_size; - uint32_t discard_padding = 0; - uint8_t skip_reason = 0; - uint8_t discard_reason = 0; - - if (ret >= 0 && got_frame) { - if (frame->format == AV_SAMPLE_FMT_NONE) - frame->format = avctx->sample_fmt; - if (!frame->ch_layout.nb_channels) { - int ret2 = av_channel_layout_copy(&frame->ch_layout, &avctx->ch_layout); - if (ret2 < 0) { - ret = ret2; - got_frame = 0; - } - } -#if FF_API_OLD_CHANNEL_LAYOUT -FF_DISABLE_DEPRECATION_WARNINGS - if (!frame->channel_layout) - frame->channel_layout = avctx->ch_layout.order == AV_CHANNEL_ORDER_NATIVE ? - avctx->ch_layout.u.mask : 0; - if (!frame->channels) - frame->channels = avctx->ch_layout.nb_channels; -FF_ENABLE_DEPRECATION_WARNINGS -#endif - if (!frame->sample_rate) - frame->sample_rate = avctx->sample_rate; - } - - side= av_packet_get_side_data(avci->last_pkt_props, AV_PKT_DATA_SKIP_SAMPLES, &side_size); - if(side && side_size>=10) { - avci->skip_samples = AV_RL32(side); - avci->skip_samples = FFMAX(0, avci->skip_samples); - discard_padding = AV_RL32(side + 4); - av_log(avctx, AV_LOG_DEBUG, "skip %d / discard %d samples due to side data\n", - avci->skip_samples, (int)discard_padding); - skip_reason = AV_RL8(side + 8); - discard_reason = AV_RL8(side + 9); - } - - if ((frame->flags & AV_FRAME_FLAG_DISCARD) && got_frame && - !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { - avci->skip_samples = FFMAX(0, avci->skip_samples - frame->nb_samples); - got_frame = 0; - *discarded_samples += frame->nb_samples; - } - - if (avci->skip_samples > 0 && got_frame && - !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { - if(frame->nb_samples <= avci->skip_samples){ - got_frame = 0; - *discarded_samples += frame->nb_samples; - avci->skip_samples -= frame->nb_samples; - av_log(avctx, AV_LOG_DEBUG, "skip whole frame, skip left: %d\n", - avci->skip_samples); - } else { - av_samples_copy(frame->extended_data, frame->extended_data, 0, avci->skip_samples, - frame->nb_samples - avci->skip_samples, avctx->ch_layout.nb_channels, frame->format); - if(avctx->pkt_timebase.num && avctx->sample_rate) { - int64_t diff_ts = av_rescale_q(avci->skip_samples, - (AVRational){1, avctx->sample_rate}, - avctx->pkt_timebase); - if(frame->pts!=AV_NOPTS_VALUE) - frame->pts += diff_ts; - if(frame->pkt_dts!=AV_NOPTS_VALUE) - frame->pkt_dts += diff_ts; - if (frame->duration >= diff_ts) - frame->duration -= diff_ts; - } else { - av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for skipped samples.\n"); - } - av_log(avctx, AV_LOG_DEBUG, "skip %d/%d samples\n", - avci->skip_samples, frame->nb_samples); - *discarded_samples += avci->skip_samples; - frame->nb_samples -= avci->skip_samples; - avci->skip_samples = 0; - } - } - if (discard_padding > 0 && discard_padding <= frame->nb_samples && got_frame && - !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { - if (discard_padding == frame->nb_samples) { - *discarded_samples += frame->nb_samples; - got_frame = 0; - } else { - if(avctx->pkt_timebase.num && avctx->sample_rate) { - int64_t diff_ts = av_rescale_q(frame->nb_samples - discard_padding, - (AVRational){1, avctx->sample_rate}, - avctx->pkt_timebase); - frame->duration = diff_ts; - } else { - av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for discarded samples.\n"); - } - av_log(avctx, AV_LOG_DEBUG, "discard %d/%d samples\n", - (int)discard_padding, frame->nb_samples); - frame->nb_samples -= discard_padding; - } - } - - if ((avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL) && got_frame) { - AVFrameSideData *fside = av_frame_new_side_data(frame, AV_FRAME_DATA_SKIP_SAMPLES, 10); - if (fside) { - AV_WL32(fside->data, avci->skip_samples); - AV_WL32(fside->data + 4, discard_padding); - AV_WL8(fside->data + 8, skip_reason); - AV_WL8(fside->data + 9, discard_reason); - avci->skip_samples = 0; - } - } - } + if (got_frame) { + if (avctx->codec->type == AVMEDIA_TYPE_VIDEO) { + ret = (frame->flags & AV_FRAME_FLAG_DISCARD) ? AVERROR(EAGAIN) : 0; + } else if (avctx->codec->type == AVMEDIA_TYPE_AUDIO) + ret = discard_samples(avctx, frame, discarded_samples); + } else + ret = AVERROR(EAGAIN); - if (!got_frame) + if (!got_frame || ret == AVERROR(EAGAIN)) av_frame_unref(frame); - if (ret >= 0 && avctx->codec->type == AVMEDIA_TYPE_VIDEO) - ret = pkt->size; + if (consumed < 0) + ret = consumed; + if (consumed >= 0 && avctx->codec->type == AVMEDIA_TYPE_VIDEO) + consumed = pkt->size; + + if (!ret) + av_assert0(frame->buf[0]); + if (ret == AVERROR(EAGAIN)) + ret = 0; - /* do not stop draining when actual_got_frame != 0 or ret < 0 */ - /* got_frame == 0 but actual_got_frame != 0 when frame is discarded */ - if (avci->draining && !actual_got_frame) { + /* do not stop draining when got_frame != 0 or ret < 0 */ + if (avci->draining && !got_frame) { if (ret < 0) { /* prevent infinite loop if a decoder wrongly always return error on draining */ /* reasonable nb_errors_max = maximum b frames + thread count */ @@ -490,11 +502,9 @@ FF_ENABLE_DEPRECATION_WARNINGS } } - if (ret >= pkt->size || ret < 0) { + if (consumed >= pkt->size || ret < 0) { av_packet_unref(pkt); } else { - int consumed = ret; - pkt->data += consumed; pkt->size -= consumed; pkt->pts = AV_NOPTS_VALUE; @@ -509,10 +519,7 @@ FF_ENABLE_DEPRECATION_WARNINGS } } - if (got_frame) - av_assert0(frame->buf[0]); - - return ret < 0 ? ret : 0; + return ret; } #if CONFIG_LCMS2