From patchwork Mon Jul 10 21:54:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 42584 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:6da1:b0:131:a7d0:bc6d with SMTP id gl33csp1019995pzb; Mon, 10 Jul 2023 14:55:17 -0700 (PDT) X-Google-Smtp-Source: APBJJlF87O9Gnf/1vSgx+ev3vlKmzmaPRIj0XAgtBOZsYn6LSkzmmEWChZjtiK9v7U8ghELRWvA5 X-Received: by 2002:aa7:d1c3:0:b0:51e:1a5d:bc8c with SMTP id g3-20020aa7d1c3000000b0051e1a5dbc8cmr10013211edp.42.1689026116871; Mon, 10 Jul 2023 14:55:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689026116; cv=none; d=google.com; s=arc-20160816; b=CcKUdKV2wrQ+pOIO08v/lrlMXgjYAFHWAU5W21sB6PfEnBRUA1ifg07XhaBYFJcZMc t5APC3nPnzDt/HFXoag6Unt+hymy5dS74a5Ht+ozXunpRCH9RZ8vGC3C9kj1MLsoCmTe uoVHycStFAJUifJkazxfjIcrwzTAbcH8uOuGDEF5ECA7BIzqdF/ZdxnWZNUMKI5BPEvn DSNJUq/lONwDBYEmTRDlk3ngVKBtUt5LuZgSXrbXzpu4xgrDqRGPp8YCW3bP//XiMvL5 iCGvRTqh4nlTgG5uUXYS/sE28aGYBsZd9Fn/j8bK3MVja6OHwpuFAqQ+31dnQk6lt/L6 Du/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=vXfE6ILGJ8j7U7QLBN9OdPUs+KRxiPMa8Vw0NJFpjLk=; fh=J3zlMo7rVW2t2IQYogliBcMNuBh6YQg7NRChcrschf4=; b=bW8+7UvaA3I79KSCGFCahaMfG6Dkju0szul5rTQ5aFdZn74WTZnCAmkxqLSR8wBWcj a8gvhqtsKjiHQdVo+s+o5nQIG3t2t+blVV42ZuWPHWWWsLnQWDCSCkqbOPR2tvBRd93U Ryd2fny+p+mKDcZRVndj48XzVAq94SaFU0YwpSkwmXMZUlA2pQqAtrWDuGwm0AGEtRko XkJ17Qrx0u/2GkHsXWXaNxqHW4jU4DGwPe7sUeKgu6dNtrOuYL9ByBFmgH72pZ5LMs0d LcrbYDZ8lZysil7+arMcx9w6U57wiWd9uoUJdgY6Lg3idQkTf8y9yYqpeokC4GcSJp2/ RF9Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b=JFQBYN+t; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e20-20020a50fb94000000b0051c81230138si473493edq.220.2023.07.10.14.55.16; Mon, 10 Jul 2023 14:55:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b=JFQBYN+t; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CCA0968C5A5; Tue, 11 Jul 2023 00:55:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oa1-f53.google.com (mail-oa1-f53.google.com [209.85.160.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EFD3568C364 for ; Tue, 11 Jul 2023 00:55:05 +0300 (EEST) Received: by mail-oa1-f53.google.com with SMTP id 586e51a60fabf-1b060bce5b0so4323909fac.3 for ; Mon, 10 Jul 2023 14:55:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689026104; x=1691618104; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=flmRdsBumCZOcQRbwsyNuW8KT31QSZzZoNs8979ZN1c=; b=JFQBYN+t32WRoKbv3LIZ4kJW0Tq6fpE8tTxbghBYcwQmF5RRbSBWYJOtYGdIHdsFQI U+qtqYHiVOHqe+6Pb5wwNITcqXS0rXEhXLnINOS2EOiEuvKkqImZMj8sPJ4SQvVZVrZm fGGFlNHWOIg4POvbMRlzQ9ZZzJZxHLNfgswbz4z7xhT0MVkLF6h466Zyeiw72R7UiIAk FE6kNtAuY5sqdzB+EaPRF2RkQvKtw6vVQVOn9zRoVb/raubNor7ghDgMXD9fqJYe+UV0 256GkZhfrzBji4GXONKzCYwkx/hBkfg0dNACKh6wgI2Shj3wx5dMcMactV2RIK7kbI5I Ocgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689026104; x=1691618104; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=flmRdsBumCZOcQRbwsyNuW8KT31QSZzZoNs8979ZN1c=; b=WoQGc92YW7DAdMo89KPAX35CPnNUDoJrN+sQhBgUMoAefMbLjG+ywd3pJVcOyDbjl5 ZK8xX9xhKuJeI9G6Q17zY6wqrGdye5Ec/2PfzK+HyfiSkwXtx36IZaoB+OxICAhPMiSG CrNz/59N5LlfenLt5yvFcBJeNP/hVekmudGv4wZ4PZPM+n9HFfejhYRUi+BYp6siV/mY Y6NPM6zLoK2wCMhHTiWAK6pwujpSCI5DHTVPcojsSLEBQJ5l9HePmybuGRRqXKwyscF0 VoPa/bw/mGWbRCmzXi6ETHs9EIx8aQ2zLR1yhbEHXZqweVmzL/laKgfllAnAJn0C2/Hv TbwQ== X-Gm-Message-State: ABy/qLaLRDyiaOxl6irfvjOPBuVDnUBO7i+7Wh5gmMqoe1j+o+ogqOpZ 1XTyovr+jOBx218dE+M0D8097o9G24s= X-Received: by 2002:a05:6870:5250:b0:1aa:1c3f:808 with SMTP id o16-20020a056870525000b001aa1c3f0808mr14399922oai.55.1689026103569; Mon, 10 Jul 2023 14:55:03 -0700 (PDT) Received: from localhost.localdomain (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id dy45-20020a056870c7ad00b0019ed19a8659sm405446oab.8.2023.07.10.14.55.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:55:03 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Mon, 10 Jul 2023 18:54:51 -0300 Message-ID: <20230710215453.34421-1-jamrial@gmail.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 1/3] avcodec/decode: move processing discard samples to its own function X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FyIoK1mBwA6d Signed-off-by: James Almer --- libavcodec/decode.c | 270 +++++++++++++++++++++++--------------------- 1 file changed, 140 insertions(+), 130 deletions(-) diff --git a/libavcodec/decode.c b/libavcodec/decode.c index a47abeca06..c9da8f685c 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -289,6 +289,123 @@ static int64_t guess_correct_pts(AVCodecContext *ctx, return pts; } +static int discard_samples(AVCodecContext *avctx, AVFrame *frame, int64_t *discarded_samples) +{ + AVCodecInternal *avci = avctx->internal; + int ret = 0; + uint8_t *side; + size_t side_size; + uint32_t discard_padding = 0; + uint8_t skip_reason = 0; + uint8_t discard_reason = 0; + + side = av_packet_get_side_data(avci->last_pkt_props, AV_PKT_DATA_SKIP_SAMPLES, &side_size); + if (side && side_size >= 10) { + avci->skip_samples = AV_RL32(side); + avci->skip_samples = FFMAX(0, avci->skip_samples); + discard_padding = AV_RL32(side + 4); + av_log(avctx, AV_LOG_DEBUG, "skip %d / discard %d samples due to side data\n", + avci->skip_samples, (int)discard_padding); + skip_reason = AV_RL8(side + 8); + discard_reason = AV_RL8(side + 9); + } + + if (!frame->buf[0]) + return AVERROR(EAGAIN); + + if (frame->format == AV_SAMPLE_FMT_NONE) + frame->format = avctx->sample_fmt; + if (!frame->ch_layout.nb_channels) { + int ret2 = av_channel_layout_copy(&frame->ch_layout, &avctx->ch_layout); + if (ret2 < 0) { + ret = ret2; + } + } +#if FF_API_OLD_CHANNEL_LAYOUT +FF_DISABLE_DEPRECATION_WARNINGS + if (!frame->channel_layout) + frame->channel_layout = avctx->ch_layout.order == AV_CHANNEL_ORDER_NATIVE ? + avctx->ch_layout.u.mask : 0; + if (!frame->channels) + frame->channels = avctx->ch_layout.nb_channels; +FF_ENABLE_DEPRECATION_WARNINGS +#endif + if (!frame->sample_rate) + frame->sample_rate = avctx->sample_rate; + + if ((frame->flags & AV_FRAME_FLAG_DISCARD) && + !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + avci->skip_samples = FFMAX(0, avci->skip_samples - frame->nb_samples); + *discarded_samples += frame->nb_samples; + return AVERROR(EAGAIN); + } + + if (avci->skip_samples > 0 && + !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + if (frame->nb_samples <= avci->skip_samples){ + *discarded_samples += frame->nb_samples; + avci->skip_samples -= frame->nb_samples; + av_log(avctx, AV_LOG_DEBUG, "skip whole frame, skip left: %d\n", + avci->skip_samples); + return AVERROR(EAGAIN); + } else { + av_samples_copy(frame->extended_data, frame->extended_data, 0, avci->skip_samples, + frame->nb_samples - avci->skip_samples, avctx->ch_layout.nb_channels, frame->format); + if (avctx->pkt_timebase.num && avctx->sample_rate) { + int64_t diff_ts = av_rescale_q(avci->skip_samples, + (AVRational){1, avctx->sample_rate}, + avctx->pkt_timebase); + if (frame->pts != AV_NOPTS_VALUE) + frame->pts += diff_ts; + if (frame->pkt_dts != AV_NOPTS_VALUE) + frame->pkt_dts += diff_ts; + if (frame->duration >= diff_ts) + frame->duration -= diff_ts; + } else + av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for skipped samples.\n"); + + av_log(avctx, AV_LOG_DEBUG, "skip %d/%d samples\n", + avci->skip_samples, frame->nb_samples); + *discarded_samples += avci->skip_samples; + frame->nb_samples -= avci->skip_samples; + avci->skip_samples = 0; + } + } + + if (discard_padding > 0 && discard_padding <= frame->nb_samples && + !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + if (discard_padding == frame->nb_samples) { + *discarded_samples += frame->nb_samples; + return AVERROR(EAGAIN); + } else { + if (avctx->pkt_timebase.num && avctx->sample_rate) { + int64_t diff_ts = av_rescale_q(frame->nb_samples - discard_padding, + (AVRational){1, avctx->sample_rate}, + avctx->pkt_timebase); + frame->duration = diff_ts; + } else + av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for discarded samples.\n"); + + av_log(avctx, AV_LOG_DEBUG, "discard %d/%d samples\n", + (int)discard_padding, frame->nb_samples); + frame->nb_samples -= discard_padding; + } + } + + if ((avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + AVFrameSideData *fside = av_frame_new_side_data(frame, AV_FRAME_DATA_SKIP_SAMPLES, 10); + if (fside) { + AV_WL32(fside->data, avci->skip_samples); + AV_WL32(fside->data + 4, discard_padding); + AV_WL8(fside->data + 8, skip_reason); + AV_WL8(fside->data + 9, discard_reason); + avci->skip_samples = 0; + } + } + + return ret; +} + /* * The core of the receive_frame_wrapper for the decoders implementing * the simple API. Certain decoders might consume partial packets without @@ -300,7 +417,7 @@ static inline int decode_simple_internal(AVCodecContext *avctx, AVFrame *frame, AVCodecInternal *avci = avctx->internal; AVPacket *const pkt = avci->in_pkt; const FFCodec *const codec = ffcodec(avctx->codec); - int got_frame, actual_got_frame; + int got_frame, consumed; int ret; if (!pkt->data && !avci->draining) { @@ -323,9 +440,9 @@ static inline int decode_simple_internal(AVCodecContext *avctx, AVFrame *frame, got_frame = 0; if (HAVE_THREADS && avctx->active_thread_type & FF_THREAD_FRAME) { - ret = ff_thread_decode_frame(avctx, frame, &got_frame, pkt); + consumed = ff_thread_decode_frame(avctx, frame, &got_frame, pkt); } else { - ret = codec->cb.decode(avctx, frame, &got_frame, pkt); + consumed = codec->cb.decode(avctx, frame, &got_frame, pkt); if (!(codec->caps_internal & FF_CODEC_CAP_SETS_PKT_DTS)) frame->pkt_dts = pkt->dts; @@ -347,132 +464,30 @@ FF_ENABLE_DEPRECATION_WARNINGS } } emms_c(); - actual_got_frame = got_frame; - if (avctx->codec->type == AVMEDIA_TYPE_VIDEO) { - if (frame->flags & AV_FRAME_FLAG_DISCARD) - got_frame = 0; - } else if (avctx->codec->type == AVMEDIA_TYPE_AUDIO) { - uint8_t *side; - size_t side_size; - uint32_t discard_padding = 0; - uint8_t skip_reason = 0; - uint8_t discard_reason = 0; - - if (ret >= 0 && got_frame) { - if (frame->format == AV_SAMPLE_FMT_NONE) - frame->format = avctx->sample_fmt; - if (!frame->ch_layout.nb_channels) { - int ret2 = av_channel_layout_copy(&frame->ch_layout, &avctx->ch_layout); - if (ret2 < 0) { - ret = ret2; - got_frame = 0; - } - } -#if FF_API_OLD_CHANNEL_LAYOUT -FF_DISABLE_DEPRECATION_WARNINGS - if (!frame->channel_layout) - frame->channel_layout = avctx->ch_layout.order == AV_CHANNEL_ORDER_NATIVE ? - avctx->ch_layout.u.mask : 0; - if (!frame->channels) - frame->channels = avctx->ch_layout.nb_channels; -FF_ENABLE_DEPRECATION_WARNINGS -#endif - if (!frame->sample_rate) - frame->sample_rate = avctx->sample_rate; - } - - side= av_packet_get_side_data(avci->last_pkt_props, AV_PKT_DATA_SKIP_SAMPLES, &side_size); - if(side && side_size>=10) { - avci->skip_samples = AV_RL32(side); - avci->skip_samples = FFMAX(0, avci->skip_samples); - discard_padding = AV_RL32(side + 4); - av_log(avctx, AV_LOG_DEBUG, "skip %d / discard %d samples due to side data\n", - avci->skip_samples, (int)discard_padding); - skip_reason = AV_RL8(side + 8); - discard_reason = AV_RL8(side + 9); - } - - if ((frame->flags & AV_FRAME_FLAG_DISCARD) && got_frame && - !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { - avci->skip_samples = FFMAX(0, avci->skip_samples - frame->nb_samples); - got_frame = 0; - *discarded_samples += frame->nb_samples; - } - - if (avci->skip_samples > 0 && got_frame && - !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { - if(frame->nb_samples <= avci->skip_samples){ - got_frame = 0; - *discarded_samples += frame->nb_samples; - avci->skip_samples -= frame->nb_samples; - av_log(avctx, AV_LOG_DEBUG, "skip whole frame, skip left: %d\n", - avci->skip_samples); - } else { - av_samples_copy(frame->extended_data, frame->extended_data, 0, avci->skip_samples, - frame->nb_samples - avci->skip_samples, avctx->ch_layout.nb_channels, frame->format); - if(avctx->pkt_timebase.num && avctx->sample_rate) { - int64_t diff_ts = av_rescale_q(avci->skip_samples, - (AVRational){1, avctx->sample_rate}, - avctx->pkt_timebase); - if(frame->pts!=AV_NOPTS_VALUE) - frame->pts += diff_ts; - if(frame->pkt_dts!=AV_NOPTS_VALUE) - frame->pkt_dts += diff_ts; - if (frame->duration >= diff_ts) - frame->duration -= diff_ts; - } else { - av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for skipped samples.\n"); - } - av_log(avctx, AV_LOG_DEBUG, "skip %d/%d samples\n", - avci->skip_samples, frame->nb_samples); - *discarded_samples += avci->skip_samples; - frame->nb_samples -= avci->skip_samples; - avci->skip_samples = 0; - } - } - - if (discard_padding > 0 && discard_padding <= frame->nb_samples && got_frame && - !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { - if (discard_padding == frame->nb_samples) { - *discarded_samples += frame->nb_samples; - got_frame = 0; - } else { - if(avctx->pkt_timebase.num && avctx->sample_rate) { - int64_t diff_ts = av_rescale_q(frame->nb_samples - discard_padding, - (AVRational){1, avctx->sample_rate}, - avctx->pkt_timebase); - frame->duration = diff_ts; - } else { - av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for discarded samples.\n"); - } - av_log(avctx, AV_LOG_DEBUG, "discard %d/%d samples\n", - (int)discard_padding, frame->nb_samples); - frame->nb_samples -= discard_padding; - } - } + if (!got_frame) + av_frame_unref(frame); - if ((avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL) && got_frame) { - AVFrameSideData *fside = av_frame_new_side_data(frame, AV_FRAME_DATA_SKIP_SAMPLES, 10); - if (fside) { - AV_WL32(fside->data, avci->skip_samples); - AV_WL32(fside->data + 4, discard_padding); - AV_WL8(fside->data + 8, skip_reason); - AV_WL8(fside->data + 9, discard_reason); - avci->skip_samples = 0; - } - } - } + if (avctx->codec->type == AVMEDIA_TYPE_VIDEO) { + ret = (!got_frame || frame->flags & AV_FRAME_FLAG_DISCARD) ? AVERROR(EAGAIN) : 0; + } else if (avctx->codec->type == AVMEDIA_TYPE_AUDIO) + ret = discard_samples(avctx, frame, discarded_samples); - if (!got_frame) + if (ret == AVERROR(EAGAIN)) av_frame_unref(frame); - if (ret >= 0 && avctx->codec->type == AVMEDIA_TYPE_VIDEO) - ret = pkt->size; + if (consumed < 0) + ret = consumed; + if (consumed >= 0 && avctx->codec->type == AVMEDIA_TYPE_VIDEO) + consumed = pkt->size; + + if (!ret) + av_assert0(frame->buf[0]); + if (ret == AVERROR(EAGAIN)) + ret = 0; - /* do not stop draining when actual_got_frame != 0 or ret < 0 */ - /* got_frame == 0 but actual_got_frame != 0 when frame is discarded */ - if (avci->draining && !actual_got_frame) { + /* do not stop draining when got_frame != 0 or ret < 0 */ + if (avci->draining && !got_frame) { if (ret < 0) { /* prevent infinite loop if a decoder wrongly always return error on draining */ /* reasonable nb_errors_max = maximum b frames + thread count */ @@ -490,11 +505,9 @@ FF_ENABLE_DEPRECATION_WARNINGS } } - if (ret >= pkt->size || ret < 0) { + if (consumed >= pkt->size || ret < 0) { av_packet_unref(pkt); } else { - int consumed = ret; - pkt->data += consumed; pkt->size -= consumed; pkt->pts = AV_NOPTS_VALUE; @@ -509,10 +522,7 @@ FF_ENABLE_DEPRECATION_WARNINGS } } - if (got_frame) - av_assert0(frame->buf[0]); - - return ret < 0 ? ret : 0; + return ret; } #if CONFIG_LCMS2