From patchwork Wed Jul 12 22:07:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 42616 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:6da1:b0:131:a7d0:bc6d with SMTP id gl33csp2496061pzb; Wed, 12 Jul 2023 15:07:52 -0700 (PDT) X-Google-Smtp-Source: APBJJlEiCVQhgyeEORc3P6cyxngcADcq2HWO3j6fnE5Jvp19XwpCfNLYVYo8fNOWZHi59eLWmVfl X-Received: by 2002:ac2:498e:0:b0:4fb:74da:989a with SMTP id f14-20020ac2498e000000b004fb74da989amr16515656lfl.3.1689199671923; Wed, 12 Jul 2023 15:07:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689199671; cv=none; d=google.com; s=arc-20160816; b=kCeoc3xxE69y8c4RsuKZk/+JLE8G7fK9QqrvK+GFXzsz6lsJ2tIUP+SJgINx/zwwgP q59TQ+wbvYgONZ3y9rJsNZFrfNOO0ZANQkk88HKtiNx7KtJl8pt7/wz9Kqd7O1V0ms6b vXqLQ8ziG57a+dA1vN1FrsPRW6LSi4rJlsJcQC0xlS5g32RAUeAC9GdrVKuuTheAIBUJ aPNsbVJrXlcDX7iiYzJVsUrJxCAoYPHLQBB/GfjqGQQFW3lu4kU3daUSbSHLV1JzIRMb sNWuwlCFsAK/EhYATWASCeJbhKh2p3hKgBLiCPpzSEaYDnO2SV4r2GA8exYYDmT/zRvm f3vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=TXqRBrtTAbFL3rVWkyFz7VRzhinPnqje3pw46boI5lo=; fh=J3zlMo7rVW2t2IQYogliBcMNuBh6YQg7NRChcrschf4=; b=KVHxlYeIHTbuEdDac+ZjgfAj5gvrEJXJbMgsOEhaGRXaTj93z0PIkp88JueA3vTnip a/AsnM2pFAQ489ub9kbBWDONCnQNPMfjMB6gTuYaSdklOmTOmP/0/Iy8pFkwcHYsgR3i 94pQnJk7BM0cbvewm5/7vBPwzbwM7crXR02Viu+hf03wP6Xn9c9Bah1RVnTVW57064Uc q/iHlSskwCS8RoLsx1Vy4cZGqG5uWqnGY37A9mdPDR8o9KmXBplH3N/S3LDlrnq6Jr1F P6dCxzcGVYNlXmdIEexpZfHwtiR9cZyADrRKjTn/+T3P4uDJ/y9HTFhoXJyLHo4dr95m LqYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b="L/ygB3vg"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k23-20020aa7d8d7000000b0051debcd957esi5995730eds.351.2023.07.12.15.07.51; Wed, 12 Jul 2023 15:07:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b="L/ygB3vg"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1947E68C499; Thu, 13 Jul 2023 01:07:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oa1-f45.google.com (mail-oa1-f45.google.com [209.85.160.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C4A8068C1FF for ; Thu, 13 Jul 2023 01:07:39 +0300 (EEST) Received: by mail-oa1-f45.google.com with SMTP id 586e51a60fabf-1b0249f1322so6032982fac.3 for ; Wed, 12 Jul 2023 15:07:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689199658; x=1691791658; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=ossJJw7/xAvyx7DYOLYH/nI62IJwzhJaRGfDtVjki4o=; b=L/ygB3vgZ3SeuhqZTozq4c8wCncVQasDO3G690It6PHo+2suUAtksL2DV8y77NWFcK mAm12iGhMptXSV1Kooo66feCgh0Wo/404YJ1bDzmr5/oXDEPdPUwtO1vz4uR5AyeyKwe oAEYZuC5SkC7fTdSHuVmrPX545NpMQuyhl69iXnvoJZUwxj1uOhlHg36sFyOQPxk4t+5 557HtbOYx/eyDR3fiTnbf0hWp1O2IoszK8zAQRLGytaC1cz5CVsIwc/1AeMZvxLibcbo 0oy9ofHNfFEVyuDWsRwx30pTZ/nAYRoVi8aCgJ1l5bDXg1Op6BamBOjfYQcP3qR2wBC3 6WTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689199658; x=1691791658; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ossJJw7/xAvyx7DYOLYH/nI62IJwzhJaRGfDtVjki4o=; b=VMJIUL/tHNPTCqv3NKaKwosQGU/vSxbwX+ZBFh5AYHZRTnlgKsuQJg0jRzRShZo2Ad x3cZcee48BHKQw7P0eEYsJOTwfBuaiMeJfAGD9sW5skX4zcvbKM+y6aW1qBH/jOkTShW GLoJ1GzTOix46sxj0Ev7f3IR4v3RsoEgPF4dNQ5oQYQ1rlQ3ZUX8aPQ4zsbpWLVMC/bl ZU+vvyMoKmfQmBgt1exPPGCJDqpZkjB7gLnIRrpL4fp7lqDEtYs9SSNQIhlY650zR5Ho NOegxqqpzW/hsPipzQA01kGPzOx1UyiZiAZelf6AmEdZSvyFV7QRIKhvyr0N4qPuTrQJ DOvA== X-Gm-Message-State: ABy/qLb0LsGbG8fgSE8ipg6iHu+e6PVHabHfuGQRLYGg3Uz+uglYXoet aIx5aV5GaJwZ/diTPLEYHJPVnh/Bwm8= X-Received: by 2002:a05:6871:453:b0:1b0:2203:2bed with SMTP id e19-20020a056871045300b001b022032bedmr20140307oag.59.1689199657730; Wed, 12 Jul 2023 15:07:37 -0700 (PDT) Received: from localhost.localdomain (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id v13-20020a056870e28d00b001b3d5b06b6asm2433756oad.23.2023.07.12.15.07.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Jul 2023 15:07:37 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Wed, 12 Jul 2023 19:07:09 -0300 Message-ID: <20230712220712.46338-1-jamrial@gmail.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 1/4] avcodec/decode: move processing discard samples to its own function X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: oBOPDZIGFudw Signed-off-by: James Almer --- libavcodec/decode.c | 269 +++++++++++++++++++++++--------------------- 1 file changed, 141 insertions(+), 128 deletions(-) diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 239ad70b41..4478c8c4fe 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -289,6 +289,123 @@ static int64_t guess_correct_pts(AVCodecContext *ctx, return pts; } +static int discard_samples(AVCodecContext *avctx, AVFrame *frame, int64_t *discarded_samples) +{ + AVCodecInternal *avci = avctx->internal; + int ret = 0; + uint8_t *side; + size_t side_size; + uint32_t discard_padding = 0; + uint8_t skip_reason = 0; + uint8_t discard_reason = 0; + + side = av_packet_get_side_data(avci->last_pkt_props, AV_PKT_DATA_SKIP_SAMPLES, &side_size); + if (side && side_size >= 10) { + avci->skip_samples = AV_RL32(side); + avci->skip_samples = FFMAX(0, avci->skip_samples); + discard_padding = AV_RL32(side + 4); + av_log(avctx, AV_LOG_DEBUG, "skip %d / discard %d samples due to side data\n", + avci->skip_samples, (int)discard_padding); + skip_reason = AV_RL8(side + 8); + discard_reason = AV_RL8(side + 9); + } + + if (!frame->buf[0]) + return AVERROR(EAGAIN); + + if (frame->format == AV_SAMPLE_FMT_NONE) + frame->format = avctx->sample_fmt; + if (!frame->ch_layout.nb_channels) { + int ret2 = av_channel_layout_copy(&frame->ch_layout, &avctx->ch_layout); + if (ret2 < 0) { + ret = ret2; + } + } +#if FF_API_OLD_CHANNEL_LAYOUT +FF_DISABLE_DEPRECATION_WARNINGS + if (!frame->channel_layout) + frame->channel_layout = avctx->ch_layout.order == AV_CHANNEL_ORDER_NATIVE ? + avctx->ch_layout.u.mask : 0; + if (!frame->channels) + frame->channels = avctx->ch_layout.nb_channels; +FF_ENABLE_DEPRECATION_WARNINGS +#endif + if (!frame->sample_rate) + frame->sample_rate = avctx->sample_rate; + + if ((frame->flags & AV_FRAME_FLAG_DISCARD) && + !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + avci->skip_samples = FFMAX(0, avci->skip_samples - frame->nb_samples); + *discarded_samples += frame->nb_samples; + return AVERROR(EAGAIN); + } + + if (avci->skip_samples > 0 && + !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + if (frame->nb_samples <= avci->skip_samples){ + *discarded_samples += frame->nb_samples; + avci->skip_samples -= frame->nb_samples; + av_log(avctx, AV_LOG_DEBUG, "skip whole frame, skip left: %d\n", + avci->skip_samples); + return AVERROR(EAGAIN); + } else { + av_samples_copy(frame->extended_data, frame->extended_data, 0, avci->skip_samples, + frame->nb_samples - avci->skip_samples, avctx->ch_layout.nb_channels, frame->format); + if (avctx->pkt_timebase.num && avctx->sample_rate) { + int64_t diff_ts = av_rescale_q(avci->skip_samples, + (AVRational){1, avctx->sample_rate}, + avctx->pkt_timebase); + if (frame->pts != AV_NOPTS_VALUE) + frame->pts += diff_ts; + if (frame->pkt_dts != AV_NOPTS_VALUE) + frame->pkt_dts += diff_ts; + if (frame->duration >= diff_ts) + frame->duration -= diff_ts; + } else + av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for skipped samples.\n"); + + av_log(avctx, AV_LOG_DEBUG, "skip %d/%d samples\n", + avci->skip_samples, frame->nb_samples); + *discarded_samples += avci->skip_samples; + frame->nb_samples -= avci->skip_samples; + avci->skip_samples = 0; + } + } + + if (discard_padding > 0 && discard_padding <= frame->nb_samples && + !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + if (discard_padding == frame->nb_samples) { + *discarded_samples += frame->nb_samples; + return AVERROR(EAGAIN); + } else { + if (avctx->pkt_timebase.num && avctx->sample_rate) { + int64_t diff_ts = av_rescale_q(frame->nb_samples - discard_padding, + (AVRational){1, avctx->sample_rate}, + avctx->pkt_timebase); + frame->duration = diff_ts; + } else + av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for discarded samples.\n"); + + av_log(avctx, AV_LOG_DEBUG, "discard %d/%d samples\n", + (int)discard_padding, frame->nb_samples); + frame->nb_samples -= discard_padding; + } + } + + if ((avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { + AVFrameSideData *fside = av_frame_new_side_data(frame, AV_FRAME_DATA_SKIP_SAMPLES, 10); + if (fside) { + AV_WL32(fside->data, avci->skip_samples); + AV_WL32(fside->data + 4, discard_padding); + AV_WL8(fside->data + 8, skip_reason); + AV_WL8(fside->data + 9, discard_reason); + avci->skip_samples = 0; + } + } + + return ret; +} + /* * The core of the receive_frame_wrapper for the decoders implementing * the simple API. Certain decoders might consume partial packets without @@ -300,7 +417,7 @@ static inline int decode_simple_internal(AVCodecContext *avctx, AVFrame *frame, AVCodecInternal *avci = avctx->internal; AVPacket *const pkt = avci->in_pkt; const FFCodec *const codec = ffcodec(avctx->codec); - int got_frame, actual_got_frame; + int got_frame, consumed; int ret; if (!pkt->data && !avci->draining) { @@ -323,9 +440,9 @@ static inline int decode_simple_internal(AVCodecContext *avctx, AVFrame *frame, got_frame = 0; if (HAVE_THREADS && avctx->active_thread_type & FF_THREAD_FRAME) { - ret = ff_thread_decode_frame(avctx, frame, &got_frame, pkt); + consumed = ff_thread_decode_frame(avctx, frame, &got_frame, pkt); } else { - ret = codec->cb.decode(avctx, frame, &got_frame, pkt); + consumed = codec->cb.decode(avctx, frame, &got_frame, pkt); if (!(codec->caps_internal & FF_CODEC_CAP_SETS_PKT_DTS)) frame->pkt_dts = pkt->dts; @@ -347,132 +464,33 @@ FF_ENABLE_DEPRECATION_WARNINGS } } emms_c(); - actual_got_frame = got_frame; + + if (!got_frame) + av_frame_unref(frame); if (avctx->codec->type == AVMEDIA_TYPE_VIDEO) { - if (frame->flags & AV_FRAME_FLAG_DISCARD) - got_frame = 0; + ret = (!got_frame || frame->flags & AV_FRAME_FLAG_DISCARD) + ? AVERROR(EAGAIN) + : 0; } else if (avctx->codec->type == AVMEDIA_TYPE_AUDIO) { - uint8_t *side; - size_t side_size; - uint32_t discard_padding = 0; - uint8_t skip_reason = 0; - uint8_t discard_reason = 0; - - if (ret >= 0 && got_frame) { - if (frame->format == AV_SAMPLE_FMT_NONE) - frame->format = avctx->sample_fmt; - if (!frame->ch_layout.nb_channels) { - int ret2 = av_channel_layout_copy(&frame->ch_layout, &avctx->ch_layout); - if (ret2 < 0) { - ret = ret2; - got_frame = 0; - } - } -#if FF_API_OLD_CHANNEL_LAYOUT -FF_DISABLE_DEPRECATION_WARNINGS - if (!frame->channel_layout) - frame->channel_layout = avctx->ch_layout.order == AV_CHANNEL_ORDER_NATIVE ? - avctx->ch_layout.u.mask : 0; - if (!frame->channels) - frame->channels = avctx->ch_layout.nb_channels; -FF_ENABLE_DEPRECATION_WARNINGS -#endif - if (!frame->sample_rate) - frame->sample_rate = avctx->sample_rate; - } - - side= av_packet_get_side_data(avci->last_pkt_props, AV_PKT_DATA_SKIP_SAMPLES, &side_size); - if(side && side_size>=10) { - avci->skip_samples = AV_RL32(side); - avci->skip_samples = FFMAX(0, avci->skip_samples); - discard_padding = AV_RL32(side + 4); - av_log(avctx, AV_LOG_DEBUG, "skip %d / discard %d samples due to side data\n", - avci->skip_samples, (int)discard_padding); - skip_reason = AV_RL8(side + 8); - discard_reason = AV_RL8(side + 9); - } - - if ((frame->flags & AV_FRAME_FLAG_DISCARD) && got_frame && - !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { - avci->skip_samples = FFMAX(0, avci->skip_samples - frame->nb_samples); - got_frame = 0; - *discarded_samples += frame->nb_samples; - } - - if (avci->skip_samples > 0 && got_frame && - !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { - if(frame->nb_samples <= avci->skip_samples){ - got_frame = 0; - *discarded_samples += frame->nb_samples; - avci->skip_samples -= frame->nb_samples; - av_log(avctx, AV_LOG_DEBUG, "skip whole frame, skip left: %d\n", - avci->skip_samples); - } else { - av_samples_copy(frame->extended_data, frame->extended_data, 0, avci->skip_samples, - frame->nb_samples - avci->skip_samples, avctx->ch_layout.nb_channels, frame->format); - if(avctx->pkt_timebase.num && avctx->sample_rate) { - int64_t diff_ts = av_rescale_q(avci->skip_samples, - (AVRational){1, avctx->sample_rate}, - avctx->pkt_timebase); - if(frame->pts!=AV_NOPTS_VALUE) - frame->pts += diff_ts; - if(frame->pkt_dts!=AV_NOPTS_VALUE) - frame->pkt_dts += diff_ts; - if (frame->duration >= diff_ts) - frame->duration -= diff_ts; - } else { - av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for skipped samples.\n"); - } - av_log(avctx, AV_LOG_DEBUG, "skip %d/%d samples\n", - avci->skip_samples, frame->nb_samples); - *discarded_samples += avci->skip_samples; - frame->nb_samples -= avci->skip_samples; - avci->skip_samples = 0; - } - } - - if (discard_padding > 0 && discard_padding <= frame->nb_samples && got_frame && - !(avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL)) { - if (discard_padding == frame->nb_samples) { - *discarded_samples += frame->nb_samples; - got_frame = 0; - } else { - if(avctx->pkt_timebase.num && avctx->sample_rate) { - int64_t diff_ts = av_rescale_q(frame->nb_samples - discard_padding, - (AVRational){1, avctx->sample_rate}, - avctx->pkt_timebase); - frame->duration = diff_ts; - } else { - av_log(avctx, AV_LOG_WARNING, "Could not update timestamps for discarded samples.\n"); - } - av_log(avctx, AV_LOG_DEBUG, "discard %d/%d samples\n", - (int)discard_padding, frame->nb_samples); - frame->nb_samples -= discard_padding; - } - } - - if ((avctx->flags2 & AV_CODEC_FLAG2_SKIP_MANUAL) && got_frame) { - AVFrameSideData *fside = av_frame_new_side_data(frame, AV_FRAME_DATA_SKIP_SAMPLES, 10); - if (fside) { - AV_WL32(fside->data, avci->skip_samples); - AV_WL32(fside->data + 4, discard_padding); - AV_WL8(fside->data + 8, skip_reason); - AV_WL8(fside->data + 9, discard_reason); - avci->skip_samples = 0; - } - } + ret = discard_samples(avctx, frame, discarded_samples); } - if (!got_frame) + if (ret == AVERROR(EAGAIN)) av_frame_unref(frame); - if (ret >= 0 && avctx->codec->type == AVMEDIA_TYPE_VIDEO) - ret = pkt->size; + if (consumed < 0) + ret = consumed; + if (consumed >= 0 && avctx->codec->type == AVMEDIA_TYPE_VIDEO) + consumed = pkt->size; + + if (!ret) + av_assert0(frame->buf[0]); + if (ret == AVERROR(EAGAIN)) + ret = 0; - /* do not stop draining when actual_got_frame != 0 or ret < 0 */ - /* got_frame == 0 but actual_got_frame != 0 when frame is discarded */ - if (avci->draining && !actual_got_frame) { + /* do not stop draining when got_frame != 0 or ret < 0 */ + if (avci->draining && !got_frame) { if (ret < 0) { /* prevent infinite loop if a decoder wrongly always return error on draining */ /* reasonable nb_errors_max = maximum b frames + thread count */ @@ -490,11 +508,9 @@ FF_ENABLE_DEPRECATION_WARNINGS } } - if (ret >= pkt->size || ret < 0) { + if (consumed >= pkt->size || ret < 0) { av_packet_unref(pkt); } else { - int consumed = ret; - pkt->data += consumed; pkt->size -= consumed; pkt->pts = AV_NOPTS_VALUE; @@ -509,10 +525,7 @@ FF_ENABLE_DEPRECATION_WARNINGS } } - if (got_frame) - av_assert0(frame->buf[0]); - - return ret < 0 ? ret : 0; + return ret; } #if CONFIG_LCMS2