From patchwork Wed May 17 10:20:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 41672 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:ba91:b0:105:feb:71f2 with SMTP id fb17csp1078972pzb; Wed, 17 May 2023 03:23:16 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ53+TUTh1OpyK51L4/ta1mq9sTAhLofoWNuW2H9W+Sni9htnbrAtxaSTXKJt79yDM24ITP0 X-Received: by 2002:a17:907:7d9e:b0:969:f54c:dee2 with SMTP id oz30-20020a1709077d9e00b00969f54cdee2mr27159843ejc.26.1684318996501; Wed, 17 May 2023 03:23:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684318996; cv=none; d=google.com; s=arc-20160816; b=TQvlWT3Hei8XqxM0cVZKdBKZ81+lD7yVF0djJ4vjKJ5RI9kCf8xTRB14EytKhF1HVi 4VGzniww4NlUuuCg8OsOIYIq4cVOxjn7ZJcRSjw9EfulHb3jXTds2EL35kFd9oKXEIsq btVEX1X8mFYFfiEyY2pPX29X/FpCagDLlSY7qb7S00hb0+QmKoO5G2iO9/YSqbOzKq+P x59oKkpRUiQV4aSAqfTuI0o7wG1mCi9Z9v+sIt6e9nd0dbzoT3S5NcEbbA17Ptt86MME Ah4cZUxojrXvMgJon7/vdXSj1M4TjHJttFYAkydWSZs77cymTNWs3lD0EHZudQhPca3O /czg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=kZzWF9uPZTRpKf/zo/UTg8xZJGn35+/PQWtJ27hBScM=; b=w1B+0yj8AgWnc7VQtcp0l84WX+A+ML6LE4Mn0n3ylC79w1zoUWgvFqEZA3NH5f51YL G1jxWg77CVMjbY6hhUZnfexO1KYhsfH6mm2VTfd6sp0klGf4nBLK8/DjLU6IXv1kg/aG f65q2vLSijKLyKYIxp/HntHuUaFSK3WDWTmHrseEnmN5mV8UAEFMHRQwhVtAjPLqSH5X IMJzZIPwwPnl94h38bmzwrxnF5x9ETWjRgWt6Sv4yX+c0WPx5zbTJhNk42tvgls32aRK cZPPzI67ucbITJdI+/MByt5X8xJrCoAOewOpMYjO/OLh22pE5VpMlr2t0P+ryy4NGUPf yIDQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gy16-20020a170906f25000b009598cbe557csi14466616ejb.69.2023.05.17.03.23.16; Wed, 17 May 2023 03:23:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F418868C1A3; Wed, 17 May 2023 13:21:08 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A776C68C141 for ; Wed, 17 May 2023 13:20:57 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 1DD3F2406D0 for ; Wed, 17 May 2023 12:20:55 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id v_6SEbvPRBf7 for ; Wed, 17 May 2023 12:20:54 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 3A28D240D1B for ; Wed, 17 May 2023 12:20:46 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id CB8D53A222D for ; Wed, 17 May 2023 12:20:40 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Wed, 17 May 2023 12:20:24 +0200 Message-Id: <20230517102029.541-31-anton@khirnov.net> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230517102029.541-1-anton@khirnov.net> References: <20230517102029.541-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 31/36] fftools/ffmpeg_dec: restructure audio/video decoding loop X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: sJHj8L1lg+8b It currently emulates the long-removed avcodec_decode_audio4/avcodec_decode_video2 APIs, which obfuscates the actual decoding flow. Restructure the decoding calls so that they naturally follow the new avcodec_send_packet()/avcodec_receive_frame() design. This is not only significantly easier to read, but also shorter. --- fftools/ffmpeg_dec.c | 187 ++++++++++++++----------------------------- 1 file changed, 61 insertions(+), 126 deletions(-) diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index 646b587f9e..73f826c76a 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -31,7 +31,7 @@ #include "ffmpeg.h" -static void check_decode_result(InputStream *ist, int *got_output, int ret) +static void check_decode_result(InputStream *ist, int got_output, int ret) { if (ret < 0) ist->decode_errors++; @@ -39,7 +39,7 @@ static void check_decode_result(InputStream *ist, int *got_output, int ret) if (ret < 0 && exit_on_error) exit_program(1); - if (*got_output && ist->dec_ctx->codec_type != AVMEDIA_TYPE_SUBTITLE) { + if (got_output && ist->dec_ctx->codec_type != AVMEDIA_TYPE_SUBTITLE) { if (ist->decoded_frame->decode_error_flags || (ist->decoded_frame->flags & AV_FRAME_FLAG_CORRUPT)) { av_log(ist, exit_on_error ? AV_LOG_FATAL : AV_LOG_WARNING, "corrupt decoded frame\n"); @@ -49,52 +49,6 @@ static void check_decode_result(InputStream *ist, int *got_output, int ret) } } -// This does not quite work like avcodec_decode_audio4/avcodec_decode_video2. -// There is the following difference: if you got a frame, you must call -// it again with pkt=NULL. pkt==NULL is treated differently from pkt->size==0 -// (pkt==NULL means get more output, pkt->size==0 is a flush/drain packet) -static int decode(InputStream *ist, AVCodecContext *avctx, - AVFrame *frame, int *got_frame, const AVPacket *pkt) -{ - int ret; - - *got_frame = 0; - - if (pkt) { - ret = avcodec_send_packet(avctx, pkt); - // In particular, we don't expect AVERROR(EAGAIN), because we read all - // decoded frames with avcodec_receive_frame() until done. - if (ret < 0 && ret != AVERROR_EOF) - return ret; - } - - ret = avcodec_receive_frame(avctx, frame); - if (ret < 0 && ret != AVERROR(EAGAIN)) - return ret; - if (ret >= 0) { - if (ist->want_frame_data) { - FrameData *fd; - - av_assert0(!frame->opaque_ref); - frame->opaque_ref = av_buffer_allocz(sizeof(*fd)); - if (!frame->opaque_ref) { - av_frame_unref(frame); - return AVERROR(ENOMEM); - } - fd = (FrameData*)frame->opaque_ref->data; - fd->pts = frame->pts; - fd->tb = avctx->pkt_timebase; - fd->idx = avctx->frame_num - 1; - } - - frame->time_base = avctx->pkt_timebase; - - *got_frame = 1; - } - - return 0; -} - static int send_frame_to_filters(InputStream *ist, AVFrame *decoded_frame) { int i, ret; @@ -192,25 +146,10 @@ static void audio_ts_process(InputStream *ist, AVFrame *frame) frame->time_base = tb_filter; } -static int decode_audio(InputStream *ist, const AVPacket *pkt, int *got_output, - int *decode_failed) +static int decode_audio(InputStream *ist, AVFrame *decoded_frame) { - AVFrame *decoded_frame = ist->decoded_frame; - AVCodecContext *avctx = ist->dec_ctx; int ret, err = 0; - update_benchmark(NULL); - ret = decode(ist, avctx, decoded_frame, got_output, pkt); - update_benchmark("decode_audio %d.%d", ist->file_index, ist->st->index); - if (ret < 0) - *decode_failed = 1; - - if (ret != AVERROR_EOF) - check_decode_result(ist, got_output, ret); - - if (!*got_output || ret < 0) - return ret; - ist->samples_decoded += decoded_frame->nb_samples; ist->frames_decoded++; @@ -274,24 +213,10 @@ static int64_t video_duration_estimate(const InputStream *ist, const AVFrame *fr return FFMAX(ist->last_frame_duration_est, 1); } -static int decode_video(InputStream *ist, const AVPacket *pkt, int *got_output, - int eof, int *decode_failed) +static int decode_video(InputStream *ist, AVFrame *frame) { - AVFrame *frame = ist->decoded_frame; int ret = 0, err = 0; - // With fate-indeo3-2, we're getting 0-sized packets before EOF for some - // reason. This seems like a semi-critical bug. Don't trigger EOF, and - // skip the packet. - if (!eof && pkt && pkt->size == 0) - return 0; - - update_benchmark(NULL); - ret = decode(ist, ist->dec_ctx, frame, got_output, pkt); - update_benchmark("decode_video %d.%d", ist->file_index, ist->st->index); - if (ret < 0) - *decode_failed = 1; - // The following line may be required in some cases where there is no parser // or the parser does not has_b_frames correctly if (ist->par->video_delay < ist->dec_ctx->has_b_frames) { @@ -307,10 +232,6 @@ static int decode_video(InputStream *ist, const AVPacket *pkt, int *got_output, ist->par->video_delay); } - if (ret != AVERROR_EOF) - check_decode_result(ist, got_output, ret); - - if (*got_output && ret >= 0) { if (ist->dec_ctx->width != frame->width || ist->dec_ctx->height != frame->height || ist->dec_ctx->pix_fmt != frame->format) { @@ -322,10 +243,6 @@ static int decode_video(InputStream *ist, const AVPacket *pkt, int *got_output, ist->dec_ctx->height, ist->dec_ctx->pix_fmt); } - } - - if (!*got_output || ret < 0) - return ret; if(ist->top_field_first>=0) frame->flags |= AV_FRAME_FLAG_TOP_FIELD_FIRST; @@ -470,7 +387,7 @@ static int transcode_subtitles(InputStream *ist, const AVPacket *pkt) exit_program(1); } - check_decode_result(ist, &got_output, ret); + check_decode_result(ist, got_output, ret); if (ret < 0 || !got_output) { if (!pkt->size) @@ -500,40 +417,44 @@ static int send_filter_eof(InputStream *ist) int dec_packet(InputStream *ist, const AVPacket *pkt, int no_eof) { AVCodecContext *dec = ist->dec_ctx; - AVPacket *avpkt = ist->pkt; - int ret, repeating = 0; + const char *type_desc = av_get_media_type_string(dec->codec_type); + int ret; if (dec->codec_type == AVMEDIA_TYPE_SUBTITLE) return transcode_subtitles(ist, pkt ? pkt : ist->pkt); - if (pkt) { - av_packet_unref(avpkt); - ret = av_packet_ref(avpkt, pkt); - if (ret < 0) - return ret; + // With fate-indeo3-2, we're getting 0-sized packets before EOF for some + // reason. This seems like a semi-critical bug. Don't trigger EOF, and + // skip the packet. + if (pkt && pkt->size == 0) + return 0; + + ret = avcodec_send_packet(dec, pkt); + if (ret < 0 && !(ret == AVERROR_EOF && !pkt)) { + // In particular, we don't expect AVERROR(EAGAIN), because we read all + // decoded frames with avcodec_receive_frame() until done. + av_log(ist, AV_LOG_ERROR, "Error submitting %s to decoder: %s\n", + pkt ? "packet" : "EOF", av_err2str(ret)); + if (exit_on_error) + exit_program(1); + return ret; } - // while we have more to decode or while the decoder did output something on EOF while (1) { - int got_output = 0; - int decode_failed = 0; - - switch (ist->par->codec_type) { - case AVMEDIA_TYPE_AUDIO: - ret = decode_audio (ist, repeating ? NULL : avpkt, &got_output, - &decode_failed); - av_packet_unref(avpkt); - break; - case AVMEDIA_TYPE_VIDEO: - ret = decode_video (ist, repeating ? NULL : avpkt, &got_output, !pkt, - &decode_failed); + AVFrame *frame = ist->decoded_frame; - av_packet_unref(avpkt); - break; - default: av_assert0(0); - } + update_benchmark(NULL); + ret = avcodec_receive_frame(dec, frame); + update_benchmark("decode_%s %d.%d", type_desc, + ist->file_index, ist->st->index); - if (ret == AVERROR_EOF) { + if (ret != AVERROR_EOF && ret != AVERROR(EAGAIN)) + check_decode_result(ist, ret >= 0, ret); + + if (ret == AVERROR(EAGAIN)) { + av_assert0(pkt); // should never happen during flushing + return 0; + } else if (ret == AVERROR_EOF) { /* after flushing, send an EOF on all the filter inputs attached to the stream */ /* except when looping we need to flush but not to send an EOF */ if (!no_eof) { @@ -545,25 +466,39 @@ int dec_packet(InputStream *ist, const AVPacket *pkt, int no_eof) } return AVERROR_EOF; + } else if (ret < 0) { + av_log(ist, AV_LOG_ERROR, "Decoding error: %s\n", av_err2str(ret)); + if (exit_on_error) + exit_program(1); + return ret; } - if (ret < 0) { - if (decode_failed) { - av_log(NULL, AV_LOG_ERROR, "Error while decoding stream #%d:%d: %s\n", - ist->file_index, ist->st->index, av_err2str(ret)); - } else { - av_log(NULL, AV_LOG_FATAL, "Error while processing the decoded " - "data for stream #%d:%d\n", ist->file_index, ist->st->index); + if (ist->want_frame_data) { + FrameData *fd; + + av_assert0(!frame->opaque_ref); + frame->opaque_ref = av_buffer_allocz(sizeof(*fd)); + if (!frame->opaque_ref) { + av_frame_unref(frame); + report_and_exit(AVERROR(ENOMEM)); } - if (!decode_failed || exit_on_error) - exit_program(1); - return ret; + fd = (FrameData*)frame->opaque_ref->data; + fd->pts = frame->pts; + fd->tb = dec->pkt_timebase; + fd->idx = dec->frame_num - 1; } - if (!got_output) - return 0; + frame->time_base = dec->pkt_timebase; + + ret = dec->codec_type == AVMEDIA_TYPE_AUDIO ? + decode_audio(ist, frame) : + decode_video(ist, frame); - repeating = 1; + if (ret < 0) { + av_log(NULL, AV_LOG_FATAL, "Error while processing the decoded " + "data for stream #%d:%d\n", ist->file_index, ist->st->index); + exit_program(1); + } } }