From patchwork Mon Dec 5 13:39:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 39616 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp3246043pzb; Mon, 5 Dec 2022 05:39:46 -0800 (PST) X-Google-Smtp-Source: AA0mqf5imOfbVRk+IGfiY14CYGOcIjpbk6rfyU3641PBsHcxwReEFojmO5/hcA+/iskI8/6X5krY X-Received: by 2002:aa7:d6d9:0:b0:46a:f635:6bb2 with SMTP id x25-20020aa7d6d9000000b0046af6356bb2mr34427247edr.270.1670247585949; Mon, 05 Dec 2022 05:39:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670247585; cv=none; d=google.com; s=arc-20160816; b=a/aWoVbyEak7Nrh55OaP8Epk4Kk2NlZC3ihSAmD6H0Hw+JF56GBLNAnDklJ9+m6McV 0SmeT4lB36OU0Bp0PKGSgMW80LffoBtbZAHAkRVlQvWT/nS2dRvDCem+QdLXEb/zXj2x TM4ymbRaUXOU0Q8wUTyd4dcYnYYDzAiYkYGW5L2Yw5VmmrS4c+iOZaNSerzH8n/lIESu S3xSjX9pePnlAg9eX5uTWar/rJfaT4kP/6FwU85+y1M1TPH1uU0mNPKZQERasReAU/a4 9NS9sbvpJwZAGrGCEeY9CEl6ZENBXkWtSNPL3V0kxDIuYNq+ofMUtojyduMP4AJ20UM9 KtpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=SVNf3XnVL2Sbl3cU+9VyvdFkmfxR+NpRy9qD6DUPqg0=; b=pzvfQ0Qt2pXHdN7bpEBWFD3CNFkx3JnWMBGP4444+IGt0kqvIyOfKCXSty9Nw7HRbT vzJoEJ8bmqiI9hB8fG2hnjqKaXLRNMPzhdJfNftJJdihJMYVakdDLSmrsAgaZRch4AAL kaYnO+p3WCo2p5FASmCx5bxhxPnbaC2afPKEhwTHSvRcVf0V0rGt7hycY5Ui383uzVuG 27MH5ig0wP0cKDleVGkvOSSNX74bWSZPvBggYlF7psDZOZvoYp+1n5ttLCR1vMg/un7J kYA5CE5cMK/uZvLr7ZlRglFwA98e20bemITmQgeirF+tvUJ/GkZPCj1ODOVnNyo/kWe8 4W4w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=nGtYEpbj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q8-20020a056402518800b004630539ab29si13246141edd.101.2022.12.05.05.39.45; Mon, 05 Dec 2022 05:39:45 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=nGtYEpbj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CE1A168BC9F; Mon, 5 Dec 2022 15:39:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [136.243.74.85]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D882F68A417 for ; Mon, 5 Dec 2022 15:39:36 +0200 (EET) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id 608323A5BF7; Mon, 5 Dec 2022 14:39:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1670247576; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=p/J8XePN/jmZ38MKx78B8j6iqnnyH6s0Do/shexbyX4=; b=nGtYEpbjQVAa33Z0lXB/F1+asJo1M237KwVNAwu6Y/XIj+Ozlnb4lj79YuVDFoYeNaHkj4 EfmDVA83vnL4AhQE3s0xRtcmRqq+wBUBg2NHuDIU7InC/1T5EhLF01vylIQpYs75qCrbvr Ef/Sg2TWjPsaeunzTLUduvO+JRA5aBEG2iVclJU7HjthWVRTXlrg3xyh10K0ml0rARwjog Y3YplgPChm62u3OcoYxZ9+yQ23ccurmP7RBpx8bsRdw/8MgZO3DLg36qFvdN76p2ksxDkH nG3jxlqt27pdi1Rk5o/hLkgugCRjRBud4n+oS27zG8bS7myXHn1UuacwZLkgHw== From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Mon, 5 Dec 2022 14:39:37 +0100 Message-Id: <20221205133938.505-1-timo@rothenpieler.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avcodec/thread: add support for frame threading receive_frame based decoders X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ZP894EJ/fJwZ This is fairly basic and makes a lot of assumptions, but it works for the most simple cases. For one, it only ever fetches exactly one packet per call to receive_frame. Right now it's impossible for there to ever be more than one, but the API allows for more, which might need handled in the future. It also basically translates the new API back to the old, since that's how the frame threading code operates. Which feels backwards in regards to the new API, but it was the path with least resistance in implementing this. --- libavcodec/decode.c | 6 +++- libavcodec/pthread_frame.c | 68 ++++++++++++++++++++++++++++++++++++-- libavcodec/thread.h | 17 ++++++++++ 3 files changed, 88 insertions(+), 3 deletions(-) diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 6be2d3d6ed..72a8253aae 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -577,7 +577,11 @@ static int decode_receive_frame_internal(AVCodecContext *avctx, AVFrame *frame) av_assert0(!frame->buf[0]); if (codec->cb_type == FF_CODEC_CB_TYPE_RECEIVE_FRAME) { - ret = codec->cb.receive_frame(avctx, frame); + if (HAVE_THREADS && avctx->active_thread_type & FF_THREAD_FRAME) + ret = ff_thread_receive_frame(avctx, frame); + else + ret = codec->cb.receive_frame(avctx, frame); + if (ret != AVERROR(EAGAIN)) av_packet_unref(avci->last_pkt_props); } else diff --git a/libavcodec/pthread_frame.c b/libavcodec/pthread_frame.c index df82a4125f..8f704e35d3 100644 --- a/libavcodec/pthread_frame.c +++ b/libavcodec/pthread_frame.c @@ -92,6 +92,7 @@ typedef struct PerThreadContext { AVCodecContext *avctx; ///< Context used to decode packets passed to this thread. AVPacket *avpkt; ///< Input packet (for decoding) or output (for encoding). + int avpkt_read; ///< Indicates if the packet has been read for this recv_frame call already. AVFrame *frame; ///< Output frame (for decoding) or input (for encoding). int got_frame; ///< The output of got_picture_ptr from the last avcodec_decode_video() call. @@ -237,8 +238,14 @@ FF_ENABLE_DEPRECATION_WARNINGS } av_frame_unref(p->frame); - p->got_frame = 0; - p->result = codec->cb.decode(avctx, p->frame, &p->got_frame, p->avpkt); + if (codec->cb_type == FF_CODEC_CB_TYPE_RECEIVE_FRAME) { + p->avpkt_read = 0; + p->result = codec->cb.receive_frame(avctx, p->frame); + p->got_frame = !p->result; + } else { + p->got_frame = 0; + p->result = codec->cb.decode(avctx, p->frame, &p->got_frame, p->avpkt); + } if ((p->result < 0 || !p->got_frame) && p->frame->buf[0]) ff_thread_release_buffer(avctx, p->frame); @@ -621,6 +628,58 @@ finish: return err; } +int ff_thread_receive_frame(AVCodecContext *avctx, AVFrame *frame) +{ + AVPacket *const avpkt = avctx->internal->in_pkt; + int got_picture = 0; + int draining = 0; + int ret; + + av_packet_unref(avpkt); + ret = ff_decode_get_packet(avctx, avpkt); + if (ret < 0 && ret != AVERROR_EOF) + return ret; + draining = ret == AVERROR_EOF; + + ret = ff_thread_decode_frame(avctx, frame, &got_picture, avpkt); + + if (ret == avpkt->size) { + if (got_picture) { + return 0; + } else if (draining) { + return AVERROR_EOF; + } + + return AVERROR(EAGAIN); + } else if (ret < 0) { + return ret; + } + + return AVERROR_BUG; +} + +int ff_thread_decode_get_packet(AVCodecContext *avctx, AVPacket *pkt) +{ + PerThreadContext *p; + int err; + + if (!(avctx->active_thread_type & FF_THREAD_FRAME)) + return ff_decode_get_packet(avctx, pkt); + + p = avctx->internal->thread_ctx; + + if (p->avpkt_read) + return AVERROR(EAGAIN); + + err = av_packet_ref(pkt, p->avpkt); + if (err < 0) + return err; + + p->avpkt_read = 1; + + return 0; +} + void ff_thread_report_progress(ThreadFrame *f, int n, int field) { PerThreadContext *p; @@ -775,6 +834,7 @@ void ff_frame_thread_free(AVCodecContext *avctx, int thread_count) av_freep(&ctx->slice_offset); av_buffer_unref(&ctx->internal->pool); + av_packet_free(&ctx->internal->in_pkt); av_freep(&ctx->internal); av_buffer_unref(&ctx->hw_frames_ctx); } @@ -826,6 +886,10 @@ static av_cold int init_thread(PerThreadContext *p, int *threads_to_free, return AVERROR(ENOMEM); copy->internal->thread_ctx = p; + copy->internal->in_pkt = av_packet_alloc(); + if (!copy->internal->in_pkt) + return AVERROR(ENOMEM); + copy->delay = avctx->delay; if (codec->priv_data_size) { diff --git a/libavcodec/thread.h b/libavcodec/thread.h index d5673f25ea..76e7d44bd4 100644 --- a/libavcodec/thread.h +++ b/libavcodec/thread.h @@ -52,6 +52,15 @@ void ff_thread_flush(AVCodecContext *avctx); int ff_thread_decode_frame(AVCodecContext *avctx, AVFrame *picture, int *got_picture_ptr, AVPacket *avpkt); +/** + * Receive a new frame from a decoding thread. + * Returns the next available frame, or AVERROR(EAGAIN) if + * none is available. + * + * Parameters are the same as FFCodec.receive_frame. + */ +int ff_thread_receive_frame(AVCodecContext *avctx, AVFrame *frame); + /** * If the codec defines update_thread_context(), call this * when they are ready for the next thread to start decoding @@ -99,6 +108,14 @@ int ff_thread_get_buffer(AVCodecContext *avctx, AVFrame *f, int flags); */ void ff_thread_release_buffer(AVCodecContext *avctx, AVFrame *f); +/** + * Wrapper around ff_decode_get_packet() for frame-multithreaded codecs. + * Call this functions instead of ff_decode_get_packet(). + * + * Parameters are the same as ff_decode_get_packet(). + */ +int ff_thread_decode_get_packet(AVCodecContext *avctx, AVPacket *pkt); + int ff_thread_init(AVCodecContext *s); int ff_slice_thread_execute_with_mainfunc(AVCodecContext *avctx, int (*action_func2)(AVCodecContext *c, void *arg, int jobnr, int threadnr),