From patchwork Tue Feb 28 12:01:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 40541 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp1540123pzb; Tue, 28 Feb 2023 04:01:44 -0800 (PST) X-Google-Smtp-Source: AK7set/2ktkanyJZGToBJsN7jpm7EJK55JO/u74kDRYSHufGnNU3HAnDNv0vVbFFUDiQfrHiw8gv X-Received: by 2002:a17:907:3da7:b0:7e0:eed0:8beb with SMTP id he39-20020a1709073da700b007e0eed08bebmr3017269ejc.41.1677585704696; Tue, 28 Feb 2023 04:01:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677585704; cv=none; d=google.com; s=arc-20160816; b=A5d5y6Hjp62Q3ht1HvyL3kPF5dwJSt7+ibN8HnzLJFviYlD7uja8GBpGnFeksuudBe QPbsgVOtb93kSHR8qC9jq6dt58gKwxM8JJXzxkEe3O+6fahIRcSeOFo4hEdC76fZc7YQ Ye8A5pTQ906KQ87SPgPaDj/eE6qw9/zyaepHzxIuFnnVzIwx2mIMQe+3AvglG2FCJlxv e/ECeq+ionjGtqVMI967doguHct1EYgppa4sX/SbPsCPcH9wer6N00AGXnrvjUmQ3xCo VIDPaI9JtQkBBpxeaB4dvSvhHWPbARPILznE0Iz/Z6Qjcc8P288KUftxvsW8V8SfNacJ 5kpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=FaqtGuCptr+AnRSusN/uwDYzDhYW6M3XXH147h0Uc7g=; b=sFo3AHS8M+Dhd6hXfpNDex+FLKXG1dA5YOyZZYm9/I9jmgdtputOoVQYgcT8uZ5oMx Y9DIIuWAVrRQIQjBNLUlYnpJgglokWxC1OC0jwGQP0vAMccTAAYdWR54dLIN9Ii3eWrz +AXVRixLq1bLdECRlTAropQ5qus4MhVP6bJMar8SCoykq+sd+cCc6YEqWs6uRnGdyyjv O7kQlvaJZkjtAS4/96VhM69FsIptqhtO8EHM4/4Hkil4jcS+QKhbU3IIZFDcpBLpC7Np X2C3/tQkdoXOAQ9Mv7uTN8rqN4PbaPAm8OK2HrPfHtIb4lJ+MNiOT5m1kuDstDbG5AEL Fb9w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v1-20020aa7dbc1000000b004ad15704e7csi11588957edt.376.2023.02.28.04.01.42; Tue, 28 Feb 2023 04:01:44 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D760968AAB6; Tue, 28 Feb 2023 14:01:29 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A08D868A92B for ; Tue, 28 Feb 2023 14:01:22 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 664ED2404EC for ; Tue, 28 Feb 2023 13:01:22 +0100 (CET) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id MuBGUExm5MGs for ; Tue, 28 Feb 2023 13:01:21 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id B77C12404EE for ; Tue, 28 Feb 2023 13:01:20 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 5358C3A0404 for ; Tue, 28 Feb 2023 13:01:14 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Tue, 28 Feb 2023 13:01:01 +0100 Message-Id: <20230228120104.2347-5-anton@khirnov.net> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230228120104.2347-1-anton@khirnov.net> References: <20230228120104.2347-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/8] lavc/libvpxenc: handle frame durations and AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: m1/HnBCDOY9r --- libavcodec/libvpxenc.c | 139 +++++++++++++++++++++++++++++------------ libavcodec/version.h | 2 +- 2 files changed, 100 insertions(+), 41 deletions(-) diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c index 77921badba..af16e53deb 100644 --- a/libavcodec/libvpxenc.c +++ b/libavcodec/libvpxenc.c @@ -68,6 +68,14 @@ struct FrameListData { typedef struct FrameData { int64_t pts; + int64_t duration; + +#if FF_API_REORDERED_OPAQUE + int64_t reordered_opaque; +#endif + void *frame_opaque; + AVBufferRef *frame_opaque_ref; + AVBufferRef *hdr10_plus; } FrameData; @@ -329,32 +337,101 @@ static av_cold void free_frame_list(struct FrameListData *list) } } +static void frame_data_uninit(FrameData *fd) +{ + av_buffer_unref(&fd->frame_opaque_ref); + av_buffer_unref(&fd->hdr10_plus); +} + static av_cold void fifo_free(AVFifo **fifo) { FrameData fd; while (av_fifo_read(*fifo, &fd, 1) >= 0) - av_buffer_unref(&fd.hdr10_plus); + frame_data_uninit(&fd); av_fifo_freep2(fifo); } -static int frame_data_apply(AVFifo *fifo, AVPacket *pkt) +static int frame_data_submit(AVCodecContext *avctx, AVFifo *fifo, + const AVFrame *frame) +{ + VPxContext *ctx = avctx->priv_data; + const struct vpx_codec_enc_cfg *enccfg = ctx->encoder.config.enc; + + FrameData fd = { .pts = frame->pts }; + + AVFrameSideData *av_uninit(sd); + int ret; + +#if CONFIG_LIBVPX_VP9_ENCODER + // Keep HDR10+ if it has bit depth higher than 8 and + // it has PQ trc (SMPTE2084). + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_DYNAMIC_HDR_PLUS); + if (avctx->codec_id == AV_CODEC_ID_VP9 && sd && + enccfg->g_bit_depth > 8 && avctx->color_trc == AVCOL_TRC_SMPTE2084) { + fd.hdr10_plus = av_buffer_ref(sd->buf); + if (!fd.hdr10_plus) + return AVERROR(ENOMEM); + } +#endif + + fd.duration = frame->duration; + fd.frame_opaque = frame->opaque; + if (avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE && frame->opaque_ref) { + ret = av_buffer_replace(&fd.frame_opaque_ref, frame->opaque_ref); + if (ret < 0) + goto fail; + } +#if FF_API_REORDERED_OPAQUE +FF_DISABLE_DEPRECATION_WARNINGS + fd.reordered_opaque = frame->reordered_opaque; +FF_ENABLE_DEPRECATION_WARNINGS +#endif + + ret = av_fifo_write(fifo, &fd, 1); + if (ret < 0) + goto fail; + + return 0; +fail: + frame_data_uninit(&fd); + return ret; +} + +static int frame_data_apply(AVCodecContext *avctx, AVFifo *fifo, AVPacket *pkt) { FrameData fd; uint8_t *data; if (!pkt || av_fifo_peek(fifo, &fd, 1, 0) < 0) return 0; - if (!fd.hdr10_plus || fd.pts != pkt->pts) + if (fd.pts != pkt->pts) return 0; av_fifo_drain2(fifo, 1); - data = av_packet_new_side_data(pkt, AV_PKT_DATA_DYNAMIC_HDR10_PLUS, fd.hdr10_plus->size); - if (!data) { +#if FF_API_REORDERED_OPAQUE +FF_DISABLE_DEPRECATION_WARNINGS + avctx->reordered_opaque = fd.reordered_opaque; +FF_ENABLE_DEPRECATION_WARNINGS +#endif + + pkt->duration = fd.duration; + if (avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE) { + pkt->opaque = fd.frame_opaque; + pkt->opaque_ref = fd.frame_opaque_ref; + fd.frame_opaque_ref = NULL; + } + av_buffer_unref(&fd.frame_opaque_ref); + + if (fd.hdr10_plus) { + data = av_packet_new_side_data(pkt, AV_PKT_DATA_DYNAMIC_HDR10_PLUS, fd.hdr10_plus->size); + if (!data) { + av_buffer_unref(&fd.hdr10_plus); + return AVERROR(ENOMEM); + } + + memcpy(data, fd.hdr10_plus->data, fd.hdr10_plus->size); av_buffer_unref(&fd.hdr10_plus); - return AVERROR(ENOMEM); } - memcpy(data, fd.hdr10_plus->data, fd.hdr10_plus->size); - av_buffer_unref(&fd.hdr10_plus); return 0; } @@ -914,17 +991,14 @@ static av_cold int vpx_init(AVCodecContext *avctx, return AVERROR(EINVAL); } + ctx->fifo = av_fifo_alloc2(1, sizeof(FrameData), AV_FIFO_FLAG_AUTO_GROW); + if (!ctx->fifo) + return AVERROR(ENOMEM); + #if CONFIG_LIBVPX_VP9_ENCODER if (avctx->codec_id == AV_CODEC_ID_VP9) { if (set_pix_fmt(avctx, codec_caps, &enccfg, &flags, &img_fmt)) return AVERROR(EINVAL); - // Keep HDR10+ if it has bit depth higher than 8 and - // it has PQ trc (SMPTE2084). - if (enccfg.g_bit_depth > 8 && avctx->color_trc == AVCOL_TRC_SMPTE2084) { - ctx->fifo = av_fifo_alloc2(1, sizeof(FrameData), AV_FIFO_FLAG_AUTO_GROW); - if (!ctx->fifo) - return AVERROR(ENOMEM); - } } #endif @@ -1285,11 +1359,9 @@ static int storeframe(AVCodecContext *avctx, struct FrameListData *cx_frame, AV_WB64(side_data, 1); memcpy(side_data + 8, alpha_cx_frame->buf, alpha_cx_frame->sz); } - if (ctx->fifo) { - int err = frame_data_apply(ctx->fifo, pkt); - if (err < 0) - return err; - } + ret = frame_data_apply(avctx, ctx->fifo, pkt); + if (ret < 0) + return ret; return pkt->size; } @@ -1703,24 +1775,9 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket *pkt, } } - if (ctx->fifo) { - AVFrameSideData *hdr10_plus_metadata; - // Add HDR10+ metadata to queue. - hdr10_plus_metadata = av_frame_get_side_data(frame, AV_FRAME_DATA_DYNAMIC_HDR_PLUS); - if (hdr10_plus_metadata) { - int err; - FrameData data; - data.pts = frame->pts; - data.hdr10_plus = av_buffer_ref(hdr10_plus_metadata->buf); - if (!data.hdr10_plus) - return AVERROR(ENOMEM); - err = av_fifo_write(ctx->fifo, &data, 1); - if (err < 0) { - av_buffer_unref(&data.hdr10_plus); - return err; - } - } - } + res = frame_data_submit(avctx, ctx->fifo, frame); + if (res < 0) + return res; } // this is for encoding with preset temporal layering patterns defined in @@ -1953,7 +2010,8 @@ const FFCodec ff_libvpx_vp8_encoder = { .p.type = AVMEDIA_TYPE_VIDEO, .p.id = AV_CODEC_ID_VP8, .p.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY | - AV_CODEC_CAP_OTHER_THREADS, + AV_CODEC_CAP_OTHER_THREADS | + AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE, .priv_data_size = sizeof(VPxContext), .init = vp8_init, FF_CODEC_ENCODE_CB(vpx_encode), @@ -1986,7 +2044,8 @@ FFCodec ff_libvpx_vp9_encoder = { .p.type = AVMEDIA_TYPE_VIDEO, .p.id = AV_CODEC_ID_VP9, .p.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY | - AV_CODEC_CAP_OTHER_THREADS, + AV_CODEC_CAP_OTHER_THREADS | + AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE, .p.profiles = NULL_IF_CONFIG_SMALL(ff_vp9_profiles), .p.priv_class = &class_vp9, .p.wrapper_name = "libvpx", diff --git a/libavcodec/version.h b/libavcodec/version.h index 06631ffa8c..789d9047c2 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -30,7 +30,7 @@ #include "version_major.h" #define LIBAVCODEC_VERSION_MINOR 4 -#define LIBAVCODEC_VERSION_MICRO 100 +#define LIBAVCODEC_VERSION_MICRO 101 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \