From patchwork Tue May 25 10:11:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brad Hards X-Patchwork-Id: 27942 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp3739968iof; Tue, 25 May 2021 03:12:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxtaYP3cCZBVcXNd0yI2pchie7jPdOpeerlklGG+niU96WI6GWFZrnsT+1km7BMN/rP9Z4L X-Received: by 2002:a17:906:2e82:: with SMTP id o2mr28658706eji.388.1621937547819; Tue, 25 May 2021 03:12:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1621937547; cv=none; d=google.com; s=arc-20160816; b=s7kO0vtBwUrimCYQ/WoJJLp/BzZq4YdNaZP+lYvDS/inmgB7cJHGYTU4QshhQmiso9 XIciFel3tYAq8qjLqV0aMPJY9mUc7WB0vm8t6Kr0DwpRw3+fgGhzCyRZ++B2vguC8QD1 vYETTmdHq3wIceWSfNidvdyFZpA6Q/X2+hqtEulGHqOzRzalJSGlkV5E37DUBF3x0Ite f2WGJV/DEvTm4NH02sXuSHa2gznXnOtXOozqla7yohU/BGfKJnbFsBilKw8Z1Y/xEFKx MLQDX8UzbcOIGH+w+V/RkuN5xn6oImG74NNgNs2PC1M59S2GEY7Ef1hmiai2ZWbwyxuL d+bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=LuwpQZjzQ0fYFsTcFrGqdUf7vSdGQTfoBTSkSMDGW24=; b=M/nXV1hNG6zNNbQNWj/1c1wEwMApGccGbaYaz26DqpLxJCmvO7HBtlynvPqpg+U4yk B4ef2V26bus1h+Xb3lMrAfJVwbe08M2m/JJkjDkc8CHMpZ6bk3/qGnuldECIdqQlIZqV 5QxVLl+DxCZLICu6GHpo0ZSyrBJDjaPHDGLgme9+WZfgOLrV3LZO9abDN/jNp6Z7dG7h EdhWUyXpsB5pEh1B1d911m43q+hoPnxMJOszpPdUWmjuKKth8zeP+PoEn1DrgjNUto/Y JKVp6gTiGubie8a+G/2vzasT37Cy34Y6penoJte8lvlmYYkNWRnx8ovNpvLMsdhUgTat 2u5w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p4si6126109ejr.271.2021.05.25.03.12.27; Tue, 25 May 2021 03:12:27 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 27D07689DC4; Tue, 25 May 2021 13:12:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from cp1.nerdvana.net.au (cp1-new.nerdvana.net.au [113.212.99.70]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C5DA1689C6C for ; Tue, 25 May 2021 13:12:04 +0300 (EEST) Received: from ppp115-205.static.internode.on.net ([150.101.115.205]:48640 helo=audax.fritz.box) by cp1.nerdvana.net.au with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1llU2a-000Bs9-SB; Tue, 25 May 2021 20:12:00 +1000 From: Brad Hards To: ffmpeg-devel@ffmpeg.org Date: Tue, 25 May 2021 20:11:57 +1000 Message-Id: <20210525101157.21519-2-bradh@frogmouth.net> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210525101157.21519-1-bradh@frogmouth.net> References: <20210525101157.21519-1-bradh@frogmouth.net> MIME-Version: 1.0 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - cp1.nerdvana.net.au X-AntiAbuse: Original Domain - ffmpeg.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - frogmouth.net X-Get-Message-Sender-Via: cp1.nerdvana.net.au: authenticated_id: bradh@frogmouth.net X-Authenticated-Sender: cp1.nerdvana.net.au: bradh@frogmouth.net X-Source: X-Source-Args: X-Source-Dir: Subject: [FFmpeg-devel] [PATCH v6] avcodec/nvenc: write out user data unregistered SEI X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Brad Hards Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: B4Pen/HI84rh Signed-off-by: Brad Hards --- libavcodec/nvenc.c | 73 +++++++++++++++++++++++++++++++++++++--------- libavcodec/nvenc.h | 2 ++ 2 files changed, 62 insertions(+), 13 deletions(-) diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c index 0dcd93a99c..9c2664a99e 100644 --- a/libavcodec/nvenc.c +++ b/libavcodec/nvenc.c @@ -1610,6 +1610,8 @@ av_cold int ff_nvenc_encode_close(AVCodecContext *avctx) av_frame_free(&ctx->frame); + av_freep(&ctx->sei_data); + if (ctx->nvencoder) { p_nvenc->nvEncDestroyEncoder(ctx->nvencoder); @@ -2170,9 +2172,9 @@ static int nvenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) NVENCSTATUS nv_status; NvencSurface *tmp_out_surf, *in_surf; int res, res2; - NV_ENC_SEI_PAYLOAD sei_data[8]; int sei_count = 0; int i; + int total_unregistered_sei = 0; NvencContext *ctx = avctx->priv_data; NvencDynLoadFunctions *dl_fn = &ctx->nvenc_dload_funcs; @@ -2185,6 +2187,8 @@ static int nvenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) return AVERROR(EINVAL); if (frame && frame->buf[0]) { + void *a53_data = NULL; + void *tc_data = NULL; in_surf = get_free_frame(ctx); if (!in_surf) return AVERROR(EAGAIN); @@ -2230,7 +2234,6 @@ static int nvenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) pic_params.inputTimeStamp = frame->pts; if (ctx->a53_cc && av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC)) { - void *a53_data = NULL; size_t a53_size = 0; if (ff_alloc_a53_sei(frame, 0, (void**)&a53_data, &a53_size) < 0) { @@ -2238,15 +2241,23 @@ static int nvenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) } if (a53_data) { - sei_data[sei_count].payloadSize = (uint32_t)a53_size; - sei_data[sei_count].payloadType = 4; - sei_data[sei_count].payload = (uint8_t*)a53_data; - sei_count ++; + void *tmp = av_fast_realloc(ctx->sei_data, + &(ctx->sei_data_size), + sizeof(NV_ENC_SEI_PAYLOAD)); + if (!tmp) { + av_free(a53_data); + return AVERROR(ENOMEM); + } else { + ctx->sei_data = tmp; + ctx->sei_data[sei_count].payloadSize = (uint32_t)a53_size; + ctx->sei_data[sei_count].payloadType = 4; + ctx->sei_data[sei_count].payload = (uint8_t*)a53_data; + sei_count ++; + } } } if (ctx->s12m_tc && av_frame_get_side_data(frame, AV_FRAME_DATA_S12M_TIMECODE)) { - void *tc_data = NULL; size_t tc_size = 0; if (ff_alloc_timecode_sei(frame, avctx->framerate, 0, (void**)&tc_data, &tc_size) < 0) { @@ -2254,14 +2265,50 @@ static int nvenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) } if (tc_data) { - sei_data[sei_count].payloadSize = (uint32_t)tc_size; - sei_data[sei_count].payloadType = SEI_TYPE_TIME_CODE; - sei_data[sei_count].payload = (uint8_t*)tc_data; - sei_count ++; + void *tmp = av_fast_realloc(ctx->sei_data, + &(ctx->sei_data_size), + (sei_count + 1) * sizeof(NV_ENC_SEI_PAYLOAD)); + if (!tmp) { + av_free(a53_data); + av_free(tc_data); + return AVERROR(ENOMEM); + } else { + ctx->sei_data = tmp; + ctx->sei_data[sei_count].payloadSize = (uint32_t)tc_size; + ctx->sei_data[sei_count].payloadType = SEI_TYPE_TIME_CODE; + ctx->sei_data[sei_count].payload = (uint8_t*)tc_data; + sei_count ++; + } + } + } + + for (int j = 0; j < frame->nb_side_data; j++) + if (frame->side_data[j]->type == AV_FRAME_DATA_SEI_UNREGISTERED) + total_unregistered_sei++; + if (total_unregistered_sei > 0) { + void *tmp = av_fast_realloc(ctx->sei_data, + &(ctx->sei_data_size), + (sei_count + total_unregistered_sei) * sizeof(NV_ENC_SEI_PAYLOAD)); + if (!tmp) { + av_free(a53_data); + av_free(tc_data); + return AVERROR(ENOMEM); + } else { + ctx->sei_data = tmp; + for (int j = 0; j < frame->nb_side_data; j++) { + AVFrameSideData *side_data = frame->side_data[j]; + if (side_data->type == AV_FRAME_DATA_SEI_UNREGISTERED) { + ctx->sei_data[sei_count].payloadSize = side_data->size; + ctx->sei_data[sei_count].payloadType = SEI_TYPE_USER_DATA_UNREGISTERED; + ctx->sei_data[sei_count].payload = av_memdup(side_data->data, side_data->size); + if (ctx->sei_data[sei_count].payload) + sei_count ++; + } + } } } - nvenc_codec_specific_pic_params(avctx, &pic_params, sei_data, sei_count); + nvenc_codec_specific_pic_params(avctx, &pic_params, ctx->sei_data, sei_count); } else { pic_params.encodePicFlags = NV_ENC_PIC_FLAG_EOS; } @@ -2273,7 +2320,7 @@ static int nvenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) nv_status = p_nvenc->nvEncEncodePicture(ctx->nvencoder, &pic_params); for ( i = 0; i < sei_count; i++) - av_freep(&sei_data[i].payload); + av_freep(&(ctx->sei_data[i].payload)); res = nvenc_pop_context(avctx); if (res < 0) diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h index 314c270e74..d28f02b54f 100644 --- a/libavcodec/nvenc.h +++ b/libavcodec/nvenc.h @@ -165,6 +165,8 @@ typedef struct NvencContext AVFifoBuffer *output_surface_queue; AVFifoBuffer *output_surface_ready_queue; AVFifoBuffer *timestamp_list; + NV_ENC_SEI_PAYLOAD *sei_data; + int sei_data_size; struct { void *ptr;