From patchwork Sun Aug 21 17:20:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 37374 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:139a:b0:8f:1db5:eae2 with SMTP id w26csp899193pzh; Sun, 21 Aug 2022 10:21:50 -0700 (PDT) X-Google-Smtp-Source: AA6agR6/xpW55lxqZ799/ZdsCcnEKh2rTzY4d9kONtOk7Ddv9rupNBmCFEjWoDm4RHBsf6CksrZe X-Received: by 2002:a17:907:6da8:b0:73d:6d4d:338 with SMTP id sb40-20020a1709076da800b0073d6d4d0338mr3469348ejc.342.1661102509900; Sun, 21 Aug 2022 10:21:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1661102509; cv=none; d=google.com; s=arc-20160816; b=LCPTe48gslf8DkoT4PPsVdUINFtzJUVVgfve/MB0+dMS3ZzuqvHb3iDLkxP9Nizok7 Hspkf1TqXQ7fHdm+a8hISAcWtwRB6ahylB+0ZhOpuMpv9ldJqb8rCE70MPIn6+ljFp7C 6KoiyNzMkw3ZlOT43f6EscCJuE8E0JlzGOlN/oIQ2WwReQ4g269OsV/pV7cB/5spBchf /Y85aVmoris5TvKVkYhgAumiQHw5m0OqzpCnpkaMb5Do1W+P6V8KPnPW8EcVIbSyIQFm jwYz9ULL3PrcMJfUa9TfjoD89aVjA5ltjaU1qtZQRgRO5FjsiIeZiYwzqEHACCICr8qr HuHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=aF5Rr6Rwr7KKVsa1uTPT2btrZZIVkSz3Lgc2CLiUuWw=; b=li10kSs8qBVLmioN6jop5XW6vdaGvhtVN9N7R5vBeK44d5nrbggNwhN0rp+sQRCFXk QWxE3E2p1iVcSYub7O426YL62dFWZ7HMYpxfre8ekYoPeoLBfsM7/3JJpOckmoySu77W bgqtbaICh+awoyQSeUiey7PqlyJYWwEL8K8dqYaER/5HSVbHnpBfAy9AaWxZKVkPtXUk ayAlORVriz92iyRZcqdaWQ9WBECPnDwFiJWnEls4jPAE1RY/fQYOFI0E9A4USMGEAYaf DCTQ4GcqxkJGKkMDO/bxMo9tsd5jL5Ro1hhCtjRX2hi3O5J4QT8T7clAwim42KbNuhV/ wA+A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=OTk84Sio; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id eb10-20020a0564020d0a00b00445bd289a71si9001211edb.322.2022.08.21.10.21.49; Sun, 21 Aug 2022 10:21:49 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=OTk84Sio; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 96A4668B9B3; Sun, 21 Aug 2022 20:21:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-vs1-f42.google.com (mail-vs1-f42.google.com [209.85.217.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 10CB168B94D for ; Sun, 21 Aug 2022 20:21:39 +0300 (EEST) Received: by mail-vs1-f42.google.com with SMTP id 190so781345vsz.7 for ; Sun, 21 Aug 2022 10:21:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc; bh=MSXgkxY7nD8c0tjp7pBEhPm9HXWYmaTnw271FSY4Cr8=; b=OTk84SiorDgaIGwW8wYbuKywh/0LW+F5r0ML9OM68mHHVRMHHIaK9u/aJWz5lhy51v uIDjGgMJpeV5GL46xjqaE+BVk5cVVbWhFw9a3PwNCQoXW5QblXeJcuOvImPRAc5DW2Qz SYyxjdZwhjo27tTvJoXidSiJsemgubEt3LS4aMnLVYJydzMZDFXjr9kIUn2tvSB21fav tLr2Zd0KeaMipXXmqWodQcK9UaE2WHI7wbKX9xJnzwcuTvI0s+RA3YmDk1/A0WvQA7dY C7m71/Wr05uoH4NbY4gMRT8qwIzE4Ie02qwmyiabUf7E1A09wbykFrem3Ge4bmW4+PvJ 4gXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc; bh=MSXgkxY7nD8c0tjp7pBEhPm9HXWYmaTnw271FSY4Cr8=; b=Gdezn+jsGs8GI/Y2HiCi0q0WZ4varV4SvZeXh/uDzbreBolwuyEbMtVHVI3EUGkf97 zcBA9f8h6ARW2ZCG0d4w8veLxWTTa3iv1+6jsIaw6MakKbicPveBOvftPkYBgnWf+Hj0 dEVJimvwrK+fWiHtCJvh5qTvrWadHQSryhzsU2bpOkuOlhsmLBd7iz77rXJeepuZ4bfy zcRZ5YMezzb4CSfJ2rn4wQeYvS+2skLDICQ89jXWzp/UUcgNTZe1uHLlhwPARLseSHna inBdn+tWxLDaml+eSu9UUlhyDjc9W/mHfnE2e8PwqvZ0BIDrT2l/be9J9tVOEWxPoahC f6Dg== X-Gm-Message-State: ACgBeo3jpCvKQO4NAp9Y1OkE7cgLyuGqc1vR7YEv4onp9UVWFSFhj+Ro tsSF1Ya0+cfAYWesKLPc2A4VcP3g1pc= X-Received: by 2002:a05:6102:1494:b0:37e:2dc5:870d with SMTP id d20-20020a056102149400b0037e2dc5870dmr5536445vsv.6.1661102496832; Sun, 21 Aug 2022 10:21:36 -0700 (PDT) Received: from localhost.localdomain ([191.97.187.183]) by smtp.gmail.com with ESMTPSA id l8-20020a056122200800b0037751d442a8sm8890453vkd.2.2022.08.21.10.21.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Aug 2022 10:21:35 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Aug 2022 14:20:47 -0300 Message-Id: <20220821172047.1709-1-jamrial@gmail.com> X-Mailer: git-send-email 2.37.2 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/libvpx: fix assembling vp9 packets with alpha channel X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 2dg6VYOluVik There's no warranty that vpx_codec_encode() will generate a list with the same amount of packets for both the yuv planes encoder and the alpha plane encoder, so queueing packets based on what the main encoder returns will fail when the amount of packets in both lists differ. Queue all data packets for every vpx_codec_encode() call from both encoders before attempting to assemble output AVPackets out of them. Fixes ticket #9884 Signed-off-by: James Almer --- libavcodec/libvpxenc.c | 83 ++++++++++++++++++++---------------------- 1 file changed, 40 insertions(+), 43 deletions(-) diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c index 5b7c7735a1..e08df5fb96 100644 --- a/libavcodec/libvpxenc.c +++ b/libavcodec/libvpxenc.c @@ -56,8 +56,6 @@ struct FrameListData { void *buf; /**< compressed data buffer */ size_t sz; /**< length of compressed data */ - void *buf_alpha; - size_t sz_alpha; int64_t pts; /**< time stamp to show frame (in timebase units) */ unsigned long duration; /**< duration to show frame @@ -87,6 +85,7 @@ typedef struct VPxEncoderContext { int have_sse; /**< true if we have pending sse[] */ uint64_t frame_number; struct FrameListData *coded_frame_list; + struct FrameListData *alpha_coded_frame_list; int cpu_used; int sharpness; @@ -311,8 +310,6 @@ static void coded_frame_add(void *list, struct FrameListData *cx_frame) static av_cold void free_coded_frame(struct FrameListData *cx_frame) { av_freep(&cx_frame->buf); - if (cx_frame->buf_alpha) - av_freep(&cx_frame->buf_alpha); av_freep(&cx_frame); } @@ -446,6 +443,7 @@ static av_cold int vpx_free(AVCodecContext *avctx) av_freep(&ctx->twopass_stats.buf); av_freep(&avctx->stats_out); free_frame_list(ctx->coded_frame_list); + free_frame_list(ctx->alpha_coded_frame_list); if (ctx->hdr10_plus_fifo) free_hdr10_plus_fifo(&ctx->hdr10_plus_fifo); return 0; @@ -1205,7 +1203,6 @@ static av_cold int vpx_init(AVCodecContext *avctx, static inline void cx_pktcpy(struct FrameListData *dst, const struct vpx_codec_cx_pkt *src, - const struct vpx_codec_cx_pkt *src_alpha, VPxContext *ctx) { dst->pts = src->data.frame.pts; @@ -1229,13 +1226,6 @@ static inline void cx_pktcpy(struct FrameListData *dst, } else { dst->frame_number = -1; /* sanity marker */ } - if (src_alpha) { - dst->buf_alpha = src_alpha->data.frame.buf; - dst->sz_alpha = src_alpha->data.frame.sz; - } else { - dst->buf_alpha = NULL; - dst->sz_alpha = 0; - } } /** @@ -1246,7 +1236,7 @@ static inline void cx_pktcpy(struct FrameListData *dst, * @return a negative AVERROR on error */ static int storeframe(AVCodecContext *avctx, struct FrameListData *cx_frame, - AVPacket *pkt) + struct FrameListData *alpha_cx_frame, AVPacket *pkt) { VPxContext *ctx = avctx->priv_data; int ret = ff_get_encode_buffer(avctx, pkt, cx_frame->sz, 0); @@ -1279,16 +1269,16 @@ static int storeframe(AVCodecContext *avctx, struct FrameListData *cx_frame, avctx->error[i] += cx_frame->sse[i + 1]; cx_frame->have_sse = 0; } - if (cx_frame->sz_alpha > 0) { + if (alpha_cx_frame) { side_data = av_packet_new_side_data(pkt, AV_PKT_DATA_MATROSKA_BLOCKADDITIONAL, - cx_frame->sz_alpha + 8); + alpha_cx_frame->sz + 8); if (!side_data) { av_packet_unref(pkt); return AVERROR(ENOMEM); } AV_WB64(side_data, 1); - memcpy(side_data + 8, cx_frame->buf_alpha, cx_frame->sz_alpha); + memcpy(side_data + 8, alpha_cx_frame->buf, alpha_cx_frame->sz); } if (cx_frame->frame_number != -1) { if (ctx->hdr10_plus_fifo) { @@ -1309,40 +1299,37 @@ static int storeframe(AVCodecContext *avctx, struct FrameListData *cx_frame, * @return AVERROR(EINVAL) on output size error * @return AVERROR(ENOMEM) on coded frame queue data allocation error */ -static int queue_frames(AVCodecContext *avctx, AVPacket *pkt_out) +static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, + struct FrameListData **frame_list, AVPacket *pkt_out) { VPxContext *ctx = avctx->priv_data; const struct vpx_codec_cx_pkt *pkt; - const struct vpx_codec_cx_pkt *pkt_alpha = NULL; const void *iter = NULL; - const void *iter_alpha = NULL; int size = 0; - if (ctx->coded_frame_list) { - struct FrameListData *cx_frame = ctx->coded_frame_list; + if (!ctx->is_alpha && *frame_list) { + struct FrameListData *cx_frame = *frame_list; /* return the leading frame if we've already begun queueing */ - size = storeframe(avctx, cx_frame, pkt_out); + size = storeframe(avctx, cx_frame, NULL, pkt_out); if (size < 0) return size; - ctx->coded_frame_list = cx_frame->next; + *frame_list = cx_frame->next; free_coded_frame(cx_frame); } /* consume all available output from the encoder before returning. buffers are only good through the next vpx_codec call */ - while ((pkt = vpx_codec_get_cx_data(&ctx->encoder, &iter)) && - (!ctx->is_alpha || - (pkt_alpha = vpx_codec_get_cx_data(&ctx->encoder_alpha, &iter_alpha)))) { + while (pkt = vpx_codec_get_cx_data(encoder, &iter)) { switch (pkt->kind) { case VPX_CODEC_CX_FRAME_PKT: - if (!size) { + if (!ctx->is_alpha && !size) { struct FrameListData cx_frame; /* avoid storing the frame when the list is empty and we haven't yet provided a frame for output */ av_assert0(!ctx->coded_frame_list); - cx_pktcpy(&cx_frame, pkt, pkt_alpha, ctx); - size = storeframe(avctx, &cx_frame, pkt_out); + cx_pktcpy(&cx_frame, pkt, ctx); + size = storeframe(avctx, &cx_frame, NULL, pkt_out); if (size < 0) return size; } else { @@ -1353,7 +1340,7 @@ static int queue_frames(AVCodecContext *avctx, AVPacket *pkt_out) "Frame queue element alloc failed\n"); return AVERROR(ENOMEM); } - cx_pktcpy(cx_frame, pkt, pkt_alpha, ctx); + cx_pktcpy(cx_frame, pkt, ctx); cx_frame->buf = av_malloc(cx_frame->sz); if (!cx_frame->buf) { @@ -1364,23 +1351,14 @@ static int queue_frames(AVCodecContext *avctx, AVPacket *pkt_out) return AVERROR(ENOMEM); } memcpy(cx_frame->buf, pkt->data.frame.buf, pkt->data.frame.sz); - if (ctx->is_alpha) { - cx_frame->buf_alpha = av_malloc(cx_frame->sz_alpha); - if (!cx_frame->buf_alpha) { - av_log(avctx, AV_LOG_ERROR, - "Data buffer alloc (%"SIZE_SPECIFIER" bytes) failed\n", - cx_frame->sz_alpha); - av_free(cx_frame); - return AVERROR(ENOMEM); - } - memcpy(cx_frame->buf_alpha, pkt_alpha->data.frame.buf, pkt_alpha->data.frame.sz); - } - coded_frame_add(&ctx->coded_frame_list, cx_frame); + coded_frame_add(frame_list, cx_frame); } break; case VPX_CODEC_STATS_PKT: { struct vpx_fixed_buf *stats = &ctx->twopass_stats; int err; + if (!pkt_out) + break; if ((err = av_reallocp(&stats->buf, stats->sz + pkt->data.twopass_stats.sz)) < 0) { @@ -1394,6 +1372,8 @@ static int queue_frames(AVCodecContext *avctx, AVPacket *pkt_out) break; } case VPX_CODEC_PSNR_PKT: + if (!pkt_out) + break; av_assert0(!ctx->have_sse); ctx->sse[0] = pkt->data.psnr.sse[0]; ctx->sse[1] = pkt->data.psnr.sse[1]; @@ -1788,7 +1768,24 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket *pkt, } } - coded_size = queue_frames(avctx, pkt); + coded_size = queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, pkt); + if (ctx->is_alpha) { + queue_frames(avctx, &ctx->encoder_alpha, &ctx->alpha_coded_frame_list, NULL); + + if (ctx->coded_frame_list && ctx->alpha_coded_frame_list) { + struct FrameListData *cx_frame = ctx->coded_frame_list; + struct FrameListData *alpha_cx_frame = ctx->alpha_coded_frame_list; + av_assert0(!coded_size); + /* return the leading frame if we've already begun queueing */ + coded_size = storeframe(avctx, cx_frame, alpha_cx_frame, pkt); + if (coded_size < 0) + return coded_size; + ctx->coded_frame_list = cx_frame->next; + ctx->alpha_coded_frame_list = alpha_cx_frame->next; + free_coded_frame(cx_frame); + free_coded_frame(alpha_cx_frame); + } + } if (!frame && avctx->flags & AV_CODEC_FLAG_PASS1) { unsigned int b64_size = AV_BASE64_SIZE(ctx->twopass_stats.sz);