From patchwork Fri Oct 16 13:16:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 23014 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1E3C244B01D for ; Fri, 16 Oct 2020 16:23:24 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0112668BA5E; Fri, 16 Oct 2020 16:23:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3ADFE68BA5E for ; Fri, 16 Oct 2020 16:23:17 +0300 (EEST) Received: by mail-lj1-f177.google.com with SMTP id a4so2384369lji.12 for ; Fri, 16 Oct 2020 06:23:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=mdlEB1IlazveMz0P28818fnHIhs2NGqJFuETBQO/8q8=; b=rC78yh3qFAt8WCwi2jj22rioeeY+9o1Nd31hb4z1vaXMw00JJt8X37UOxFTOM8OTej 18kBRTBJSS6B64a0kf1ZJEPQR/ZXCvUoER0hlrY7FZ6qREVrjBpQCYdhLQzNFRjk4KBl ZtKdmDuX+5V1+90mg3O5agGekxKFb9nBIFHKkNQNTvRI67cq2RBRpC2SI5V8YRP96I6L qWpteNS9oxOsML2vA8hQOFxdwcOXVThBt6FwVDA7oTHX9iu4ivlPEuiCbPV04H2swoGn Dpc8GJ5cdnyGkzm8C/EY8PPfS7wYczum+E69l24GnLJHOJUOnIy/99L8tJkI0CIiV0oJ jzsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=mdlEB1IlazveMz0P28818fnHIhs2NGqJFuETBQO/8q8=; b=aoHAo51w5EFx4+hUUaiFVEWkMT1VEVH6LoHGLDK0b4CstFHS6DKgEF8dHq1VtNOWUj 41rIAjamSePbtGL/FG3t67hhzEVbgORIzrtdgN+K8UUhY0PsHEagi2PasMRM75n7qlAi TN2yh6+0IoLDFYJMSpaynJVfjDgAv9FcQqmbls4z+MtrYzNLYVFQaftb1vM2sLGbrAza oz1B8wXcty37XoFbSZCPPqfqQnnxHyUW7XdfeEM3ceYVwtvvSTJFfM9rggm1JvAd8Ema O7r1yIG9M2CWu/5dFN9Q7CFLoMrpoPT1P5S93/2GPXuF15BCaHOmE0dioEyHLOkJwlxl nvAg== X-Gm-Message-State: AOAM530uIXciPOnAl+Xz5IRcZczaiCTScW5u6jqrnjVxiYT7NwJWemOV cMTjDi3jNB+zrVPF1mh/Vs/Kvsw9HHg= X-Google-Smtp-Source: ABdhPJxARBhqCBiyILIGHKC1CKalJvUdd1dJ/qRXPPRqu9a2K6uEwpoqMvBM1sSHZI30v41DJjz4IA== X-Received: by 2002:a19:78e:: with SMTP id 136mr1222399lfh.389.1602854214284; Fri, 16 Oct 2020 06:16:54 -0700 (PDT) Received: from localhost.localdomain (n89ridqjqdjpsztph-2.v6.elisa-mobile.fi. [2001:999:11:8336:994:7f72:f89e:9125]) by smtp.gmail.com with ESMTPSA id x13sm280594lfe.101.2020.10.16.06.16.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 06:16:53 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Fri, 16 Oct 2020 16:16:43 +0300 Message-Id: <20201016131649.4361-1-jeebjp@gmail.com> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 0/6] ffmpeg: late A/V encoder init, AVFrame metadata usage X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This patch set started with a very simple wish to not have to set color related values manually each time when utilizing ffmpeg.c. As of the third iteration, the following changes were done since the second: 1. A simple mistake was corrected, fixing `debug_ts`. 2. As I noticed such a change enabling a fix for the interlaced flag writing for Y4M, switched the location of the field order and interlaced/progressive logic to where the encoder is initialized. 3. First attempt at fixing cases where the difference between stream copy and re-encoding leads to the muxer queue filling up, breaking cases where a stream with lots of small packets (such as audio) is copied, and a seek ends up multiple seconds before the actual requested seek time. Unfortunately, audio still needs two locations where the encoder is initialized, due to how avfilter_graph_request_oldest peeks and already puts one AVFrame to be available from the filter graph (which is then utilized as-is as an early return inside both av_buffersink_get_frame_flags and av_buffersink_get_samples). If this would be improved in lavfi (or the call to avfilter_graph_request_oldest removed), we could at least remove one of these. Currently limited to using values for video and started with the basic values, more can be added later if needed. This probably fixes some trac issues, but with a quick look I couldn't find anything that explicitly was due to lack of video color metadata passthrough. Jan Example 1: I have an RGB 3-D render, which I would like to encode into BT.709 YCbCr. The video filter I'm generally using for this (zscale) does flag the matrix in the output AVFrame. Yet to have the video encoder have the correct metadata set, I have to set the value(s) manually. With this patch set, the value(s) from the first AVFrame fed to do_video_out will be utilized. Example 2: I have an input video that sets one or more of the following: matrix/primaries/transfer function/range/chroma location. I just want to re-encode it. All of this metadata gets stripped. With this patch set, the value(s) from the first AVFrame fed to do_video_out will be utilized. Example 3: I have a video which has incorrect metadata tagged. Before, I had to set the correct data data manually. With this patch set, since ffmpeg.c takes color related options as dictionary keys, the AVFrame values will only be utilized if the user has not set the option for a given stream. Thus, this use case still works. Jan Ekström (6): ffmpeg: deduplicate init_output_stream usage logic ffmpeg: move AVFrame time base adjustment into a function ffmpeg: move A/V non-streamcopy initialization to a later point ffmpeg: pass decoded or filtered AVFrame to output stream initialization ffmpeg: move field order decision making to encoder initialization ffmpeg: add a data size threshold for muxing queue size doc/ffmpeg.texi | 5 + fftools/ffmpeg.c | 249 ++++++++++++------ fftools/ffmpeg.h | 11 + fftools/ffmpeg_opt.c | 8 + .../fate/concat-demuxer-extended-lavf-mxf_d10 | 2 +- .../fate/concat-demuxer-simple1-lavf-mxf_d10 | 2 +- tests/ref/fate/rgb24-mkv | 4 +- tests/ref/lavf/mxf_d10 | 2 +- tests/ref/lavf/mxf_dv25 | 2 +- tests/ref/lavf/mxf_dvcpro50 | 2 +- tests/ref/lavf/mxf_opatom | 2 +- 11 files changed, 202 insertions(+), 87 deletions(-)