From patchwork Sun Sep 13 10:26:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 22334 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E48AA4495BC for ; Sun, 13 Sep 2020 13:49:43 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C61FF68BBEF; Sun, 13 Sep 2020 13:49:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2337168BA8F for ; Sun, 13 Sep 2020 13:49:38 +0300 (EEST) Received: by mail-ed1-f47.google.com with SMTP id a12so14734093eds.13 for ; Sun, 13 Sep 2020 03:49:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ApqY2GFm7cLyGPwYdGkrLON6omuMlw0vrZ9e2pLJyGM=; b=pVfqi3YS4Dl9pCkEP57Ebq/zQ5Dl8ExllEFoY9agfu2E44VVauv/8+G14ihumQlbKI uUjuxDq8hh3mmSkhVzhW1FMpZcVp5SAWUHvbElrG9JKQdCZDUFditGhywI3aYSlOuH4h 7WralKjbJLYvjtXP76u71hl/6RMePM/sI+aaPKvEgqYOEpqu8Le7Xzucc/orIQRE5ij+ 2POxsw8VlvKFrswUiXnkJ8IYgykBIxp3+kXsL3ZiYiBHlTtvvb4LZxRleTP5oLD2uHLX mqQjrKCDqiDIdM+prktdWhrncjisoigQbc5lebdZAb9IU6l87SdOsPOoJizM0dJoJsbI oKrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ApqY2GFm7cLyGPwYdGkrLON6omuMlw0vrZ9e2pLJyGM=; b=pA50tOVn1Qj7+wbM/Fa0K0M/9RN4rxkWEyEiXlm+YP1428QS23m7zfKLBnQ1b8GOO+ fD6Mogx2S37tYtkpwJmM3uANVa8ZHxKFI1eRdzZ0NCJIOiJ/BMPCA+HEdyGeJjlJlmoz oMDDK0pya2gNeLUE/04jQ1Zry3sME1ZdIoNG6rEq9xcuuyDdRAjJn3g+2XbfVNsKZpm8 BGa6/OU+JOD1J/piVqzpqYZOitcnT20hi/VC4Udx+8CrueSBa+8LK6kAem/mmVwrGZae 4xFx3fZR5+cAcfQws29s9U9UAy4eKQy4/eVuSRLQZPjHWLaq/KrxrqErMK8jTCYnJxTs GqpA== X-Gm-Message-State: AOAM533JX8IMbPbo1agREIGBGBTvYLgqzeoEdFtPH1lq0yCu9dcyyMDW vhAdGdcs4ihh+ejhPPo0JBqIqPZH/nk= X-Google-Smtp-Source: ABdhPJzVxnyhNZVsQo0YcAs5WxMLp4byOoaCFmwtMad2XDrREUYtbxBKGo8luM4uc8TY0m+JiyddXg== X-Received: by 2002:a05:651c:484:: with SMTP id s4mr3630703ljc.391.1599992785702; Sun, 13 Sep 2020 03:26:25 -0700 (PDT) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id n3sm2674790ljj.59.2020.09.13.03.26.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 13 Sep 2020 03:26:25 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Sun, 13 Sep 2020 13:26:19 +0300 Message-Id: <20200913102622.168011-2-jeebjp@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200913102622.168011-1-jeebjp@gmail.com> References: <20200913102622.168011-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/4] ffmpeg: deduplicate init_output_stream usage logic X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Adds a wrapper function, which handles any errors depending on how fatal a failure would be. --- fftools/ffmpeg.c | 51 ++++++++++++++++++++++++------------------------ 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 84306818a2..cb7644de6a 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -1392,6 +1392,26 @@ static void do_video_stats(OutputStream *ost, int frame_size) static int init_output_stream(OutputStream *ost, char *error, int error_len); +static int init_output_stream_wrapper(OutputStream *ost, unsigned int fatal) +{ + int ret = AVERROR_BUG; + char error[1024] = {0}; + + if (ost->initialized) + return 0; + + ret = init_output_stream(ost, error, sizeof(error)); + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", + ost->file_index, ost->index, error); + + if (fatal) + exit_program(1); + } + + return ret; +} + static void finish_output_stream(OutputStream *ost) { OutputFile *of = output_files[ost->file_index]; @@ -1428,15 +1448,7 @@ static int reap_filters(int flush) continue; filter = ost->filter->filter; - if (!ost->initialized) { - char error[1024] = ""; - ret = init_output_stream(ost, error, sizeof(error)); - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", - ost->file_index, ost->index, error); - exit_program(1); - } - } + init_output_stream_wrapper(ost, 1); if (!ost->filtered_frame && !(ost->filtered_frame = av_frame_alloc())) { return AVERROR(ENOMEM); @@ -1860,7 +1872,6 @@ static void flush_encoders(void) // Maybe we should just let encoding fail instead. if (!ost->initialized) { FilterGraph *fg = ost->filter->graph; - char error[1024] = ""; av_log(NULL, AV_LOG_WARNING, "Finishing stream %d:%d without any data written to it.\n", @@ -1886,12 +1897,7 @@ static void flush_encoders(void) finish_output_stream(ost); } - ret = init_output_stream(ost, error, sizeof(error)); - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", - ost->file_index, ost->index, error); - exit_program(1); - } + init_output_stream_wrapper(ost, 1); } if (enc->codec_type != AVMEDIA_TYPE_VIDEO && enc->codec_type != AVMEDIA_TYPE_AUDIO) @@ -3669,7 +3675,7 @@ static int transcode_init(void) if (output_streams[i]->filter) continue; - ret = init_output_stream(output_streams[i], error, sizeof(error)); + ret = init_output_stream_wrapper(output_streams[i], 0); if (ret < 0) goto dump_format; } @@ -4580,15 +4586,8 @@ static int transcode_step(void) } if (ost->filter && ost->filter->graph->graph) { - if (!ost->initialized) { - char error[1024] = {0}; - ret = init_output_stream(ost, error, sizeof(error)); - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", - ost->file_index, ost->index, error); - exit_program(1); - } - } + init_output_stream_wrapper(ost, 1); + if ((ret = transcode_from_filter(ost->filter->graph, &ist)) < 0) return ret; if (!ist) From patchwork Sun Sep 13 10:26:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 22333 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 9B98E44B525 for ; Sun, 13 Sep 2020 13:34:31 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7A71268BBDE; Sun, 13 Sep 2020 13:34:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3CB6268BBB6 for ; Sun, 13 Sep 2020 13:34:24 +0300 (EEST) Received: by mail-wr1-f67.google.com with SMTP id w5so15557860wrp.8 for ; Sun, 13 Sep 2020 03:34:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=xFYAcJhkV718VcfZUdwhm+3AZWsZPZMIv+8l7ql/1vc=; b=C53XyPDMAFuB3SJ5A3pcQ2piG107wTfUlb+ZAG7QoUZaoV3TCNf/EKloxoEiC9miX4 n0cfxRJc+ZskYp0Z9NsOKhv9q01gwY5lCcY9L96UChBvdx9hyn2V5W/VN6z3LEBr/zJM oPCAmF7k6yPeGZNyNVGsdqLhYTftAt9ljNc4xzc8drL0yyrPw52RdN6u8T8WuImEcJ+g dwVjJ8ZL+lP/BCKxRt4eHrd2haEFrvw77eA9Ic3eXDgKf8XPG7ILoMO3JYTybqCtPJ2s VH8W8mJhGfoX6ZM7KChMRAI+2hVWGNiXoPugihRYVMuOL4Tk4VVGBcugujxQfVzt0xeN nNlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xFYAcJhkV718VcfZUdwhm+3AZWsZPZMIv+8l7ql/1vc=; b=nCMyFABvX41NmxmEkZ7j6h3pHQsYKQl4Iu0AENKHjQudeFiVDk8Qp+XRpUmArljqD/ tf0OUQ2hICq4KkEs0dxo9n/NKv4Lm8XG/imZ1pdjYQGTN7BpHLlkstk9dGHcG0zWBDh3 seA+/Ipun/A0jmmA++jv/oVbri33Li4z0yV8Eml2AoevBdbsOKFx5Y5IpcIINljsrgmN AbzSAHlWlijXVRWiaFr2V4r6P6GLCcCL1hZH5lIJYGOQBOC4XkACN2ogZT/CAcEWYlBh 3NDR88+sfCW29DIp0qrz1QxPoHhy7QPQFicBihHVHFQpZK/8vOJKRMC2vKVJYGbNQqoP Czeg== X-Gm-Message-State: AOAM5312WC+hGjvJB1dCzcw4GX407eS4OgLnB1Pd5pr+98uYc4n0m4wl tLY8O/EeIqlA18etN6e2Lzr0sByXJZE= X-Google-Smtp-Source: ABdhPJwfbeectil4VmEIHOcb51DYYWb2+D8w1lSc2eqlc026hb2hbumPWh8dszQpHW7bzKrpc9Zjfg== X-Received: by 2002:a19:7e92:: with SMTP id z140mr3133922lfc.299.1599992786660; Sun, 13 Sep 2020 03:26:26 -0700 (PDT) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id n3sm2674790ljj.59.2020.09.13.03.26.25 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 13 Sep 2020 03:26:26 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Sun, 13 Sep 2020 13:26:20 +0300 Message-Id: <20200913102622.168011-3-jeebjp@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200913102622.168011-1-jeebjp@gmail.com> References: <20200913102622.168011-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/4] ffmpeg: move AVFrame time base adjustment into a function X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This will have to be called later for video down the line. --- fftools/ffmpeg.c | 77 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 52 insertions(+), 25 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index cb7644de6a..7e6c0a962b 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -897,6 +897,55 @@ static int check_recording_time(OutputStream *ost) return 1; } +static double adjust_frame_pts_to_encoder_tb(OutputFile *of, OutputStream *ost, + AVFrame *frame) +{ + if (!frame || frame->pts == AV_NOPTS_VALUE || + !ost->filter || !ost->filter->graph->graph) + return AV_NOPTS_VALUE; + + double float_pts = AV_NOPTS_VALUE; // this is identical to frame.pts but with higher precision + AVFilterContext *filter = ost->filter->filter; + AVCodecContext *enc = ost->enc_ctx; + + int64_t start_time = (of->start_time == AV_NOPTS_VALUE) ? 0 : of->start_time; + AVRational filter_tb = av_buffersink_get_time_base(filter); + AVRational tb = enc->time_base; + int extra_bits = av_clip(29 - av_log2(tb.den), 0, 16); + + tb.den <<= extra_bits; + av_log(NULL, AV_LOG_VERBOSE, + "%s: frame pts: %"PRId64" start_time: %"PRId64", " + "filter_tb: %d/%d, tb: %d/%d\n", + __FUNCTION__, frame->pts, start_time, + filter_tb.num, filter_tb.den, + tb.num, tb.den); + + float_pts = + av_rescale_q(frame->pts, filter_tb, tb) - + av_rescale_q(start_time, AV_TIME_BASE_Q, tb); + float_pts /= 1 << extra_bits; + // avoid exact midoints to reduce the chance of rounding differences, this can be removed in case the fps code is changed to work with integers + float_pts += FFSIGN(float_pts) * 1.0 / (1<<17); + + frame->pts = + av_rescale_q(frame->pts, filter_tb, enc->time_base) - + av_rescale_q(start_time, AV_TIME_BASE_Q, enc->time_base); + + av_log(NULL, AV_LOG_VERBOSE, + "%s: post-adjustment PTS: %"PRId64"\n", + __FUNCTION__, frame->pts); + + if (debug_ts) { + av_log(NULL, AV_LOG_INFO, "filter -> pts:%s pts_time:%s exact:%f time_base:%d/%d\n", + av_ts2str(frame->pts), av_ts2timestr(frame->pts, &enc->time_base), + float_pts, + enc->time_base.num, enc->time_base.den); + } + + return float_pts; +} + static void do_audio_out(OutputFile *of, OutputStream *ost, AVFrame *frame) { @@ -1473,37 +1522,15 @@ static int reap_filters(int flush) av_frame_unref(filtered_frame); continue; } - if (filtered_frame->pts != AV_NOPTS_VALUE) { - int64_t start_time = (of->start_time == AV_NOPTS_VALUE) ? 0 : of->start_time; - AVRational filter_tb = av_buffersink_get_time_base(filter); - AVRational tb = enc->time_base; - int extra_bits = av_clip(29 - av_log2(tb.den), 0, 16); - - tb.den <<= extra_bits; - float_pts = - av_rescale_q(filtered_frame->pts, filter_tb, tb) - - av_rescale_q(start_time, AV_TIME_BASE_Q, tb); - float_pts /= 1 << extra_bits; - // avoid exact midoints to reduce the chance of rounding differences, this can be removed in case the fps code is changed to work with integers - float_pts += FFSIGN(float_pts) * 1.0 / (1<<17); - - filtered_frame->pts = - av_rescale_q(filtered_frame->pts, filter_tb, enc->time_base) - - av_rescale_q(start_time, AV_TIME_BASE_Q, enc->time_base); - } + + float_pts = adjust_frame_pts_to_encoder_tb(of, ost, + filtered_frame); switch (av_buffersink_get_type(filter)) { case AVMEDIA_TYPE_VIDEO: if (!ost->frame_aspect_ratio.num) enc->sample_aspect_ratio = filtered_frame->sample_aspect_ratio; - if (debug_ts) { - av_log(NULL, AV_LOG_INFO, "filter -> pts:%s pts_time:%s exact:%f time_base:%d/%d\n", - av_ts2str(filtered_frame->pts), av_ts2timestr(filtered_frame->pts, &enc->time_base), - float_pts, - enc->time_base.num, enc->time_base.den); - } - do_video_out(of, ost, filtered_frame, float_pts); break; case AVMEDIA_TYPE_AUDIO: From patchwork Sun Sep 13 10:26:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 22331 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4FFB6441480 for ; Sun, 13 Sep 2020 13:33:07 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 214AE68BBF1; Sun, 13 Sep 2020 13:33:07 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f68.google.com (mail-ed1-f68.google.com [209.85.208.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EAEEC68BBDE for ; Sun, 13 Sep 2020 13:32:59 +0300 (EEST) Received: by mail-ed1-f68.google.com with SMTP id c8so14758708edv.5 for ; Sun, 13 Sep 2020 03:32:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=cG0fRhxBm4nJyjHujZpI1sEMWl3YAy+hZi6lqd9FM6A=; b=j4L7Jfu76JrFQcHC1ZEDs/dzT8Yx+SGG1cIbmlSBgs+HN20r9Cqy6ewyMWisaucfpL fhNPQm12KuWaZyGnU25h3K70a9E/EZUTqmnjyeF4JoS4kz43Mw0JzPGpxkyk7D/hco3p xQ2tyds6IuBAShKMJF6Y+/vCpff1sVdBDydwbBtPKO1OOQ33LVoN9mjc45C6IgqUVp4F XPvcGwAKHFsTGCfecJKp8h0rqpgjuVAiX/9nbqPXM/l1iTYzEdHwLH71gW4x3+oonmG7 9/nYluRPGWhe7mCJ9IWJ4/CbETgGRcQ54Bfgaqfv4VJ4EsdiigDDd3cPB87kmr/wjd3c sn3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cG0fRhxBm4nJyjHujZpI1sEMWl3YAy+hZi6lqd9FM6A=; b=LAcrVOYEQXzP06XWsarZ7bLyIrsmsOw1s4QbQ84GL1tVdTsVfxjf/FdVZxppd5l/K8 QdPTFLZTYB+Mjug3v1q0EfN/fQfluygrATzftvrzhgiIQW7r2hCIDgQq1n0oDEOHz3NF Jz8+SIJ5aDy/yk//PsztdwYjZTPrcZLDzb5bqh65ZPjciI8aOwI7QK+v0WpaXnbf06VS s/jyIU1pNiIdjLlL2mw39/7anAMTqwSb1/S5aZS2lOmbFzzQAHwCLwV1YZZRZDJYFBQx RlEA+uPf0zOLHJEgcrqPZGLICrxEsq3V3eR468i14z4TABk9+ZDDlgv8zJzEMI132sL9 bKxw== X-Gm-Message-State: AOAM532jceOZJc4dRvHsJRGnv5BlnAYOBLcbEClMIo61fysp4EyyI/Oz 94vAZe+UXfZNfE4vUL3TukiJV3kHHL0= X-Google-Smtp-Source: ABdhPJzexGL7rKs7cn+6K/aZJpkyDKOHELWGrisDr+bMike0LdWCKWiMmJZXcUypPXBxCBEsmas14A== X-Received: by 2002:a19:c18d:: with SMTP id r135mr3555615lff.160.1599992787584; Sun, 13 Sep 2020 03:26:27 -0700 (PDT) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id n3sm2674790ljj.59.2020.09.13.03.26.26 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 13 Sep 2020 03:26:26 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Sun, 13 Sep 2020 13:26:21 +0300 Message-Id: <20200913102622.168011-4-jeebjp@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200913102622.168011-1-jeebjp@gmail.com> References: <20200913102622.168011-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/4] ffmpeg: move A/V non-streamcopy initialization to a later point X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" - For video, this means a single initialization point in do_video_out. - For audio we unfortunately need to do it in two places just before the buffer sink is utilized (if av_buffersink_get_samples would still work according to its specification after a call to avfilter_graph_request_oldest was made, we could at least remove the one in transcode_step). Other adjustments to make things work: - As the AVFrame PTS adjustment to encoder time base needs the encoder to be initialized, so it is now moved to do_{video,audio}_out, right after the encoder has been initialized. Due to this, the additional parameter in do_video_out is removed as it is no longer necessary. --- fftools/ffmpeg.c | 111 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 76 insertions(+), 35 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 7e6c0a962b..66d7da695a 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -946,6 +946,28 @@ static double adjust_frame_pts_to_encoder_tb(OutputFile *of, OutputStream *ost, return float_pts; } +static int init_output_stream(OutputStream *ost, char *error, int error_len); + +static int init_output_stream_wrapper(OutputStream *ost, unsigned int fatal) +{ + int ret = AVERROR_BUG; + char error[1024] = {0}; + + if (ost->initialized) + return 0; + + ret = init_output_stream(ost, error, sizeof(error)); + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", + ost->file_index, ost->index, error); + + if (fatal) + exit_program(1); + } + + return ret; +} + static void do_audio_out(OutputFile *of, OutputStream *ost, AVFrame *frame) { @@ -957,6 +979,8 @@ static void do_audio_out(OutputFile *of, OutputStream *ost, pkt.data = NULL; pkt.size = 0; + adjust_frame_pts_to_encoder_tb(of, ost, frame); + if (!check_recording_time(ost)) return; @@ -1091,8 +1115,7 @@ static void do_subtitle_out(OutputFile *of, static void do_video_out(OutputFile *of, OutputStream *ost, - AVFrame *next_picture, - double sync_ipts) + AVFrame *next_picture) { int ret, format_video_sync; AVPacket pkt; @@ -1102,10 +1125,14 @@ static void do_video_out(OutputFile *of, int nb_frames, nb0_frames, i; double delta, delta0; double duration = 0; + double sync_ipts = AV_NOPTS_VALUE; int frame_size = 0; InputStream *ist = NULL; AVFilterContext *filter = ost->filter->filter; + init_output_stream_wrapper(ost, 1); + sync_ipts = adjust_frame_pts_to_encoder_tb(of, ost, next_picture); + if (ost->source_index >= 0) ist = input_streams[ost->source_index]; @@ -1439,28 +1466,6 @@ static void do_video_stats(OutputStream *ost, int frame_size) } } -static int init_output_stream(OutputStream *ost, char *error, int error_len); - -static int init_output_stream_wrapper(OutputStream *ost, unsigned int fatal) -{ - int ret = AVERROR_BUG; - char error[1024] = {0}; - - if (ost->initialized) - return 0; - - ret = init_output_stream(ost, error, sizeof(error)); - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", - ost->file_index, ost->index, error); - - if (fatal) - exit_program(1); - } - - return ret; -} - static void finish_output_stream(OutputStream *ost) { OutputFile *of = output_files[ost->file_index]; @@ -1497,7 +1502,17 @@ static int reap_filters(int flush) continue; filter = ost->filter->filter; - init_output_stream_wrapper(ost, 1); + /* + * Unlike video, with audio the audio frame size matters. + * Currently we are fully reliant on the lavfi filter chain to + * do the buffering deed for us, and thus the frame size parameter + * needs to be set accordingly. Where does one get the required + * frame size? From the initialized AVCodecContext of an audio + * encoder. Thus, if we have gotten to an audio stream, initialize + * the encoder earlier than receiving the first AVFrame. + */ + if (av_buffersink_get_type(filter) == AVMEDIA_TYPE_AUDIO) + init_output_stream_wrapper(ost, 1); if (!ost->filtered_frame && !(ost->filtered_frame = av_frame_alloc())) { return AVERROR(ENOMEM); @@ -1505,7 +1520,6 @@ static int reap_filters(int flush) filtered_frame = ost->filtered_frame; while (1) { - double float_pts = AV_NOPTS_VALUE; // this is identical to filtered_frame.pts but with higher precision ret = av_buffersink_get_frame_flags(filter, filtered_frame, AV_BUFFERSINK_FLAG_NO_REQUEST); if (ret < 0) { @@ -1514,7 +1528,7 @@ static int reap_filters(int flush) "Error in av_buffersink_get_frame_flags(): %s\n", av_err2str(ret)); } else if (flush && ret == AVERROR_EOF) { if (av_buffersink_get_type(filter) == AVMEDIA_TYPE_VIDEO) - do_video_out(of, ost, NULL, AV_NOPTS_VALUE); + do_video_out(of, ost, NULL); } break; } @@ -1523,15 +1537,12 @@ static int reap_filters(int flush) continue; } - float_pts = adjust_frame_pts_to_encoder_tb(of, ost, - filtered_frame); - switch (av_buffersink_get_type(filter)) { case AVMEDIA_TYPE_VIDEO: if (!ost->frame_aspect_ratio.num) enc->sample_aspect_ratio = filtered_frame->sample_aspect_ratio; - do_video_out(of, ost, filtered_frame, float_pts); + do_video_out(of, ost, filtered_frame); break; case AVMEDIA_TYPE_AUDIO: if (!(enc->codec->capabilities & AV_CODEC_CAP_PARAM_CHANGE) && @@ -3696,10 +3707,19 @@ static int transcode_init(void) goto dump_format; } - /* open each encoder */ + /* + * initialize stream copy and subtitle/data streams. + * Encoded AVFrame based streams will get initialized as follows: + * - when the first AVFrame is received in do_video_out + * - just before the first AVFrame is received in either transcode_step + * or reap_filters due to us requiring the filter chain buffer sink + * to be configured with the correct audio frame size, which is only + * known after the encoder is initialized. + */ for (i = 0; i < nb_output_streams; i++) { - // skip streams fed from filtergraphs until we have a frame for them - if (output_streams[i]->filter) + if (!output_streams[i]->stream_copy && + (output_streams[i]->enc_ctx->codec_type == AVMEDIA_TYPE_VIDEO || + output_streams[i]->enc_ctx->codec_type == AVMEDIA_TYPE_AUDIO)) continue; ret = init_output_stream_wrapper(output_streams[i], 0); @@ -4613,7 +4633,28 @@ static int transcode_step(void) } if (ost->filter && ost->filter->graph->graph) { - init_output_stream_wrapper(ost, 1); + /* + * Similar case to the early audio initialization in reap_filters. + * Audio is special in ffmpeg.c currently as we depend on lavfi's + * audio frame buffering/creation to get the output audio frame size + * in samples correct. The audio frame size for the filter chain is + * configured during the output stream initialization. + * + * Apparently avfilter_graph_request_oldest (called in + * transcode_from_filter just down the line) peeks. Peeking already + * puts one frame "ready to be given out", which means that any + * update in filter buffer sink configuration afterwards will not + * help us. + * + * And yes, even av_buffersink_get_samples is affected, + * As it internally utilizes the same early exit for peeked frames. + * In other words, if either av_buffersink_get_samples with + * avfilter_graph_request_oldest will start playing ball, or we add + * our own audio buffering to handle frame size mismatches, both of + * these early exits can be gotten rid of. + */ + if (av_buffersink_get_type(ost->filter->filter) == AVMEDIA_TYPE_AUDIO) + init_output_stream_wrapper(ost, 1); if ((ret = transcode_from_filter(ost->filter->graph, &ist)) < 0) return ret; From patchwork Sun Sep 13 10:26:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 22335 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 47DE844B525 for ; Sun, 13 Sep 2020 13:54:19 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1A65768BBF6; Sun, 13 Sep 2020 13:54:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7C76C68A3AF for ; Sun, 13 Sep 2020 13:54:12 +0300 (EEST) Received: by mail-ej1-f41.google.com with SMTP id lo4so19231824ejb.8 for ; Sun, 13 Sep 2020 03:54:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=sOy1QMl8A8YXoOClgcfT6P3i24inWmpAXE0njYjNCx4=; b=e2diIwl2j1GeFomhjdM9sVrVF5D4JLEVkCqTxzq0nN96GVpItLCfReEu9tE52AKhtw cofOucGl8C+JoJZcP06x0b1F2+6rTXcMCK9vCEitg/SWczYqu6vF2uCh1eQ4w6Ajfi3V aZ2O62njfnxz3WTQO6arCWiVaCMC/cA2IG4kgugRuVh9p0/dUM8TB3HqD9sQlqLJIEMd VMVO6/8Ny2TMGhXuIHlTjVoV9BS9ocnRtvKiHb32pRsi0FIk5WeolkZmEb39wFhRXXYK yKh20cvrD4xKSHDB128zwN3aZGQkzhVkhZ2ku8ofPITB+X50G9vxKXu0LMGBJSaliCgt 3Yqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sOy1QMl8A8YXoOClgcfT6P3i24inWmpAXE0njYjNCx4=; b=mGWl7+1IcBLS6eh+ImFoUPiziuch3syyCDHGQXodeH9sAvq0wHTJZn9sW7Pe64fBs+ NF5fqgeoyn2VLlNxIGOC8MTnqHMdkVryxZCYS+/h+mvuQE5m6puD5RdAtc+uvckLtA8K BNIOQ7Wq1m+0yLI1iBGAtvW3z8inQ8u4QEC7xu227/c4As7LSGqDAH6qpG5Me5A+DG0h IeHapVnPcs3zLIFiSgDOUSBgME1tHUeM9xufSt3avLIAVcagkinXCnuyhD18zbmLRS42 vY6Mi9c4Qzfs75mkoYgiT1AN8wBcOdi8nMxS8Sv8vVnSb0a/X4f/NX9fhtm5e7cyDi0k xVbQ== X-Gm-Message-State: AOAM533YFbgzuXcFxZMJylbu/agJmmbsvHRoF+t/I+O1jguu9u7wUVph Y/fViUOU31Z1KQd5WTx/n2e63UFhjtE= X-Google-Smtp-Source: ABdhPJylCWVxLFAcskXNzvCqqHhLKY05lc3C/fhXe8Q6Z+It7SRnMfDiQdHuyeRxU5FCBXWFe6jc+w== X-Received: by 2002:a19:ed17:: with SMTP id y23mr3139690lfy.595.1599992788624; Sun, 13 Sep 2020 03:26:28 -0700 (PDT) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id n3sm2674790ljj.59.2020.09.13.03.26.27 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 13 Sep 2020 03:26:27 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Sun, 13 Sep 2020 13:26:22 +0300 Message-Id: <20200913102622.168011-5-jeebjp@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200913102622.168011-1-jeebjp@gmail.com> References: <20200913102622.168011-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/4] ffmpeg: pass decoded or filtered AVFrame to output stream initialization X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Additionally, reap the first rewards by being able to set the color related encoding values based on the passed AVFrame. The only tests that seem to have changed their results with this change seem to be the MXF tests. There, the muxer writes the limited/full range flag to the output container if the encoder is not set to "unspecified". --- fftools/ffmpeg.c | 42 +++++++++++++++++++++++++++---------- tests/ref/lavf/mxf_d10 | 2 +- tests/ref/lavf/mxf_dv25 | 2 +- tests/ref/lavf/mxf_dvcpro50 | 2 +- tests/ref/lavf/mxf_opatom | 2 +- 5 files changed, 35 insertions(+), 15 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 66d7da695a..5425ba245d 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -946,9 +946,11 @@ static double adjust_frame_pts_to_encoder_tb(OutputFile *of, OutputStream *ost, return float_pts; } -static int init_output_stream(OutputStream *ost, char *error, int error_len); +static int init_output_stream(OutputStream *ost, AVFrame *frame, + char *error, int error_len); -static int init_output_stream_wrapper(OutputStream *ost, unsigned int fatal) +static int init_output_stream_wrapper(OutputStream *ost, AVFrame *frame, + unsigned int fatal) { int ret = AVERROR_BUG; char error[1024] = {0}; @@ -956,7 +958,7 @@ static int init_output_stream_wrapper(OutputStream *ost, unsigned int fatal) if (ost->initialized) return 0; - ret = init_output_stream(ost, error, sizeof(error)); + ret = init_output_stream(ost, frame, error, sizeof(error)); if (ret < 0) { av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", ost->file_index, ost->index, error); @@ -1130,7 +1132,7 @@ static void do_video_out(OutputFile *of, InputStream *ist = NULL; AVFilterContext *filter = ost->filter->filter; - init_output_stream_wrapper(ost, 1); + init_output_stream_wrapper(ost, next_picture, 1); sync_ipts = adjust_frame_pts_to_encoder_tb(of, ost, next_picture); if (ost->source_index >= 0) @@ -1512,7 +1514,7 @@ static int reap_filters(int flush) * the encoder earlier than receiving the first AVFrame. */ if (av_buffersink_get_type(filter) == AVMEDIA_TYPE_AUDIO) - init_output_stream_wrapper(ost, 1); + init_output_stream_wrapper(ost, NULL, 1); if (!ost->filtered_frame && !(ost->filtered_frame = av_frame_alloc())) { return AVERROR(ENOMEM); @@ -1935,7 +1937,7 @@ static void flush_encoders(void) finish_output_stream(ost); } - init_output_stream_wrapper(ost, 1); + init_output_stream_wrapper(ost, NULL, 1); } if (enc->codec_type != AVMEDIA_TYPE_VIDEO && enc->codec_type != AVMEDIA_TYPE_AUDIO) @@ -3307,7 +3309,7 @@ static void init_encoder_time_base(OutputStream *ost, AVRational default_time_ba enc_ctx->time_base = default_time_base; } -static int init_output_stream_encode(OutputStream *ost) +static int init_output_stream_encode(OutputStream *ost, AVFrame *frame) { InputStream *ist = get_input_stream(ost); AVCodecContext *enc_ctx = ost->enc_ctx; @@ -3404,6 +3406,23 @@ static int init_output_stream_encode(OutputStream *ost) enc_ctx->bits_per_raw_sample = FFMIN(dec_ctx->bits_per_raw_sample, av_pix_fmt_desc_get(enc_ctx->pix_fmt)->comp[0].depth); + if (frame) { + if (!av_dict_get(ost->encoder_opts, "color_range", NULL, 0)) + enc_ctx->color_range = frame->color_range; + + if (!av_dict_get(ost->encoder_opts, "color_primaries", NULL, 0)) + enc_ctx->color_primaries = frame->color_primaries; + + if (!av_dict_get(ost->encoder_opts, "color_trc", NULL, 0)) + enc_ctx->color_trc = frame->color_trc; + + if (!av_dict_get(ost->encoder_opts, "colorspace", NULL, 0)) + enc_ctx->colorspace = frame->colorspace; + + if (!av_dict_get(ost->encoder_opts, "chroma_sample_location", NULL, 0)) + enc_ctx->chroma_sample_location = frame->chroma_location; + } + enc_ctx->framerate = ost->frame_rate; ost->st->avg_frame_rate = ost->frame_rate; @@ -3461,7 +3480,8 @@ static int init_output_stream_encode(OutputStream *ost) return 0; } -static int init_output_stream(OutputStream *ost, char *error, int error_len) +static int init_output_stream(OutputStream *ost, AVFrame *frame, + char *error, int error_len) { int ret = 0; @@ -3470,7 +3490,7 @@ static int init_output_stream(OutputStream *ost, char *error, int error_len) AVCodecContext *dec = NULL; InputStream *ist; - ret = init_output_stream_encode(ost); + ret = init_output_stream_encode(ost, frame); if (ret < 0) return ret; @@ -3722,7 +3742,7 @@ static int transcode_init(void) output_streams[i]->enc_ctx->codec_type == AVMEDIA_TYPE_AUDIO)) continue; - ret = init_output_stream_wrapper(output_streams[i], 0); + ret = init_output_stream_wrapper(output_streams[i], NULL, 0); if (ret < 0) goto dump_format; } @@ -4654,7 +4674,7 @@ static int transcode_step(void) * these early exits can be gotten rid of. */ if (av_buffersink_get_type(ost->filter->filter) == AVMEDIA_TYPE_AUDIO) - init_output_stream_wrapper(ost, 1); + init_output_stream_wrapper(ost, NULL, 1); if ((ret = transcode_from_filter(ost->filter->graph, &ist)) < 0) return ret; diff --git a/tests/ref/lavf/mxf_d10 b/tests/ref/lavf/mxf_d10 index aea469bb58..85e337d157 100644 --- a/tests/ref/lavf/mxf_d10 +++ b/tests/ref/lavf/mxf_d10 @@ -1,3 +1,3 @@ -e597f73ef9c9819710d2f815813eb91f *tests/data/lavf/lavf.mxf_d10 +36fc2a640368f6d33987d2b2d39df966 *tests/data/lavf/lavf.mxf_d10 5332013 tests/data/lavf/lavf.mxf_d10 tests/data/lavf/lavf.mxf_d10 CRC=0x6c74d488 diff --git a/tests/ref/lavf/mxf_dv25 b/tests/ref/lavf/mxf_dv25 index db6b76c6f8..d4559df862 100644 --- a/tests/ref/lavf/mxf_dv25 +++ b/tests/ref/lavf/mxf_dv25 @@ -1,3 +1,3 @@ -0fc964fa22bc8b3a389b81b9a2efccb3 *tests/data/lavf/lavf.mxf_dv25 +57623b3b968c0bb0d8a0bbaeef6fe657 *tests/data/lavf/lavf.mxf_dv25 3834413 tests/data/lavf/lavf.mxf_dv25 tests/data/lavf/lavf.mxf_dv25 CRC=0xbdaf7f52 diff --git a/tests/ref/lavf/mxf_dvcpro50 b/tests/ref/lavf/mxf_dvcpro50 index 09999914bf..8bcf67d17f 100644 --- a/tests/ref/lavf/mxf_dvcpro50 +++ b/tests/ref/lavf/mxf_dvcpro50 @@ -1,3 +1,3 @@ -aa81ea83af44a69e73849e327cc4bd12 *tests/data/lavf/lavf.mxf_dvcpro50 +6e82b4cc962199e2593e30851ff7ccfb *tests/data/lavf/lavf.mxf_dvcpro50 7431213 tests/data/lavf/lavf.mxf_dvcpro50 tests/data/lavf/lavf.mxf_dvcpro50 CRC=0xe3bbe4b4 diff --git a/tests/ref/lavf/mxf_opatom b/tests/ref/lavf/mxf_opatom index 05794a4e5e..1aa843a22a 100644 --- a/tests/ref/lavf/mxf_opatom +++ b/tests/ref/lavf/mxf_opatom @@ -1,3 +1,3 @@ -06a1816aa91c733e1ef7e45d82e4f1d3 *tests/data/lavf/lavf.mxf_opatom +d5f56215c2b16dee204f03bfa653dd1b *tests/data/lavf/lavf.mxf_opatom 4717625 tests/data/lavf/lavf.mxf_opatom tests/data/lavf/lavf.mxf_opatom CRC=0xf55aa22a