From patchwork Fri Oct 16 13:16:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 23013 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1D0DC44A855 for ; Fri, 16 Oct 2020 16:22:57 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 020A068BA1D; Fri, 16 Oct 2020 16:22:57 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 185A068BA16 for ; Fri, 16 Oct 2020 16:22:50 +0300 (EEST) Received: by mail-ej1-f50.google.com with SMTP id dt13so3092937ejb.12 for ; Fri, 16 Oct 2020 06:22:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ApqY2GFm7cLyGPwYdGkrLON6omuMlw0vrZ9e2pLJyGM=; b=iogkdwcm6bEkDZKBFnOMt82+CYu/8ZpALgJc4rJiZF847kqkF4TmhZI7C3+lFsJ4vf ig36j4AP0hAn52njlOF5aEorIvavdgnN72AxoxVCjbQ+ovkQMeSsTouXFvrGNl2f2dDY HDvVhPVneg+5eCvIeWSeLYGJ0v5b0fypFksoMdTnQl0GfTUzj1QT7J7DPq4shg4sg0PF 05wTxQyANZ20eCUB6m5oRHsJTsEQ1Y+FTPbmzN9bSL87xXBzEaB9GwM1T0S4+reu15Hj oxVW0Yy8pm1XktFBxN4+w44f4uKPh8yK1Wx9Q3YJ722Z1z6LwMy7bjqlK3gLknNZaYJm pJpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ApqY2GFm7cLyGPwYdGkrLON6omuMlw0vrZ9e2pLJyGM=; b=CweYxohWnzH8pOrF1c8ALdXA/eoy0TE61aSTUWW08ryGrrG5Z400YOUuf/jgfNs6+6 CPaAa3FU5tO03dUyXQpQ1szmtoo2R8kL+AU2NtirshIqW4WISehDJXfPv3sfXFtcOsV6 1W+nwRUr+KeMou9qTL77lBTNSc01UmFIk6wUwhFOVBS+D46txWMAnfXE1ypp8kLs/lGH D8jMsurzj2VmFSwHBaa3KIDK8XLkCDn1xIC+JW79LmS+UzRvNKDreGVlATXMRf8+s9iI nE1Zz3T9BrYcqDx93GofeHH73+nOeNMNqVgPuaHoLE7UJArQH5tBW9rzTAx8Os+y68wU h7rQ== X-Gm-Message-State: AOAM533yyIlJszAvfdMknhyFlrfjxkA/GPN5K489blMBHsXkHIQ/9+JT 2AS2rp2Qcy3dsY4UoYLZJcVpa49l9Uk= X-Google-Smtp-Source: ABdhPJy6JV4/cUzos7nN449g1XxRfO/FGf1riNm5iClWYvSbTcywxIG7JsSvLwXTs2Xs9ImKM+oxRQ== X-Received: by 2002:a19:e015:: with SMTP id x21mr1471016lfg.586.1602854215916; Fri, 16 Oct 2020 06:16:55 -0700 (PDT) Received: from localhost.localdomain (n89ridqjqdjpsztph-2.v6.elisa-mobile.fi. [2001:999:11:8336:994:7f72:f89e:9125]) by smtp.gmail.com with ESMTPSA id x13sm280594lfe.101.2020.10.16.06.16.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 06:16:55 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Fri, 16 Oct 2020 16:16:44 +0300 Message-Id: <20201016131649.4361-2-jeebjp@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201016131649.4361-1-jeebjp@gmail.com> References: <20201016131649.4361-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 1/6] ffmpeg: deduplicate init_output_stream usage logic X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Adds a wrapper function, which handles any errors depending on how fatal a failure would be. --- fftools/ffmpeg.c | 51 ++++++++++++++++++++++++------------------------ 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 84306818a2..cb7644de6a 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -1392,6 +1392,26 @@ static void do_video_stats(OutputStream *ost, int frame_size) static int init_output_stream(OutputStream *ost, char *error, int error_len); +static int init_output_stream_wrapper(OutputStream *ost, unsigned int fatal) +{ + int ret = AVERROR_BUG; + char error[1024] = {0}; + + if (ost->initialized) + return 0; + + ret = init_output_stream(ost, error, sizeof(error)); + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", + ost->file_index, ost->index, error); + + if (fatal) + exit_program(1); + } + + return ret; +} + static void finish_output_stream(OutputStream *ost) { OutputFile *of = output_files[ost->file_index]; @@ -1428,15 +1448,7 @@ static int reap_filters(int flush) continue; filter = ost->filter->filter; - if (!ost->initialized) { - char error[1024] = ""; - ret = init_output_stream(ost, error, sizeof(error)); - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", - ost->file_index, ost->index, error); - exit_program(1); - } - } + init_output_stream_wrapper(ost, 1); if (!ost->filtered_frame && !(ost->filtered_frame = av_frame_alloc())) { return AVERROR(ENOMEM); @@ -1860,7 +1872,6 @@ static void flush_encoders(void) // Maybe we should just let encoding fail instead. if (!ost->initialized) { FilterGraph *fg = ost->filter->graph; - char error[1024] = ""; av_log(NULL, AV_LOG_WARNING, "Finishing stream %d:%d without any data written to it.\n", @@ -1886,12 +1897,7 @@ static void flush_encoders(void) finish_output_stream(ost); } - ret = init_output_stream(ost, error, sizeof(error)); - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", - ost->file_index, ost->index, error); - exit_program(1); - } + init_output_stream_wrapper(ost, 1); } if (enc->codec_type != AVMEDIA_TYPE_VIDEO && enc->codec_type != AVMEDIA_TYPE_AUDIO) @@ -3669,7 +3675,7 @@ static int transcode_init(void) if (output_streams[i]->filter) continue; - ret = init_output_stream(output_streams[i], error, sizeof(error)); + ret = init_output_stream_wrapper(output_streams[i], 0); if (ret < 0) goto dump_format; } @@ -4580,15 +4586,8 @@ static int transcode_step(void) } if (ost->filter && ost->filter->graph->graph) { - if (!ost->initialized) { - char error[1024] = {0}; - ret = init_output_stream(ost, error, sizeof(error)); - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", - ost->file_index, ost->index, error); - exit_program(1); - } - } + init_output_stream_wrapper(ost, 1); + if ((ret = transcode_from_filter(ost->filter->graph, &ist)) < 0) return ret; if (!ist) From patchwork Fri Oct 16 13:16:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 23015 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id A43CB44B20D for ; Fri, 16 Oct 2020 16:24:09 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 844AE68BAAC; Fri, 16 Oct 2020 16:24:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f65.google.com (mail-lf1-f65.google.com [209.85.167.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5D17D68B903 for ; Fri, 16 Oct 2020 16:24:02 +0300 (EEST) Received: by mail-lf1-f65.google.com with SMTP id l28so2843480lfp.10 for ; Fri, 16 Oct 2020 06:24:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Np0qDXgaZex9jai3MfKaTNk9A1UTLQqCFBFqKrQv3zo=; b=M46xt/E1a3Med4/YwDk1KDHEZZ/o1RTOPeNUh5VGKl5U55g72P4KjS3iNw1YqsSYlP UdF122ob+VRsroKQhlkpH6Mv9YbRmKPWIqXRxmIa2qIrrWxvcV1pcYn/G+etkF7XPqyl IkVfVgOZDJF/MU/e0mcRfj52SvZvIxfGHDVusD1waxSKi0+13lNeoopEK2pQmo311wyb kv9vQ3xljahsrgEjnq7wveFi5BVhVK8Yff5iVucAZ5sVUid4RBIUssZh1VAaw3HpEZsE qsGBBmvcK9eMvRruOWrEKavM8GAa7NyPs9KS/qLIZwkb9wz/mN9I+f9JJpj89lpQSUOf Y2gA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Np0qDXgaZex9jai3MfKaTNk9A1UTLQqCFBFqKrQv3zo=; b=oJ0UhYaGoZbWdWGw6Zbd+/9nAEqq7OD8e86GV0u1dDhg5kQO1g/xdHm8tiVEiiZByp ZuNmf4zV6oOLg9X802T1xsT46VMOtu8chEYRuF7RJwCptXkJHKy83pmp5FmkNnYiO1JK 5DkbJ8ZrAQQr0x7+TkdzvjUODnFoI/NP7njoKpxuLzCbgJBakbtoZ3YxgX74vYXsimxw MfWPnT0+IR126qzOpCFg7BX7ldF5cpWnbe4ZTLWk5gt8qt57ay92Ikj2vmMLV+hkUAWI E31Rw3Pm0X+QOAD49oltqy8892LRYty8qIUXTWjK+Tqym9CBKDMM5qLRz4FrM4a8s4Vh aKgg== X-Gm-Message-State: AOAM532BTqGgFyhdUw1nPsHGKnR9ZDxxNOYpp3GKzw5PsrlRX79Z4S+z IEU214hyvxUMD4VhQ/P7VGxGNNY6+1w= X-Google-Smtp-Source: ABdhPJyLfHCdCZKFbIm/onE2Hem6TgCn3dJNj47wHMqzhpTfBpmnpYA7iaMHWfjusNA3OVSYoGrTFg== X-Received: by 2002:a19:8606:: with SMTP id i6mr1256296lfd.263.1602854217411; Fri, 16 Oct 2020 06:16:57 -0700 (PDT) Received: from localhost.localdomain (n89ridqjqdjpsztph-2.v6.elisa-mobile.fi. [2001:999:11:8336:994:7f72:f89e:9125]) by smtp.gmail.com with ESMTPSA id x13sm280594lfe.101.2020.10.16.06.16.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 06:16:56 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Fri, 16 Oct 2020 16:16:45 +0300 Message-Id: <20201016131649.4361-3-jeebjp@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201016131649.4361-1-jeebjp@gmail.com> References: <20201016131649.4361-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 2/6] ffmpeg: move AVFrame time base adjustment into a function X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This will have to be called later for video down the line. --- fftools/ffmpeg.c | 72 +++++++++++++++++++++++++++++++----------------- 1 file changed, 47 insertions(+), 25 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index cb7644de6a..0d8ed26912 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -897,6 +897,50 @@ static int check_recording_time(OutputStream *ost) return 1; } +static double adjust_frame_pts_to_encoder_tb(OutputFile *of, OutputStream *ost, + AVFrame *frame) +{ + double float_pts = AV_NOPTS_VALUE; // this is identical to frame.pts but with higher precision + AVCodecContext *enc = ost->enc_ctx; + if (!frame || frame->pts == AV_NOPTS_VALUE || + !enc || !ost->filter || !ost->filter->graph->graph) + goto early_exit; + + { + AVFilterContext *filter = ost->filter->filter; + + int64_t start_time = (of->start_time == AV_NOPTS_VALUE) ? 0 : of->start_time; + AVRational filter_tb = av_buffersink_get_time_base(filter); + AVRational tb = enc->time_base; + int extra_bits = av_clip(29 - av_log2(tb.den), 0, 16); + + tb.den <<= extra_bits; + float_pts = + av_rescale_q(frame->pts, filter_tb, tb) - + av_rescale_q(start_time, AV_TIME_BASE_Q, tb); + float_pts /= 1 << extra_bits; + // avoid exact midoints to reduce the chance of rounding differences, this can be removed in case the fps code is changed to work with integers + float_pts += FFSIGN(float_pts) * 1.0 / (1<<17); + + frame->pts = + av_rescale_q(frame->pts, filter_tb, enc->time_base) - + av_rescale_q(start_time, AV_TIME_BASE_Q, enc->time_base); + } + +early_exit: + + if (debug_ts) { + av_log(NULL, AV_LOG_INFO, "filter -> pts:%s pts_time:%s exact:%f time_base:%d/%d\n", + frame ? av_ts2str(frame->pts) : "NULL", + frame ? av_ts2timestr(frame->pts, &enc->time_base) : "NULL", + float_pts, + enc ? enc->time_base.num : -1, + enc ? enc->time_base.den : -1); + } + + return float_pts; +} + static void do_audio_out(OutputFile *of, OutputStream *ost, AVFrame *frame) { @@ -1473,37 +1517,15 @@ static int reap_filters(int flush) av_frame_unref(filtered_frame); continue; } - if (filtered_frame->pts != AV_NOPTS_VALUE) { - int64_t start_time = (of->start_time == AV_NOPTS_VALUE) ? 0 : of->start_time; - AVRational filter_tb = av_buffersink_get_time_base(filter); - AVRational tb = enc->time_base; - int extra_bits = av_clip(29 - av_log2(tb.den), 0, 16); - - tb.den <<= extra_bits; - float_pts = - av_rescale_q(filtered_frame->pts, filter_tb, tb) - - av_rescale_q(start_time, AV_TIME_BASE_Q, tb); - float_pts /= 1 << extra_bits; - // avoid exact midoints to reduce the chance of rounding differences, this can be removed in case the fps code is changed to work with integers - float_pts += FFSIGN(float_pts) * 1.0 / (1<<17); - - filtered_frame->pts = - av_rescale_q(filtered_frame->pts, filter_tb, enc->time_base) - - av_rescale_q(start_time, AV_TIME_BASE_Q, enc->time_base); - } + + float_pts = adjust_frame_pts_to_encoder_tb(of, ost, + filtered_frame); switch (av_buffersink_get_type(filter)) { case AVMEDIA_TYPE_VIDEO: if (!ost->frame_aspect_ratio.num) enc->sample_aspect_ratio = filtered_frame->sample_aspect_ratio; - if (debug_ts) { - av_log(NULL, AV_LOG_INFO, "filter -> pts:%s pts_time:%s exact:%f time_base:%d/%d\n", - av_ts2str(filtered_frame->pts), av_ts2timestr(filtered_frame->pts, &enc->time_base), - float_pts, - enc->time_base.num, enc->time_base.den); - } - do_video_out(of, ost, filtered_frame, float_pts); break; case AVMEDIA_TYPE_AUDIO: From patchwork Fri Oct 16 13:16:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 23017 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1E8DD44B2DD for ; Fri, 16 Oct 2020 16:25:19 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 017D668BAA9; Fri, 16 Oct 2020 16:25:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f193.google.com (mail-lj1-f193.google.com [209.85.208.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7A12E68BAA0 for ; Fri, 16 Oct 2020 16:25:12 +0300 (EEST) Received: by mail-lj1-f193.google.com with SMTP id y16so2449468ljk.1 for ; Fri, 16 Oct 2020 06:25:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=QZDujKRqxy50dU9Ez/Jd2dZb6cEWubUIb9i7xwQFgr0=; b=FXCPFc3TaynmtXQuKnUxZckh1uj/789mVaaCj9+3o4X2LxS6zQ1MgHJbSy0XNRGbn6 LM+wNwKUSOkaqClssTnn3YdOKSxsMg8KJf3W/txlZkJSnSo46Lns8R4vRwW6Sl9Gvf7P Vc973WMTZB6OyFwy2lnTpwlhqnHTiDanPpTd4my6xSnTPgNMZl/Gv5d4lbn5Ua+OxqxE UvyGYqIubaqz9m3SV6riR6kZA3GrEq2r1wBZTwn7pvDe6l6QLv5YGKH413veGVrcNYNL 0Zx+AXEvlUZzPLzVY1MvkpnecTkUpQbmnnAUx9DJThpl8LLIhJ7VmkZzF8LhJJyzmfeq ivWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QZDujKRqxy50dU9Ez/Jd2dZb6cEWubUIb9i7xwQFgr0=; b=fZ43NRAXzdlTezmMRVkTVNvEWV0yIhPowTHqEPMqLWoVv+fCX01K/sai6lUrAkH9zb qoajpiLEXYVC/LtRVUdMUKnwR1fW8MxdphT1dLr+81mDjYZThbIdrEnRTTq7vRGJQtYY WZJPznSI6TvjQf4lobFbYqBgCVxedolWMD0C0fBayadBv9A6P098fR7oY5Yg8hqr1sdL iaXGdFfIz6kj2OpG9lCHqFxPucE73x9ArQHwdXXOwHq5fAsMF5IvLWmnXjPiAdx6DAfX 1hKptRn7OPLY8+qpZJ9Oo7gHVEfykCmSgVEYUzLD2gEWT84LlkyoE6WtnWSgGR2aRt6T kWpQ== X-Gm-Message-State: AOAM5308WN4T6zyQ8rfvySfh0N7GIbZBYPpkfhutS/+5MhpZBtDhe/VV oA6LC5JmE5NwOemQwpbLc1RjrkWC8kc= X-Google-Smtp-Source: ABdhPJwnD/jtTrbf2MB6p/JouOWXMOAzYdn6/Cd7HV+aDtWW4LqU1Hwm3agRJS+XQ2ksQAkr1eZ2Nw== X-Received: by 2002:a05:6512:3305:: with SMTP id k5mr1410401lfe.472.1602854219066; Fri, 16 Oct 2020 06:16:59 -0700 (PDT) Received: from localhost.localdomain (n89ridqjqdjpsztph-2.v6.elisa-mobile.fi. [2001:999:11:8336:994:7f72:f89e:9125]) by smtp.gmail.com with ESMTPSA id x13sm280594lfe.101.2020.10.16.06.16.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 06:16:58 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Fri, 16 Oct 2020 16:16:46 +0300 Message-Id: <20201016131649.4361-4-jeebjp@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201016131649.4361-1-jeebjp@gmail.com> References: <20201016131649.4361-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 3/6] ffmpeg: move A/V non-streamcopy initialization to a later point X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" - For video, this means a single initialization point in do_video_out. - For audio we unfortunately need to do it in two places just before the buffer sink is utilized (if av_buffersink_get_samples would still work according to its specification after a call to avfilter_graph_request_oldest was made, we could at least remove the one in transcode_step). Other adjustments to make things work: - As the AVFrame PTS adjustment to encoder time base needs the encoder to be initialized, so it is now moved to do_{video,audio}_out, right after the encoder has been initialized. Due to this, the additional parameter in do_video_out is removed as it is no longer necessary. --- fftools/ffmpeg.c | 112 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 77 insertions(+), 35 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 0d8ed26912..08db67a6ab 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -941,6 +941,28 @@ early_exit: return float_pts; } +static int init_output_stream(OutputStream *ost, char *error, int error_len); + +static int init_output_stream_wrapper(OutputStream *ost, unsigned int fatal) +{ + int ret = AVERROR_BUG; + char error[1024] = {0}; + + if (ost->initialized) + return 0; + + ret = init_output_stream(ost, error, sizeof(error)); + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", + ost->file_index, ost->index, error); + + if (fatal) + exit_program(1); + } + + return ret; +} + static void do_audio_out(OutputFile *of, OutputStream *ost, AVFrame *frame) { @@ -952,6 +974,8 @@ static void do_audio_out(OutputFile *of, OutputStream *ost, pkt.data = NULL; pkt.size = 0; + adjust_frame_pts_to_encoder_tb(of, ost, frame); + if (!check_recording_time(ost)) return; @@ -1086,8 +1110,7 @@ static void do_subtitle_out(OutputFile *of, static void do_video_out(OutputFile *of, OutputStream *ost, - AVFrame *next_picture, - double sync_ipts) + AVFrame *next_picture) { int ret, format_video_sync; AVPacket pkt; @@ -1097,10 +1120,14 @@ static void do_video_out(OutputFile *of, int nb_frames, nb0_frames, i; double delta, delta0; double duration = 0; + double sync_ipts = AV_NOPTS_VALUE; int frame_size = 0; InputStream *ist = NULL; AVFilterContext *filter = ost->filter->filter; + init_output_stream_wrapper(ost, 1); + sync_ipts = adjust_frame_pts_to_encoder_tb(of, ost, next_picture); + if (ost->source_index >= 0) ist = input_streams[ost->source_index]; @@ -1434,28 +1461,6 @@ static void do_video_stats(OutputStream *ost, int frame_size) } } -static int init_output_stream(OutputStream *ost, char *error, int error_len); - -static int init_output_stream_wrapper(OutputStream *ost, unsigned int fatal) -{ - int ret = AVERROR_BUG; - char error[1024] = {0}; - - if (ost->initialized) - return 0; - - ret = init_output_stream(ost, error, sizeof(error)); - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", - ost->file_index, ost->index, error); - - if (fatal) - exit_program(1); - } - - return ret; -} - static void finish_output_stream(OutputStream *ost) { OutputFile *of = output_files[ost->file_index]; @@ -1492,7 +1497,17 @@ static int reap_filters(int flush) continue; filter = ost->filter->filter; - init_output_stream_wrapper(ost, 1); + /* + * Unlike video, with audio the audio frame size matters. + * Currently we are fully reliant on the lavfi filter chain to + * do the buffering deed for us, and thus the frame size parameter + * needs to be set accordingly. Where does one get the required + * frame size? From the initialized AVCodecContext of an audio + * encoder. Thus, if we have gotten to an audio stream, initialize + * the encoder earlier than receiving the first AVFrame. + */ + if (av_buffersink_get_type(filter) == AVMEDIA_TYPE_AUDIO) + init_output_stream_wrapper(ost, 1); if (!ost->filtered_frame && !(ost->filtered_frame = av_frame_alloc())) { return AVERROR(ENOMEM); @@ -1500,7 +1515,6 @@ static int reap_filters(int flush) filtered_frame = ost->filtered_frame; while (1) { - double float_pts = AV_NOPTS_VALUE; // this is identical to filtered_frame.pts but with higher precision ret = av_buffersink_get_frame_flags(filter, filtered_frame, AV_BUFFERSINK_FLAG_NO_REQUEST); if (ret < 0) { @@ -1509,7 +1523,7 @@ static int reap_filters(int flush) "Error in av_buffersink_get_frame_flags(): %s\n", av_err2str(ret)); } else if (flush && ret == AVERROR_EOF) { if (av_buffersink_get_type(filter) == AVMEDIA_TYPE_VIDEO) - do_video_out(of, ost, NULL, AV_NOPTS_VALUE); + do_video_out(of, ost, NULL); } break; } @@ -1518,15 +1532,12 @@ static int reap_filters(int flush) continue; } - float_pts = adjust_frame_pts_to_encoder_tb(of, ost, - filtered_frame); - switch (av_buffersink_get_type(filter)) { case AVMEDIA_TYPE_VIDEO: if (!ost->frame_aspect_ratio.num) enc->sample_aspect_ratio = filtered_frame->sample_aspect_ratio; - do_video_out(of, ost, filtered_frame, float_pts); + do_video_out(of, ost, filtered_frame); break; case AVMEDIA_TYPE_AUDIO: if (!(enc->codec->capabilities & AV_CODEC_CAP_PARAM_CHANGE) && @@ -3691,10 +3702,19 @@ static int transcode_init(void) goto dump_format; } - /* open each encoder */ + /* + * initialize stream copy and subtitle/data streams. + * Encoded AVFrame based streams will get initialized as follows: + * - when the first AVFrame is received in do_video_out + * - just before the first AVFrame is received in either transcode_step + * or reap_filters due to us requiring the filter chain buffer sink + * to be configured with the correct audio frame size, which is only + * known after the encoder is initialized. + */ for (i = 0; i < nb_output_streams; i++) { - // skip streams fed from filtergraphs until we have a frame for them - if (output_streams[i]->filter) + if (!output_streams[i]->stream_copy && + (output_streams[i]->enc_ctx->codec_type == AVMEDIA_TYPE_VIDEO || + output_streams[i]->enc_ctx->codec_type == AVMEDIA_TYPE_AUDIO)) continue; ret = init_output_stream_wrapper(output_streams[i], 0); @@ -4608,7 +4628,29 @@ static int transcode_step(void) } if (ost->filter && ost->filter->graph->graph) { - init_output_stream_wrapper(ost, 1); + /* + * Similar case to the early audio initialization in reap_filters. + * Audio is special in ffmpeg.c currently as we depend on lavfi's + * audio frame buffering/creation to get the output audio frame size + * in samples correct. The audio frame size for the filter chain is + * configured during the output stream initialization. + * + * Apparently avfilter_graph_request_oldest (called in + * transcode_from_filter just down the line) peeks. Peeking already + * puts one frame "ready to be given out", which means that any + * update in filter buffer sink configuration afterwards will not + * help us. And yes, even if it would be utilized, + * av_buffersink_get_samples is affected, as it internally utilizes + * the same early exit for peeked frames. + * + * In other words, if avfilter_graph_request_oldest would not make + * further filter chain configuration or usage of + * av_buffersink_get_samples useless (by just causing the return + * of the peeked AVFrame as-is), we could get rid of this additional + * early encoder initialization. + */ + if (av_buffersink_get_type(ost->filter->filter) == AVMEDIA_TYPE_AUDIO) + init_output_stream_wrapper(ost, 1); if ((ret = transcode_from_filter(ost->filter->graph, &ist)) < 0) return ret; From patchwork Fri Oct 16 13:16:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 23012 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1480944A855 for ; Fri, 16 Oct 2020 16:22:55 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D773D68B9E7; Fri, 16 Oct 2020 16:22:54 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f67.google.com (mail-ed1-f67.google.com [209.85.208.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0A01768B7A6 for ; Fri, 16 Oct 2020 16:22:48 +0300 (EEST) Received: by mail-ed1-f67.google.com with SMTP id o18so2351165edq.4 for ; Fri, 16 Oct 2020 06:22:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=oiZrx07Fw4xoKYULlA0i4IbHikgiZKR4pSk2S3qvgJ0=; b=FB6uvoSPFQGEOJ8KtYVK5INNWPmpRLNeALtsCcx7S93a171r48XW72wa7c+w89Gfxw bUWLKCh4AaJb47jqQldfjWH4YydS98VzfWtaMRe6RZuqCXm0/+PQ9NhN62nYOhi+ngzV ZyybOZCGt5M99nKK5rpoCRJpj3XyqGfLteaebULAWe1vwCd9sBh6+7+IWSFfgmxsopou WZF+K79NwiksC8lDm+xE68JJs/M0YR1/W/N4GC+71FvSareGbDYla5YimWmoIWejUzsF 42tkY82nugTJ5ExLOuEd1wg79Q/2msImiRViq6ZCHFZbEp2uXBZ6IOZSpS69SQT1lG3q 4hCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oiZrx07Fw4xoKYULlA0i4IbHikgiZKR4pSk2S3qvgJ0=; b=n4dujY02p/yNmAQnbknUNvosG9wwVolireizIxYSr+/9ItryHheBLJK0zbNwgaevaU IcWPOtnuudR+/H9dB3TF/a4h5bF0v5VE35K1V7pF8DmvVQQw5Eef3wE1vY+1TgKalhxP Ca39adTPnUP6xCea1Rx25fknRv1981gcNwQ+uyjVLT81hXRPMf60qj7E4yh3E0Bm2zjH PGvjooXEajv8qlLet9XBxmXjltGHRzF5v4ofH9nfo8ggtue+5bTfvo6uWc4Fl5LO4Kwx wxldHx9E00pOaSPHIYdbIzTcr/FXc+n8CFEYlLB304LLB+AzElxB5rFIoZ0+JlnObIQd Iy+w== X-Gm-Message-State: AOAM5320DClwAOL7xm3GlVWu26qXrvjlR7l6/wbYJmiP9lHnnVcl+v7l 1Bwn0nHX05pWp0sxrKxEw8g1UJtEJiQ= X-Google-Smtp-Source: ABdhPJzSZl0BX3DaLlrCKxyFVNwGe5lWQg1TfvAjXH2rVPH0a1mp5wckININf6TcOLEOd7KL/PP0Xg== X-Received: by 2002:ac2:5473:: with SMTP id e19mr1412605lfn.544.1602854220612; Fri, 16 Oct 2020 06:17:00 -0700 (PDT) Received: from localhost.localdomain (n89ridqjqdjpsztph-2.v6.elisa-mobile.fi. [2001:999:11:8336:994:7f72:f89e:9125]) by smtp.gmail.com with ESMTPSA id x13sm280594lfe.101.2020.10.16.06.16.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 06:17:00 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Fri, 16 Oct 2020 16:16:47 +0300 Message-Id: <20201016131649.4361-5-jeebjp@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201016131649.4361-1-jeebjp@gmail.com> References: <20201016131649.4361-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 4/6] ffmpeg: pass decoded or filtered AVFrame to output stream initialization X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Additionally, reap the first rewards by being able to set the color related encoding values based on the passed AVFrame. The only tests that seem to have changed their results with this change seem to be the MXF tests. There, the muxer writes the limited/full range flag to the output container if the encoder is not set to "unspecified". --- fftools/ffmpeg.c | 42 +++++++++++++++++++++++++++---------- tests/ref/lavf/mxf_d10 | 2 +- tests/ref/lavf/mxf_dv25 | 2 +- tests/ref/lavf/mxf_dvcpro50 | 2 +- tests/ref/lavf/mxf_opatom | 2 +- 5 files changed, 35 insertions(+), 15 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 08db67a6ab..b2e210c814 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -941,9 +941,11 @@ early_exit: return float_pts; } -static int init_output_stream(OutputStream *ost, char *error, int error_len); +static int init_output_stream(OutputStream *ost, AVFrame *frame, + char *error, int error_len); -static int init_output_stream_wrapper(OutputStream *ost, unsigned int fatal) +static int init_output_stream_wrapper(OutputStream *ost, AVFrame *frame, + unsigned int fatal) { int ret = AVERROR_BUG; char error[1024] = {0}; @@ -951,7 +953,7 @@ static int init_output_stream_wrapper(OutputStream *ost, unsigned int fatal) if (ost->initialized) return 0; - ret = init_output_stream(ost, error, sizeof(error)); + ret = init_output_stream(ost, frame, error, sizeof(error)); if (ret < 0) { av_log(NULL, AV_LOG_ERROR, "Error initializing output stream %d:%d -- %s\n", ost->file_index, ost->index, error); @@ -1125,7 +1127,7 @@ static void do_video_out(OutputFile *of, InputStream *ist = NULL; AVFilterContext *filter = ost->filter->filter; - init_output_stream_wrapper(ost, 1); + init_output_stream_wrapper(ost, next_picture, 1); sync_ipts = adjust_frame_pts_to_encoder_tb(of, ost, next_picture); if (ost->source_index >= 0) @@ -1507,7 +1509,7 @@ static int reap_filters(int flush) * the encoder earlier than receiving the first AVFrame. */ if (av_buffersink_get_type(filter) == AVMEDIA_TYPE_AUDIO) - init_output_stream_wrapper(ost, 1); + init_output_stream_wrapper(ost, NULL, 1); if (!ost->filtered_frame && !(ost->filtered_frame = av_frame_alloc())) { return AVERROR(ENOMEM); @@ -1930,7 +1932,7 @@ static void flush_encoders(void) finish_output_stream(ost); } - init_output_stream_wrapper(ost, 1); + init_output_stream_wrapper(ost, NULL, 1); } if (enc->codec_type != AVMEDIA_TYPE_VIDEO && enc->codec_type != AVMEDIA_TYPE_AUDIO) @@ -3302,7 +3304,7 @@ static void init_encoder_time_base(OutputStream *ost, AVRational default_time_ba enc_ctx->time_base = default_time_base; } -static int init_output_stream_encode(OutputStream *ost) +static int init_output_stream_encode(OutputStream *ost, AVFrame *frame) { InputStream *ist = get_input_stream(ost); AVCodecContext *enc_ctx = ost->enc_ctx; @@ -3399,6 +3401,23 @@ static int init_output_stream_encode(OutputStream *ost) enc_ctx->bits_per_raw_sample = FFMIN(dec_ctx->bits_per_raw_sample, av_pix_fmt_desc_get(enc_ctx->pix_fmt)->comp[0].depth); + if (frame) { + if (!av_dict_get(ost->encoder_opts, "color_range", NULL, 0)) + enc_ctx->color_range = frame->color_range; + + if (!av_dict_get(ost->encoder_opts, "color_primaries", NULL, 0)) + enc_ctx->color_primaries = frame->color_primaries; + + if (!av_dict_get(ost->encoder_opts, "color_trc", NULL, 0)) + enc_ctx->color_trc = frame->color_trc; + + if (!av_dict_get(ost->encoder_opts, "colorspace", NULL, 0)) + enc_ctx->colorspace = frame->colorspace; + + if (!av_dict_get(ost->encoder_opts, "chroma_sample_location", NULL, 0)) + enc_ctx->chroma_sample_location = frame->chroma_location; + } + enc_ctx->framerate = ost->frame_rate; ost->st->avg_frame_rate = ost->frame_rate; @@ -3456,7 +3475,8 @@ static int init_output_stream_encode(OutputStream *ost) return 0; } -static int init_output_stream(OutputStream *ost, char *error, int error_len) +static int init_output_stream(OutputStream *ost, AVFrame *frame, + char *error, int error_len) { int ret = 0; @@ -3465,7 +3485,7 @@ static int init_output_stream(OutputStream *ost, char *error, int error_len) AVCodecContext *dec = NULL; InputStream *ist; - ret = init_output_stream_encode(ost); + ret = init_output_stream_encode(ost, frame); if (ret < 0) return ret; @@ -3717,7 +3737,7 @@ static int transcode_init(void) output_streams[i]->enc_ctx->codec_type == AVMEDIA_TYPE_AUDIO)) continue; - ret = init_output_stream_wrapper(output_streams[i], 0); + ret = init_output_stream_wrapper(output_streams[i], NULL, 0); if (ret < 0) goto dump_format; } @@ -4650,7 +4670,7 @@ static int transcode_step(void) * early encoder initialization. */ if (av_buffersink_get_type(ost->filter->filter) == AVMEDIA_TYPE_AUDIO) - init_output_stream_wrapper(ost, 1); + init_output_stream_wrapper(ost, NULL, 1); if ((ret = transcode_from_filter(ost->filter->graph, &ist)) < 0) return ret; diff --git a/tests/ref/lavf/mxf_d10 b/tests/ref/lavf/mxf_d10 index aea469bb58..85e337d157 100644 --- a/tests/ref/lavf/mxf_d10 +++ b/tests/ref/lavf/mxf_d10 @@ -1,3 +1,3 @@ -e597f73ef9c9819710d2f815813eb91f *tests/data/lavf/lavf.mxf_d10 +36fc2a640368f6d33987d2b2d39df966 *tests/data/lavf/lavf.mxf_d10 5332013 tests/data/lavf/lavf.mxf_d10 tests/data/lavf/lavf.mxf_d10 CRC=0x6c74d488 diff --git a/tests/ref/lavf/mxf_dv25 b/tests/ref/lavf/mxf_dv25 index db6b76c6f8..d4559df862 100644 --- a/tests/ref/lavf/mxf_dv25 +++ b/tests/ref/lavf/mxf_dv25 @@ -1,3 +1,3 @@ -0fc964fa22bc8b3a389b81b9a2efccb3 *tests/data/lavf/lavf.mxf_dv25 +57623b3b968c0bb0d8a0bbaeef6fe657 *tests/data/lavf/lavf.mxf_dv25 3834413 tests/data/lavf/lavf.mxf_dv25 tests/data/lavf/lavf.mxf_dv25 CRC=0xbdaf7f52 diff --git a/tests/ref/lavf/mxf_dvcpro50 b/tests/ref/lavf/mxf_dvcpro50 index 09999914bf..8bcf67d17f 100644 --- a/tests/ref/lavf/mxf_dvcpro50 +++ b/tests/ref/lavf/mxf_dvcpro50 @@ -1,3 +1,3 @@ -aa81ea83af44a69e73849e327cc4bd12 *tests/data/lavf/lavf.mxf_dvcpro50 +6e82b4cc962199e2593e30851ff7ccfb *tests/data/lavf/lavf.mxf_dvcpro50 7431213 tests/data/lavf/lavf.mxf_dvcpro50 tests/data/lavf/lavf.mxf_dvcpro50 CRC=0xe3bbe4b4 diff --git a/tests/ref/lavf/mxf_opatom b/tests/ref/lavf/mxf_opatom index 05794a4e5e..1aa843a22a 100644 --- a/tests/ref/lavf/mxf_opatom +++ b/tests/ref/lavf/mxf_opatom @@ -1,3 +1,3 @@ -06a1816aa91c733e1ef7e45d82e4f1d3 *tests/data/lavf/lavf.mxf_opatom +d5f56215c2b16dee204f03bfa653dd1b *tests/data/lavf/lavf.mxf_opatom 4717625 tests/data/lavf/lavf.mxf_opatom tests/data/lavf/lavf.mxf_opatom CRC=0xf55aa22a From patchwork Fri Oct 16 13:16:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 23016 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E9A3C44B20D for ; Fri, 16 Oct 2020 16:24:56 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C5D0968BA27; Fri, 16 Oct 2020 16:24:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E5B8968B739 for ; Fri, 16 Oct 2020 16:24:49 +0300 (EEST) Received: by mail-ed1-f47.google.com with SMTP id o18so2360376edq.4 for ; Fri, 16 Oct 2020 06:24:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=PU4/7zjxp0+xS8XuUAJfv4PSnski3aJb/MMEZ7+XwY8=; b=q+6fmdw3KiObMkhl/CGh6BfPpS3SmTTnBiEi2nIjBlTa7zKLWiIjpXPEQ5GKsTLgIp GNE6fLx0r9w6qg1n4sI+ao9cziEUYLKUsG+qVYrt8iznpreJ9enfNKPgaoTjmfEJWZto IJtdH1tkSnKwj8UUzlCcDhTIAkx+VqcAXvB+tastnycanzC7cwq+phH6gL0hjGzJjdJB nxyXz6jM0+tlFYXLaAtG2ZNNZPkHPaMj7liBPfGNB3ZM3NkIkTpNAY4UPCTp/cB6FveI INPD4DBP7emahGHJoNTOlWJU6DIE2SUeAUeaOHq1gvMeGwjixq+cFVS/CvRCpF1wEp8R avqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PU4/7zjxp0+xS8XuUAJfv4PSnski3aJb/MMEZ7+XwY8=; b=H4kBTGG9lm+XfUuPJVkDW29zRloqykLTYjJEgWCd3rbBJbyRSC8x1g+tAJYwo1/gut GAhneWKMQNg9NDnD5mCDeTfWJk+3dSpXKe/h7QCD1a7ZtQt5xOCGvZdHLV3iwteaRCOi h2KFyesp7aVK3QxuD/apd2hUtTNxtMM+b4/Cd+uYwxAH5haDl9+BBSlUygm5gLlYNf0H ItVp9nNhKbJvSSHkSZwXwIdv+99Bebzn5zN8iNVeeVnU36HsUg6HX81G4eTTIL0fYLHp ECce3MBYRAWk4t4TC4mvVxUJe9KsL4ddQz2H5xIrjlhHtZI6UPcEQJSMGrGpN5V7WkmU ZiCw== X-Gm-Message-State: AOAM530jD2OmWTRvNsbUtaiV4OF1uPSer+DKVBLu66NtcHeHJ0clTKzE BS7YqjQUnOTRuP/L8x+ffaRyotbXijI= X-Google-Smtp-Source: ABdhPJxXLiFfX1VnLKA1SFTOriRuhcD80Ip0EjE5ugztfj0JubyxcE7ToI4WoI/9dAbcOPVsJZ5PRg== X-Received: by 2002:a2e:9c84:: with SMTP id x4mr1480995lji.326.1602854222285; Fri, 16 Oct 2020 06:17:02 -0700 (PDT) Received: from localhost.localdomain (n89ridqjqdjpsztph-2.v6.elisa-mobile.fi. [2001:999:11:8336:994:7f72:f89e:9125]) by smtp.gmail.com with ESMTPSA id x13sm280594lfe.101.2020.10.16.06.17.00 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 06:17:01 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Fri, 16 Oct 2020 16:16:48 +0300 Message-Id: <20201016131649.4361-6-jeebjp@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201016131649.4361-1-jeebjp@gmail.com> References: <20201016131649.4361-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 5/6] ffmpeg: move field order decision making to encoder initialization X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" We now have the possibility of getting AVFrames here, and we should not touch the muxer's codecpar after writing the header. --- fftools/ffmpeg.c | 27 ++++++++++--------- .../fate/concat-demuxer-extended-lavf-mxf_d10 | 2 +- .../fate/concat-demuxer-simple1-lavf-mxf_d10 | 2 +- tests/ref/fate/rgb24-mkv | 4 +-- tests/ref/lavf/mxf_d10 | 2 +- 5 files changed, 19 insertions(+), 18 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index b2e210c814..1c95890f08 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -1117,7 +1117,6 @@ static void do_video_out(OutputFile *of, int ret, format_video_sync; AVPacket pkt; AVCodecContext *enc = ost->enc_ctx; - AVCodecParameters *mux_par = ost->st->codecpar; AVRational frame_rate; int nb_frames, nb0_frames, i; double delta, delta0; @@ -1279,18 +1278,6 @@ static void do_video_out(OutputFile *of, if (!check_recording_time(ost)) return; - if (enc->flags & (AV_CODEC_FLAG_INTERLACED_DCT | AV_CODEC_FLAG_INTERLACED_ME) && - ost->top_field_first >= 0) - in_picture->top_field_first = !!ost->top_field_first; - - if (in_picture->interlaced_frame) { - if (enc->codec->id == AV_CODEC_ID_MJPEG) - mux_par->field_order = in_picture->top_field_first ? AV_FIELD_TT:AV_FIELD_BB; - else - mux_par->field_order = in_picture->top_field_first ? AV_FIELD_TB:AV_FIELD_BT; - } else - mux_par->field_order = AV_FIELD_PROGRESSIVE; - in_picture->quality = enc->global_quality; in_picture->pict_type = 0; @@ -3435,6 +3422,20 @@ static int init_output_stream_encode(OutputStream *ost, AVFrame *frame) enc_ctx->field_order = AV_FIELD_TT; } + if (frame) { + if (enc_ctx->flags & (AV_CODEC_FLAG_INTERLACED_DCT | AV_CODEC_FLAG_INTERLACED_ME) && + ost->top_field_first >= 0) + frame->top_field_first = !!ost->top_field_first; + + if (frame->interlaced_frame) { + if (enc_ctx->codec->id == AV_CODEC_ID_MJPEG) + enc_ctx->field_order = frame->top_field_first ? AV_FIELD_TT:AV_FIELD_BB; + else + enc_ctx->field_order = frame->top_field_first ? AV_FIELD_TB:AV_FIELD_BT; + } else + enc_ctx->field_order = AV_FIELD_PROGRESSIVE; + } + if (ost->forced_keyframes) { if (!strncmp(ost->forced_keyframes, "expr:", 5)) { ret = av_expr_parse(&ost->forced_keyframes_pexpr, ost->forced_keyframes+5, diff --git a/tests/ref/fate/concat-demuxer-extended-lavf-mxf_d10 b/tests/ref/fate/concat-demuxer-extended-lavf-mxf_d10 index e3e76f217a..f6efc00ca4 100644 --- a/tests/ref/fate/concat-demuxer-extended-lavf-mxf_d10 +++ b/tests/ref/fate/concat-demuxer-extended-lavf-mxf_d10 @@ -1 +1 @@ -d66177ea3922692bc91cd0f8aa907650 *tests/data/fate/concat-demuxer-extended-lavf-mxf_d10.ffprobe +84496cfe2d668db395280ea67e5c6fbe *tests/data/fate/concat-demuxer-extended-lavf-mxf_d10.ffprobe diff --git a/tests/ref/fate/concat-demuxer-simple1-lavf-mxf_d10 b/tests/ref/fate/concat-demuxer-simple1-lavf-mxf_d10 index 79ce1e2306..8f3f2e5265 100644 --- a/tests/ref/fate/concat-demuxer-simple1-lavf-mxf_d10 +++ b/tests/ref/fate/concat-demuxer-simple1-lavf-mxf_d10 @@ -78,5 +78,5 @@ video|0|34|1.360000|34|1.360000|1|0.040000|N/A|N/A|150000|1924096|K_|1 Strings Metadata audio|1|65280|1.360000|65280|1.360000|1920|0.040000|N/A|N/A|7680|2074624|K_|1 Strings Metadata -0|mpeg2video|0|video|1/25|[0][0][0][0]|0x0000|720|608|0|0|0|0|1:1|45:38|yuv422p|5|tv|unknown|unknown|unknown|topleft|tt|N/A|1|N/A|25/1|25/1|1/25|0|0.000000|N/A|N/A|30000000|N/A|N/A|N/A|N/A|35|0|0|0|0|0|0|0|0|0|0|0|0|0x060A2B340101010501010D001300000000000000000000000000000000000001 +0|mpeg2video|0|video|1/25|[0][0][0][0]|0x0000|720|608|0|0|0|0|1:1|45:38|yuv422p|5|tv|unknown|unknown|unknown|topleft|tb|N/A|1|N/A|25/1|25/1|1/25|0|0.000000|N/A|N/A|30000000|N/A|N/A|N/A|N/A|35|0|0|0|0|0|0|0|0|0|0|0|0|0x060A2B340101010501010D001300000000000000000000000000000000000001 1|pcm_s16le|unknown|audio|1/48000|[0][0][0][0]|0x0000|s16|48000|2|unknown|16|N/A|0/0|0/0|1/48000|0|0.000000|N/A|N/A|1536000|N/A|N/A|N/A|N/A|35|0|0|0|0|0|0|0|0|0|0|0|0|0x060A2B340101010501010D001300000000000000000000000000000000000001 diff --git a/tests/ref/fate/rgb24-mkv b/tests/ref/fate/rgb24-mkv index 34d028cbfd..3b14cd0ef0 100644 --- a/tests/ref/fate/rgb24-mkv +++ b/tests/ref/fate/rgb24-mkv @@ -1,5 +1,5 @@ -fdc02d700dbe99315a9f0d928a9b935e *tests/data/fate/rgb24-mkv.matroska -58213 tests/data/fate/rgb24-mkv.matroska +fde8903c4df0ba8235dafcfd8a2f368c *tests/data/fate/rgb24-mkv.matroska +58216 tests/data/fate/rgb24-mkv.matroska #tb 0: 1/10 #media_type 0: video #codec_id 0: rawvideo diff --git a/tests/ref/lavf/mxf_d10 b/tests/ref/lavf/mxf_d10 index 85e337d157..30701619e0 100644 --- a/tests/ref/lavf/mxf_d10 +++ b/tests/ref/lavf/mxf_d10 @@ -1,3 +1,3 @@ -36fc2a640368f6d33987d2b2d39df966 *tests/data/lavf/lavf.mxf_d10 +da0ebbebb50a530b14c0f06017f464b3 *tests/data/lavf/lavf.mxf_d10 5332013 tests/data/lavf/lavf.mxf_d10 tests/data/lavf/lavf.mxf_d10 CRC=0x6c74d488 From patchwork Fri Oct 16 13:16:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 23018 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 01CB744B368 for ; Fri, 16 Oct 2020 16:44:48 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CF3A968BA30; Fri, 16 Oct 2020 16:44:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2A27E68B8E3 for ; Fri, 16 Oct 2020 16:44:41 +0300 (EEST) Received: by mail-ed1-f52.google.com with SMTP id dn5so2414857edb.10 for ; Fri, 16 Oct 2020 06:44:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=TknSccbbzzO885SIb7eJPX7tqB1oeUklCMAbAmZyhbY=; b=DQrR7GrK9vBd9byaFvoafwsTI1XNaA20GXkPmhcxJbzWqkmkwTh/Lfpyxmw+0KQs2j I7eU10vZIqW0ENT2PfJPX5RMPUxYMYxCRlqd4Hl5J9BEwIt36dIlwXvXXE0ctvEXSSNl bn8KCW8dzt6IMalbEEyAqDMbQfXr7aJCv7WnQNRX1hn/ITY86MHh1zCjdUcZGqeFWcdh I1OFNqr61Q1C6d9FDi7M/srxCiHGW6E7tKqkFjlvhRvHQrMLwPVZWbm6azYYsrrmNUh+ VkN8t0VMZGqnf7Yiom3CybHJEDNyzRn4bfR+aaQMhCiOLqo/WNGf5t9Pb9ZMKc0KQyvB wHmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TknSccbbzzO885SIb7eJPX7tqB1oeUklCMAbAmZyhbY=; b=s/NsNj6hpX+AKZmp9UyNB7k6Vbip7DgZJXgg9LWUKKIPDswGjaJgnbpZrEAqNYSURG 6ST21AQdn0U7Z9e0D/8n2S/yWkzy7ZYP19h9mlIFXmk/SgOw3mol4bqxSeEGmLWfD0mz qm17Dzj0DIHnm/IQ16Z0cABnNwRJ3jhjzuZX1doPxw642tJPxuybmjgFMMD71P9KdpQ9 tHApQAt9TS5dJitGL2O0P1DYnIru1AheyopMOt9yuVzuU6roJIMSQmY05cqAw1HCzfK7 GItldRjSKeLrtG4l/9/HD/bzbH9yXvMUfM0LFzhj0bUzUuwW656BHNI/sJSGF7mCwvfX Bodg== X-Gm-Message-State: AOAM530vmJW18UayHiw0SVPVFUGB15ija7si4fTQ/G0Glns7+9+WQXb+ dBauYsR1Iobu/fIseaFES/ulSD2WuPY= X-Google-Smtp-Source: ABdhPJz3M3Uc982rbOgb9GCD/bHloDNlvc8mumzXm27AjtrEGcWcNd3qr2roswZQegEgqONm2EQnYA== X-Received: by 2002:a19:254:: with SMTP id 81mr1203068lfc.405.1602854224113; Fri, 16 Oct 2020 06:17:04 -0700 (PDT) Received: from localhost.localdomain (n89ridqjqdjpsztph-2.v6.elisa-mobile.fi. [2001:999:11:8336:994:7f72:f89e:9125]) by smtp.gmail.com with ESMTPSA id x13sm280594lfe.101.2020.10.16.06.17.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 06:17:03 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Fri, 16 Oct 2020 16:16:49 +0300 Message-Id: <20201016131649.4361-7-jeebjp@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201016131649.4361-1-jeebjp@gmail.com> References: <20201016131649.4361-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 6/6] ffmpeg: add a data size threshold for muxing queue size X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This way the old max queue size limit based behavior for streams where each individual packet is large is kept, while for smaller streams more packets can be buffered (current default is at 50 megabytes per stream). For some explanation, by default ffmpeg copies packets from before the appointed seek point and puts them into the local muxing queue. Before, it getting utilized was much less likely since as soon as the filter chain was initialized, the encoder (and thus output stream) was also initialized. Now, since we have pushed the encoder initialization to when the first AVFrame is decoded and filtered - which only happens after the exact seek point is hit - this queue is seeing much more usage. This attempts to fix cases such as where: - seek point ends up being 5 seconds before requested time. - audio is set to copy, and thus immediately begins filling the muxing queue. - video is being encoded, and thus all received packets are skipped until the requested time is hit. --- doc/ffmpeg.texi | 5 +++++ fftools/ffmpeg.c | 11 +++++++++-- fftools/ffmpeg.h | 11 +++++++++++ fftools/ffmpeg_opt.c | 8 ++++++++ 4 files changed, 33 insertions(+), 2 deletions(-) diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index 96b3257e79..95d6463685 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -1746,6 +1746,11 @@ this buffer, in packets, for the matching output stream. The default value of this option should be high enough for most uses, so only touch this option if you are sure that you need it. +@item -muxing_queue_data_threshold @var{bytes} (@emph{output,per-stream}) +This is a minimum threshold until which the muxing queue size is not taken into +account. Defaults to 50 megabytes per stream, and is based on the overall size +of packets passed to the muxer. + @item -auto_conversion_filters (@emph{global}) Enable automatically inserting format conversion filters in all filter graphs, including those defined by @option{-vf}, @option{-af}, diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 1c95890f08..c57f0a43ad 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -725,8 +725,13 @@ static void write_packet(OutputFile *of, AVPacket *pkt, OutputStream *ost, int u AVPacket tmp_pkt = {0}; /* the muxer is not initialized yet, buffer the packet */ if (!av_fifo_space(ost->muxing_queue)) { - int new_size = FFMIN(2 * av_fifo_size(ost->muxing_queue), - ost->max_muxing_queue_size); + unsigned int are_we_over_size = + (ost->muxing_queue_data_size + pkt->size) > ost->muxing_queue_data_threshold; + int new_size = are_we_over_size ? + FFMIN(2 * av_fifo_size(ost->muxing_queue), + ost->max_muxing_queue_size) : + 2 * av_fifo_size(ost->muxing_queue); + if (new_size <= av_fifo_size(ost->muxing_queue)) { av_log(NULL, AV_LOG_ERROR, "Too many packets buffered for output stream %d:%d.\n", @@ -741,6 +746,7 @@ static void write_packet(OutputFile *of, AVPacket *pkt, OutputStream *ost, int u if (ret < 0) exit_program(1); av_packet_move_ref(&tmp_pkt, pkt); + ost->muxing_queue_data_size += tmp_pkt.size; av_fifo_generic_write(ost->muxing_queue, &tmp_pkt, sizeof(tmp_pkt), NULL); return; } @@ -3013,6 +3019,7 @@ static int check_init_output_file(OutputFile *of, int file_index) while (av_fifo_size(ost->muxing_queue)) { AVPacket pkt; av_fifo_generic_read(ost->muxing_queue, &pkt, sizeof(pkt), NULL); + ost->muxing_queue_data_size -= pkt.size; write_packet(of, &pkt, ost, 1); } } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 8665218dcf..3b54dab7fc 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -215,6 +215,8 @@ typedef struct OptionsContext { int nb_passlogfiles; SpecifierOpt *max_muxing_queue_size; int nb_max_muxing_queue_size; + SpecifierOpt *muxing_queue_data_threshold; + int nb_muxing_queue_data_threshold; SpecifierOpt *guess_layout_max; int nb_guess_layout_max; SpecifierOpt *apad; @@ -547,6 +549,15 @@ typedef struct OutputStream { /* the packets are buffered here until the muxer is ready to be initialized */ AVFifoBuffer *muxing_queue; + /* + * The size of the AVPackets' buffers in queue. + * Updated when a packet is either pushed or pulled from the queue. + */ + size_t muxing_queue_data_size; + + /* Threshold after which max_muxing_queue_size will be in effect */ + size_t muxing_queue_data_threshold; + /* packet picture type */ int pict_type; diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 19f719e3ff..39e0a31ea2 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -87,6 +87,7 @@ static const char *opt_name_canvas_sizes[] = {"canvas_size", NULL}; static const char *opt_name_pass[] = {"pass", NULL}; static const char *opt_name_passlogfiles[] = {"passlogfile", NULL}; static const char *opt_name_max_muxing_queue_size[] = {"max_muxing_queue_size", NULL}; +static const char *opt_name_muxing_queue_data_threshold[] = {"muxing_queue_data_threshold", NULL}; static const char *opt_name_guess_layout_max[] = {"guess_layout_max", NULL}; static const char *opt_name_apad[] = {"apad", NULL}; static const char *opt_name_discard[] = {"discard", NULL}; @@ -1564,6 +1565,11 @@ static OutputStream *new_output_stream(OptionsContext *o, AVFormatContext *oc, e MATCH_PER_STREAM_OPT(max_muxing_queue_size, i, ost->max_muxing_queue_size, oc, st); ost->max_muxing_queue_size *= sizeof(AVPacket); + ost->muxing_queue_data_size = 0; + + ost->muxing_queue_data_threshold = 50*1024*1024; + MATCH_PER_STREAM_OPT(muxing_queue_data_threshold, i, ost->muxing_queue_data_threshold, oc, st); + if (oc->oformat->flags & AVFMT_GLOBALHEADER) ost->enc_ctx->flags |= AV_CODEC_FLAG_GLOBAL_HEADER; @@ -3760,6 +3766,8 @@ const OptionDef options[] = { { "max_muxing_queue_size", HAS_ARG | OPT_INT | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT, { .off = OFFSET(max_muxing_queue_size) }, "maximum number of packets that can be buffered while waiting for all streams to initialize", "packets" }, + { "muxing_queue_data_threshold", HAS_ARG | OPT_INT | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT, { .off = OFFSET(muxing_queue_data_threshold) }, + "set the threshold after which max_muxing_queue_size is taken into account", "bytes" }, /* data codec support */ { "dcodec", HAS_ARG | OPT_DATA | OPT_PERFILE | OPT_EXPERT | OPT_INPUT | OPT_OUTPUT, { .func_arg = opt_data_codec },