From patchwork Sun Dec 4 17:12:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39581 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp2708264pzb; Sun, 4 Dec 2022 09:13:22 -0800 (PST) X-Google-Smtp-Source: AA0mqf5ZfXJVuFV5IQh93sOmKN3X8Y1RML8uUwMiApR6lcvQesOykg2xgqFS4SL8wuPaOJmDHDwl X-Received: by 2002:a17:907:a688:b0:7ba:ba67:f2f with SMTP id vv8-20020a170907a68800b007baba670f2fmr42156654ejc.199.1670174002706; Sun, 04 Dec 2022 09:13:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670174002; cv=none; d=google.com; s=arc-20160816; b=BAbNtkwbh1lKssvBD2Dypq5fS+AOQquVdv306YRLk6w30+jpt1Mxk/azPkeI7rga2r jeEgx9Dl51GLhVvIzBc/uNGgtD1076oXu+Wg0K5ebR3WeMddoeFoNMf22ULRywdF3ae6 QowTn2B5FgxFcoEs2/PjMX8qLsJorTJLEMVS0Sw/9Zw+cFFoh3Bjz4Up0WIxHyspvLgO /Ks5CnTk1cqrnwuZZmadf5qPPiPT2Rk2xeGQsne2Q7vz/P7npQjt0nHyv3AuZRiUfN/r 8WYowrK5sYfqTy4dvAyJEYMCMOSvuBv6//d9MAYGAL+sRRRsauSLBXX7WfdHOfSiTE2D XD7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=AxeanIbXaN7rh3zr7wxBO5hZjpNcmlJtuKC06mORrHM=; b=gzYcXcS5JP2e64FGTnRjaA1TT54zSp3u23uIMflnRx1QAk+laSLUdAIIN7vgUqXJmA XjbH757CMC0nXB5zIolcjQn4MvUcC0CS+MXG9g2UftoFSROv2mB/dHjY30Zy5YlTywLU 8utb3f1TkVMqosHvTEMsjT6KTESyrEbUSzOWM8Bx/WGhJwFGCN1fKarTzY2TqOD1JUlf bvqtx1pI4O3SQvKH99T0EPC/L2P+O/JCAkpeKb7b8zjuzSU+JEkZ/RXRT90BJiupF49t GTzyHyklzmOuwArhB+0+4LUMhhSW7ETyJITtaR3m6gkV4kgcw2NOd1YiaZ6rscOUkzD7 NZxQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=cQnKUDV8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id vk9-20020a170907cbc900b007aeab6a1174si8840344ejc.633.2022.12.04.09.13.22; Sun, 04 Dec 2022 09:13:22 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=cQnKUDV8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 41EB56801B0; Sun, 4 Dec 2022 19:13:07 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-64.mail.qq.com (out162-62-57-64.mail.qq.com [162.62.57.64]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C6BF068B7BA for ; Sun, 4 Dec 2022 19:12:58 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1670173976; bh=8zq9GMBEMvQyQI4kY+SwJU2IG9ONVPQ8b9w7Bc67v8U=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=cQnKUDV8owRAwW3Q62RzYJgTksJNmij//tYEN73GrbjKVstw2f5/RzETSywzuZAjO 8/BC+dHUSgiixRbA/8VILoZ8+1CCfD/C1bXmKCkgIyb8Qli/URX4R2sUd/2bcQcyUQ jq1YON/WEILETeydkmb23i+a3Y+GA6M8jMDAlAtg= Received: from localhost.localdomain ([14.153.79.182]) by newxmesmtplogicsvrszb1-0.qq.com (NewEsmtp) with SMTP id 334B3853; Mon, 05 Dec 2022 01:12:52 +0800 X-QQ-mid: xmsmtpt1670173975tivkjaish Message-ID: X-QQ-XMAILINFO: MFWpArBVhhGTKcLgicnQJFhp4AMLCYLFI008rvaHy1+fuvZMHOi0ogBkLs1eF3 uWKzCAp0Cf0HXXTXcyYgf5XN3YggCevHl1o7+BBOjIkFy8q9dKZ1wISX4WV1IyVbRvUsaTKE0Mzl LPaN5s3Ym6v9s0vFUGfSJhuEVD9JoN/eSmz1r7j5izXTPpioI0iLX16/ej9XaqtGiX636nm2l7v6 zFut+DhI2bOGUeV8V9E9mwgDjtAVR1MxT2pbA1UvfZus9VmkCwpH4DuhOjF1qJWhMutnanlQSTJL BlZ7FpKaKr87oghCk/o8fW1j8cMcpgimse0UW5Ww1dAopKlCDs1JYUZbFaDQJfvP/Tk+FD54m3Pd ZDKo05JmbMctksgIKlUNjdlzWGMGM3kQF9Wiol6RA75YCzlZoTZ3boq5EZIisf5Bc4rUTLCOTbF2 d3Z+2sRbYez59di9tpdYDORqzHFWJTxIHWjxpeILM9G/m23alOkKlJCiGFlkokjJj4TIRPXWUv48 +lTGJmqIhFobTJHDqvX8C/G0qGCkPitbppSR/wkWEzJBap/j5a0OSiE2WLCbBn+23N4ZZTRJKqbr y2dRmpS3fLMHd3W25oEn3MwXKeUlOGuE87k/pISZ1ZN3yGlfzlhT95g13rsVsf5zVSqDM/3UYomE UG+2vrwgq9xEAFMvmJnCQpr58OZEL0Km8skMM8WMt12TA/OjwkP/vYCUpRG7H+O3L7FFH3eW4hCm VPsRMemM4lX0XHvOiiGWjAcfIlpy8ul5ylPIQMYBhyP7vdCyJhji9NWfeInV60KSFICDuwDWYfu3 pVhmprVb2NDpqdtkXyZHTPl0h9IodWM3rc+Lr/uU/dshkMgq2L/ZjUcKiGoZ9vNIIVM2t7XQURQ/ rtm64nr2KGdlE/wlsr59860ll8krV6xoBbECCvmsHYcTqCmOcc7Qy5/Riz9yAABa6mwak3rblvl9 0IN0bkJOe7Zvj9Yt+hDV7jKc0uzSyTmKQ+xL95y7s= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 5 Dec 2022 01:12:26 +0800 X-OQ-MSGID: <20221204171228.50160-5-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221204171228.50160-1-quinkblack@foxmail.com> References: <20221204171228.50160-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/7] avcodec/mediacodecenc: remove the strategy to create DTS X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Tdjb+YXaAVti From: Zhao Zhili Use input PTS as DTS has multiple problems: 1. If there is no reordering, it's better to just use the output PTS as DTS, since encoder may change the timestamp value (do it on purpose or rounding error). 2. If there is reordering, input PTS should be shift a few frames as DTS to satisfy the requirement of PTS >= DTS. I can't find a reliable way to determine how many frames to be shift. For example, we don't known if the encoder use hierarchical B frames. The max_num_reorder_frames can be get from VUI, but VUI is optional. 3. Encoder dropping frames makes the case worse. Android has an BITRATE_MODE_CBR_FD option to allow it explicitly. --- libavcodec/mediacodecenc.c | 28 +--------------------------- 1 file changed, 1 insertion(+), 27 deletions(-) diff --git a/libavcodec/mediacodecenc.c b/libavcodec/mediacodecenc.c index dce4a96d1f..4b0397d1ea 100644 --- a/libavcodec/mediacodecenc.c +++ b/libavcodec/mediacodecenc.c @@ -64,18 +64,6 @@ typedef struct MediaCodecEncContext { uint8_t *extradata; int extradata_size; - - // Since MediaCodec doesn't output DTS, use a timestamp queue to save pts - // of AVFrame and generate DTS for AVPacket. - // - // This doesn't work when use Surface as input, in that case frames can be - // sent to encoder without our notice. One exception is frames come from - // our MediaCodec decoder wrapper, since we can control it's render by - // av_mediacodec_release_buffer. - int64_t timestamps[32]; - int ts_head; - int ts_tail; - int eof_sent; AVFrame *frame; @@ -368,11 +356,6 @@ static int mediacodec_receive(AVCodecContext *avctx, } memcpy(pkt->data + extradata_size, out_buf + out_info.offset, out_info.size); pkt->pts = av_rescale_q(out_info.presentationTimeUs, AV_TIME_BASE_Q, avctx->time_base); - if (s->ts_tail != s->ts_head) { - pkt->dts = s->timestamps[s->ts_tail]; - s->ts_tail = (s->ts_tail + 1) % FF_ARRAY_ELEMS(s->timestamps); - } - if (out_info.flags & ff_AMediaCodec_getBufferFlagKeyFrame(codec)) pkt->flags |= AV_PKT_FLAG_KEY; ret = 0; @@ -437,14 +420,8 @@ static int mediacodec_send(AVCodecContext *avctx, return ff_AMediaCodec_signalEndOfInputStream(codec); } - - if (frame->data[3]) { - pts = av_rescale_q(frame->pts, avctx->time_base, AV_TIME_BASE_Q); - s->timestamps[s->ts_head] = frame->pts; - s->ts_head = (s->ts_head + 1) % FF_ARRAY_ELEMS(s->timestamps); - + if (frame->data[3]) av_mediacodec_release_buffer((AVMediaCodecBuffer *)frame->data[3], 1); - } return 0; } @@ -463,9 +440,6 @@ static int mediacodec_send(AVCodecContext *avctx, copy_frame_to_buffer(avctx, frame, input_buf, input_size); pts = av_rescale_q(frame->pts, avctx->time_base, AV_TIME_BASE_Q); - - s->timestamps[s->ts_head] = frame->pts; - s->ts_head = (s->ts_head + 1) % FF_ARRAY_ELEMS(s->timestamps); } else { flags |= ff_AMediaCodec_getBufferFlagEndOfStream(codec); s->eof_sent = 1;