From patchwork Wed Dec 7 09:31:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39636 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp415257pzb; Wed, 7 Dec 2022 01:32:20 -0800 (PST) X-Google-Smtp-Source: AA0mqf7ztDOl32MJ/Ys4AgrLJZsY9bzULVQ0DQz1SwyriKC+JOVSODfsuCYmThqf1Nt4551pyqwF X-Received: by 2002:aa7:c1d0:0:b0:46b:431f:f2d4 with SMTP id d16-20020aa7c1d0000000b0046b431ff2d4mr13669041edp.198.1670405540033; Wed, 07 Dec 2022 01:32:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670405540; cv=none; d=google.com; s=arc-20160816; b=ivDQURnlQRskSxNqLRhxsywqctPQA2LhVBh/a0tQc6P7P8xMlf74iwEB6h4ibvMY1x FjML0BPs+Hlksqd8RanqkPjwemMAemDfMjqhbSzIqbNOBlBF3Weux8PeJfJkaXuO2ri6 m+774c/sIXIKauZtkY2OAb9ObkKUrfG222n3mkYzxaO5/qSxEdU/m1ydsvFFlIQK8aMM BOSCH6648v7wr7W4xC5PA7+RVKntMxMTpyk5RhEScX/tOGTjFQMhrGkeEXmCiozGkJUg j6TSM15FRB45wjKWnXd7pL8mK0RZGTksDcncjHEJqRi2Rtd4Jrs9vx31/BpQmBrB+bKX YE5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=30UtV/9PhXtePmjvQrgJkZLvFt0/SGGtSzF1i4JDwo0=; b=w+Ga7EoU1qyGU1jPy2nU9HpeDwdbLVIYsYt9exRQxk3XxvWaqh5UMponxsNKwrET5h jzxwti4NFfGXRN2JqsORo7T1fQZ7EK5itz2Bn8YLdh1wCQBIE34GGYpwreQZCAGhh9D5 +ZxE3N0vQB735vACQUHy0gKFlKQsGISInhCfGHawM5AHQNnSu1vpF+VpK2eU4dQc46Q3 ByWeUaAQEZiM8WdNGxn7WoAUO9dbPGpzMdjlVXMgcINS6ZSc/iMY1COw37GKggry7b2N dQ0/r8uJfXYhiU/KpVhijpjo3o00XH6iIrRXGXdPTRGEUL+K9rUh277z4u56k/PcZnK/ AAog== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="UfHM/+xV"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c23-20020a056402159700b004537a3c4982si3509549edv.601.2022.12.07.01.32.17; Wed, 07 Dec 2022 01:32:20 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="UfHM/+xV"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F20B568A99E; Wed, 7 Dec 2022 11:32:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-210.mail.qq.com (out162-62-57-210.mail.qq.com [162.62.57.210]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5857768BA90 for ; Wed, 7 Dec 2022 11:32:04 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1670405522; bh=yOSJ4SXhktbN/yhgiGcvH0W/tBvUUpRObQ8woZHHHII=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=UfHM/+xVGeAC0Xu8YDcja/mUukZmuf0UOFM+azmCFpPNEtUcXWUQjJwHPjkPqmDEl 8KMn66pe5tVwXBnbI3rWN63SWXiwJo2be8oMdzyOROWEiHhFwWNj184ezfYWeY5kMI gOGhG84Ooi2/bmaE27E8An9G1D3bBEKfMZui2aTk= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrszb1-0.qq.com (NewEsmtp) with SMTP id 7F9B046E; Wed, 07 Dec 2022 17:31:57 +0800 X-QQ-mid: xmsmtpt1670405521tc2srdryf Message-ID: X-QQ-XMAILINFO: MFWpArBVhhGTkxJPFv/wd4dFsUKacApK1k8n/LIbElBZNZPKurJbcEi4yyYn+f u6y4n/Dk+P02Zm6KDonKP541d8IAno624MfdWi6/ITKp4ILiXgozizBnM3tLULOq3Bct/dBO8N5j Rco1akbdHm/127Tz9nM8sxlk/qUABQvyIoEWr6FtrbDG2Bs9NuQ0IeEv+PAxb6sFY+ZlXoFMMIHI 3+GKT1Lm4TJ4ROtchVPNiFi1W4w/9PQd9AlRqVI+fQawWczt65o5M+KCZysL9/9h+9B0ouaHziOM gzzG3nV49H12Q4yMvka5jX963yQvOdiInQUoUqoZaBUgNcWSQUXxTroa4/2jsDQr9q/0udqEHrIX 09PO0pFu+veZ3CSTrlGiStwqZ/RraJG9luWh9bkGzqh9zJFDM9blwd3cA9t04ioWD/SA6JXYWMtO ELBcb1ZJtvZAin8IGPwwDnpdnfSNXU085er71Nn1PcL3IervhWTxTscRqiCAOcxvB7xfOUeLHW5y +LA9VCyA2ZqtOqbzIxXtfa2a+C47TinnYnCdv43nKyec6NgZN8ylhdbUEu0keaWC+YMi0P8Ha9Ot roEc0B922da7zAQIqL9DMJodr+KUIEBp57GImjM37PaugLHuLhBKeCKb9doTsWszh/4HQvA7z/jp zt4KVTIEch6/PE24JLjhlm0DeS1ExWHuSXJqxHPKip07W4/Lhbsuo1rT0umOkjTVTxetIdYKKOo4 ZiVnrySWyuHPs3gKrfR3aCrXuYQ7SM5T1N5fkpk4xaedq78qzDIhJHyrFCTIKwEWqkwf2IWzokr6 FUpQPUV7X7ti6fWsqNGCHK1hmWQbrSbIreZKbxb1nuFy+sg5/BEuD5GMxvCVhRMo0FVD8ukhXHju wOQrgar2ladQLvISK8J6PpdrMqHB24SRwuP9SvPFjj6pRgqKyiD2ZIMeEXXgPOZovRqjyRRuiicU JXfT3biWAMcjE8EOHN6PyzLXiqPrnz From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Wed, 7 Dec 2022 17:31:20 +0800 X-OQ-MSGID: <20221207093122.553668-6-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221207093122.553668-1-quinkblack@foxmail.com> References: <20221207093122.553668-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 5/7] avcodec/mediacodecenc: remove the strategy to create DTS X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: mHRoyLRcZ0Oc From: Zhao Zhili Use input PTS as DTS has multiple problems: 1. If there is no reordering, it's better to just use the output PTS as DTS, since encoder may change the timestamp value (do it on purpose or rounding error). 2. If there is reordering, input PTS should be shift a few frames as DTS to satisfy the requirement of PTS >= DTS. I can't find a reliable way to determine how many frames to be shift. For example, we don't known if the encoder use hierarchical B frames. The max_num_reorder_frames can be get from VUI, but VUI is optional. 3. Encoder dropping frames makes the case worse. Android has an BITRATE_MODE_CBR_FD option to allow it explicitly. --- libavcodec/mediacodecenc.c | 28 +--------------------------- 1 file changed, 1 insertion(+), 27 deletions(-) diff --git a/libavcodec/mediacodecenc.c b/libavcodec/mediacodecenc.c index 7f2ae88285..8e28a50e0d 100644 --- a/libavcodec/mediacodecenc.c +++ b/libavcodec/mediacodecenc.c @@ -64,18 +64,6 @@ typedef struct MediaCodecEncContext { uint8_t *extradata; int extradata_size; - - // Since MediaCodec doesn't output DTS, use a timestamp queue to save pts - // of AVFrame and generate DTS for AVPacket. - // - // This doesn't work when use Surface as input, in that case frames can be - // sent to encoder without our notice. One exception is frames come from - // our MediaCodec decoder wrapper, since we can control it's render by - // av_mediacodec_release_buffer. - int64_t timestamps[32]; - int ts_head; - int ts_tail; - int eof_sent; AVFrame *frame; @@ -368,11 +356,6 @@ static int mediacodec_receive(AVCodecContext *avctx, } memcpy(pkt->data + extradata_size, out_buf + out_info.offset, out_info.size); pkt->pts = av_rescale_q(out_info.presentationTimeUs, AV_TIME_BASE_Q, avctx->time_base); - if (s->ts_tail != s->ts_head) { - pkt->dts = s->timestamps[s->ts_tail]; - s->ts_tail = (s->ts_tail + 1) % FF_ARRAY_ELEMS(s->timestamps); - } - if (out_info.flags & ff_AMediaCodec_getBufferFlagKeyFrame(codec)) pkt->flags |= AV_PKT_FLAG_KEY; ret = 0; @@ -437,14 +420,8 @@ static int mediacodec_send(AVCodecContext *avctx, return ff_AMediaCodec_signalEndOfInputStream(codec); } - - if (frame->data[3]) { - pts = av_rescale_q(frame->pts, avctx->time_base, AV_TIME_BASE_Q); - s->timestamps[s->ts_head] = frame->pts; - s->ts_head = (s->ts_head + 1) % FF_ARRAY_ELEMS(s->timestamps); - + if (frame->data[3]) av_mediacodec_release_buffer((AVMediaCodecBuffer *)frame->data[3], 1); - } return 0; } @@ -463,9 +440,6 @@ static int mediacodec_send(AVCodecContext *avctx, copy_frame_to_buffer(avctx, frame, input_buf, input_size); pts = av_rescale_q(frame->pts, avctx->time_base, AV_TIME_BASE_Q); - - s->timestamps[s->ts_head] = frame->pts; - s->ts_head = (s->ts_head + 1) % FF_ARRAY_ELEMS(s->timestamps); } else { flags |= ff_AMediaCodec_getBufferFlagEndOfStream(codec); s->eof_sent = 1;