From patchwork Fri Oct 4 17:54:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 52050 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:938f:0:b0:48e:c0f8:d0de with SMTP id z15csp582296vqg; Fri, 4 Oct 2024 10:55:56 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWb+8ZO3JP1hGppw69t9Xu1f7103U8RBuxlVrvZ6F0pipAY6vEyKP+AROA5X9RXcsmfAxLaDq93PEVvR1cx5VSA@gmail.com X-Google-Smtp-Source: AGHT+IF8wJbBgE4ovt5lEvHRlFI9MXV6x1QnPp9Ix4u8w0MkAJwcvMSzvEFhwhweEf4Z3Iwe8tCS X-Received: by 2002:a2e:be9b:0:b0:2ef:307d:1f with SMTP id 38308e7fff4ca-2faf3c00318mr5425811fa.1.1728064556672; Fri, 04 Oct 2024 10:55:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1728064556; cv=none; d=google.com; s=arc-20240605; b=P8el0b44KAIFdYowEqmyWO671qWbb1kZiWkENEySkDxrwD1zZ8VVjOYCeVcM8PrMPL 2/M1H6MeuKUYY2gnOV/rQmgZ/ieB9KmHoNE9bwJ3wBrrYLMBYmCDahhG0rMBeDI9llH/ BuzqiMTjMZPwPguVHLhQ68szpQZ2gi16oVzmbfYADFRBH2ERx9qYmVSvFbXTMpuZrY1Z +DN+kOuhseEwsSHcvv3IbCBhs18uARrVTbKnRjexP5GyVezWxk+fk7FGyBAu0DmVwTJH DrSNyzMKSgavFs508tEdY2jXYgYtRR1vCwyPO7QhwNccryOshS3CPTIr1ioKm41QSLlV eCFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:date:to:from:message-id :dkim-signature:delivered-to; bh=D00whelKNbDRydjQ7WxJAW28yD1ve0m0nLUfhxfQ9Q8=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=S0E4K/6dfVoSlWaMDEimKYu6/nxY80mjsmR+MnL05EjTuocHBor5TqkTD4V70xIEp9 BFBrC4kznBVd51XmezucYDMAIxU83Y5R9Ho3VMRGGYsioKiONIYg6fgeJitQ21twJAOl YUDfF2Acs+jq7hAdRXFM/0gD9+jVBacbyUivEaNbwgFqy64ApXREX3/m0u/xK5XHxg/y UCgDelTb4+Ncj1pkjAttFqDG+RlevnUpEqm+CcXt0hvyy6FNAFFHaK60FWoYWD4uBEh2 xrXrUZEMfEzyw97TsbSSFth+m7ojLVdn6NZgjoXZ3W6TEgyceB1MP4bh10srldBaZ8oi 7Geg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=mfl5PQcx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2faf9b22f80si865191fa.280.2024.10.04.10.55.56; Fri, 04 Oct 2024 10:55:56 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=mfl5PQcx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2CD4568D56A; Fri, 4 Oct 2024 20:55:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-221-242.mail.qq.com (out203-205-221-242.mail.qq.com [203.205.221.242]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 30E5468D56A for ; Fri, 4 Oct 2024 20:55:43 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1728064540; bh=tUsoqHHOZxktkV7jOGs7oSyRepXcYP7iGwgKHiZJrJU=; h=From:To:Cc:Subject:Date; b=mfl5PQcxnKcRnxYUUQE577Gp+JxoZ8ZG5sFmJ0vR8rQvnTUFvgREqZ+iphCdkVXDu BGMwTo/Qttv+ARD0S6hJ3Ub4DwePr6zDXcWF5YyOdcDVl+e10aF+ox8TF+0dZrbhmo SCXdmptGtH0mBvMHenBFwQj4u3boCTqGCVT6cWJw= Received: from ZHILIZHAO-MB1.tencent.com ([113.118.105.142]) by newxmesmtplogicsvrszb16-1.qq.com (NewEsmtp) with SMTP id D97A6077; Sat, 05 Oct 2024 01:54:23 +0800 X-QQ-mid: xmsmtpt1728064463t2r43u4y2 Message-ID: X-QQ-XMAILINFO: Nb13eR/nWe3klgbbsNhuRGMORzDz3YN60JqEPV3tcL6ByvbE9tYl9GvRLuotFt yzoxb+VC14HxIempQ3vpHOtJKRKJvNwtOHGjD7D7V6JeLb9Bl4ewQXgqB5v3H2CNc/syfPHVHksn +JtULIxJnGzPGjQ6Bs1v3SHN9tBcAkM0RvDtRx63iSOkQy1HE55c+m2hjh3X6YA3BJQmXaKgI7mr Sa1uwufUy+yWp9jRnZoWRtMOFjcuSr6L/q0DkJnGUNoYkYmL2iEs1e6VMCq2lIgD/r2kxRyKr8C3 dMzPcsvfEc/z1ImRwnVip9P6GtS+U276zmvK+TcCR7MaSuSFyYJ1TJ4fgzJWe+hcNJbJSnkCEDsl IBiZj6vn2jlNMZfcPyN+8Y4uuMkJ7m+j03RE+TAu0IYbNWuVn/xD7J4m7x5PhzeDRgPbmZ18I62T aJKl7Ub/tRVrpoJIlCNGwwDv4PbPlzMy/Twno0XXNOzindU0FMdfKBDLaKmYbTYkmIN/ukYQsGK2 3vSLG3cu8HNS+Hj5e4usrsQdm45vSj9rIK3MQZgUGelSpce/Z/xknICvnu+PthJMiiMbeSCUTqEp 2EbjBQLCHZXTNGqujhQw1A+Y98CdA3gDXEfEMVvWq6s/+swDYUfpf1RbrctKRl+v4M+iyRZre+Nv nSRE8jS9mktyO6o7tgXt5QVWUJavuskFfjBlddW6VpAG9RGwFCK2tWOR9CGupmfTZAiCfO+IPjLB fXVx3+lzRX77Z9XaEkwxf3sJwgqJ9KHbMQ7fOFoicx/fh6rupAXQsQLFKNMJnU+Iu1HzoC8O+Hrh KvNBPrIZ5xxrC20swcbNC0GxbJL7Ot3f8nR647m1CkARGtBmxYq/1Uxb4XTgnHa9Inc4SLqVCUHN 0DAl9xllTI0OQFIIFTAtFa2eZOmAMoCdqhd932DtuW17KSpTXedDyrMCDlxh4HWsYz8yDcirh0im Kx2gAErxmTrCpLzchfv0BYJjJqoI+Pw9SjBIajhac= X-QQ-XMRINFO: OWPUhxQsoeAVDbp3OJHYyFg= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sat, 5 Oct 2024 01:54:21 +0800 X-OQ-MSGID: <20241004175421.1157-1-quinkblack@foxmail.com> X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/mediacodecenc: Extract configOBUs from AV1CodecConfigurationRecord X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ykNDa4mfzmNS From: Zhao Zhili MediaCodec can generate AV1CodecConfigurationRecord, which shouldn't be put into packet->data. Skip four bytes and extract configOBUs if it exist. --- I did some test on Pixel 8 Pro. AV1 hardware encoding works with a lot of bugs: 1. It's broken for width non-aligned to 16. For width 1080 and pixel format YUV420P, MediaCodec use 1080 as stride. For pixel format NV12, MediaCodec use 1088 as stride. There is no API to get the stride info. AMEDIAFORMAT_KEY_STRIDE doesn't work. And set stride to MediaCodec has no effect from my test, at least on that device. We know the buffer size provided by MediaCodec, but we still cannot get stride by buf_size / height: 1) For YUV420P, buf_size = 1080 * height 2) For NV12, buf_size = 1080 + 1088 * (height - 1). Yes, buf_size doesn't count last line's padding :( 2. After flush (which means reset for MediaCodec), it doesn't reset GOP info, so it output a few non-key frames before generate a key frame. This breaks what I did for AV_CODEC_FLAG_GLOBAL_HEADER: send a dummy frame, send EOF, get packet and extract extra data, then reset MediaCodec state by AMediaCodec_flush. libavcodec/mediacodecenc.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/libavcodec/mediacodecenc.c b/libavcodec/mediacodecenc.c index e76ea81236..3880ea2fe9 100644 --- a/libavcodec/mediacodecenc.c +++ b/libavcodec/mediacodecenc.c @@ -429,6 +429,16 @@ static int mediacodec_receive(AVCodecContext *avctx, AVPacket *pkt) } if (out_info.flags & ff_AMediaCodec_getBufferFlagCodecConfig(codec)) { + if (avctx->codec_id == AV_CODEC_ID_AV1) { + // Skip AV1CodecConfigurationRecord without configOBUs + if (out_info.size <= 4) { + ff_AMediaCodec_releaseOutputBuffer(codec, index, false); + return mediacodec_receive(avctx, pkt); + } + out_info.size -= 4; + out_info.offset += 4; + } + ret = av_reallocp(&s->extradata, out_info.size); if (ret) goto bailout;