From patchwork Wed Mar 6 13:35:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 46852 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d90e:b0:19e:cdac:8cce with SMTP id jd14csp389074pzb; Wed, 6 Mar 2024 05:35:28 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCXKD+bnT7EK8mCqQB7YAaiN9lv/4EwCqEmPuH3SBcTiamzBTtFX9ztg5YHIqUsHxQMAU+XLiJkTX8zz+E5g22PCMwkbpWBdY1ts1g== X-Google-Smtp-Source: AGHT+IGbgUYgUUGQUk40Htdo3dVHU61x3fj5c7t4PqJe0ZkVxPEGkBTlrnWC9Lz3a9txiN6pQ+cz X-Received: by 2002:a17:906:e0d6:b0:a45:5328:8432 with SMTP id gl22-20020a170906e0d600b00a4553288432mr5810028ejb.50.1709732127734; Wed, 06 Mar 2024 05:35:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1709732127; cv=none; d=google.com; s=arc-20160816; b=lsttXAbchYV5Zk8Z9G6j0DBBya9beOiB4SfFG8GyhueT1kzJu4HQPLC2aGRyxi+jlP 2XL72LhdkLR9YunUIH/zF5+ic4OHfa/u0bDLdjN4x7UKIJx8D8vwoxCUTmnQQLO5P2Ea PzuEMk4OP2AEN1pm6Eap1UlP4GYVPQKSQX9sAcGaikpm9uoF5KmdXY3fbvfFBN4D8hxS 8rSKRosZD5kdd0eQu1IRIrV+IYXr6PRn7IMWdEdlc4onlkdxJIkUvI7DIZ7kACZDLNw7 RpcBNWQmm8FdsPZ8U5Q76T7le4p/tqTIY2Rf/ICk7UOvH6k2OHWLJ77xiKRmNCu5Yw/9 P3Jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:date:to:from:message-id :dkim-signature:delivered-to; bh=CJO+YpwPqFx3ukInM5/sAcBJMga7eXeNXCFXEXWueSI=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=VVHpHUo8S/6rrAjPvZ4b9QeL/Od/AAbcO763QRW0OtjVSwUzbDzTVFSdppgOYp8Y5U LA5EtU96fxbu6AW1TDMpz0SdAMfePbRlufP5GBxRrs1CWmCIxePYGImlvFzfIYOQhqP6 1W9zmWDiMDwun89JDP8UTsdN54qiXXft05Gp1LAwuh4WD9yoxSbxm7UDWcsWR4jy8J0o RzF1eWXROiDtSGrxuenQIoiyBabcdrUCXnm39VqfEC5xfQbaoB9RaJIo87eQRYlFkydU owaFzTL4EIfKv0+xgc43qiMdzJ/KEnmJk6PRccYT0bUrQpOfgXRsiIx+gwHGCcrL1q3c hYvA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=mRg9Zo78; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y5-20020a170906470500b00a443be8cd2asi5777618ejq.209.2024.03.06.05.35.27; Wed, 06 Mar 2024 05:35:27 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=mRg9Zo78; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DE45E68CB11; Wed, 6 Mar 2024 15:35:22 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-252.mail.qq.com (out162-62-57-252.mail.qq.com [162.62.57.252]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E694D68C4F0 for ; Wed, 6 Mar 2024 15:35:14 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1709732103; bh=1f/r3OcO+Pv5TJYt4bBOR0N1JtKwLuWGobqR5VJhuXc=; h=From:To:Cc:Subject:Date; b=mRg9Zo78bN30RfAbi5PTMtBqJTEqsWwui0VBxvvVqxAkgb8exKEj2zA0Sh/rBTh8l wMUuxIyelsCB+fQMWGhkcpa3iN/GfHEd9OXr2VVrR5w0yYYZgURXI82WLYtdrsDZaH PymUhP1U+VewkfAPNsa44RtCTYb3k1Ftu1wm5iXY= Received: from ZHILIZHAO-MB1.tencent.com ([119.147.10.186]) by newxmesmtplogicsvrszc5-0.qq.com (NewEsmtp) with SMTP id 8C21647A; Wed, 06 Mar 2024 21:35:02 +0800 X-QQ-mid: xmsmtpt1709732102thoilyvev Message-ID: X-QQ-XMAILINFO: NIergYn61mJ1jHTzGyH/VCrzc+RU/WLF5AJ7/iDS71f1VOjE+7yF84HyBnmM1O k+3Ezrsir9W2nHShut11XC1MOJ03nDgK6bNI3iNqVcVBSmiY3NQ//D5WlhvB/tPywezHnxHe7ucb uBTkK02QSibjsxRRZKC5zhWQtOfOHbIj3E0jmLWpY8bvDgu/oSnK19giS9jpqfMIpFbWRskhMQsi fqNrBSXf8jSw+sN+KgLxcjcf0Mxm0iB4eTcDfbpaKZId+U8E44NQWlJqWbVrKN+lkRrw7gKBq+iV RqdhfPNmOtcQ1CISfAsJpc4RnNvXfFpmvtHQUTrEvpKdFj0k3BshVdaVG6L1TQ8t9FcAU67Bmf5C IkqYk9sKixjLa+yVdBKYnxKzOdC2ux9DqhJ0jivItOumTEoEKGMqoqCooLS+BYX+QrtHGsGDYfzx N+W3cfbikAA7kUmect08n5bX6znhpzzHX+IJFD3x+kmrDf11V4mNuGq5ez1HrXZTjpasHmqvUFge KSFvg8/xhPXAsIaxG8dO6uIdpYcQMqITRdi2gIjzPxUCCReTHc1AKTx9BFL3yjTgYK6+b27CH1S9 oAwkQ+zq7Gcf4rJEM3KJW1IKesoMOj5vu05Vv0p9QvgaHQOGLFwDcDlfwIpZIp+6iY5UkpeDG2X6 MEZ26GUsHfC1HCMsYYnOXgKtcF4YVAl51Xc++F8VlxueoYTmygb0ZBROf25dgmhwLpxBxFe1tnbA BcVi9s7B2OXe0UIrkuAZ6DsCLWig/phv/8I+RRx5Re2dw+yfquYdxpzzJ4M0f8pt4KRiycLA5OeW QTBsT2829hnOn6VDZ1e52orpkyjevwYEjbRyQcRMyVxpeXKcEM28NQSNgyLm/CAvvf9nk5ee1sY+ fjM5fuKEK4MW3AJVgbtk6dTn37UfIZL7AAYj4aOf0X6TVRrFQV2rWxkOfzDo7paI3VTKkEtbUi43 qxLTYzvPxPpAWpjucDRiUmWVdiwNXJMI6bgwVcrh5Q9RUyC9WN8TivwXnEx5O5 X-QQ-XMRINFO: Nq+8W0+stu50PRdwbJxPCL0= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Wed, 6 Mar 2024 21:35:00 +0800 X-OQ-MSGID: <20240306133500.15523-1-quinkblack@foxmail.com> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/libx264: fix extradata when config annexb=0 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Kza/PWkNtshd From: Zhao Zhili --- configure | 2 +- libavcodec/libx264.c | 148 ++++++++++++++++++++++++++++++++++++------- 2 files changed, 125 insertions(+), 25 deletions(-) diff --git a/configure b/configure index bbf1a70731..ef58c3fe97 100755 --- a/configure +++ b/configure @@ -3491,7 +3491,7 @@ libwebp_encoder_deps="libwebp" libwebp_anim_encoder_deps="libwebp" libx262_encoder_deps="libx262" libx264_encoder_deps="libx264" -libx264_encoder_select="atsc_a53" +libx264_encoder_select="atsc_a53 h264parse" libx264rgb_encoder_deps="libx264" libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" diff --git a/libavcodec/libx264.c b/libavcodec/libx264.c index 10d646bd76..10dfda125d 100644 --- a/libavcodec/libx264.c +++ b/libavcodec/libx264.c @@ -34,9 +34,12 @@ #include "avcodec.h" #include "codec_internal.h" #include "encode.h" +#include "get_bits.h" +#include "h264_ps.h" #include "internal.h" #include "packet_internal.h" #include "atsc_a53.h" +#include "put_bits.h" #include "sei.h" #include @@ -865,6 +868,124 @@ static int convert_pix_fmt(enum AVPixelFormat pix_fmt) return 0; } +static int set_avcc_extradata(AVCodecContext *avctx, x264_nal_t *nal, int nnal) +{ + x264_nal_t *sps_nal = NULL; + x264_nal_t *pps_nal = NULL; + GetBitContext gbc; + PutBitContext pbc; + H264ParamSets ps = { 0 }; + uint8_t *p; + int ret, profile; + + /* We know it's in the order of SPS/PPS/SEI, but it's not documented in x264 API. + * The x264 param i_sps_id implies there is a single pair of SPS/PPS. + */ + for (int i = 0; i < nnal; i++) { + if (nal[i].i_type == NAL_SPS) + sps_nal = &nal[i]; + else if (nal[i].i_type == NAL_PPS) + pps_nal = &nal[i]; + } + av_assert0(sps_nal); + av_assert0(pps_nal); + + // +11 for AVCDecoderConfigurationRecord, will shrink to the real size finally + avctx->extradata_size = sps_nal->i_payload + pps_nal->i_payload + 11; + avctx->extradata = av_mallocz(avctx->extradata_size + AV_INPUT_BUFFER_MIN_SIZE); + if (!avctx->extradata) + return AVERROR(ENOMEM); + + // Skip size part + p = sps_nal->p_payload + 4; + init_get_bits8(&gbc, p, sps_nal->i_payload - 4); + skip_bits(&gbc, 8); + ret = ff_h264_decode_seq_parameter_set(&gbc, avctx, &ps, 0); + if (ret < 0) + return ret; + + for (int i = 0; i < FF_ARRAY_ELEMS(ps.sps_list); i++) { + if (ps.sps_list[i]) { + ps.sps = ps.sps_list[i]; + break; + } + } + + // Now create AVCDecoderConfigurationRecord + init_put_bits(&pbc, avctx->extradata, avctx->extradata_size); + put_bits(&pbc, 8, 1); // version + put_bits(&pbc, 8, ps.sps->profile_idc); // AVCProfileIndication + put_bits(&pbc, 8, ps.sps->constraint_set_flags); // profile_compatibility + put_bits(&pbc, 8, ps.sps->level_idc); // AVCLevelIndication + put_bits(&pbc, 11, 0x7FF); + + p = sps_nal->p_payload + 4; + put_bits(&pbc, 5, 1); // numOfSequenceParameterSets + put_bits(&pbc, 16, sps_nal->i_payload - 4); // sequenceParameterSetLength + for (int i = 0; i < sps_nal->i_payload - 4; i++) + put_bits(&pbc, 8, p[i]); + + p = pps_nal->p_payload + 4; + put_bits(&pbc, 8, 1); // numOfPictureParameterSets + put_bits(&pbc, 16, pps_nal->i_payload - 4); // pictureParameterSetLength + for (int i = 0; i < pps_nal->i_payload - 4; i++) + put_bits(&pbc, 8, p[i]); + + profile = ps.sps->profile_idc; + if (profile != 66 && profile != 77 && profile != 88) { + put_bits(&pbc, 6, 0x3F); + put_bits(&pbc, 2, ps.sps->chroma_format_idc); + put_bits(&pbc, 5, 0x1F); + put_bits(&pbc, 3, ps.sps->bit_depth_luma - 8); + put_bits(&pbc, 5, 0x1F); + put_bits(&pbc, 3, ps.sps->bit_depth_chroma - 8); + put_bits(&pbc, 8, 0); + } + flush_put_bits(&pbc); + avctx->extradata_size = put_bytes_output(&pbc); + + ff_h264_ps_uninit(&ps); + + return 0; +} + +static int set_extradata(AVCodecContext *avctx) +{ + X264Context *x4 = avctx->priv_data; + x264_nal_t *nal; + uint8_t *p; + int nnal, s; + + s = x264_encoder_headers(x4->enc, &nal, &nnal); + if (s < 0) + return AVERROR_EXTERNAL; + + if (!x4->params.b_annexb) + return set_avcc_extradata(avctx, nal, nnal); + + avctx->extradata = p = av_mallocz(s + AV_INPUT_BUFFER_PADDING_SIZE); + if (!p) + return AVERROR(ENOMEM); + + for (int i = 0; i < nnal; i++) { + /* Don't put the SEI in extradata. */ + if (nal[i].i_type == NAL_SEI) { + av_log(avctx, AV_LOG_INFO, "%s\n", nal[i].p_payload + 25); + x4->sei_size = nal[i].i_payload; + x4->sei = av_malloc(x4->sei_size); + if (!x4->sei) + return AVERROR(ENOMEM); + memcpy(x4->sei, nal[i].p_payload, nal[i].i_payload); + continue; + } + memcpy(p, nal[i].p_payload, nal[i].i_payload); + p += nal[i].i_payload; + } + avctx->extradata_size = p - avctx->extradata; + + return 0; +} + #define PARSE_X264_OPT(name, var)\ if (x4->var && x264_param_parse(&x4->params, name, x4->var) < 0) {\ av_log(avctx, AV_LOG_ERROR, "Error parsing option '%s' with value '%s'.\n", name, x4->var);\ @@ -1233,30 +1354,9 @@ FF_ENABLE_DEPRECATION_WARNINGS return AVERROR_EXTERNAL; if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { - x264_nal_t *nal; - uint8_t *p; - int nnal, s, i; - - s = x264_encoder_headers(x4->enc, &nal, &nnal); - avctx->extradata = p = av_mallocz(s + AV_INPUT_BUFFER_PADDING_SIZE); - if (!p) - return AVERROR(ENOMEM); - - for (i = 0; i < nnal; i++) { - /* Don't put the SEI in extradata. */ - if (nal[i].i_type == NAL_SEI) { - av_log(avctx, AV_LOG_INFO, "%s\n", nal[i].p_payload+25); - x4->sei_size = nal[i].i_payload; - x4->sei = av_malloc(x4->sei_size); - if (!x4->sei) - return AVERROR(ENOMEM); - memcpy(x4->sei, nal[i].p_payload, nal[i].i_payload); - continue; - } - memcpy(p, nal[i].p_payload, nal[i].i_payload); - p += nal[i].i_payload; - } - avctx->extradata_size = p - avctx->extradata; + ret = set_extradata(avctx); + if (ret < 0) + return ret; } cpb_props = ff_encode_add_cpb_side_data(avctx);