From patchwork Tue Mar 12 14:09:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 46997 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dc95:b0:1a1:738b:6bc0 with SMTP id ky21csp1891422pzb; Tue, 12 Mar 2024 07:11:04 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX1ul7U14RwhxnuwPfi+wwUOzuyT8q1kdD0GZlCteU1tM/7Efm41pCqUHL3CJLpkGoaDQCjrLxBSOlhKecKwhzmLD0oTrjCgqdYUw== X-Google-Smtp-Source: AGHT+IHWe6FWvoDD/wzq2fV8QM+F+O8TOROXYyIBEQ3WN7LcSnIt0IWyFkPPy/2Ds1dcgYCEujLk X-Received: by 2002:a17:906:7f08:b0:a45:200e:171a with SMTP id d8-20020a1709067f0800b00a45200e171amr2786279ejr.33.1710252664481; Tue, 12 Mar 2024 07:11:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710252664; cv=none; d=google.com; s=arc-20160816; b=c8gn/aCBc0tyRtXXhP8fPCOUI8SCQQdWerG3zECKbwUwYZT8wfYH6YUHWe+obJBYlg vguVqeSUHlhL4e3hXqKRfjifmsRzNu8QoPsv2+ZRkCry0pl28BGEt+WvXed+qEepTh4Q vnEYoBeKhomj884OnvAreDmHDmfOLeVu+AprTdK2+kI+KdXNivgRq6Xl3pIi0UpR/w6q HIGQrgS+N6nD2Cdg5p102JhZXFrz2HcgRpfpuGbgJLjK2WgYUxC0u2TB4e4ooOtP2ZIz EWX0qStNLKsAM0oW9VVktbvG9s8J7gntlN2U5uNGha+sytDhq+X2lu3pmYQlnDL268Cp UbXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:date:to:from:message-id :dkim-signature:delivered-to; bh=5sSlE719PGbRfSbxz4kpBYMYk+mwr/kIxH/pCzJGPYs=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=jBfUK5jQ0IdLWQig5W1ysQC/UUyQu5AXVCh13K4JesJZ7EBonSVIeLT8OeFM+p1vXn sRZXAaMlh4G5OLXnVuMeQoWfwN3CXdhjmqce4OAnih5c6KXevYTZWlSfyDFIPMxARnrr HBX0L7WKfNPrirhSogED40wDv6COOfvQLRN7+ZO37ay5vbaTScaOID1FLsmOJKM+8Tf/ ikDx8AsfUqnUvFrtsvW0XLjMtTpbPqVdF3F7Hlc4UbERmOqq+sttBvvYhdxgXVfTuN5H R/c9HrJdPSblZ+xPguMU19moJRybr1QB49pFl0j/0kr6Nyo/tSOEzi2/xHZQ5yWk9b4+ gEcA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=ByReyKyu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i24-20020a17090671d800b00a461af7f053si2429674ejk.807.2024.03.12.07.11.03; Tue, 12 Mar 2024 07:11:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=ByReyKyu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CEBB068D0B7; Tue, 12 Mar 2024 16:10:59 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-49.mail.qq.com (out162-62-57-49.mail.qq.com [162.62.57.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4184068CFF2 for ; Tue, 12 Mar 2024 16:10:51 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1710252641; bh=tTSV3ReMZd+mhRyKhuhQahI5Vcj6zyP0zICph6bTyEo=; h=From:To:Cc:Subject:Date; b=ByReyKyuCiJ9X+mDlj3hALdhFD1+c9doeAEQYsvJvJhL8qdROjisqfa2P1qBAFQzg hYlFwYK0xUKjQZrBVABifLghTS9UpAGPiWUy1wtRA+1xN8a3L1nRJ6OFOz2g4EaOCm gN/2I1jXGXM0uSjqidrdCaeFx2mPlnS/dhR5tCeI= Received: from localhost.localdomain ([119.147.10.188]) by newxmesmtplogicsvrsza1-0.qq.com (NewEsmtp) with SMTP id 2A8394E1; Tue, 12 Mar 2024 22:10:40 +0800 X-QQ-mid: xmsmtpt1710252640twlfz54sk Message-ID: X-QQ-XMAILINFO: NPETcIssjKadEBUYZREMryAOFpehCS1nAEZJxENqVh7GpF5diUeNmqjTl05eU/ QS5Z0Wrxc22tfO1a7FFNLBU1ycjIgVlAmn5axNe9h4tL6ojrXO7nRtIW++VEvLGgN75iKTmPFUeJ QzX5lSchOWtNtwXgLyeWNv2x7/9PwBsCd1HsqQCzyW6Z/piePaiW8q2/PXsvQvRLb3HpbZyMzuea 4H7BVFotYThIdIo8Q9/Vg2aPQOq8gCup/Bxhnp7GinJ+3rAeZJi9H9r6uLDw2sbTeAMLSf8bWWYD HPHEvJHteUJAyzhLZOa0iPqdkNTSVEvsJZB+OqRj8ixR1ctdQ+sbPu4z3dxkBVWinQumvmzs9QdD HgahdPN+beVSVZCPJg8M9FdnzAsej3uyfysiG+PNibVnEtGf5PonPvT+Fkj0xuRGErP6qdsMbvfJ +95LO2F/VCppx19aw94+MNVZMIjjwhHGVCqu5NEi0K0+J04SFqCq6UI+651tucf6Z4AsyGtcPTYP QEm6yWSiiPK3gPxlY108tHO5pC43/6/OmIg13qOFURFjlIdQWZqoC62fxGfDOpiSF+wEdMGMgzot q3KKuTznHibnZs1DzO7U6NiFIS5Mf1NjeEUP+ciI6QLcyOxK9/Xfv9CVKeBSqYYbDbld0lKNTWjC s+wAOIfTdvssD8uAufRrs//DKb9xTqYYV1AophLJA3At/qBsJGnIsnFUuOcqTv2IUMeWmWJha51Y heEbjaFgD51yja70zOHvP9N65YTglx4wM4M0/D9cSTVbQqEQYeTQbaeL2C1gPBpgZ83TI/oL1N3H H4xBky91aDQ9LGBxddmZJD3JR9C7dI1Tqc2ypszfmtsTCyAsYImxQOv8vYNeG5teOhax7Go67+cT J1NrP2NYMSPT5Ee+07JQuhboVjRdiVYSUV6aDSPK9odcBDtzkWFSz39N5V6SJn50AmWeqj57Xhh9 WIELMYsqMeWWrf/3CyGG7j6kigeKHg+RZ6dAgP77BuCq5eFKJcxw== X-QQ-XMRINFO: NI4Ajvh11aEj8Xl/2s1/T8w= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Tue, 12 Mar 2024 22:09:45 +0800 X-OQ-MSGID: <20240312140945.5940-1-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4] avcodec/libx264: fix extradata when config annexb=0 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: lVPUBssWNANo From: Zhao Zhili --- v4: Fix missing SEI in set_avcc_extradata v3: Remove unnecessary inclusion configure | 2 +- libavcodec/libx264.c | 153 ++++++++++++++++++++++++++++++++++++------- 2 files changed, 130 insertions(+), 25 deletions(-) diff --git a/configure b/configure index db7dc89755..24cb897d28 100755 --- a/configure +++ b/configure @@ -3491,7 +3491,7 @@ libwebp_encoder_deps="libwebp" libwebp_anim_encoder_deps="libwebp" libx262_encoder_deps="libx262" libx264_encoder_deps="libx264" -libx264_encoder_select="atsc_a53" +libx264_encoder_select="atsc_a53 h264parse" libx264rgb_encoder_deps="libx264" libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" diff --git a/libavcodec/libx264.c b/libavcodec/libx264.c index 10d646bd76..e7d16997d2 100644 --- a/libavcodec/libx264.c +++ b/libavcodec/libx264.c @@ -34,6 +34,7 @@ #include "avcodec.h" #include "codec_internal.h" #include "encode.h" +#include "h264_ps.h" #include "internal.h" #include "packet_internal.h" #include "atsc_a53.h" @@ -865,6 +866,131 @@ static int convert_pix_fmt(enum AVPixelFormat pix_fmt) return 0; } +static int save_sei(AVCodecContext *avctx, x264_nal_t *nal) +{ + X264Context *x4 = avctx->priv_data; + + av_log(avctx, AV_LOG_INFO, "%s\n", nal->p_payload + 25); + x4->sei_size = nal->i_payload; + x4->sei = av_malloc(x4->sei_size); + if (!x4->sei) + return AVERROR(ENOMEM); + + memcpy(x4->sei, nal->p_payload, nal->i_payload); + + return 0; +} + +static int set_avcc_extradata(AVCodecContext *avctx, x264_nal_t *nal, int nnal) +{ + X264Context *x4 = avctx->priv_data; + x264_nal_t *sps_nal = NULL; + x264_nal_t *pps_nal = NULL; + uint8_t *p, *sps; + int ret; + + /* We know it's in the order of SPS/PPS/SEI, but it's not documented in x264 API. + * The x264 param i_sps_id implies there is a single pair of SPS/PPS. + */ + for (int i = 0; i < nnal; i++) { + switch (nal[i].i_type) { + case NAL_SPS: + sps_nal = &nal[i]; + break; + case NAL_PPS: + pps_nal = &nal[i]; + break; + case NAL_SEI: + ret = save_sei(avctx, &nal[i]); + if (ret < 0) + return ret; + break; + } + } + if (!sps_nal || !pps_nal) + return AVERROR_EXTERNAL; + + avctx->extradata_size = sps_nal->i_payload + pps_nal->i_payload + 7; + avctx->extradata = av_mallocz(avctx->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE); + if (!avctx->extradata) + return AVERROR(ENOMEM); + + // Now create AVCDecoderConfigurationRecord + p = avctx->extradata; + // Skip size part + sps = sps_nal->p_payload + 4; + *p++ = 1; // version + *p++ = sps[1]; // AVCProfileIndication + *p++ = sps[2]; // profile_compatibility + *p++ = sps[3]; // AVCLevelIndication + *p++ = 0xFF; + *p++ = 0xE0 | 0x01; // 3 bits reserved (111) + 5 bits number of sps + memcpy(p, sps_nal->p_payload + 2, sps_nal->i_payload - 2); + // Make sps has AV_INPUT_BUFFER_PADDING_SIZE padding, so it can be used + // with GetBitContext + sps = p + 2; + p += sps_nal->i_payload - 2; + *p++ = 1; + memcpy(p, pps_nal->p_payload + 2, pps_nal->i_payload - 2); + p += pps_nal->i_payload - 2; + + if (sps[3] != 66 && sps[3] != 77 && sps[3] != 88) { + GetBitContext gbc; + H264ParamSets ps = { 0 }; + + init_get_bits8(&gbc, sps, sps_nal->i_payload - 4); + skip_bits(&gbc, 8); + ret = ff_h264_decode_seq_parameter_set(&gbc, avctx, &ps, 1); + if (ret < 0) + return ret; + + ps.sps = ps.sps_list[x4->params.i_sps_id]; + *p++ = 0xFC | ps.sps->chroma_format_idc; + *p++ = 0xF8 | (ps.sps->bit_depth_luma - 8); + *p++ = 0xF8 | (ps.sps->bit_depth_chroma - 8); + *p++ = 0; + ff_h264_ps_uninit(&ps); + } + av_assert0(avctx->extradata + avctx->extradata_size >= p); + avctx->extradata_size = p - avctx->extradata; + + return 0; +} + +static int set_extradata(AVCodecContext *avctx) +{ + X264Context *x4 = avctx->priv_data; + x264_nal_t *nal; + uint8_t *p; + int nnal, s; + + s = x264_encoder_headers(x4->enc, &nal, &nnal); + if (s < 0) + return AVERROR_EXTERNAL; + + if (!x4->params.b_annexb) + return set_avcc_extradata(avctx, nal, nnal); + + avctx->extradata = p = av_mallocz(s + AV_INPUT_BUFFER_PADDING_SIZE); + if (!p) + return AVERROR(ENOMEM); + + for (int i = 0; i < nnal; i++) { + /* Don't put the SEI in extradata. */ + if (nal[i].i_type == NAL_SEI) { + s = save_sei(avctx, &nal[i]); + if (s < 0) + return s; + continue; + } + memcpy(p, nal[i].p_payload, nal[i].i_payload); + p += nal[i].i_payload; + } + avctx->extradata_size = p - avctx->extradata; + + return 0; +} + #define PARSE_X264_OPT(name, var)\ if (x4->var && x264_param_parse(&x4->params, name, x4->var) < 0) {\ av_log(avctx, AV_LOG_ERROR, "Error parsing option '%s' with value '%s'.\n", name, x4->var);\ @@ -1233,30 +1359,9 @@ FF_ENABLE_DEPRECATION_WARNINGS return AVERROR_EXTERNAL; if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { - x264_nal_t *nal; - uint8_t *p; - int nnal, s, i; - - s = x264_encoder_headers(x4->enc, &nal, &nnal); - avctx->extradata = p = av_mallocz(s + AV_INPUT_BUFFER_PADDING_SIZE); - if (!p) - return AVERROR(ENOMEM); - - for (i = 0; i < nnal; i++) { - /* Don't put the SEI in extradata. */ - if (nal[i].i_type == NAL_SEI) { - av_log(avctx, AV_LOG_INFO, "%s\n", nal[i].p_payload+25); - x4->sei_size = nal[i].i_payload; - x4->sei = av_malloc(x4->sei_size); - if (!x4->sei) - return AVERROR(ENOMEM); - memcpy(x4->sei, nal[i].p_payload, nal[i].i_payload); - continue; - } - memcpy(p, nal[i].p_payload, nal[i].i_payload); - p += nal[i].i_payload; - } - avctx->extradata_size = p - avctx->extradata; + ret = set_extradata(avctx); + if (ret < 0) + return ret; } cpb_props = ff_encode_add_cpb_side_data(avctx);