From patchwork Sun Mar 17 03:34:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 47124 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:958e:b0:1a3:31a3:7958 with SMTP id iu14csp322431pzb; Sat, 16 Mar 2024 20:35:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX7UIuzEcl3z5buUxdHuYvGCdBD8vIizABpwVX+UjvAHmfigroK2LTeiw5SrBFNfZJhSTjqQcSyR8S0XsT4gzYwJBVLwF8DVwE2BQ== X-Google-Smtp-Source: AGHT+IEpU2CrwHNJP1u6wb1ZVWcCQ4k+JoQYqrbkPSb+wVP/nokiA8WfGrkMR4Nmtw3pIs9sEdHS X-Received: by 2002:a5d:44c6:0:b0:33e:793b:a2aa with SMTP id z6-20020a5d44c6000000b0033e793ba2aamr2420530wrr.47.1710646522785; Sat, 16 Mar 2024 20:35:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710646522; cv=none; d=google.com; s=arc-20160816; b=XA89ANzCbhuVQ5xjDQLqECHA4wnN0DemeNG55kuMGMEjP3+zrj9cAfcBF+T0q88Zs6 rrWD/x4nv+GjQ1XvoBKQd8wKYO5z2lApCq3Upf/m+nnOR0byRUbJUq7KFdNIwH9gGjIz VroTN+F/p5tE9bRFJDlCfdAuwrNiV6YLyImLKMgfgpQmy3ZvzmkEXjHz43+wmHb1uyUL MwbSoJteyyJp5f6wLTaCTdFmcYJUH/LzHylfZuD/O+QwQ72wJAxrHOjW8Zhaabb6LzYb E1lD2oIBMP/4BAM8sYZtsqLWUpTytwNG6oH1kofBcHX38EMN9h85wU9Rww2sank4jR7J GXeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:date:to:from:message-id :dkim-signature:delivered-to; bh=XGrWUKCPunU7HV3MxNmlo4ErnUkDKlK2NyBbjCkfKdc=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=ATDQyYjoAcofkmOkpckXOJVOsj94a6rX6cuTulRMLRpuH53fYgo2sKtbaGwbRXLSDY GOZwmbFfTGnhRmAljQD11bkTjvSIZNXG1XLew9A4BFuhC1VLSzJEmtQfJvZ/Hh07Y6nt RK43w13Ne+2rBVy1nUcITLi5HUSGiOaJti8clFUtSEsRQJWRR4zWTvAWkmj9YTsNSep+ /4UYL/DfO1WKPraXLTFifc6FKusxgybBROZwMKfa94fvYbQCNJqUDSjv+0NyXor1e/nM dHAE67TfOAH2fak/eM0AGyMr0XKmiWsv8Uv9cvcaWKz3o4h8CLVkA9bXXft6csXg8YAw Llgw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=KZJ8TBop; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qu9-20020a170907110900b00a469c3f1041si1389285ejb.621.2024.03.16.20.35.22; Sat, 16 Mar 2024 20:35:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=KZJ8TBop; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E6C3468D063; Sun, 17 Mar 2024 05:35:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-211.mail.qq.com (out162-62-58-211.mail.qq.com [162.62.58.211]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D0785680BC0 for ; Sun, 17 Mar 2024 05:35:10 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1710646499; bh=aYMG04Yu+1tyw9Q3ntww42blufA8qiGum0RqTPau9HM=; h=From:To:Cc:Subject:Date; b=KZJ8TBopM5rCd+T/onz7eKxV76uvpYSg+Bv4Pj9uZL7EFUBug6onuaBlzLFfgsI4t 2J6u4w1aN2ntO+772A228XwCNIOw4ATxJsI5p337gTgUr6dzkpFlSIP3KvQM6XEZh2 C/krfr5EmhoGiLkKQqEYlCZmVkyfAIMkcWuWi3Cg= Received: from ZHILIZHAO-MB1.tencent.com ([121.35.186.33]) by newxmesmtplogicsvrszb9-1.qq.com (NewEsmtp) with SMTP id 8BA906BF; Sun, 17 Mar 2024 11:34:58 +0800 X-QQ-mid: xmsmtpt1710646498tfenkqjzj Message-ID: X-QQ-XMAILINFO: MBjwNRQMz5zUyr7eSmv4qxYUhTKagxB+BZAPpZDf7LS5VotE3hCSvz8HVFvzkF V32FVDg7gHhVFqK4L1yn0FRvBfoubscixu+XBIYNpWWF0w+IeXrt+oEGp8SmiykotqCDkfVFUJbP hkP5LlwaGKa+YaHG7B9C1hsnQ55jJ7NS8igSJuuHwAhjNNQ+vITqPZ40xV+Snf1O2rC5WZylnYcc Qo2Syy7RNu9Nxntgti2RybBUqtvNe5ewNJvJGFPGodtzlW6Y6shtOLpyUB22TEnUQgLPOWBRsnCg kU210H3+L62cqj0uJEAIoGUg5J7JOnLqmdLcdPXGFdFat1/IKg1+5eZpisbmH9UKz5dJ078YtUfx Un/sR3vavQ5/2LQATv0lDtpq8vCDKjg3rofcaQden0Air/rwzHsnzT+BXAXzquzlI5u8cAKCy7OD Ee4KD9lIACZtMtOi/DY7qeyf903YtqCVH6It1JeDCfIR3GrOxwCvaY8P85i+Tk+UWJqUT7gJaovb DPnm7mc/2oEZxqR2+Fq0gkdpEqiKx421G6fRei9zFafLhE9Rw5JvYWQWN+NvvKeW1PO1/GRG4+fL RIUcLK6qsEAB5XaJTc9rZ/M29gtYMB9AWE6FIueXFJ4VjdbKDlFzNsWYtZc8Iqhn6JshsTmyviwx UhafBCc6Ex2RBzNenGx/SpzT/sU3gCVUaIYy6IYs2+6P28kW2YKmeMv7mGOqPv7FamBk15xe2mpt z9r6E0jzpwafIjZls4HI7HrEgCu/ol1Il1eMK+00PYiNAxM4/91UDUh+BLRDm4xfH68Bx+bGOgba f1M1HF/qLGctbFZnbD+7SPQ2cIYCjqq4RMlUqAU5VTmbkgGQ9mieX/3Wbonstw9eyeykKUScyv1j i/Ozf1Wqw1cJH8LRor0vvtbYxJMqrkPB1vPeM7AegJilQBa3SpfZ1a8g7Puq0Enep01MLFOyMGvw 4/ccoCrl7YTmy6fVJ5NswUBAJr3HMNwBIYF3es3wI= X-QQ-XMRINFO: NI4Ajvh11aEj8Xl/2s1/T8w= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 17 Mar 2024 11:34:57 +0800 X-OQ-MSGID: <20240317033457.85906-1-quinkblack@foxmail.com> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v6] avcodec/libx264: fix extradata when config annexb=0 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IO1h+lCqMB0d From: Zhao Zhili --- v6: use av_assert2 select golomb in configure conditional compile in case of CONFIG_LIBX264_ENCODER=0 v5: Decode chroma_format_idc directly instead of ff_h264_decode_seq_parameter_set v4: Fix missing SEI in set_avcc_extradata v3: Remove unnecessary inclusion configure | 2 +- libavcodec/libx264.c | 166 ++++++++++++++++++++++++++++++++++++------- 2 files changed, 143 insertions(+), 25 deletions(-) diff --git a/configure b/configure index 2b4c4ec9a2..d9d2183a47 100755 --- a/configure +++ b/configure @@ -3484,7 +3484,7 @@ libwebp_encoder_deps="libwebp" libwebp_anim_encoder_deps="libwebp" libx262_encoder_deps="libx262" libx264_encoder_deps="libx264" -libx264_encoder_select="atsc_a53" +libx264_encoder_select="atsc_a53 golomb" libx264rgb_encoder_deps="libx264" libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" diff --git a/libavcodec/libx264.c b/libavcodec/libx264.c index 0997c4e134..ddec06d960 100644 --- a/libavcodec/libx264.c +++ b/libavcodec/libx264.c @@ -38,6 +38,7 @@ #include "packet_internal.h" #include "atsc_a53.h" #include "sei.h" +#include "golomb.h" #include #include @@ -847,6 +848,144 @@ static int convert_pix_fmt(enum AVPixelFormat pix_fmt) return 0; } +static int save_sei(AVCodecContext *avctx, x264_nal_t *nal) +{ + X264Context *x4 = avctx->priv_data; + + av_log(avctx, AV_LOG_INFO, "%s\n", nal->p_payload + 25); + x4->sei_size = nal->i_payload; + x4->sei = av_malloc(x4->sei_size); + if (!x4->sei) + return AVERROR(ENOMEM); + + memcpy(x4->sei, nal->p_payload, nal->i_payload); + + return 0; +} + +#if CONFIG_LIBX264_ENCODER +static int set_avcc_extradata(AVCodecContext *avctx, x264_nal_t *nal, int nnal) +{ + X264Context *x4 = avctx->priv_data; + x264_nal_t *sps_nal = NULL; + x264_nal_t *pps_nal = NULL; + uint8_t *p, *sps; + int ret; + + /* We know it's in the order of SPS/PPS/SEI, but it's not documented in x264 API. + * The x264 param i_sps_id implies there is a single pair of SPS/PPS. + */ + for (int i = 0; i < nnal; i++) { + switch (nal[i].i_type) { + case NAL_SPS: + sps_nal = &nal[i]; + break; + case NAL_PPS: + pps_nal = &nal[i]; + break; + case NAL_SEI: + ret = save_sei(avctx, &nal[i]); + if (ret < 0) + return ret; + break; + } + } + if (!sps_nal || !pps_nal) + return AVERROR_EXTERNAL; + + avctx->extradata_size = sps_nal->i_payload + pps_nal->i_payload + 7; + avctx->extradata = av_mallocz(avctx->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE); + if (!avctx->extradata) + return AVERROR(ENOMEM); + + // Now create AVCDecoderConfigurationRecord + p = avctx->extradata; + // Skip size part + sps = sps_nal->p_payload + 4; + *p++ = 1; // version + *p++ = sps[1]; // AVCProfileIndication + *p++ = sps[2]; // profile_compatibility + *p++ = sps[3]; // AVCLevelIndication + *p++ = 0xFF; + *p++ = 0xE0 | 0x01; // 3 bits reserved (111) + 5 bits number of sps + memcpy(p, sps_nal->p_payload + 2, sps_nal->i_payload - 2); + // Make sps has AV_INPUT_BUFFER_PADDING_SIZE padding, so it can be used + // with GetBitContext + sps = p + 2; + p += sps_nal->i_payload - 2; + *p++ = 1; + memcpy(p, pps_nal->p_payload + 2, pps_nal->i_payload - 2); + p += pps_nal->i_payload - 2; + + if (sps[3] != 66 && sps[3] != 77 && sps[3] != 88) { + GetBitContext gbc; + int chroma_format_idc; + int bit_depth_luma_minus8, bit_depth_chroma_minus8; + + /* It's not possible to have emulation prevention byte before + * bit_depth_chroma_minus8 due to the range of sps id, chroma_format_idc + * and so on. So we can read directly without need to escape emulation + * prevention byte. + * + * +4 to skip until sps id. + */ + init_get_bits8(&gbc, sps + 4, sps_nal->i_payload - 4 - 4); + // Skip sps id + get_ue_golomb_31(&gbc); + chroma_format_idc = get_ue_golomb_31(&gbc); + if (chroma_format_idc == 3) + skip_bits1(&gbc); + bit_depth_luma_minus8 = get_ue_golomb_31(&gbc); + bit_depth_chroma_minus8 = get_ue_golomb_31(&gbc); + + *p++ = 0xFC | chroma_format_idc; + *p++ = 0xF8 | bit_depth_luma_minus8; + *p++ = 0xF8 | bit_depth_chroma_minus8; + *p++ = 0; + } + av_assert2(avctx->extradata + avctx->extradata_size >= p); + avctx->extradata_size = p - avctx->extradata; + + return 0; +} +#endif + +static int set_extradata(AVCodecContext *avctx) +{ + X264Context *x4 = avctx->priv_data; + x264_nal_t *nal; + uint8_t *p; + int nnal, s; + + s = x264_encoder_headers(x4->enc, &nal, &nnal); + if (s < 0) + return AVERROR_EXTERNAL; + +#if CONFIG_LIBX264_ENCODER + if (!x4->params.b_annexb) + return set_avcc_extradata(avctx, nal, nnal); +#endif + + avctx->extradata = p = av_mallocz(s + AV_INPUT_BUFFER_PADDING_SIZE); + if (!p) + return AVERROR(ENOMEM); + + for (int i = 0; i < nnal; i++) { + /* Don't put the SEI in extradata. */ + if (nal[i].i_type == NAL_SEI) { + s = save_sei(avctx, &nal[i]); + if (s < 0) + return s; + continue; + } + memcpy(p, nal[i].p_payload, nal[i].i_payload); + p += nal[i].i_payload; + } + avctx->extradata_size = p - avctx->extradata; + + return 0; +} + #define PARSE_X264_OPT(name, var)\ if (x4->var && x264_param_parse(&x4->params, name, x4->var) < 0) {\ av_log(avctx, AV_LOG_ERROR, "Error parsing option '%s' with value '%s'.\n", name, x4->var);\ @@ -1215,30 +1354,9 @@ FF_ENABLE_DEPRECATION_WARNINGS return AVERROR_EXTERNAL; if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { - x264_nal_t *nal; - uint8_t *p; - int nnal, s, i; - - s = x264_encoder_headers(x4->enc, &nal, &nnal); - avctx->extradata = p = av_mallocz(s + AV_INPUT_BUFFER_PADDING_SIZE); - if (!p) - return AVERROR(ENOMEM); - - for (i = 0; i < nnal; i++) { - /* Don't put the SEI in extradata. */ - if (nal[i].i_type == NAL_SEI) { - av_log(avctx, AV_LOG_INFO, "%s\n", nal[i].p_payload+25); - x4->sei_size = nal[i].i_payload; - x4->sei = av_malloc(x4->sei_size); - if (!x4->sei) - return AVERROR(ENOMEM); - memcpy(x4->sei, nal[i].p_payload, nal[i].i_payload); - continue; - } - memcpy(p, nal[i].p_payload, nal[i].i_payload); - p += nal[i].i_payload; - } - avctx->extradata_size = p - avctx->extradata; + ret = set_extradata(avctx); + if (ret < 0) + return ret; } cpb_props = ff_encode_add_cpb_side_data(avctx);