From patchwork Fri Nov 17 17:24:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 44691 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:2a20:b0:181:818d:5e7f with SMTP id e32csp450545pzh; Fri, 17 Nov 2023 01:25:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IFnoBxlqwYFUTu5/ORfMH+RVhSqAFoV+STMJ+XnzAeJyIjXbIcvUC3BYWmvsWB7W7AmuN6P X-Received: by 2002:a17:906:e12:b0:9a5:a0c6:9e8e with SMTP id l18-20020a1709060e1200b009a5a0c69e8emr12406364eji.31.1700213109838; Fri, 17 Nov 2023 01:25:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700213109; cv=none; d=google.com; s=arc-20160816; b=ntFpnW/rjqeo6uuCefesqXtyrXpB7L6Nxh5xX5WelNiG+ZJ+F9Mn+os56TjY/Q0JdN wGhdjk4zE3jriBiw9RkEvU78aRn/JtZ+Hgw6XcQFJVZcJ87oMjTlZNUEQe9MxVcrPKUY tdMiQlyoTmrJkk6VXFPdqDsIem8chlPcMvOES4q3bZ0XT06QFOwUs/hkY9ivmuRzV2p8 +H/pKJJ5wQIC3DCpo8SBvUINaRlTnKLmSqjIQZ5i0UHgcIw8A6OLBoZd5zqMhgW8JZS4 NI7lbczc7/uOrOcPhTdsbxDB6oUYam6E+SSsQ3LlObs/l2MFryeyCOTikGUF35TlQOdx w1BQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=kVlZJfpf96eWgSXOWTywqr81JyzN9dini4k/1DWE82A=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=sVagRuCYXoSYrtK8TdkXsKwly6saCdudsKqxfMo+B5ve2az+0qwP8XfzyAZRgtgvyl PfC0cawAqur7BMETnikf+mOAnnG6lKiSnk0XtJCn7tC65VasVIrZOBwmAXkzT9b0E7F2 PqIHpGPZRCiAEYXWOGxnAemxPllfDld7OC9gq9p5nHlsg8YyUDtqW06J9/l4JWSYSQF2 2ptUxqzrjHZhRTyq5x0luAmlPI+bNuc1tfxzXCP9Em1tpqASl9qq7zClM2ZSCl1PSBFK SXgiP/MEt1q+MZp7l6+eTwTbo223GJUNHfIPCH4blABUuZOoEvnVU6tNiMIZSlk0wvEW quLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=Gg92r8iJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j21-20020a170906255500b009b65b665a89si712378ejb.788.2023.11.17.01.25.08; Fri, 17 Nov 2023 01:25:09 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=Gg92r8iJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1DFC768CD07; Fri, 17 Nov 2023 11:25:04 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-221-173.mail.qq.com (out203-205-221-173.mail.qq.com [203.205.221.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 181FB68CCC0 for ; Fri, 17 Nov 2023 11:24:55 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1700213086; bh=8w3SvwkBmBI+AAzqdsMcRHeMC8zIA8uKTvLi5L9bTy0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Gg92r8iJGAGwU1QpwLmr0CUr4t8aOGhuCWoWlNS/gz+A4iEBmpwCqvCPt2IlrebKj a4f+jYXFYdJVFFH7hV3dsg7SOWBh44hsU/EEUl103XOCcjnSg4TmtlHnNsV5tzi+q1 zk/FHN8cYRnL6Uh1OzaErOK+DOPllAu+s7bw1dzs= Received: from localhost.localdomain ([113.108.77.61]) by newxmesmtplogicsvrszb6-0.qq.com (NewEsmtp) with SMTP id 62C14297; Fri, 17 Nov 2023 17:24:44 +0800 X-QQ-mid: xmsmtpt1700213086td2igmpxa Message-ID: X-QQ-XMAILINFO: OakVxA/3u/8RIy3+QUV56RUdAT3ekJt9J+7E3MZfYEvV6BL15DhP3809E3rPRY hZcSkH+VPb9n2lGBHmML8Cnd3SMoi7LjRqqruQkne/ttuSoPBFagkBK76Yc23LfhY2UlM8PvdXu7 UHWn4Y+MIx0U0NhvxGRa2e8uT1188wCuqQgqE2/+CaByh3Ha0kjX6NhrYEgnMNObWG65rMJ6yIab wf32SwVUjx8QD83OT2qs/hT/Jxf6FehYX4ksFUdDRYl9EQ5xA+UTqSmk38a1r6eokeLn2Pll6SAB uh3HxrznZVZ+8Rxusl50XBrNG0uWIVdZA7FhVXWZddKTIOAi57QoDTrl8wTc+ij2csUfar3/U+1W 0MuVkO8WqjcLJm4eelDn6GzBZGOczrXI8rWYfls9fyM8OaFCzq5ol4/HAiRF8FE/weTzGnezP0G2 ffUmaJgIit4Mffq1tMKX8QOIJOBWv1bZK2k8YUoHB+cPHne7HMpWYTCZ/SBpNGrqLvKk5V3kh0E5 rY9tV0bUDVk0ipLRhkXZ38SSpaZ43x8T3urkmOCafdx1eT2mjskH3RI+ZK9jtyHozNFPiu6FXvws pR3sGcI66vTxnfXyedPYxlozjQt2PUKThAAWb+L5YQ/qEEBlNfdjD8xT6LPG0LFLuIiHVSh37c9K 5CQ8cmc+tXRWPP0sHlsuAQtC88ZOjWfM3IUZL3uuuSY8GzeVnszvaxDeKRxRTjpX0UpZt0+fqdov hjGlq3/b6paI2GZARKmY3cRF6rbyLps9k2NUpLTCxyp3mJoNCO3IpRk7S0H4jeVk4MKI8DgCRfwt 1/1v4Xu6kneGp6womGxYjbvifLuP3eg0AUpk+xb+EPcUfF3824JccbiZQpoiim9aJzp7R6OCTGHw ibQtoaELLZAEBxHxGZeS/b1X7/fF6lRc8x6xRV6Gyy2V1k5C1wk46PlZM8uqXzPKXUIv64ccy12Q J7Y+3EX+krneq6wmyZ3AqgcW0M9mmDSAnPJUcb0Infckpajklnqg== X-QQ-XMRINFO: MSVp+SPm3vtS1Vd6Y4Mggwc= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sat, 18 Nov 2023 01:24:40 +0800 X-OQ-MSGID: <20231117172441.288316-4-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231117172441.288316-1-quinkblack@foxmail.com> References: <20231117172441.288316-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 4/5] avcodec/h264_mp4toannexb_bsf: process new extradata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PBOSggG5XtqF From: Zhao Zhili For fate-h264_mp4toannexb_ticket5927 and fate-h264_mp4toannexb_ticket5927_2, they work by accident previously. The sample file has two 'avc1' entries, and video samples use the second one. It means packets should be decoded with new extradata in side data. Before this patch, only extradata was kept in the output, new extradata has been dropped. The output can be decoded because the two extradata are almost the same, except level indication. This patch fixed the issue, and add another fate test. --- libavcodec/h264_mp4toannexb_bsf.c | 37 ++++++++++++------- tests/fate/h264.mak | 4 +- .../fate/h264-bsf-mp4toannexb-new-extradata | 9 +++++ tests/ref/fate/h264_mp4toannexb_ticket5927 | 8 ++-- tests/ref/fate/h264_mp4toannexb_ticket5927_2 | 8 ++-- 5 files changed, 43 insertions(+), 23 deletions(-) create mode 100644 tests/ref/fate/h264-bsf-mp4toannexb-new-extradata diff --git a/libavcodec/h264_mp4toannexb_bsf.c b/libavcodec/h264_mp4toannexb_bsf.c index 4073c780c5..120241c892 100644 --- a/libavcodec/h264_mp4toannexb_bsf.c +++ b/libavcodec/h264_mp4toannexb_bsf.c @@ -80,7 +80,8 @@ static void count_or_copy(uint8_t **out, uint64_t *out_size, *out_size += start_code_size + in_size; } -static int h264_extradata_to_annexb(AVBSFContext *ctx) +static int h264_extradata_to_annexb(AVBSFContext *ctx, + uint8_t *extradata, int extradata_size) { H264BSFContext *s = ctx->priv_data; GetByteContext ogb, *gb = &ogb; @@ -91,7 +92,7 @@ static int h264_extradata_to_annexb(AVBSFContext *ctx) const int padding = AV_INPUT_BUFFER_PADDING_SIZE; int length_size, pps_offset = 0; - bytestream2_init(gb, ctx->par_in->extradata, ctx->par_in->extradata_size); + bytestream2_init(gb, extradata, extradata_size); bytestream2_skipu(gb, 4); @@ -169,7 +170,13 @@ pps: ctx->par_out->extradata = out; ctx->par_out->extradata_size = total_size; - return length_size; + s->length_size = length_size; + s->new_idr = 1; + s->idr_sps_seen = 0; + s->idr_pps_seen = 0; + s->extradata_parsed = 1; + + return 0; } static int h264_mp4toannexb_save_ps(uint8_t **dst, int *dst_size, @@ -203,9 +210,7 @@ static int h264_mp4toannexb_save_ps(uint8_t **dst, int *dst_size, static int h264_mp4toannexb_init(AVBSFContext *ctx) { - H264BSFContext *s = ctx->priv_data; int extra_size = ctx->par_in->extradata_size; - int ret; /* retrieve sps and pps NAL units from extradata */ if (!extra_size || @@ -214,15 +219,9 @@ static int h264_mp4toannexb_init(AVBSFContext *ctx) av_log(ctx, AV_LOG_VERBOSE, "The input looks like it is Annex B already\n"); } else if (extra_size >= 7) { - ret = h264_extradata_to_annexb(ctx); - if (ret < 0) - return ret; - - s->length_size = ret; - s->new_idr = 1; - s->idr_sps_seen = 0; - s->idr_pps_seen = 0; - s->extradata_parsed = 1; + return h264_extradata_to_annexb(ctx, + ctx->par_in->extradata, + ctx->par_in->extradata_size); } else { av_log(ctx, AV_LOG_ERROR, "Invalid extradata size: %d\n", extra_size); return AVERROR_INVALIDDATA; @@ -241,11 +240,21 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) uint8_t *out; uint64_t out_size; int ret; + size_t extradata_size; + uint8_t *extradata; ret = ff_bsf_get_packet(ctx, &in); if (ret < 0) return ret; + extradata = av_packet_get_side_data(in, AV_PKT_DATA_NEW_EXTRADATA, + &extradata_size); + if (extradata) { + ret = h264_extradata_to_annexb(ctx, extradata, extradata_size); + if (ret < 0) + goto fail; + } + /* nothing to filter */ if (!s->extradata_parsed) { av_packet_move_ref(opkt, in); diff --git a/tests/fate/h264.mak b/tests/fate/h264.mak index c7e0d0a84e..010dd9abf6 100644 --- a/tests/fate/h264.mak +++ b/tests/fate/h264.mak @@ -225,7 +225,8 @@ FATE_H264-$(call FRAMECRC, MOV, H264) += fate-h264-unescaped-extradata FATE_H264-$(call FRAMECRC, MOV, H264) += fate-h264-twofields-packet FATE_H264-$(call DEMMUX, MOV, H264, H264_MP4TOANNEXB_BSF) += fate-h264-bsf-mp4toannexb \ - fate-h264-bsf-mp4toannexb-2 + fate-h264-bsf-mp4toannexb-2 \ + fate-h264-bsf-mp4toannexb-new-extradata \ FATE_H264-$(call FRAMECRC, MATROSKA, H264) += fate-h264-direct-bff FATE_H264-$(call FRAMECRC, FLV, H264, SCALE_FILTER) += fate-h264-brokensps-2580 @@ -432,6 +433,7 @@ fate-h264-bsf-mp4toannexb: CMD = md5 -i $(TARGET_SAMPLES) fate-h264-bsf-mp4toannexb-2: CMD = md5 -i $(TARGET_SAMPLES)/h264/ps_prefix_first_idr.mp4 -c:v copy -f h264 fate-h264-bsf-mp4toannexb-2: CMP = oneline fate-h264-bsf-mp4toannexb-2: REF = cffcfa6a2d0b58c9de1f5785f099f41d +fate-h264-bsf-mp4toannexb-new-extradata: CMD = stream_remux mov $(TARGET_SAMPLES)/h264/extradata-reload-multi-stsd.mov "" h264 "-map 0:v" fate-h264-crop-to-container: CMD = framemd5 -i $(TARGET_SAMPLES)/h264/crop-to-container-dims-canon.mov fate-h264-direct-bff: CMD = framecrc -i $(TARGET_SAMPLES)/h264/direct-bff.mkv diff --git a/tests/ref/fate/h264-bsf-mp4toannexb-new-extradata b/tests/ref/fate/h264-bsf-mp4toannexb-new-extradata new file mode 100644 index 0000000000..a0423ad295 --- /dev/null +++ b/tests/ref/fate/h264-bsf-mp4toannexb-new-extradata @@ -0,0 +1,9 @@ +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 256x128 +#sar 0: 1/1 +0, 0, 0, 1, 49152, 0x08745db9 +0, 1, 1, 1, 49152, 0x96bf5e58 +0, 2, 2, 1, 49152, 0x8fe31b6d +0, 3, 3, 1, 49152, 0x0b621cc3 diff --git a/tests/ref/fate/h264_mp4toannexb_ticket5927 b/tests/ref/fate/h264_mp4toannexb_ticket5927 index 95e35c4d80..eb0b5cf283 100644 --- a/tests/ref/fate/h264_mp4toannexb_ticket5927 +++ b/tests/ref/fate/h264_mp4toannexb_ticket5927 @@ -1,12 +1,12 @@ -a3b02fd09392e01619cebc959d4d9ff2 *tests/data/fate/h264_mp4toannexb_ticket5927.h264 +edddeef7901b2bd8d55625b8105b579f *tests/data/fate/h264_mp4toannexb_ticket5927.h264 595583 tests/data/fate/h264_mp4toannexb_ticket5927.h264 -#extradata 0: 33, 0x84fe08f8 +#extradata 0: 33, 0x84e308f7 #tb 0: 1/1200000 #media_type 0: video #codec_id 0: h264 #dimensions 0: 1920x1080 #sar 0: 0/1 -0, -48000, -9223372036854775808, 48000, 247993, 0x1ce821ea +0, -48000, -9223372036854775808, 48000, 247993, 0x541321e9 0, 0, -9223372036854775808, 48000, 43354, 0xa05dca6f, F=0x0 0, 48000, -9223372036854775808, 48000, 11423, 0x5e8086dd, F=0x0 0, 96000, -9223372036854775808, 48000, 50798, 0x145fbe4f, F=0x0 @@ -18,4 +18,4 @@ a3b02fd09392e01619cebc959d4d9ff2 *tests/data/fate/h264_mp4toannexb_ticket5927.h2 0, 384000, -9223372036854775808, 48000, 54483, 0xefead99f, F=0x0 0, 432000, -9223372036854775808, 48000, 13705, 0x23cd27e8, F=0x0 0, 480000, -9223372036854775808, 48000, 22308, 0x4093b5af, F=0x0 -0, 528000, -9223372036854775808, 48000, 6369, 0x858b2aa1 +0, 528000, -9223372036854775808, 48000, 6369, 0x6cca2aa0 diff --git a/tests/ref/fate/h264_mp4toannexb_ticket5927_2 b/tests/ref/fate/h264_mp4toannexb_ticket5927_2 index 8db6a7e54a..8c3613ee79 100644 --- a/tests/ref/fate/h264_mp4toannexb_ticket5927_2 +++ b/tests/ref/fate/h264_mp4toannexb_ticket5927_2 @@ -1,12 +1,12 @@ -a3b02fd09392e01619cebc959d4d9ff2 *tests/data/fate/h264_mp4toannexb_ticket5927_2.h264 +edddeef7901b2bd8d55625b8105b579f *tests/data/fate/h264_mp4toannexb_ticket5927_2.h264 595583 tests/data/fate/h264_mp4toannexb_ticket5927_2.h264 -#extradata 0: 33, 0x84fe08f8 +#extradata 0: 33, 0x84e308f7 #tb 0: 1/1200000 #media_type 0: video #codec_id 0: h264 #dimensions 0: 1920x1080 #sar 0: 0/1 -0, -48000, -9223372036854775808, 48000, 247993, 0x1ce821ea +0, -48000, -9223372036854775808, 48000, 247993, 0x541321e9 0, 0, -9223372036854775808, 48000, 43354, 0xa05dca6f, F=0x0 0, 48000, -9223372036854775808, 48000, 11423, 0x5e8086dd, F=0x0 0, 96000, -9223372036854775808, 48000, 50798, 0x145fbe4f, F=0x0 @@ -18,4 +18,4 @@ a3b02fd09392e01619cebc959d4d9ff2 *tests/data/fate/h264_mp4toannexb_ticket5927_2. 0, 384000, -9223372036854775808, 48000, 54483, 0xefead99f, F=0x0 0, 432000, -9223372036854775808, 48000, 13705, 0x23cd27e8, F=0x0 0, 480000, -9223372036854775808, 48000, 22308, 0x4093b5af, F=0x0 -0, 528000, -9223372036854775808, 48000, 6369, 0x858b2aa1 +0, 528000, -9223372036854775808, 48000, 6369, 0x6cca2aa0