From patchwork Fri May 19 16:41:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 41722 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:ba91:b0:105:feb:71f2 with SMTP id fb17csp1130200pzb; Fri, 19 May 2023 01:41:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ71jwmiu6ZeZwzX0vyRQLlBYPnuR3DqbpTbDmfvposXebkKGgszYW/vHhDMo4hZNoqAWIIu X-Received: by 2002:aa7:da16:0:b0:510:f3a6:d50 with SMTP id r22-20020aa7da16000000b00510f3a60d50mr746151eds.36.1684485719251; Fri, 19 May 2023 01:41:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684485719; cv=none; d=google.com; s=arc-20160816; b=HxXepRhNFCbBj8p382hj+v2FGI8x7o+BFEEa5xNyHoPolD+QlXu0BCT4OMfncgSweq a7CaAJo5hUaFLct34YKvCUzoIAtB3IqsWuiHgqvyUXpXs48kNFv66mDl2GMoCtPDaCIT m6Ah+7CzjfSr5Uj/mJwXfJYRO0n0RHIik7rNyJEFuNdewTLu28GFloJnClel2f7bi6Mo 9gF/edA03bglGa1Ay8G1gzwFmiHJUvkZaq1Dq/axo0hERmpA3qnxMoEkJ5+0W1iAJdE2 9lYQDNLIirTM0D6R7YRlmbWRd4DRmWoJy5+Z7F/yIGgxKyNuANrJWpmiV+F1Wjp2emg2 h3kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=WUkQUgKFLlPYnHDTVj2k0rI7IyrwbykDHscuF7i0ups=; b=xm6l/Mdquk/Air8zmolDPsLRbU6ex8U7F8RV9Kal1d/kv4rPOXXV3vWlD4XmdQNU1u 3/G8L+GbSHSYGwKZkG0l3G1u5kk8Wqdqm1VdQZ9/RbV6FcGBG//0rSWcDB2AJGtH71Aq 48aOrNGEtpF/1iM0roZ0EueM5lPEfWKtedXcjE642Z9Im/0wGOW8rGY2xZkOihct45ZK 0ALNZiJIU9wBwLO+6ZRvficGo2cMj8Wz6DjmZCp5wNkVvoU4yva0l8yBVJDXCquEAq7s I1mnzVnrQOrCmcfkKNmlsHXwraw8KTjhQtP+bBZ0gXetoLp2MK/YryQ7HyRxIgJzZvcI PzDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=BdoA+lYg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b14-20020a05640202ce00b0050dbe83a003si2129049edx.10.2023.05.19.01.41.58; Fri, 19 May 2023 01:41:59 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=BdoA+lYg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1669E68C162; Fri, 19 May 2023 11:41:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-221-240.mail.qq.com (out203-205-221-240.mail.qq.com [203.205.221.240]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D0EFE68C125 for ; Fri, 19 May 2023 11:41:36 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1684485689; bh=5MUZe4cdoxTbBzZH71EiKkhgnDIdxzuxgaL9if0nePE=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=BdoA+lYg083B9q6SbValosBQlOJUMQF6x4e6N88m97v8KMLW4CGVOTQVXklje+sJa lq64IcjINviDBzvIStnSGLH8kWOgJ1lMpMVu2y/WaVqrp0Yw0ZYePIBRQ5IQNJfadD fy27S1l6fLudCLppfMNnoBYt7hmdAW49NnVDC/lQ= Received: from localhost.localdomain ([119.147.10.201]) by newxmesmtplogicsvrszc2-0.qq.com (NewEsmtp) with SMTP id A5B2749A; Fri, 19 May 2023 16:41:27 +0800 X-QQ-mid: xmsmtpt1684485689tpo87j61w Message-ID: X-QQ-XMAILINFO: OZZSS56D9fAj2m8DG7brdtUgcTjyfZaUh3Eus/g3gTaTDjVToWqqIk97wdOU5I n2d04o78GClYPUga2YT6Ba+zC6V+xj1gTIl/Pjd86Zax9UzKKPwYRkZZ4c+sMTc0lSSHq1hkXvML L8uo1EO1CIYKkQtHbDgheXGIEvF53jPBRkr/Ybd/8/sxI4bF30uLmxhirxZUxSNcln9BOxb3Kiwb 26lZ7FlMMKasB8hFuEoV+PaZ5BJhmoE1H9LgAMZfe69eHN3yc7pO94w0ITP2DkcZ5PAf654Kp+kb jf1A8bpIvWJhzJ4e/Uqv5UbcbR+DyTUv66dbcLKAsqMba7bn9UtkGpNuYGH13gCliXVW+KINlTCF 43BOyGpY1ZbNY7L5ET6io/Z4G8VQFeVvlcv1d4Ak3BIiWWcM93SacCZKxyS+kQ+7frvtogN/TYUI ZGO/yYtg6nx+4MTdlfzXoYKktboSb18XDRd6eLbaYBzvFM0E2vHV88Rp/QacH7fR3Az0NkFddjCD JGttR1hB4KtVaxR7kzHseeyAYa0Ab9qMSQnpSfgfn5YSMz2Tey7DBPR3CaHGnCGSWCohevPGdix4 FMeR8ga2dcoEPJLmIEdJzjSdjVsyGsMz7nb3WH/LAisRXAF3FDQx4k9FNtZhAaTFwB4jGtnvPyoe PEX0gBgaFPgnDO/a/aXV560Iv1pkhXqpVeoZLPTIwUc54dnqNL4NA5gz1qA0QLM2YhBo2cM0dR45 EYsyx2Ct/0ZUk05UIzSUWr9NU7BYBh0MH8ncYZH74znRhCLtUL4PUGIGaIDrbEoKQbtmqrVmIb9m 1A+euLmKZo3dvzlnFSdo46ooWuINe/C1wyWUBF9zGy2ueGbFya/Ois8DvnrI3+Wipg4WiKOxLkDR OwfAXHItBD6tkFoD8AZY9l8RO7r4mY7UbkQEK1331E9Nf8i7zhL0kk7SHnHP60yvuph0+W59pyC1 fR/iFPX6du3vklzy19OfxnhVgckcyCGfkhLvAX5LAtX2I0gMRPJVW7iAer9WVj6DVGoSuzsqnEVP ikh+nLQRGUoWKca2S/+w0cTo3Prw0hkmQ+qN6JVRanshAxj3UT From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sat, 20 May 2023 00:41:32 +0800 X-OQ-MSGID: <20230519164133.3278597-4-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230519164133.3278597-1-quinkblack@foxmail.com> References: <20230519164133.3278597-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/5] avcodec/h264_mp4toannexb_bsf: process new extradata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: mq7dLahI01xX From: Zhao Zhili For fate-h264_mp4toannexb_ticket5927 and fate-h264_mp4toannexb_ticket5927_2, they work by accident previously. The sample file has two 'avc1' entries, and video samples use the second one. It means packets should be decoded with new extradata in side data. Before this patch, only extradata was kept in the output, new extradata has been dropped. The output can be decoded because the two extradata are almost the same, except level indication. This patch fixed the issue, and add another fate test. --- libavcodec/h264_mp4toannexb_bsf.c | 37 ++++++++++++------- tests/fate/h264.mak | 4 +- .../fate/h264-bsf-mp4toannexb-new-extradata | 9 +++++ tests/ref/fate/h264_mp4toannexb_ticket5927 | 8 ++-- tests/ref/fate/h264_mp4toannexb_ticket5927_2 | 8 ++-- 5 files changed, 43 insertions(+), 23 deletions(-) create mode 100644 tests/ref/fate/h264-bsf-mp4toannexb-new-extradata diff --git a/libavcodec/h264_mp4toannexb_bsf.c b/libavcodec/h264_mp4toannexb_bsf.c index 4073c780c5..120241c892 100644 --- a/libavcodec/h264_mp4toannexb_bsf.c +++ b/libavcodec/h264_mp4toannexb_bsf.c @@ -80,7 +80,8 @@ static void count_or_copy(uint8_t **out, uint64_t *out_size, *out_size += start_code_size + in_size; } -static int h264_extradata_to_annexb(AVBSFContext *ctx) +static int h264_extradata_to_annexb(AVBSFContext *ctx, + uint8_t *extradata, int extradata_size) { H264BSFContext *s = ctx->priv_data; GetByteContext ogb, *gb = &ogb; @@ -91,7 +92,7 @@ static int h264_extradata_to_annexb(AVBSFContext *ctx) const int padding = AV_INPUT_BUFFER_PADDING_SIZE; int length_size, pps_offset = 0; - bytestream2_init(gb, ctx->par_in->extradata, ctx->par_in->extradata_size); + bytestream2_init(gb, extradata, extradata_size); bytestream2_skipu(gb, 4); @@ -169,7 +170,13 @@ pps: ctx->par_out->extradata = out; ctx->par_out->extradata_size = total_size; - return length_size; + s->length_size = length_size; + s->new_idr = 1; + s->idr_sps_seen = 0; + s->idr_pps_seen = 0; + s->extradata_parsed = 1; + + return 0; } static int h264_mp4toannexb_save_ps(uint8_t **dst, int *dst_size, @@ -203,9 +210,7 @@ static int h264_mp4toannexb_save_ps(uint8_t **dst, int *dst_size, static int h264_mp4toannexb_init(AVBSFContext *ctx) { - H264BSFContext *s = ctx->priv_data; int extra_size = ctx->par_in->extradata_size; - int ret; /* retrieve sps and pps NAL units from extradata */ if (!extra_size || @@ -214,15 +219,9 @@ static int h264_mp4toannexb_init(AVBSFContext *ctx) av_log(ctx, AV_LOG_VERBOSE, "The input looks like it is Annex B already\n"); } else if (extra_size >= 7) { - ret = h264_extradata_to_annexb(ctx); - if (ret < 0) - return ret; - - s->length_size = ret; - s->new_idr = 1; - s->idr_sps_seen = 0; - s->idr_pps_seen = 0; - s->extradata_parsed = 1; + return h264_extradata_to_annexb(ctx, + ctx->par_in->extradata, + ctx->par_in->extradata_size); } else { av_log(ctx, AV_LOG_ERROR, "Invalid extradata size: %d\n", extra_size); return AVERROR_INVALIDDATA; @@ -241,11 +240,21 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) uint8_t *out; uint64_t out_size; int ret; + size_t extradata_size; + uint8_t *extradata; ret = ff_bsf_get_packet(ctx, &in); if (ret < 0) return ret; + extradata = av_packet_get_side_data(in, AV_PKT_DATA_NEW_EXTRADATA, + &extradata_size); + if (extradata) { + ret = h264_extradata_to_annexb(ctx, extradata, extradata_size); + if (ret < 0) + goto fail; + } + /* nothing to filter */ if (!s->extradata_parsed) { av_packet_move_ref(opkt, in); diff --git a/tests/fate/h264.mak b/tests/fate/h264.mak index d75aebb253..cf08cb9e54 100644 --- a/tests/fate/h264.mak +++ b/tests/fate/h264.mak @@ -225,7 +225,8 @@ FATE_H264-$(call FRAMECRC, MOV, H264) += fate-h264-unescaped-extradata FATE_H264-$(call FRAMECRC, MOV, H264) += fate-h264-twofields-packet FATE_H264-$(call DEMMUX, MOV, H264, H264_MP4TOANNEXB_BSF) += fate-h264-bsf-mp4toannexb \ - fate-h264-bsf-mp4toannexb-2 + fate-h264-bsf-mp4toannexb-2 \ + fate-h264-bsf-mp4toannexb-new-extradata \ FATE_H264-$(call FRAMECRC, MATROSKA, H264) += fate-h264-direct-bff FATE_H264-$(call FRAMECRC, FLV, H264, SCALE_FILTER) += fate-h264-brokensps-2580 @@ -432,6 +433,7 @@ fate-h264-bsf-mp4toannexb: CMD = md5 -i $(TARGET_SAMPLES) fate-h264-bsf-mp4toannexb-2: CMD = md5 -i $(TARGET_SAMPLES)/h264/ps_prefix_first_idr.mp4 -c:v copy -f h264 fate-h264-bsf-mp4toannexb-2: CMP = oneline fate-h264-bsf-mp4toannexb-2: REF = 872608399a9aac7cf91a7c447f366f32 +fate-h264-bsf-mp4toannexb-new-extradata: CMD = stream_remux mov $(TARGET_SAMPLES)/h264/extradata-reload-multi-stsd.mov h264 "-map 0:v" fate-h264-crop-to-container: CMD = framemd5 -i $(TARGET_SAMPLES)/h264/crop-to-container-dims-canon.mov fate-h264-direct-bff: CMD = framecrc -i $(TARGET_SAMPLES)/h264/direct-bff.mkv diff --git a/tests/ref/fate/h264-bsf-mp4toannexb-new-extradata b/tests/ref/fate/h264-bsf-mp4toannexb-new-extradata new file mode 100644 index 0000000000..a0423ad295 --- /dev/null +++ b/tests/ref/fate/h264-bsf-mp4toannexb-new-extradata @@ -0,0 +1,9 @@ +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 256x128 +#sar 0: 1/1 +0, 0, 0, 1, 49152, 0x08745db9 +0, 1, 1, 1, 49152, 0x96bf5e58 +0, 2, 2, 1, 49152, 0x8fe31b6d +0, 3, 3, 1, 49152, 0x0b621cc3 diff --git a/tests/ref/fate/h264_mp4toannexb_ticket5927 b/tests/ref/fate/h264_mp4toannexb_ticket5927 index 95e35c4d80..eb0b5cf283 100644 --- a/tests/ref/fate/h264_mp4toannexb_ticket5927 +++ b/tests/ref/fate/h264_mp4toannexb_ticket5927 @@ -1,12 +1,12 @@ -a3b02fd09392e01619cebc959d4d9ff2 *tests/data/fate/h264_mp4toannexb_ticket5927.h264 +edddeef7901b2bd8d55625b8105b579f *tests/data/fate/h264_mp4toannexb_ticket5927.h264 595583 tests/data/fate/h264_mp4toannexb_ticket5927.h264 -#extradata 0: 33, 0x84fe08f8 +#extradata 0: 33, 0x84e308f7 #tb 0: 1/1200000 #media_type 0: video #codec_id 0: h264 #dimensions 0: 1920x1080 #sar 0: 0/1 -0, -48000, -9223372036854775808, 48000, 247993, 0x1ce821ea +0, -48000, -9223372036854775808, 48000, 247993, 0x541321e9 0, 0, -9223372036854775808, 48000, 43354, 0xa05dca6f, F=0x0 0, 48000, -9223372036854775808, 48000, 11423, 0x5e8086dd, F=0x0 0, 96000, -9223372036854775808, 48000, 50798, 0x145fbe4f, F=0x0 @@ -18,4 +18,4 @@ a3b02fd09392e01619cebc959d4d9ff2 *tests/data/fate/h264_mp4toannexb_ticket5927.h2 0, 384000, -9223372036854775808, 48000, 54483, 0xefead99f, F=0x0 0, 432000, -9223372036854775808, 48000, 13705, 0x23cd27e8, F=0x0 0, 480000, -9223372036854775808, 48000, 22308, 0x4093b5af, F=0x0 -0, 528000, -9223372036854775808, 48000, 6369, 0x858b2aa1 +0, 528000, -9223372036854775808, 48000, 6369, 0x6cca2aa0 diff --git a/tests/ref/fate/h264_mp4toannexb_ticket5927_2 b/tests/ref/fate/h264_mp4toannexb_ticket5927_2 index 8db6a7e54a..8c3613ee79 100644 --- a/tests/ref/fate/h264_mp4toannexb_ticket5927_2 +++ b/tests/ref/fate/h264_mp4toannexb_ticket5927_2 @@ -1,12 +1,12 @@ -a3b02fd09392e01619cebc959d4d9ff2 *tests/data/fate/h264_mp4toannexb_ticket5927_2.h264 +edddeef7901b2bd8d55625b8105b579f *tests/data/fate/h264_mp4toannexb_ticket5927_2.h264 595583 tests/data/fate/h264_mp4toannexb_ticket5927_2.h264 -#extradata 0: 33, 0x84fe08f8 +#extradata 0: 33, 0x84e308f7 #tb 0: 1/1200000 #media_type 0: video #codec_id 0: h264 #dimensions 0: 1920x1080 #sar 0: 0/1 -0, -48000, -9223372036854775808, 48000, 247993, 0x1ce821ea +0, -48000, -9223372036854775808, 48000, 247993, 0x541321e9 0, 0, -9223372036854775808, 48000, 43354, 0xa05dca6f, F=0x0 0, 48000, -9223372036854775808, 48000, 11423, 0x5e8086dd, F=0x0 0, 96000, -9223372036854775808, 48000, 50798, 0x145fbe4f, F=0x0 @@ -18,4 +18,4 @@ a3b02fd09392e01619cebc959d4d9ff2 *tests/data/fate/h264_mp4toannexb_ticket5927_2. 0, 384000, -9223372036854775808, 48000, 54483, 0xefead99f, F=0x0 0, 432000, -9223372036854775808, 48000, 13705, 0x23cd27e8, F=0x0 0, 480000, -9223372036854775808, 48000, 22308, 0x4093b5af, F=0x0 -0, 528000, -9223372036854775808, 48000, 6369, 0x858b2aa1 +0, 528000, -9223372036854775808, 48000, 6369, 0x6cca2aa0