From patchwork Wed Oct 16 02:50:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andriy Gelman X-Patchwork-Id: 15784 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1E0FE447FC7 for ; Wed, 16 Oct 2019 05:51:11 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 097F068A41B; Wed, 16 Oct 2019 05:51:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f195.google.com (mail-qt1-f195.google.com [209.85.160.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 18E1A68A3ED for ; Wed, 16 Oct 2019 05:51:03 +0300 (EEST) Received: by mail-qt1-f195.google.com with SMTP id u40so33858418qth.11 for ; Tue, 15 Oct 2019 19:51:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=QGKyAI+PJ/I0rgtj5bPCEieGeEgp54DgaMKVnw1WhHw=; b=eRor/PKo62arTrIoVNSmtg3DlbpePfmLuJ0D6Y2Nkg7f963+M/XJKhtzde8QhX7I/H qNlaY22TJnYkQTRLVOfKDkixm3T5YUmpO0tcd4tbj7AE+IEwvn/C6d88l9EtNDDj3EkC fhhlIxwvqxVzb+Od4tZtBZBLtn3B4InV0FvBj1hZMwWcOEKQdzRDR7tj3ua7WkiNAZJM Qh0Sal3VGHzSMAjeY4Gg3hGEQwMrQv4EDL7fBTfRMP3jxeSt0FyYL4FXOTCYb9h4WV7e HX1D4tmjvkbYY1Zj4wt4cVqETaY8K7mnKWeaUAcNVujK5JAIiUqfOZ2sczcihRWUkMLt p/yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=QGKyAI+PJ/I0rgtj5bPCEieGeEgp54DgaMKVnw1WhHw=; b=EyMPOV/zilVYHgpWyHZdtLwp9L+cBI37QfnC8txZT4t6gRr2XaR5lozmFc+iBSxaXa HlInhH55JiuVUF6xzNBl4LrUj8GbptJNSUo8c2eIBiXUfSEtu/+dyuOzF5GGTUABO9sb 33uioeZVprNYrspLPsv7ExybYORKvxndzO0mSLAM8XK4ft57Zy8Q58PwmDZm8BysLDK2 RYcncpvqhnQ68m9io8lmCmBBwMdn3oPFMqn6NCLM2YPrLkImecMwI/Mo4lkQU/XlD/vE DvL9U3/FjnM88aQnEHKhvlfA7w6qgmWUenz5WWpn8XPPrGTM8tuqL8Cz1KWLzUFC8ccr OO6g== X-Gm-Message-State: APjAAAXA4mZUf2F2RC/2rhZEUjX7Fr8/4J9i1+KC8WyTsWjOll5Ety7A /euMsP+CqGDNn5AV1bz4uQmE3jET X-Google-Smtp-Source: APXvYqwCQ9L1YKcSF7MfvdTHnaY1izsaN3m/27lgHJ0+uiNwRzBy3xeem5k+umqpXpKb219gqiomew== X-Received: by 2002:ac8:b04:: with SMTP id e4mr43079457qti.272.1571194261307; Tue, 15 Oct 2019 19:51:01 -0700 (PDT) Received: from localhost.localdomain (c-71-232-27-28.hsd1.ma.comcast.net. [71.232.27.28]) by smtp.gmail.com with ESMTPSA id 10sm657542qkp.39.2019.10.15.19.51.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Oct 2019 19:51:00 -0700 (PDT) From: Andriy Gelman X-Google-Original-From: Andriy Gelman To: ffmpeg-devel@ffmpeg.org Date: Tue, 15 Oct 2019 22:50:39 -0400 Message-Id: <20191016025040.31273-2-andriy.gelman@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191016025040.31273-1-andriy.gelman@gmail.com> References: <20191016025040.31273-1-andriy.gelman@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v6 1/3] hevc_mp4toannexb: Insert correct parameter sets before IRAP X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andriy Gelman Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Andriy Gelman Fixes #7799 Currently, the mp4toannexb filter always inserts the same extradata at the start of the first IRAP unit. As in ticket #7799, this can lead to decoding errors if modified parameter sets are signalled in-band. Decoding errors/visual artifacts are also present in the following fate-suite/hevc-conformance datasets for hevc->mp4->hevc conversion: -RAP_B_Bossen_1.bit -RPS_C_ericsson_5.bit -SLIST_A_Sony_4.bit -SLIST_B_Sony_8.bit -SLIST_C_Sony_3.bit -SLIST_D_Sony_9.bit -TSKIP_A_MS_2.bit This commit solves these errors by keeping track of VPS/SPS/PPS parameter sets during the conversion. The correct combination is inserted at the start of the first IRAP. SEIs from extradata are inserted before each IRAP. This commit also makes an update to the hevc-bsf-mp4toannexb fate test since the result before this patch contained duplicate parameter sets in-band. --- libavcodec/hevc_mp4toannexb_bsf.c | 503 ++++++++++++++++++++++++++++-- tests/fate/hevc.mak | 2 +- 2 files changed, 472 insertions(+), 33 deletions(-) diff --git a/libavcodec/hevc_mp4toannexb_bsf.c b/libavcodec/hevc_mp4toannexb_bsf.c index 09bce5b34c..1ca5f13807 100644 --- a/libavcodec/hevc_mp4toannexb_bsf.c +++ b/libavcodec/hevc_mp4toannexb_bsf.c @@ -23,19 +23,224 @@ #include "libavutil/intreadwrite.h" #include "libavutil/mem.h" +#include "libavutil/avassert.h" #include "avcodec.h" #include "bsf.h" #include "bytestream.h" #include "hevc.h" +#include "h2645_parse.h" +#include "hevc_ps.h" +#include "golomb.h" #define MIN_HEVCC_LENGTH 23 +#define PROFILE_WITHOUT_IDC_BITS 88 +#define IS_IRAP(s) ((s)->type >= 16 && (s)->type <= 23) +#define IS_PARAMSET(s) ((s)->type >= 32 && (s)->type <= 34) + + /*reserved VCLs not included*/ +#define IS_VCL(s) ((s)->type <= 9 || ((s)->type >= 16 && (s)->type <= 21)) +#define HEVC_NAL_HEADER_BITS 16 + +/* + * Copies data from input buffer to output buffer. Appends annexb startcode. + * out must be allocated at least out_len + in_len + write_startcode * 4. + */ +#define WRITE_NAL(out, out_len, in, in_len, write_startcode) do { \ + if ((write_startcode)) { \ + AV_WB32((out) + (out_len), 1); \ + (out_len) += 4; \ + } \ + memcpy((out) + (out_len), (in), (in_len)); \ + (out_len) += (in_len); \ +} while (0) + +typedef struct Param { + uint8_t *raw_data; /* raw data to store param set payload*/ + int raw_size; /* size of raw_data*/ + size_t allocated_size; /* allocated size of raw_data*/ + int ref; /* stores the ref of the higher level parameter set*/ + int is_signalled; /* indicates whether this param has already been signalled in the cvs*/ +} Param; + +/* modified version of HEVCParamSets to store bytestream and reference to previous level*/ +typedef struct ParamSets { + Param vps_list[HEVC_MAX_VPS_COUNT]; + Param sps_list[HEVC_MAX_SPS_COUNT]; + Param pps_list[HEVC_MAX_PPS_COUNT]; + + Param sei; /* cached SEIs from extradatat*/ +} ParamSets; typedef struct HEVCBSFContext { uint8_t length_size; int extradata_parsed; + ParamSets ps; /* cached VPS/SPS/PPS parameter sets + SEI from extradata*/ } HEVCBSFContext; +static int update_cached_paramset(AVBSFContext *ctx, Param *cached_param, + H2645NAL *nal, int ref) +{ + int ret; + + if (cached_param->raw_data && cached_param->raw_size == nal->raw_size + 4 && + !memcmp(cached_param->raw_data + 4, nal->raw_data, nal->raw_size)) { + av_log(ctx, AV_LOG_DEBUG, "NAL unit: %d. Copy already exists in parameter set.\n", nal->type); + } else { + if (nal->raw_size + 4 > cached_param->allocated_size) { + ret = av_reallocp(&cached_param->raw_data, nal->raw_size + 4); + if (ret < 0) + return ret; + cached_param->allocated_size = nal->raw_size + 4; + } + cached_param->raw_size = 0; + WRITE_NAL(cached_param->raw_data, cached_param->raw_size, nal->raw_data, nal->raw_size, 1); + cached_param->ref = ref; + cached_param->is_signalled = 0; + av_assert1(cached_param->raw_size == nal->raw_size + 4); + } + return 0; +} + +static int parse_vps(AVBSFContext *ctx, H2645NAL *nal) +{ + int vps_id, ret; + + HEVCBSFContext *s = ctx->priv_data; + ParamSets *ps = &s->ps; + + GetBitContext *gb = &nal->gb; + + vps_id = get_bits(gb, 4); + + if (get_bits_left(gb) < 0) { + av_log(ctx, AV_LOG_ERROR, "Over-read bitstream in VPS id: %d\n", vps_id); + return AVERROR_INVALIDDATA; + } + + av_log(ctx, AV_LOG_TRACE, "Updating VPS id: %d\n", vps_id); + ret = update_cached_paramset(ctx, &ps->vps_list[vps_id], nal, 0); + return ret; +} + +static int parse_sps(AVBSFContext *ctx, H2645NAL *nal) +{ + int i, ret; + int sps_id, vps_ref, max_sub_layers_minus1; + + HEVCBSFContext *s = ctx->priv_data; + ParamSets *ps = &s->ps; + + uint8_t sub_layer_profile_present_flag[HEVC_MAX_SUB_LAYERS]; + uint8_t sub_layer_level_present_flag[HEVC_MAX_SUB_LAYERS]; + + GetBitContext *gb = &nal->gb; + + vps_ref = get_bits(gb, 4); + + max_sub_layers_minus1 = get_bits(gb, 3); + skip_bits1(gb); /* sps_temporal_id_nesting_flag*/ + skip_bits(gb, PROFILE_WITHOUT_IDC_BITS); /* profile_tier_level*/ + skip_bits(gb, 8); /* general_level_idc*/ + + for (i = 0; i < max_sub_layers_minus1; i++) { + sub_layer_profile_present_flag[i] = get_bits1(gb); + sub_layer_level_present_flag[i] = get_bits1(gb); + } + + if (max_sub_layers_minus1 > 0) + for (i = max_sub_layers_minus1; i < 8; i++) + skip_bits(gb, 2); /* reserved_zero_2bits[i]*/ + + for (i = 0; i < max_sub_layers_minus1; i++) { + if (sub_layer_profile_present_flag[i]) + skip_bits(gb, PROFILE_WITHOUT_IDC_BITS); /* profile_tier_level*/ + if (sub_layer_level_present_flag[i]) + skip_bits(gb, 8); /* sub_layer_level_idc*/ + } + + /*we only need the sps_id index*/ + sps_id = get_ue_golomb(gb); + if (sps_id < 0 || sps_id >= HEVC_MAX_SPS_COUNT) { + av_log(ctx, AV_LOG_ERROR, "SPS id out of range: %d\n", sps_id); + return AVERROR_INVALIDDATA; + } + + if (get_bits_left(gb) < 0) { + av_log(ctx, AV_LOG_ERROR, "Over-read bitstream in SPS id: %d\n", sps_id); + return AVERROR_INVALIDDATA; + } + + av_log(ctx, AV_LOG_TRACE, "Updating SPS id: %d, VPS ref: %d\n", sps_id, vps_ref); + ret = update_cached_paramset(ctx, &ps->sps_list[sps_id], nal, vps_ref); + return ret; +} + +static int parse_pps(AVBSFContext *ctx, H2645NAL *nal) +{ + int ret; + int pps_id, sps_ref; + + HEVCBSFContext *s = ctx->priv_data; + ParamSets *ps = &s->ps; + + GetBitContext *gb = &nal->gb; + + pps_id = get_ue_golomb(gb); + if (pps_id < 0 || pps_id >= HEVC_MAX_PPS_COUNT) { + av_log(ctx, AV_LOG_ERROR, "PPS id out of range: %d\n", pps_id); + return AVERROR_INVALIDDATA; + } + + sps_ref = get_ue_golomb(gb); + if (sps_ref < 0 || sps_ref >= HEVC_MAX_SPS_COUNT) { + av_log(ctx, AV_LOG_ERROR, "SPS id out of range: %d\n", sps_ref); + return AVERROR_INVALIDDATA; + } + + if (get_bits_left(gb) < 0) { + av_log(ctx, AV_LOG_ERROR, "Over-read bitstream in PPS id: %d\n", pps_id); + return AVERROR_INVALIDDATA; + } + + av_log(ctx, AV_LOG_TRACE, "Updating PPS id: %d, SPS ref: %d\n", pps_id, sps_ref); + ret = update_cached_paramset(ctx, &ps->pps_list[pps_id], nal, sps_ref); + return ret; +} + +static int append_sei_annexb(Param *sei, H2645NAL *nal) +{ + int ret; + + ret = av_reallocp(&sei->raw_data, sei->allocated_size + nal->raw_size + 4); + if (ret < 0) + return ret; + sei->allocated_size += nal->raw_size + 4; + + WRITE_NAL(sei->raw_data, sei->raw_size, nal->raw_data, nal->raw_size, 1); + av_assert1(sei->raw_size == sei->allocated_size); + return 0; +} + +static int update_paramset(AVBSFContext *ctx, H2645NAL *nal) +{ + int ret; + switch (nal->type) { + case (HEVC_NAL_VPS): + if ((ret = parse_vps(ctx, nal)) < 0) + return ret; + break; + case (HEVC_NAL_SPS): + if ((ret = parse_sps(ctx, nal)) < 0) + return ret; + break; + case (HEVC_NAL_PPS): + if ((ret = parse_pps(ctx, nal)) < 0) + return ret; + } + return 0; +} + static int hevc_extradata_to_annexb(AVBSFContext *ctx) { GetByteContext gb; @@ -97,6 +302,7 @@ fail: static int hevc_mp4toannexb_init(AVBSFContext *ctx) { HEVCBSFContext *s = ctx->priv_data; + H2645Packet pkt; int ret; if (ctx->par_in->extradata_size < MIN_HEVCC_LENGTH || @@ -104,77 +310,309 @@ static int hevc_mp4toannexb_init(AVBSFContext *ctx) AV_RB32(ctx->par_in->extradata) == 1) { av_log(ctx, AV_LOG_VERBOSE, "The input looks like it is Annex B already\n"); + return 0; } else { ret = hevc_extradata_to_annexb(ctx); if (ret < 0) return ret; s->length_size = ret; s->extradata_parsed = 1; + + memset(&pkt, 0, sizeof(H2645Packet)); + ret = ff_h2645_packet_split(&pkt, ctx->par_out->extradata, ctx->par_out->extradata_size, + ctx, 0, 0, AV_CODEC_ID_HEVC, 1, 0); + if (ret < 0) + goto done; + + for (int i = 0; i < pkt.nb_nals; ++i) { + H2645NAL *nal = &pkt.nals[i]; + + /*current segmentation algorithm includes next 0x00 from next nal unit*/ + if (nal->raw_data[nal->raw_size - 1] == 0x00) + nal->raw_size--; + + if (IS_PARAMSET(nal)) { + ret = update_paramset(ctx, nal); + if (ret < 0) + goto done; + continue; + } + + if (nal->type == HEVC_NAL_SEI_PREFIX || nal->type == HEVC_NAL_SEI_SUFFIX) { + ret = append_sei_annexb(&s->ps.sei, nal); + if (ret < 0) + goto done; + } + } } +done: + ff_h2645_packet_uninit(&pkt); + return ret; +} + +static void ps_uninit(ParamSets *ps) +{ + int i; + for (i = 0; i < FF_ARRAY_ELEMS(ps->vps_list); i++) { + av_freep(&ps->vps_list[i].raw_data); + ps->vps_list[i].allocated_size = 0; + ps->vps_list[i].raw_size = 0; + } + for (i = 0; i < FF_ARRAY_ELEMS(ps->sps_list); i++) { + av_freep(&ps->sps_list[i].raw_data); + ps->sps_list[i].allocated_size = 0; + ps->sps_list[i].raw_size = 0; + } + for (i = 0; i < FF_ARRAY_ELEMS(ps->pps_list); i++) { + av_freep(&ps->pps_list[i].raw_data); + ps->pps_list[i].allocated_size = 0; + ps->pps_list[i].raw_size = 0; + } + av_freep(&ps->sei); + ps->sei.allocated_size = 0; + ps->sei.raw_size = 0; +} + +static void hevc_mp4toannexb_close(AVBSFContext *ctx) +{ + HEVCBSFContext *s = ctx->priv_data; + ps_uninit(&s->ps); +} + +static int get_vcl_pps_id(AVBSFContext *ctx, H2645NAL *nal) +{ + GetBitContext *gb = &nal->gb; + int pps_id, ret; + + /* re-initialize because bitstream touched in write_extradata*/ + ret = init_get_bits(gb, nal->data, nal->size_bits); + if (ret < 0) + return ret; + skip_bits(gb, HEVC_NAL_HEADER_BITS); + + skip_bits1(gb); /* first_slice_segment_in_pic_flag*/ + + if (IS_IRAP(nal)) + skip_bits1(gb); /* no_output_of_prior_pics_flag*/ + + pps_id = get_ue_golomb(gb); + if (pps_id < 0 || pps_id >= HEVC_MAX_PPS_COUNT) { + av_log(ctx, AV_LOG_ERROR, "Invalid PPS id: %d\n", pps_id); + return AVERROR_INVALIDDATA; + } + return pps_id; +} + +static int write_vcl(AVBSFContext *ctx, AVPacket *pkt_out, int *prev_size, + H2645NAL *nal) +{ + HEVCBSFContext *s = ctx->priv_data; + ParamSets *ps = &s->ps; + int ret, pps_id; + Param *pps; + + /* get frame pps*/ + pps_id = get_vcl_pps_id(ctx, nal); + if (pps_id < 0) + return AVERROR_INVALIDDATA; + av_log(ctx, AV_LOG_TRACE, "VCL PPS id: %d\n", pps_id); + + /* write pps if not signalled*/ + pps = &ps->pps_list[pps_id]; + if (!pps->raw_data) { + av_log(ctx, AV_LOG_ERROR, "Invalid PPS: %d doesn't exist\n", pps_id); + return AVERROR_INVALIDDATA; + } + + if (!pps->is_signalled) { + av_log(ctx, AV_LOG_TRACE, "Referenced PPS: %d has not been signalled. Writing PPS to output stream\n", pps_id); + ret = av_grow_packet(pkt_out, pps->raw_size); + if (ret < 0) + return ret; + WRITE_NAL(pkt_out->data, *prev_size, pps->raw_data, pps->raw_size, 0); + pps->is_signalled = 1; + } + + /* write actual packet*/ + ret = av_grow_packet(pkt_out, nal->raw_size + 4); + if (ret < 0) + return ret; + WRITE_NAL(pkt_out->data, *prev_size, nal->raw_data, nal->raw_size, 1); return 0; } +static int write_extradata(AVBSFContext *ctx, AVPacket *pkt_out, int *prev_size, + H2645NAL *nal_irap) +{ + int ref, ret; + int new_extradata_size = 0; + + HEVCBSFContext *s = ctx->priv_data; + ParamSets *ps = &s->ps; + GetBitContext *gb = &nal_irap->gb; + + /* active parameter sets for the new irap*/ + const Param *vps; + const Param *sps; + Param *pps; /* non-const because we have to update flag that pps_id is signalled in cvs*/ + + skip_bits1(gb); /* first_slice_ic_pic_flag*/ + skip_bits1(gb); /* no_output_of_prior_pics_flag*/ + + ref = get_ue_golomb(gb); + if (ref < 0 || ref >= HEVC_MAX_PPS_COUNT || !ps->pps_list[ref].raw_data) { + av_log(ctx, AV_LOG_ERROR, "Invalid PPS: %d\n", ref); + return AVERROR_INVALIDDATA; + } + av_log(ctx, AV_LOG_TRACE, "Writing PPS id: %d\n", ref); + pps = &ps->pps_list[ref]; + new_extradata_size += pps->raw_size; + ref = pps->ref; + + if (!ps->sps_list[ref].raw_data) { + av_log(ctx, AV_LOG_ERROR, "Invalid SPS: %d\n", ref); + return AVERROR_INVALIDDATA; + } + av_log(ctx, AV_LOG_TRACE, "Writing SPS id: %d\n", ref); + sps = &ps->sps_list[ref]; + new_extradata_size += sps->raw_size; + ref = sps->ref; + + if (!ps->vps_list[ref].raw_data) { + av_log(ctx, AV_LOG_ERROR, "Invalid VPS: %d\n", ref); + return AVERROR_INVALIDDATA; + } + av_log(ctx, AV_LOG_TRACE, "Writing VPS id: %d\n", ref); + vps = &ps->vps_list[ref]; + new_extradata_size += vps->raw_size; + + if (ps->sei.raw_data) + new_extradata_size += ps->sei.raw_size; + + ret = av_grow_packet(pkt_out, new_extradata_size); + if (ret < 0) + return ret; + + WRITE_NAL(pkt_out->data, *prev_size, vps->raw_data, vps->raw_size, 0); + WRITE_NAL(pkt_out->data, *prev_size, sps->raw_data, sps->raw_size, 0); + WRITE_NAL(pkt_out->data, *prev_size, pps->raw_data, pps->raw_size, 0); + pps->is_signalled = 1; + + if (ps->sei.raw_data) + WRITE_NAL(pkt_out->data, *prev_size, ps->sei.raw_data, ps->sei.raw_size, 0); + + av_assert1(*prev_size == pkt_out->size); + return 0; +} + +/* + * Function converts mp4 access unit into annexb + * Output packet structure + * VPS, SPS, PPS, [SEI(from extradata)], [SEI_PREFIX(from access unit)], IRAP, [SEI_SUFFIX] + * or + * [SEI_PREFIX (from access unit)], [PPS (if not already signalled)], VCL(non-irap), [SEI_SUFFIX] + */ static int hevc_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *out) { HEVCBSFContext *s = ctx->priv_data; AVPacket *in; - GetByteContext gb; - - int got_irap = 0; - int i, ret = 0; + H2645Packet pkt; + int i, j, prev_size, ret; + int irap_done; ret = ff_bsf_get_packet(ctx, &in); if (ret < 0) return ret; + /* output the annexb nalu if extradata is not parsed*/ if (!s->extradata_parsed) { av_packet_move_ref(out, in); av_packet_free(&in); return 0; } - bytestream2_init(&gb, in->data, in->size); + memset(&pkt, 0, sizeof(H2645Packet)); + ret = ff_h2645_packet_split(&pkt, in->data, in->size, ctx, 1, s->length_size, AV_CODEC_ID_HEVC, 1, 0); + if (ret < 0) + goto done; + + irap_done = 1; /* 1 means that there is no irap or the irap has already been processed.*/ + for (i = 0; i < pkt.nb_nals; i++) { + if (IS_IRAP(&pkt.nals[i])) { + irap_done = 0; + break; + } + } + + prev_size = out->size; /* prev_size stores the current length of output packet */ + av_assert1(prev_size == 0); - while (bytestream2_get_bytes_left(&gb)) { - uint32_t nalu_size = 0; - int nalu_type; - int is_irap, add_extradata, extra_size, prev_size; + for (i = 0; i < pkt.nb_nals; i++) { + H2645NAL *nal = &pkt.nals[i]; + + if (IS_PARAMSET(nal)) { + ret = update_paramset(ctx, nal); + if (ret < 0) + goto done; + continue; + } - for (i = 0; i < s->length_size; i++) - nalu_size = (nalu_size << 8) | bytestream2_get_byte(&gb); + if (!irap_done && IS_IRAP(nal)) { /* append extradata and sei before first irap*/ - nalu_type = (bytestream2_peek_byte(&gb) >> 1) & 0x3f; + /* reset the pps signalled flag in this cvs*/ + for (j = 0; j < HEVC_MAX_PPS_COUNT; j++) + s->ps.pps_list[j].is_signalled = 0; - /* prepend extradata to IRAP frames */ - is_irap = nalu_type >= 16 && nalu_type <= 23; - add_extradata = is_irap && !got_irap; - extra_size = add_extradata * ctx->par_out->extradata_size; - got_irap |= is_irap; + ret = write_extradata(ctx, out, &prev_size, nal); + if (ret < 0) + goto done; + + /* append any past SEI prefix nals*/ + for (j = 0; j < i; j++) { + H2645NAL *nal_past = &pkt.nals[j]; + if (nal_past->type == HEVC_NAL_SEI_PREFIX) { + ret = av_grow_packet(out, nal_past->raw_size + 4); + if (ret < 0) + goto done; + WRITE_NAL(out->data, prev_size, nal_past->raw_data, nal_past->raw_size, 1); + } + } - if (SIZE_MAX - nalu_size < 4 || - SIZE_MAX - 4 - nalu_size < extra_size) { - ret = AVERROR_INVALIDDATA; - goto fail; + /* write irap nal unit*/ + ret = write_vcl(ctx, out, &prev_size, nal); + if (ret < 0) + goto done; + irap_done = 1; + continue; } - prev_size = out->size; + /* do not write any sei prefix nal units if irap exists but has not been + * processed*/ + if (nal->type == HEVC_NAL_SEI_PREFIX && !irap_done) + continue; - ret = av_grow_packet(out, 4 + nalu_size + extra_size); - if (ret < 0) - goto fail; + if (IS_VCL(nal)) { + ret = write_vcl(ctx, out, &prev_size, nal); + if (ret < 0) + goto done; + continue; + } - if (add_extradata) - memcpy(out->data + prev_size, ctx->par_out->extradata, extra_size); - AV_WB32(out->data + prev_size + extra_size, 1); - bytestream2_get_buffer(&gb, out->data + prev_size + 4 + extra_size, nalu_size); + /*copy any other nal units to the output - i.e. SEI_SUFFIX*/ + ret = av_grow_packet(out, nal->raw_size + 4); + if (ret < 0) + goto done; + WRITE_NAL(out->data, prev_size, nal->raw_data, nal->raw_size, 1); } + av_assert1(prev_size == out->size); + ret = av_packet_copy_props(out, in); - if (ret < 0) - goto fail; -fail: +done: + ff_h2645_packet_uninit(&pkt); + if (ret < 0) av_packet_unref(out); av_packet_free(&in); @@ -190,6 +628,7 @@ const AVBitStreamFilter ff_hevc_mp4toannexb_bsf = { .name = "hevc_mp4toannexb", .priv_data_size = sizeof(HEVCBSFContext), .init = hevc_mp4toannexb_init, + .close = hevc_mp4toannexb_close, .filter = hevc_mp4toannexb_filter, .codec_ids = codec_ids, }; diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak index 559c3898bc..4f812b0834 100644 --- a/tests/fate/hevc.mak +++ b/tests/fate/hevc.mak @@ -238,7 +238,7 @@ FATE_HEVC-$(call ALLYES, HEVC_DEMUXER MOV_DEMUXER HEVC_MP4TOANNEXB_BSF MOV_MUXER fate-hevc-bsf-mp4toannexb: tests/data/hevc-mp4.mov fate-hevc-bsf-mp4toannexb: CMD = md5 -i $(TARGET_PATH)/tests/data/hevc-mp4.mov -c:v copy -fflags +bitexact -f hevc fate-hevc-bsf-mp4toannexb: CMP = oneline -fate-hevc-bsf-mp4toannexb: REF = 1873662a3af1848c37e4eb25722c8df9 +fate-hevc-bsf-mp4toannexb: REF = 3c9d998a3aa2b9e0fb1c1f434952bf8b fate-hevc-skiploopfilter: CMD = framemd5 -skip_loop_filter nokey -i $(TARGET_SAMPLES)/hevc-conformance/SAO_D_Samsung_5.bit -sws_flags bitexact FATE_HEVC += fate-hevc-skiploopfilter