From patchwork Fri Mar 13 10:28:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 18165 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id C57D244B540 for ; Fri, 13 Mar 2020 12:30:24 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A925268B13A; Fri, 13 Mar 2020 12:30:24 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2CC2268B0B1 for ; Fri, 13 Mar 2020 12:30:18 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id 0CD3A29579C for ; Fri, 13 Mar 2020 11:30:18 +0100 (CET) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id 1wJb67KPmftr for ; Fri, 13 Mar 2020 11:30:17 +0100 (CET) Received: from quelana.khirnov.net (unknown [IPv6:2002:b061:f0a:201:5e:e696:5100:0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id A485D295798 for ; Fri, 13 Mar 2020 11:30:15 +0100 (CET) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id 4C9E325345 for ; Fri, 13 Mar 2020 11:30:15 +0100 (CET) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id 1yS5I2uBQjF2 for ; Fri, 13 Mar 2020 11:30:13 +0100 (CET) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id 46AD125356 for ; Fri, 13 Mar 2020 11:30:00 +0100 (CET) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id DB7C720E0361; Fri, 13 Mar 2020 11:29:56 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 13 Mar 2020 11:28:45 +0100 Message-Id: <20200313102850.23913-13-anton@khirnov.net> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200313102850.23913-1-anton@khirnov.net> References: <20200313102850.23913-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 13/18] h264_sei: parse the picture timing SEIs correctly X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Those SEIs refer to the currently active SPS. However, since the SEI NALUs precede the coded picture data in the bitstream, the active SPS is in general not known when we are decoding the SEI. Therefore, store the content of the picture timing SEIs and actually parse it when the active SPS is known. --- libavcodec/h264_parser.c | 9 +++++ libavcodec/h264_sei.c | 86 ++++++++++++++++++++++++---------------- libavcodec/h264_sei.h | 11 +++++ libavcodec/h264_slice.c | 10 +++++ 4 files changed, 82 insertions(+), 34 deletions(-) diff --git a/libavcodec/h264_parser.c b/libavcodec/h264_parser.c index 5f9a9c46ef..ec1cbc6a66 100644 --- a/libavcodec/h264_parser.c +++ b/libavcodec/h264_parser.c @@ -481,6 +481,15 @@ static inline int parse_nal_units(AVCodecParserContext *s, } } + if (p->sei.picture_timing.present) { + ret = ff_h264_sei_process_picture_timing(&p->sei.picture_timing, + sps, avctx); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Error processing the picture timing SEI\n"); + p->sei.picture_timing.present = 0; + } + } + if (sps->pic_struct_present_flag && p->sei.picture_timing.present) { switch (p->sei.picture_timing.pic_struct) { case H264_SEI_PIC_STRUCT_TOP_FIELD: diff --git a/libavcodec/h264_sei.c b/libavcodec/h264_sei.c index a2452141ca..4f536f82f2 100644 --- a/libavcodec/h264_sei.c +++ b/libavcodec/h264_sei.c @@ -54,30 +54,22 @@ void ff_h264_sei_uninit(H264SEIContext *h) av_buffer_unref(&h->a53_caption.buf_ref); } -static int decode_picture_timing(H264SEIPictureTiming *h, GetBitContext *gb, - const H264ParamSets *ps, void *logctx) +int ff_h264_sei_process_picture_timing(H264SEIPictureTiming *h, const SPS *sps, + void *logctx) { - int i; - const SPS *sps = ps->sps; - - for (i = 0; ilog2_max_frame_num) && ps->sps_list[i]) - sps = (const SPS *)ps->sps_list[i]->data; + GetBitContext gb; - if (!sps) { - av_log(logctx, AV_LOG_ERROR, "SPS unavailable in decode_picture_timing\n"); - return AVERROR_PS_NOT_FOUND; - } + init_get_bits(&gb, h->payload, h->payload_size_bits); if (sps->nal_hrd_parameters_present_flag || sps->vcl_hrd_parameters_present_flag) { - h->cpb_removal_delay = get_bits_long(gb, sps->cpb_removal_delay_length); - h->dpb_output_delay = get_bits_long(gb, sps->dpb_output_delay_length); + h->cpb_removal_delay = get_bits_long(&gb, sps->cpb_removal_delay_length); + h->dpb_output_delay = get_bits_long(&gb, sps->dpb_output_delay_length); } if (sps->pic_struct_present_flag) { unsigned int i, num_clock_ts; - h->pic_struct = get_bits(gb, 4); + h->pic_struct = get_bits(&gb, 4); h->ct_type = 0; if (h->pic_struct > H264_SEI_PIC_STRUCT_FRAME_TRIPLING) @@ -86,38 +78,38 @@ static int decode_picture_timing(H264SEIPictureTiming *h, GetBitContext *gb, num_clock_ts = sei_num_clock_ts_table[h->pic_struct]; h->timecode_cnt = 0; for (i = 0; i < num_clock_ts; i++) { - if (get_bits(gb, 1)) { /* clock_timestamp_flag */ + if (get_bits(&gb, 1)) { /* clock_timestamp_flag */ H264SEITimeCode *tc = &h->timecode[h->timecode_cnt++]; unsigned int full_timestamp_flag; unsigned int counting_type, cnt_dropped_flag; - h->ct_type |= 1 << get_bits(gb, 2); - skip_bits(gb, 1); /* nuit_field_based_flag */ - counting_type = get_bits(gb, 5); /* counting_type */ - full_timestamp_flag = get_bits(gb, 1); - skip_bits(gb, 1); /* discontinuity_flag */ - cnt_dropped_flag = get_bits(gb, 1); /* cnt_dropped_flag */ + h->ct_type |= 1 << get_bits(&gb, 2); + skip_bits(&gb, 1); /* nuit_field_based_flag */ + counting_type = get_bits(&gb, 5); /* counting_type */ + full_timestamp_flag = get_bits(&gb, 1); + skip_bits(&gb, 1); /* discontinuity_flag */ + cnt_dropped_flag = get_bits(&gb, 1); /* cnt_dropped_flag */ if (cnt_dropped_flag && counting_type > 1 && counting_type < 7) tc->dropframe = 1; - tc->frame = get_bits(gb, 8); /* n_frames */ + tc->frame = get_bits(&gb, 8); /* n_frames */ if (full_timestamp_flag) { tc->full = 1; - tc->seconds = get_bits(gb, 6); /* seconds_value 0..59 */ - tc->minutes = get_bits(gb, 6); /* minutes_value 0..59 */ - tc->hours = get_bits(gb, 5); /* hours_value 0..23 */ + tc->seconds = get_bits(&gb, 6); /* seconds_value 0..59 */ + tc->minutes = get_bits(&gb, 6); /* minutes_value 0..59 */ + tc->hours = get_bits(&gb, 5); /* hours_value 0..23 */ } else { tc->seconds = tc->minutes = tc->hours = tc->full = 0; - if (get_bits(gb, 1)) { /* seconds_flag */ - tc->seconds = get_bits(gb, 6); - if (get_bits(gb, 1)) { /* minutes_flag */ - tc->minutes = get_bits(gb, 6); - if (get_bits(gb, 1)) /* hours_flag */ - tc->hours = get_bits(gb, 5); + if (get_bits(&gb, 1)) { /* seconds_flag */ + tc->seconds = get_bits(&gb, 6); + if (get_bits(&gb, 1)) { /* minutes_flag */ + tc->minutes = get_bits(&gb, 6); + if (get_bits(&gb, 1)) /* hours_flag */ + tc->hours = get_bits(&gb, 5); } } } if (sps->time_offset_length > 0) - skip_bits(gb, + skip_bits(&gb, sps->time_offset_length); /* time_offset */ } } @@ -126,6 +118,32 @@ static int decode_picture_timing(H264SEIPictureTiming *h, GetBitContext *gb, h->ct_type, h->pic_struct); } + return 0; +} + +static int decode_picture_timing(H264SEIPictureTiming *h, GetBitContext *gb, + void *logctx) +{ + PutBitContext pb; + int n; + + if (get_bits_left(gb) > sizeof(h->payload) * 8) { + av_log(logctx, AV_LOG_ERROR, "Picture timing SEI payload too large\n"); + return AVERROR_INVALIDDATA; + } + + init_put_bits(&pb, h->payload, sizeof(h->payload)); + + while (get_bits_left(gb) >= 32) + put_bits(&pb, 32, get_bits_long(gb, 32)); + + n = get_bits_left(gb); + if (n) + put_bits(&pb, n, get_bits_long(gb, n)); + + h->payload_size_bits = put_bits_count(&pb); + flush_put_bits(&pb); + h->present = 1; return 0; } @@ -436,7 +454,7 @@ int ff_h264_sei_decode(H264SEIContext *h, GetBitContext *gb, switch (type) { case H264_SEI_TYPE_PIC_TIMING: // Picture timing SEI - ret = decode_picture_timing(&h->picture_timing, &gb_payload, ps, logctx); + ret = decode_picture_timing(&h->picture_timing, &gb_payload, logctx); break; case H264_SEI_TYPE_USER_DATA_REGISTERED: ret = decode_registered_user_data(h, &gb_payload, logctx, size); diff --git a/libavcodec/h264_sei.h b/libavcodec/h264_sei.h index a75c3aa175..f07a5055c3 100644 --- a/libavcodec/h264_sei.h +++ b/libavcodec/h264_sei.h @@ -20,6 +20,7 @@ #define AVCODEC_H264_SEI_H #include "get_bits.h" +#include "h264_ps.h" /** * SEI message types @@ -79,6 +80,10 @@ typedef struct H264SEITimeCode { } H264SEITimeCode; typedef struct H264SEIPictureTiming { + // maximum size of pic_timing according to the spec should be 274 bits + uint8_t payload[40]; + int payload_size_bits; + int present; H264_SEI_PicStructType pic_struct; @@ -202,4 +207,10 @@ void ff_h264_sei_uninit(H264SEIContext *h); */ const char *ff_h264_sei_stereo_mode(const H264SEIFramePacking *h); +/** + * Parse the contents of a picture timing message given an active SPS. + */ +int ff_h264_sei_process_picture_timing(H264SEIPictureTiming *h, const SPS *sps, + void *logctx); + #endif /* AVCODEC_H264_SEI_H */ diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index ed4f711ac5..120f282c0a 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -1134,6 +1134,16 @@ static int h264_export_frame_props(H264Context *h) /* Signal interlacing information externally. */ /* Prioritize picture timing SEI information over used * decoding process if it exists. */ + if (h->sei.picture_timing.present) { + int ret = ff_h264_sei_process_picture_timing(&h->sei.picture_timing, sps, + h->avctx); + if (ret < 0) { + av_log(h->avctx, AV_LOG_ERROR, "Error processing a picture timing SEI\n"); + if (h->avctx->err_recognition & AV_EF_EXPLODE) + return ret; + h->sei.picture_timing.present = 0; + } + } if (sps->pic_struct_present_flag && h->sei.picture_timing.present) { H264SEIPictureTiming *pt = &h->sei.picture_timing;