From patchwork Fri Oct 12 13:38:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: joshdk@ob-encoder.com X-Patchwork-Id: 10655 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 8547E4468A9 for ; Fri, 12 Oct 2018 16:38:47 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B3A2468A4CF; Fri, 12 Oct 2018 16:38:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 90C6268A4BE for ; Fri, 12 Oct 2018 16:38:21 +0300 (EEST) Received: by mail-wm1-f53.google.com with SMTP id r63-v6so12356527wma.4 for ; Fri, 12 Oct 2018 06:38:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ob-encoder-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id; bh=ouwkzZx23wlMZMAbvXrhGhZuP96+G/dQV5O0AkMXtKU=; b=lxL+kU3EytqGUbftz4d817LiYV5/S73nirSETTghZYZ2zj1pMU1f6AryxbT0tUQQVY o2Vn2v7KMm/KlYBIOliTXEMhB9hlmhxOIWPLZnuFPgA0q+3chcRCpekSfqe3T3gwUr4t m6AgY9jcgG6ebTs88QUEp3yxU3X7ssDuxsRAgNYjRq2JGB1RLzR1d2VROwGtxTIbWnWt MpzFFTtEPByz1/sy5xgSvlaUu2ELBjolP2aj7hpspim9zVJVV/crRaiXxZkHThbbx5sh d/Q21gSkWxVmvVrA07274QAFawqpyCox3McCd8aJKAeNuAISnSjGngpsMP/5wuXmhmyp +RNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=ouwkzZx23wlMZMAbvXrhGhZuP96+G/dQV5O0AkMXtKU=; b=Qem6oFB1wdK02NWfcgzbrujP63syAkP8k0HjEkxYUCDNHqO8RIPXC/q/NdgrOCttbE qlO54gl+jTYz1PsVhaHnwoQ9Vf531jUhom1NDWv/czgD3k7YjQatWqw/4XVikxpztEDW zxcIve4jNzeOo3EPgp8vNPafrORHC9dUIrJJ1ar5OsK2EXr4PtT2wFT3Ejc/d1cf/Zzq ES5TtkRMpxsUotf5ZFxOMEjhoKmS7WhVaaZANc9gZrBfEBuVfkhRwwqqFovpOBRmag70 3W+RGMxVUWggyOG/5jnb46cQsUKX1DfyKFMmUiI6YpHOLJ1dTlryLOwPkZn+QZ1Ct+F5 9SVQ== X-Gm-Message-State: ABuFfoihZMVaJtFyVf+IpY5uFw4IVyf5b8rvFBp2Ztq8siajK3qZQsHd R36iJpAJ7DcgOxQuzV6Kpsu2Ag6Fti4= X-Google-Smtp-Source: ACcGV63atPb4K9ZpUdfywyp6KGhH4QO0IE5g4bqG4Re1VjU243MtHf36Vr9H1i2b1vQWmy6jr2pwKg== X-Received: by 2002:a1c:1fcd:: with SMTP id f196-v6mr5288176wmf.19.1539351524065; Fri, 12 Oct 2018 06:38:44 -0700 (PDT) Received: from localhost.localdomain ([31.24.218.220]) by smtp.gmail.com with ESMTPSA id v129-v6sm905314wme.45.2018.10.12.06.38.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Oct 2018 06:38:43 -0700 (PDT) From: joshdk@ob-encoder.com X-Google-Original-From: joshdk@obe.tv To: ffmpeg-devel@ffmpeg.org Date: Fri, 12 Oct 2018 14:38:38 +0100 Message-Id: <20181012133841.13921-1-joshdk@obe.tv> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH v2 1/4] lavc/h264: create AVFrame side data from H.264 timecodes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Devin Heitmueller Create SMPTE ST 12-1 timecodes based on H.264 SEI picture timing info. For framerates > 30 FPS, the field flag is used in conjunction with pairs of frames which contain the same frame timestamp in S12M. Ensure the field is properly set per the spec. --- libavcodec/h264_sei.c | 37 ++++++++++++++++++++----------------- libavcodec/h264_sei.h | 9 +++++++++ libavcodec/h264_slice.c | 38 ++++++++++++++++++++++++++++++++++++++ libavutil/frame.c | 1 + libavutil/frame.h | 8 ++++++++ 5 files changed, 76 insertions(+), 17 deletions(-) diff --git a/libavcodec/h264_sei.c b/libavcodec/h264_sei.c index 43593d34d2..275224eabe 100644 --- a/libavcodec/h264_sei.c +++ b/libavcodec/h264_sei.c @@ -84,32 +84,35 @@ static int decode_picture_timing(H264SEIPictureTiming *h, GetBitContext *gb, return AVERROR_INVALIDDATA; num_clock_ts = sei_num_clock_ts_table[h->pic_struct]; - for (i = 0; i < num_clock_ts; i++) { - if (get_bits(gb, 1)) { /* clock_timestamp_flag */ + if (get_bits(gb, 1)) { /* clock_timestamp_flag */ unsigned int full_timestamp_flag; - + unsigned int counting_type, cnt_dropped_flag; h->ct_type |= 1 << get_bits(gb, 2); - skip_bits(gb, 1); /* nuit_field_based_flag */ - skip_bits(gb, 5); /* counting_type */ + skip_bits(gb, 1); /* nuit_field_based_flag */ + counting_type = get_bits(gb, 5); /* counting_type */ full_timestamp_flag = get_bits(gb, 1); - skip_bits(gb, 1); /* discontinuity_flag */ - skip_bits(gb, 1); /* cnt_dropped_flag */ - skip_bits(gb, 8); /* n_frames */ + skip_bits(gb, 1); /* discontinuity_flag */ + cnt_dropped_flag = get_bits(gb, 1); /* cnt_dropped_flag */ + if (cnt_dropped_flag && counting_type > 1 && counting_type < 7) + h->tc_dropframe = 1; + h->tc_frames = get_bits(gb, 8); /* n_frames */ if (full_timestamp_flag) { - skip_bits(gb, 6); /* seconds_value 0..59 */ - skip_bits(gb, 6); /* minutes_value 0..59 */ - skip_bits(gb, 5); /* hours_value 0..23 */ + h->fulltc_received = 1; + h->tc_seconds = get_bits(gb, 6); /* seconds_value 0..59 */ + h->tc_minutes = get_bits(gb, 6); /* minutes_value 0..59 */ + h->tc_hours = get_bits(gb, 5); /* hours_value 0..23 */ } else { - if (get_bits(gb, 1)) { /* seconds_flag */ - skip_bits(gb, 6); /* seconds_value range 0..59 */ - if (get_bits(gb, 1)) { /* minutes_flag */ - skip_bits(gb, 6); /* minutes_value 0..59 */ - if (get_bits(gb, 1)) /* hours_flag */ - skip_bits(gb, 5); /* hours_value 0..23 */ + if (get_bits(gb, 1)) { /* seconds_flag */ + h->tc_seconds = get_bits(gb, 6); + if (get_bits(gb, 1)) { /* minutes_flag */ + h->tc_minutes = get_bits(gb, 6); + if (get_bits(gb, 1)) /* hours_flag */ + h->tc_minutes = get_bits(gb, 5); } } } + if (sps->time_offset_length > 0) skip_bits(gb, sps->time_offset_length); /* time_offset */ diff --git a/libavcodec/h264_sei.h b/libavcodec/h264_sei.h index 5b7c8ef9d8..3b8806be0a 100644 --- a/libavcodec/h264_sei.h +++ b/libavcodec/h264_sei.h @@ -87,6 +87,15 @@ typedef struct H264SEIPictureTiming { * cpb_removal_delay in picture timing SEI message, see H.264 C.1.2 */ int cpb_removal_delay; + + /* When not continuously receiving full timecodes, we have to reference + the previous timecode received */ + int fulltc_received; + int tc_frames; + int tc_seconds; + int tc_minutes; + int tc_hours; + int tc_dropframe; } H264SEIPictureTiming; typedef struct H264SEIAFD { diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index d09cee4b13..f5415ba595 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -1287,6 +1287,44 @@ static int h264_export_frame_props(H264Context *h) h->avctx->properties |= FF_CODEC_PROPERTY_CLOSED_CAPTIONS; } + if (h->sei.picture_timing.fulltc_received) { + uint32_t tc = 0; + uint32_t frames; + + AVFrameSideData *tcside = av_frame_new_side_data(cur->f, + AV_FRAME_DATA_S12M_TIMECODE, + sizeof(uint32_t)); + if (!tcside) + return AVERROR(ENOMEM); + + /* For SMPTE 12-M timecodes, frame count is a special case if > 30 FPS. + See SMPTE ST 12-1:2014 Sec 12.1 for more info. */ + if (av_cmp_q(h->avctx->framerate, (AVRational) {30, 1}) == 1) { + frames = h->sei.picture_timing.tc_frames / 2; + if (h->sei.picture_timing.tc_frames % 2 == 1) { + if (av_cmp_q(h->avctx->framerate, (AVRational) {50, 1}) == 0) + tc |= (1 << 7); + else + tc |= (1 << 23); + } + } else { + frames = h->sei.picture_timing.tc_frames; + } + + tc |= h->sei.picture_timing.tc_dropframe << 30; + tc |= (frames / 10) << 28; + tc |= (frames % 10) << 24; + tc |= (h->sei.picture_timing.tc_seconds / 10) << 20; + tc |= (h->sei.picture_timing.tc_seconds % 10) << 16; + tc |= (h->sei.picture_timing.tc_minutes / 10) << 12; + tc |= (h->sei.picture_timing.tc_minutes % 10) << 8; + tc |= (h->sei.picture_timing.tc_hours / 10) << 4; + tc |= (h->sei.picture_timing.tc_hours % 10); + + memcpy(tcside->data, &tc, sizeof(uint32_t)); + h->sei.picture_timing.fulltc_received = 0; + } + if (h->sei.alternative_transfer.present && av_color_transfer_name(h->sei.alternative_transfer.preferred_transfer_characteristics) && h->sei.alternative_transfer.preferred_transfer_characteristics != AVCOL_TRC_UNSPECIFIED) { diff --git a/libavutil/frame.c b/libavutil/frame.c index 4460325a9b..92626dccf2 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -831,6 +831,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_MASTERING_DISPLAY_METADATA: return "Mastering display metadata"; case AV_FRAME_DATA_CONTENT_LIGHT_LEVEL: return "Content light level metadata"; case AV_FRAME_DATA_GOP_TIMECODE: return "GOP timecode"; + case AV_FRAME_DATA_S12M_TIMECODE: return "SMPTE 12-1 timecode"; case AV_FRAME_DATA_SPHERICAL: return "Spherical Mapping"; case AV_FRAME_DATA_ICC_PROFILE: return "ICC profile"; #if FF_API_FRAME_QP diff --git a/libavutil/frame.h b/libavutil/frame.h index 9d57d6ce66..e2a292980f 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -158,6 +158,14 @@ enum AVFrameSideDataType { */ AV_FRAME_DATA_QP_TABLE_DATA, #endif + + /** + * Timecode which conforms to SMPTE ST 12-1. The data is an array of 4 uint32_t + * where the first uint32_t describes how many (1-3) of the other timecodes are used. + * The timecode format is described in the av_timecode_get_smpte_from_framenum() + * function in libavutil/timecode.c. + */ + AV_FRAME_DATA_S12M_TIMECODE, }; enum AVActiveFormatDescription {