From patchwork Wed Jun 17 16:07:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Wang X-Patchwork-Id: 20438 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 029F34420F4 for ; Wed, 17 Jun 2020 19:07:28 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A51EC68B4DA; Wed, 17 Jun 2020 19:07:27 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 071D568B0B2 for ; Wed, 17 Jun 2020 19:07:21 +0300 (EEST) Received: by mail-pf1-f175.google.com with SMTP id d66so1349621pfd.6 for ; Wed, 17 Jun 2020 09:07:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=HSSpuaXJFYkKWFcZxpsu5vtUuwFz9JXGifb7dGFtFME=; b=MJc8Ny2ZGZOi4dXKhOaIHs+ZJ/KhQxVkOVfsXPWRIFfSzdU5tPF2TozaxjUn1Wmn9c ssAGOz7BFSEs+mqtnHHwgPrzlzZnw7WdPcF8co+1xzwZ+60/5s56t3Zt1Gv6Jo9CsAeN jl0QOfYGg1m63dybLPjsdjM6poyzqHzEvjEHjaDbjHC09yXIkAuRC0C2b0qvNR6DSvTs nYpatDJ664bZTP6njIJd3yYzcWFrjbFgjHDBABcRIcsk/iVARtPMwvvb64z7DcMYhqb9 VH+5z85HRH/OG8elwPBuCWOEKIWnSRH87QL6rUPr8PsG7rN9ZDlENVAGY5PlGi5Khda5 qULA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=HSSpuaXJFYkKWFcZxpsu5vtUuwFz9JXGifb7dGFtFME=; b=H1yRxpJIITGLsvZHdCCJ3QaSj6MGI/9INLGgMtVCLuRTOn/6cQjhv5CBX9TwYXoryQ HXLOUUGaYtyD9NKnjmfC7NqKWMAll9PyV0w7SC/6sn1azcWH9S9NECTSjvWpM7Jc/Qqh IEolfCrFJoHF02XcWIf/XkGbxayEnT8Hl6cdg6iOsr2NU0hw2dHmHnjkBkjZdpHGfiJ5 ilxD0kPIAD5zM41D9X02SEbQRTx7ajw4klhuhVKCcE5qiSaTVR8OFvig/plH8Bei7Bt7 HabRyL0E248GRdPo87wOroThnIaB5SEV2M0x9sRSwg3QR3MoyjvjaDgMNsa4byTLhdt4 E09Q== X-Gm-Message-State: AOAM533r94YwZxGGrx0Jf/x+LGcOp2i2B1yPaAG7CXA/ONo9syXWyRpk fbrPOs3eAQWdnXJe0O8RkW/J1h/0 X-Google-Smtp-Source: ABdhPJyOwsYnAhxmYybnyBBbB/QocvILTHkJJpkgPhYem3+oi6VZYMvvLmtjMBurg1oyrLjikgxyow== X-Received: by 2002:a62:75c1:: with SMTP id q184mr7507420pfc.210.1592410038946; Wed, 17 Jun 2020 09:07:18 -0700 (PDT) Received: from vpn2.localdomain ([161.117.202.209]) by smtp.gmail.com with ESMTPSA id g84sm318178pfb.113.2020.06.17.09.07.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Jun 2020 09:07:18 -0700 (PDT) From: lance.lmwang@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Thu, 18 Jun 2020 00:07:07 +0800 Message-Id: <1592410030-6553-1-git-send-email-lance.lmwang@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1592231234-6049-1-git-send-email-lance.lmwang@gmail.com> References: <1592231234-6049-1-git-send-email-lance.lmwang@gmail.com> Subject: [FFmpeg-devel] [PATCH v2 1/4] avcodec/hevc_sei: support HEVC timecode decode X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Limin Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Limin Wang Signed-off-by: Limin Wang --- libavcodec/hevc_sei.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ libavcodec/hevc_sei.h | 21 +++++++++++++++++++++ 2 files changed, 70 insertions(+) diff --git a/libavcodec/hevc_sei.c b/libavcodec/hevc_sei.c index 1736891..bf92de6 100644 --- a/libavcodec/hevc_sei.c +++ b/libavcodec/hevc_sei.c @@ -306,6 +306,53 @@ static int decode_nal_sei_alternative_transfer(HEVCSEIAlternativeTransfer *s, Ge return 0; } +static int decode_nal_sei_timecode(HEVCSEITimeCode *s, GetBitContext *gb) +{ + s->num_clock_ts = get_bits(gb, 2) & 3; + + for (int i = 0; i < s->num_clock_ts; i++) { + s->clock_timestamp_flag[i] = get_bits(gb, 1); + + if (s->clock_timestamp_flag[i]) { + s->units_field_based_flag[i] = get_bits(gb, 1); + s->counting_type[i] = get_bits(gb, 5); + s->full_timestamp_flag[i] = get_bits(gb, 1); + s->discontinuity_flag[i] = get_bits(gb, 1); + s->cnt_dropped_flag[i] = get_bits(gb, 1); + + s->n_frames[i] = get_bits(gb, 9); + + if (s->full_timestamp_flag[i]) { + s->seconds_value[i] = av_clip(get_bits(gb, 6), 0, 59); + s->minutes_value[i] = av_clip(get_bits(gb, 6), 0, 59); + s->hours_value[i] = av_clip(get_bits(gb, 5), 0, 23); + } else { + s->seconds_flag[i] = get_bits(gb, 1); + if (s->seconds_flag[i]) { + s->seconds_value[i] = av_clip(get_bits(gb, 6), 0, 59); + s->minutes_flag[i] = get_bits(gb, 1); + if (s->minutes_flag[i]) { + s->minutes_value[i] = av_clip(get_bits(gb, 6), 0, 59); + s->hours_flag[i] = get_bits(gb, 1); + if (s->hours_flag[i]) { + s->hours_value[i] = av_clip(get_bits(gb, 5), 0, 23); + } + } + } + } + + s->time_offset_length[i] = get_bits(gb, 5); + if (s->time_offset_length[i] > 0) { + s->time_offset_value[i] = get_bits(gb, s->time_offset_length[i]); + } + } + } + + s->present = 1; + return 0; +} + + static int decode_nal_sei_prefix(GetBitContext *gb, void *logctx, HEVCSEI *s, const HEVCParamSets *ps, int type, int size) { @@ -330,6 +377,8 @@ static int decode_nal_sei_prefix(GetBitContext *gb, void *logctx, HEVCSEI *s, return decode_nal_sei_user_data_unregistered(&s->unregistered, gb, size); case HEVC_SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS: return decode_nal_sei_alternative_transfer(&s->alternative_transfer, gb); + case HEVC_SEI_TYPE_TIME_CODE: + return decode_nal_sei_timecode(&s->timecode, gb); default: av_log(logctx, AV_LOG_DEBUG, "Skipped PREFIX SEI %d\n", type); skip_bits_long(gb, 8 * size); diff --git a/libavcodec/hevc_sei.h b/libavcodec/hevc_sei.h index 3618d16..5ee7a47 100644 --- a/libavcodec/hevc_sei.h +++ b/libavcodec/hevc_sei.h @@ -115,6 +115,26 @@ typedef struct HEVCSEIAlternativeTransfer { int preferred_transfer_characteristics; } HEVCSEIAlternativeTransfer; +typedef struct HEVCSEITimeCode { + int present; + uint8_t num_clock_ts; + uint8_t clock_timestamp_flag[3]; + uint8_t units_field_based_flag[3]; + uint8_t counting_type[3]; + uint8_t full_timestamp_flag[3]; + uint8_t discontinuity_flag[3]; + uint8_t cnt_dropped_flag[3]; + uint16_t n_frames[3]; + uint8_t seconds_value[3]; + uint8_t minutes_value[3]; + uint8_t hours_value[3]; + uint8_t seconds_flag[3]; + uint8_t minutes_flag[3]; + uint8_t hours_flag[3]; + uint8_t time_offset_length[3]; + int32_t time_offset_value[3]; +} HEVCSEITimeCode; + typedef struct HEVCSEI { HEVCSEIPictureHash picture_hash; HEVCSEIFramePacking frame_packing; @@ -126,6 +146,7 @@ typedef struct HEVCSEI { HEVCSEIContentLight content_light; int active_seq_parameter_set_id; HEVCSEIAlternativeTransfer alternative_transfer; + HEVCSEITimeCode timecode; } HEVCSEI; struct HEVCParamSets; From patchwork Wed Jun 17 16:07:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Wang X-Patchwork-Id: 20439 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id D81704430C4 for ; Wed, 17 Jun 2020 19:07:28 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B9F0768B4FA; Wed, 17 Jun 2020 19:07:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4E8DD68B393 for ; Wed, 17 Jun 2020 19:07:22 +0300 (EEST) Received: by mail-pf1-f196.google.com with SMTP id d66so1349659pfd.6 for ; Wed, 17 Jun 2020 09:07:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=T8JZKVWzRakP5nfC6PNqfo08HorKUHaX2wCeEg4S4nE=; b=gnZEGm1OFM5qDSYJOotbta/1GqZYt89W5lqKQyJAtmmPuITFyvcTUTabKq064CCdGb 1/T6S3NLXedRpzL14VxlRpOunNQIMFdq8BKVK066sa56Nv2Ppg/K1hNRDVGlg5X2E12U lWCwlYsvbQGrHzaghY+y5FOL5MZaoTl6bz7jNwNZQ9skitvrW6V+ezZ4V/XtzOcx6f6G SgbIZyJRPbNtCygsnTst2SuvUDmN+XzLHEZGEVWd+1xhIU/VX7c4Rm9tJBheA5wxp/4D WhDRlWPj3CU0S87KAM0hVf5O9Qc6hHQx2KgR4QZM/6S53sapb4bNcm4HVEA+FflKcZLn j+mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=T8JZKVWzRakP5nfC6PNqfo08HorKUHaX2wCeEg4S4nE=; b=DnRRdZYnyP8NkwfENaXSBlnSP/k79myTQ1hPXITcEKxFUpBlr2CY7bO4ZmKpLsETEi LyvfK3jiHhni+xJeQ+RlCtmnOIy5/PVH+lh+aHzRjB2MktrH+q1B0fQIIFbEuFQTL6DU ucYEiWR2s3KoeKyOlr4pHqJhsVy8CKrvyFdv7MRoJ/ipM3dIeeBzD4dgjJJJBXR/DnoE DlIZdHsx4cN5D771ne/GFx++30OJmSgip34OBhr7M2Vota7Z5qiC/XIqni6AxZw1Bzwe FUSxh998z2TFTuIDLPDFO7fSYPsFucYrhzthy7OGwH+0ibU6JTpiBRpNpCpMAZTmXkDR F5XQ== X-Gm-Message-State: AOAM531uo4Xp9clGZ5OYyhyQrgxBPP/1+QTiDSAH7uCRSvDq0YSuEIsD /C/8PWLi6VNykA3jfCubjdmgHU9t X-Google-Smtp-Source: ABdhPJyvm16DTBZhJCSxHwLXZtk0gLP0sivVHn5orDa/4DxYSQ56fnwCPk+/1KrJqaAo2+V51MOGpA== X-Received: by 2002:a63:4407:: with SMTP id r7mr7373736pga.274.1592410040527; Wed, 17 Jun 2020 09:07:20 -0700 (PDT) Received: from vpn2.localdomain ([161.117.202.209]) by smtp.gmail.com with ESMTPSA id g84sm318178pfb.113.2020.06.17.09.07.19 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Jun 2020 09:07:19 -0700 (PDT) From: lance.lmwang@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Thu, 18 Jun 2020 00:07:08 +0800 Message-Id: <1592410030-6553-2-git-send-email-lance.lmwang@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1592410030-6553-1-git-send-email-lance.lmwang@gmail.com> References: <1592231234-6049-1-git-send-email-lance.lmwang@gmail.com> <1592410030-6553-1-git-send-email-lance.lmwang@gmail.com> Subject: [FFmpeg-devel] [PATCH v2 2/4] avcodec/hevcdec: create AVFrame side data from HEVC timecodes like H.264 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Limin Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Limin Wang Signed-off-by: Limin Wang --- libavcodec/hevcdec.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index c9e28f5..39abbb9 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -2808,6 +2808,50 @@ static int set_side_data(HEVCContext *s) } s->sei.unregistered.nb_buf_ref = 0; + if (s->sei.timecode.present) { + uint32_t tc = 0; + uint32_t *tc_sd; + AVFrameSideData *tcside = av_frame_new_side_data(out, AV_FRAME_DATA_S12M_TIMECODE, + sizeof(uint32_t) * 4); + if (!tcside) + return AVERROR(ENOMEM); + + tc_sd = (uint32_t*)tcside->data; + tc_sd[0] = s->sei.timecode.num_clock_ts; + + for (int i = 0; i < tc_sd[0]; i++) { + uint32_t frames; + + /* For SMPTE 12-M timecodes, frame count is a special case if > 30 FPS. + See SMPTE ST 12-1:2014 Sec 12.1 for more info. */ + if (av_cmp_q(s->avctx->framerate, (AVRational) {30, 1}) == 1) { + frames = s->sei.timecode.n_frames[i] / 2; + if (s->sei.timecode.n_frames[i] % 2 == 1) { + if (av_cmp_q(s->avctx->framerate, (AVRational) {50, 1}) == 0) + tc |= (1 << 7); + else + tc |= (1 << 23); + } + } else { + frames = s->sei.timecode.n_frames[i]; + } + + tc |= s->sei.timecode.cnt_dropped_flag[i] << 30; + tc |= (frames / 10) << 28; + tc |= (frames % 10) << 24; + tc |= (s->sei.timecode.seconds_value[i] / 10) << 20; + tc |= (s->sei.timecode.seconds_value[i] % 10) << 16; + tc |= (s->sei.timecode.minutes_value[i] / 10) << 12; + tc |= (s->sei.timecode.minutes_value[i] % 10) << 8; + tc |= (s->sei.timecode.hours_value[i] / 10) << 4; + tc |= (s->sei.timecode.hours_value[i] % 10); + + tc_sd[i + 1] = tc; + } + + s->sei.timecode.num_clock_ts = 0; + } + return 0; } From patchwork Wed Jun 17 16:07:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Wang X-Patchwork-Id: 20440 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id AC6B244AB10 for ; Wed, 17 Jun 2020 19:07:30 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 99DC168B553; Wed, 17 Jun 2020 19:07:30 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DF0D168B53F for ; Wed, 17 Jun 2020 19:07:23 +0300 (EEST) Received: by mail-pf1-f195.google.com with SMTP id a127so1333636pfa.12 for ; Wed, 17 Jun 2020 09:07:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=KhQjBAuPhEthnL8ifpsTFzZUWSlKn0bMYex0ds8nSJQ=; b=ujy7mWxi5deMyO+hNQdsIEavNulM9qmEn0bv4qUkBh6JPU+I8U6s/XPlvfPCLM84A7 xtOVnq5KuDLF1f6GxLM0M7GsrjmiZm5CKWQDlCpuSx2mOO9jJs0l4s7dM0MRhdYv23wr CXJcjuy9ckOWgHGc/VhOdRZ5e6cNKVNRJaRTZU6svSuWPy5NOwtGZmBPqEa6xB/SMCma /O6KQv3qF0+xqG0it+4eDNEl2Nb5Ngqoy0HddpkZ7xtvTmx2343ykewETvEB+caVqT1X 8R+DsB+3OYvlUcKn7ltlWr53Jt8g0GVgDHBfhvZ24HzyIG+1NX24Dn4gHIQfLdiWFx92 Entg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=KhQjBAuPhEthnL8ifpsTFzZUWSlKn0bMYex0ds8nSJQ=; b=m4ah6VyETiHq1bjUsmV7XL/kP3b85B2n/z9VHhgfLf6OPDoqj7ur5zuQdQ85OO7Oug R8e1ctKllJMFSS9JThz0SesttkIVrt4dh0hLf9OBnUEWHl8CaD41QR59Mcmgf8teql8n QN+3BFW0mO0iz+F29bGBaokwAlUa37Ao3zazyvEg9N9c/sSTwC6E9P6fb2ZVvIS5Wat5 qS8tSC19OHuIkdEpSj/OMo6gVFbr35X2ikH5/xoSoa72NLvYkJMnEnNSVSDi8B9o+wcC J9syR9ET3K9IGcTWUweyM+EDee/du9TkxK3vpgS8ipPlrAhaYA7p7hkvAnsZKEHSCBjI dx6w== X-Gm-Message-State: AOAM531HijEwcdY5qKl4XfeoC5h3qhd9MU+bi+I6fPEr5wuGDUfN+/Ge pF4bNl4/j1uSX6a+61Jq0SCM2Ixo X-Google-Smtp-Source: ABdhPJwA83LD9TUuugx06FmAlj5fg9wrxy/byjV8HDtqK4S1gdxLWY6cFvgkRLRNOKQNivWXe5tvug== X-Received: by 2002:a62:196:: with SMTP id 144mr2298482pfb.316.1592410042102; Wed, 17 Jun 2020 09:07:22 -0700 (PDT) Received: from vpn2.localdomain ([161.117.202.209]) by smtp.gmail.com with ESMTPSA id g84sm318178pfb.113.2020.06.17.09.07.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Jun 2020 09:07:21 -0700 (PDT) From: lance.lmwang@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Thu, 18 Jun 2020 00:07:09 +0800 Message-Id: <1592410030-6553-3-git-send-email-lance.lmwang@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1592410030-6553-1-git-send-email-lance.lmwang@gmail.com> References: <1592231234-6049-1-git-send-email-lance.lmwang@gmail.com> <1592410030-6553-1-git-send-email-lance.lmwang@gmail.com> Subject: [FFmpeg-devel] [PATCH v2 3/4] avcodec/nvenc: add more sei data support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Limin Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Limin Wang Signed-off-by: Limin Wang --- libavcodec/nvenc.c | 36 ++++++++++++++++++++++-------------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c index e269c71..e4356ce 100644 --- a/libavcodec/nvenc.c +++ b/libavcodec/nvenc.c @@ -1776,7 +1776,8 @@ static int nvenc_upload_frame(AVCodecContext *avctx, const AVFrame *frame, static void nvenc_codec_specific_pic_params(AVCodecContext *avctx, NV_ENC_PIC_PARAMS *params, - NV_ENC_SEI_PAYLOAD *sei_data) + NV_ENC_SEI_PAYLOAD *sei_data, + int sei_count) { NvencContext *ctx = avctx->priv_data; @@ -1786,9 +1787,9 @@ static void nvenc_codec_specific_pic_params(AVCodecContext *avctx, ctx->encode_config.encodeCodecConfig.h264Config.sliceMode; params->codecPicParams.h264PicParams.sliceModeData = ctx->encode_config.encodeCodecConfig.h264Config.sliceModeData; - if (sei_data) { + if (sei_count > 0) { params->codecPicParams.h264PicParams.seiPayloadArray = sei_data; - params->codecPicParams.h264PicParams.seiPayloadArrayCnt = 1; + params->codecPicParams.h264PicParams.seiPayloadArrayCnt = sei_count; } break; @@ -1797,9 +1798,9 @@ static void nvenc_codec_specific_pic_params(AVCodecContext *avctx, ctx->encode_config.encodeCodecConfig.hevcConfig.sliceMode; params->codecPicParams.hevcPicParams.sliceModeData = ctx->encode_config.encodeCodecConfig.hevcConfig.sliceModeData; - if (sei_data) { + if (sei_count > 0) { params->codecPicParams.hevcPicParams.seiPayloadArray = sei_data; - params->codecPicParams.hevcPicParams.seiPayloadArrayCnt = 1; + params->codecPicParams.hevcPicParams.seiPayloadArrayCnt = sei_count; } break; @@ -2072,8 +2073,9 @@ int ff_nvenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) NVENCSTATUS nv_status; NvencSurface *tmp_out_surf, *in_surf; int res, res2; - NV_ENC_SEI_PAYLOAD *sei_data = NULL; - size_t sei_size; + NV_ENC_SEI_PAYLOAD sei_data[8]; + int sei_count = 0; + int i; NvencContext *ctx = avctx->priv_data; NvencDynLoadFunctions *dl_fn = &ctx->nvenc_dload_funcs; @@ -2139,18 +2141,22 @@ int ff_nvenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) pic_params.inputTimeStamp = frame->pts; if (ctx->a53_cc && av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC)) { - if (ff_alloc_a53_sei(frame, sizeof(NV_ENC_SEI_PAYLOAD), (void**)&sei_data, &sei_size) < 0) { + void *a53_data = NULL; + size_t a53_size = 0; + + if (ff_alloc_a53_sei(frame, 0, (void**)&a53_data, &a53_size) < 0) { av_log(ctx, AV_LOG_ERROR, "Not enough memory for closed captions, skipping\n"); } - if (sei_data) { - sei_data->payloadSize = (uint32_t)sei_size; - sei_data->payloadType = 4; - sei_data->payload = (uint8_t*)(sei_data + 1); + if (a53_data) { + sei_data[sei_count].payloadSize = (uint32_t)a53_size; + sei_data[sei_count].payloadType = 4; + sei_data[sei_count].payload = (uint8_t*)a53_data; + sei_count ++; } } - nvenc_codec_specific_pic_params(avctx, &pic_params, sei_data); + nvenc_codec_specific_pic_params(avctx, &pic_params, sei_data, sei_count); } else { pic_params.encodePicFlags = NV_ENC_PIC_FLAG_EOS; ctx->encoder_flushing = 1; @@ -2161,7 +2167,9 @@ int ff_nvenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) return res; nv_status = p_nvenc->nvEncEncodePicture(ctx->nvencoder, &pic_params); - av_free(sei_data); + + for ( i = 0; i < sei_count; i++) + av_freep(&sei_data[i].payload); res = nvenc_pop_context(avctx); if (res < 0) From patchwork Wed Jun 17 16:07:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Wang X-Patchwork-Id: 20441 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 6A4F644B7D1 for ; Wed, 17 Jun 2020 19:30:17 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5143968B4D9; Wed, 17 Jun 2020 19:30:17 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f194.google.com (mail-oi1-f194.google.com [209.85.167.194]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2DA2668A9DB for ; Wed, 17 Jun 2020 19:30:10 +0300 (EEST) Received: by mail-oi1-f194.google.com with SMTP id k4so2324429oik.2 for ; Wed, 17 Jun 2020 09:30:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1ac3CETL2QdQs3frGEW/HSHUUZS0PIXwq6kxWnj/8IM=; b=Sz9KSF334iJuLRPCZxemNJpcwN3z/6TK9zYVCuFFmiD5Xoh1Ra3+sVYDlhiZrKO6KE g3PFslXC+Aic4bkQwyyn6wb/uFNwNhRw/75IGjHiWw/LgnskejXEzeVBQJqqDq3xDYtc UDxpDsLJiXolzxRtx9d5p67RMpOAmtU12vKr1PgJeTE5v5keRcRhCts1BIHgN8dxBWYU 554c0rNeWoyF9K4r0ACDh4rLYY3L85SHAQRQIiVvH43trXNFwIhLhAJS6SC8iKkI9/kY ApKHE+p+K/o9k4QISNL2FsiZxdgR3CSNMjDylovSbPUAQcmbz0pSrT7+vszdoIbdCxx3 WIWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1ac3CETL2QdQs3frGEW/HSHUUZS0PIXwq6kxWnj/8IM=; b=BWNRuiPaVRoztsnbCGW8lMtSSX2136x7T47zjJGgFK5VylOD++FBrZQZYiIfKTjqJt qy4QMY9PDyXzF1X5uYmFBzkRCyuOySuSpQPO8y9SArB5iV62om/w5k47fhKwpBbJPHHs 9h2CpNVDluK915dd9I+URp3c1GqCN0/PC/GCsdMzQHxjf9HPYQi8zGRkd6ferUodAPVY uNDah1PWfh8zwahlKVHfPo8qQFZiMiK8LRzY1h9xMzFSnZV29IXOs+uDNtvjv5mPq9wJ Tli5J3OkRmJFSccERiqmvegEoEXSacjdi1Ws8o5E8bGOri8hMfgDZXcLhjdWZ1g8opYL RMBw== X-Gm-Message-State: AOAM532Wi29yjHfsVHjwIkNZTciLDCJhQRsGpZTGICCsIhsflEPYeWUX 83bw8zPrJC7ldaP3t1c+Vb4bgif7 X-Google-Smtp-Source: ABdhPJzbDiQYQtPlu896ti2svvePRpSbR7JRhD4HlG0lFh0GeuL3qcGTkK3+LISxPaaiyxa9ewAbqA== X-Received: by 2002:a17:90a:f485:: with SMTP id bx5mr8953477pjb.77.1592410043648; Wed, 17 Jun 2020 09:07:23 -0700 (PDT) Received: from vpn2.localdomain ([161.117.202.209]) by smtp.gmail.com with ESMTPSA id g84sm318178pfb.113.2020.06.17.09.07.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Jun 2020 09:07:23 -0700 (PDT) From: lance.lmwang@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Thu, 18 Jun 2020 00:07:10 +0800 Message-Id: <1592410030-6553-4-git-send-email-lance.lmwang@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1592410030-6553-1-git-send-email-lance.lmwang@gmail.com> References: <1592231234-6049-1-git-send-email-lance.lmwang@gmail.com> <1592410030-6553-1-git-send-email-lance.lmwang@gmail.com> Subject: [FFmpeg-devel] [PATCH v2 4/4] avcodec/nvenc: support for HEVC timecode passthrough X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Limin Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Limin Wang Signed-off-by: Limin Wang --- I failed to find any document about nvenc so that I can update for new option. libavcodec/nvenc.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++ libavcodec/nvenc.h | 1 + libavcodec/nvenc_hevc.c | 1 + 3 files changed, 55 insertions(+) diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c index e4356ce..68b212a 100644 --- a/libavcodec/nvenc.c +++ b/libavcodec/nvenc.c @@ -22,6 +22,8 @@ #include "config.h" #include "nvenc.h" +#include "hevc_sei.h" +#include "put_bits.h" #include "libavutil/hwcontext_cuda.h" #include "libavutil/hwcontext.h" @@ -2068,6 +2070,15 @@ static void reconfig_encoder(AVCodecContext *avctx, const AVFrame *frame) } } +static unsigned bcd2uint(uint8_t bcd) +{ + unsigned low = bcd & 0xf; + unsigned high = bcd >> 4; + if (low > 9 || high > 9) + return 0; + return low + 10*high; +} + int ff_nvenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) { NVENCSTATUS nv_status; @@ -2156,6 +2167,48 @@ int ff_nvenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) } } + if (ctx->tc && av_frame_get_side_data(frame, AV_FRAME_DATA_S12M_TIMECODE)) { + AVFrameSideData *sd = av_frame_get_side_data(frame, AV_FRAME_DATA_S12M_TIMECODE); + if (sd) { + uint32_t *tc = (uint32_t*)sd->data; + int m = tc[0] & 3; + if (tc) { + sei_data[sei_count].payloadSize = sizeof(uint32_t) * 4; + sei_data[sei_count].payloadType = HEVC_SEI_TYPE_TIME_CODE; + sei_data[sei_count].payload = av_mallocz(sei_data[sei_count].payloadSize); + if (sei_data[sei_count].payload) { + PutBitContext pb; + + init_put_bits(&pb, sei_data[sei_count].payload, sei_data[sei_count].payloadSize); + put_bits(&pb, 2, m); // num_clock_ts + + for (int j = 1; j <= m; j++) { + uint32_t tcsmpte = tc[j]; + unsigned hh = bcd2uint(tcsmpte & 0x3f); // 6-bit hours + unsigned mm = bcd2uint(tcsmpte>>8 & 0x7f); // 7-bit minutes + unsigned ss = bcd2uint(tcsmpte>>16 & 0x7f); // 7-bit seconds + unsigned ff = bcd2uint(tcsmpte>>24 & 0x3f); // 6-bit frames + unsigned drop = tcsmpte & 1<<30 && !0; // 1-bit drop if not arbitrary bit + + put_bits(&pb, 1, 1); // clock_timestamp_flag + put_bits(&pb, 1, 1); // units_field_based_flag + put_bits(&pb, 5, 0); // counting_type + put_bits(&pb, 1, 1); // full_timestamp_flag + put_bits(&pb, 1, 0); // discontinuity_flag + put_bits(&pb, 1, drop); + put_bits(&pb, 9, ff); + put_bits(&pb, 6, ss); + put_bits(&pb, 6, mm); + put_bits(&pb, 5, hh); + put_bits(&pb, 5, 0); + } + flush_put_bits(&pb); + sei_count++; + } + } + } + } + nvenc_codec_specific_pic_params(avctx, &pic_params, sei_data, sei_count); } else { pic_params.encodePicFlags = NV_ENC_PIC_FLAG_EOS; diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h index 7a415a4..f1adaeb 100644 --- a/libavcodec/nvenc.h +++ b/libavcodec/nvenc.h @@ -196,6 +196,7 @@ typedef struct NvencContext int coder; int b_ref_mode; int a53_cc; + int tc; int dpb_size; } NvencContext; diff --git a/libavcodec/nvenc_hevc.c b/libavcodec/nvenc_hevc.c index 7f12b56..6eef613 100644 --- a/libavcodec/nvenc_hevc.c +++ b/libavcodec/nvenc_hevc.c @@ -129,6 +129,7 @@ static const AVOption options[] = { { "each", "", 0, AV_OPT_TYPE_CONST, { .i64 = 1 }, 0, 0, VE, "b_ref_mode" }, { "middle", "", 0, AV_OPT_TYPE_CONST, { .i64 = 2 }, 0, 0, VE, "b_ref_mode" }, #endif + { "tc", "Use timecode (if available)", OFFSET(tc), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, VE }, { "dpb_size", "Specifies the DPB size used for encoding (0 means automatic)", OFFSET(dpb_size), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, VE }, { NULL }