From patchwork Fri Nov 3 05:17:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Romain Beauxis X-Patchwork-Id: 44488 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:671c:b0:181:818d:5e7f with SMTP id q28csp326745pzh; Thu, 2 Nov 2023 22:20:07 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGDFENtzS6vQVE9douv1WgVGZ9Z9Dw2G+MKL2xnI4R6cNv/rZpnK64jnCtokM/ViffTYneT X-Received: by 2002:a50:9e05:0:b0:543:72cf:5056 with SMTP id z5-20020a509e05000000b0054372cf5056mr7686271ede.9.1698988807061; Thu, 02 Nov 2023 22:20:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698988807; cv=none; d=google.com; s=arc-20160816; b=IVzONheWPl4ct/xfticX0C9F1lcaRHZy4y6QsrhhKxmJIBBzyyZhtlGJmL0b5lnt1/ BgfT+pNzwnxpD8Minta7/lF5AJBR07eGLDcu5Am998apOA4R+28IjVRnIUosjLJ14vzZ pyR68EHndHN3Bsb3teSnSBU8ZEKqXg1ggOchnynMq+d5Dtc2Nt2RgiE7rhZ4+vzEYhEg LQzkpvJ6Q8N12KTNoszsdDgufZC+mV3wXuKVZYrNtF4kkx6Whh2PdHmfb75L5UFzFun2 sAPON8y67gzH7mxV1aQwhZFhRORmQ6ovFk5f7/wVNwCFIwuOnkUExShLJOrLXjhVPX1i Bk/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=ceIMAD6/cqXy5/gLr/wrgeGVNBsdesWy8WcNDz48ch0=; fh=Dy2HmWbIgzG07YKb/uJ0dh3GBmODWRQaNOnRjO8Xj4c=; b=eNx1KWuQ/WFuQY8rZPFesLV8w0+1yzvDXChz+YicOCetn4Mn/DBquY2wxAKR3sCRwS MUJR4kAhCctAysJRrajyGui3q4NmaMSgxM8Ha6AVkK1ODqqAJVOPIRIO3cQKe0awxYad HZmRoYSKl4gtwPhAUq29eC3cKzi+GfjxJcvHgEHrqDrW/IG0DTn3w7ebETqw18AK6oT6 16+JyadFIuueXvSYWgKkvmpLuiY+AQBtU2BBhgItBJax3FxgzSjewJlO9hZjEkKG/ffh dzwLa/F9Etl8MWYCheX9uHATKDEHC8TaZxxs7XmFj9BqQXt93ej+9ZXJTmxHL4oUQ20z mCBw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t22-20020a50ab56000000b0053de79e8f9esi540572edc.619.2023.11.02.22.20.06; Thu, 02 Nov 2023 22:20:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DFF6768CCD8; Fri, 3 Nov 2023 07:20:02 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-vk1-f170.google.com (mail-vk1-f170.google.com [209.85.221.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5A6D168CC70 for ; Fri, 3 Nov 2023 07:19:56 +0200 (EET) Received: by mail-vk1-f170.google.com with SMTP id 71dfb90a1353d-4abf066fa12so121514e0c.1 for ; Thu, 02 Nov 2023 22:19:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698988794; x=1699593594; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=77cbu4u/kmvrqgsQZziLCJVE9mjB6kgbSysUVERTips=; b=jQ3L3jsFSeSSIcQjxuNIy6+oLzul50XNEsWCqiBYkeak+A2gX2Pgwdb5OJCghsejEW JwHXgLUnKoMXsqyTZLkiiFFFLZDtbpgcU3/GxMyPgYGNeic8iJYWCGSWq6o2AgqVn5Wz z98o4vn4LihjDm5+p1sDQGjO50RywafiCf7jNGBN5IGohh8tUVnqpz1sjJdz+vdy14jW P3970gEvSyeQ8rx1mTRV9TsV+WzeYBAh5BBvwofdc9R9fyC5nrKWLTT8ByCSOCVbfKES Kfu/wpWbb0lSB1x2tYfKvz5FIjYmNJJnMXGpCZznZDprkNyavwFoPT/pfnYzid5xhLYp l0Aw== X-Gm-Message-State: AOJu0YyOPdef3apurqup4sg3iTmerSesBb11btRXZsibWNLxmTZA6g8z tLx019gjWmydiP7uNB1+eNhdZLyn3xqfIA== X-Received: by 2002:a05:6122:4210:b0:4a8:d7e5:e480 with SMTP id cm16-20020a056122421000b004a8d7e5e480mr19076311vkb.1.1698988794147; Thu, 02 Nov 2023 22:19:54 -0700 (PDT) Received: from localhost.localdomain (ec2-184-73-39-201.compute-1.amazonaws.com. [184.73.39.201]) by smtp.gmail.com with ESMTPSA id e5-20020ad44425000000b0064f5d70d072sm420339qvt.37.2023.11.02.22.19.53 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 02 Nov 2023 22:19:53 -0700 (PDT) From: Romain Beauxis To: ffmpeg-devel@ffmpeg.org Date: Fri, 3 Nov 2023 00:17:58 -0500 Message-Id: <20231103051757.7702-1-toots@rastageeks.org> X-Mailer: git-send-email 2.39.3 (Apple Git-145) MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2] Extract av_hls_codec_attr X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Romain Beauxis Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Ye3IThUa6p+d The logic for extracting HLS codec attribute strings is very useful and can be re-used in many different situations when working with HLS streams using libavcodec/libavformat. This patch extracts the function's code and places it into a publicly available function. Differences since v1: - ff_nal_unit_extract_rbsp renamed into avpriv_nal_unit_extract_rbsp to follow the appropriate library cross-linking convention. --- libavcodec/Makefile | 2 + libavcodec/hls.c | 105 +++++++++++++++++++++++++++++++++++++++++++ libavcodec/hls.h | 42 +++++++++++++++++ libavformat/avc.c | 4 +- libavformat/avc.h | 2 +- libavformat/hevc.c | 2 +- libavformat/hlsenc.c | 83 +++------------------------------- 7 files changed, 159 insertions(+), 81 deletions(-) create mode 100644 libavcodec/hls.c create mode 100644 libavcodec/hls.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 580a8d6b54..b3b2b18980 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -16,6 +16,7 @@ HEADERS = ac3_parser.h \ dirac.h \ dv_profile.h \ dxva2.h \ + hls.h \ jni.h \ mediacodec.h \ packet.h \ @@ -47,6 +48,7 @@ OBJS = ac3_parser.o \ get_buffer.o \ imgconvert.o \ jni.o \ + hls.o \ mathtables.o \ mediacodec.o \ mpeg12framerate.o \ diff --git a/libavcodec/hls.c b/libavcodec/hls.c new file mode 100644 index 0000000000..05a6277dbc --- /dev/null +++ b/libavcodec/hls.c @@ -0,0 +1,105 @@ +/* + * HLS public API + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "hls.h" +#include "libavutil/intreadwrite.h" +#include "libavformat/avc.h" + +int av_hls_codec_attr(AVStream *st, char *attr, size_t attr_len) { + if (st->codecpar->codec_id == AV_CODEC_ID_H264) { + uint8_t *data = st->codecpar->extradata; + if (data) { + const uint8_t *p; + + if (AV_RB32(data) == 0x01 && (data[4] & 0x1F) == 7) + p = &data[5]; + else if (AV_RB24(data) == 0x01 && (data[3] & 0x1F) == 7) + p = &data[4]; + else if (data[0] == 0x01) /* avcC */ + p = &data[1]; + else + return AVERROR_INVALIDDATA; + snprintf(attr, attr_len, + "avc1.%02x%02x%02x", p[0], p[1], p[2]); + } else { + return AVERROR_INVALIDDATA; + } + } else if (st->codecpar->codec_id == AV_CODEC_ID_HEVC) { + uint8_t *data = st->codecpar->extradata; + int profile = AV_PROFILE_UNKNOWN; + int level = AV_LEVEL_UNKNOWN; + + if (st->codecpar->profile != AV_PROFILE_UNKNOWN) + profile = st->codecpar->profile; + if (st->codecpar->level != AV_LEVEL_UNKNOWN) + level = st->codecpar->level; + + /* check the boundary of data which from current position is small than extradata_size */ + while (data && (data - st->codecpar->extradata + 19) < st->codecpar->extradata_size) { + /* get HEVC SPS NAL and seek to profile_tier_level */ + if (!(data[0] | data[1] | data[2]) && data[3] == 1 && ((data[4] & 0x7E) == 0x42)) { + uint8_t *rbsp_buf; + int remain_size = 0; + int rbsp_size = 0; + /* skip start code + nalu header */ + data += 6; + /* process by reference General NAL unit syntax */ + remain_size = st->codecpar->extradata_size - (data - st->codecpar->extradata); + rbsp_buf = avpriv_nal_unit_extract_rbsp(data, remain_size, &rbsp_size, 0); + if (!rbsp_buf) + return AVERROR_INVALIDDATA; + if (rbsp_size < 13) { + av_freep(&rbsp_buf); + break; + } + /* skip sps_video_parameter_set_id u(4), + * sps_max_sub_layers_minus1 u(3), + * and sps_temporal_id_nesting_flag u(1) */ + profile = rbsp_buf[1] & 0x1f; + /* skip 8 + 8 + 32 + 4 + 43 + 1 bit */ + level = rbsp_buf[12]; + av_freep(&rbsp_buf); + break; + } + data++; + } + if (st->codecpar->codec_tag == MKTAG('h','v','c','1') && + profile != AV_PROFILE_UNKNOWN && + level != AV_LEVEL_UNKNOWN) { + snprintf(attr, attr_len, "%s.%d.4.L%d.B01", av_fourcc2str(st->codecpar->codec_tag), profile, level); + } else + return AVERROR_INVALIDDATA; + } else if (st->codecpar->codec_id == AV_CODEC_ID_MP2) { + snprintf(attr, attr_len, "mp4a.40.33"); + } else if (st->codecpar->codec_id == AV_CODEC_ID_MP3) { + snprintf(attr, attr_len, "mp4a.40.34"); + } else if (st->codecpar->codec_id == AV_CODEC_ID_AAC) { + /* TODO : For HE-AAC, HE-AACv2, the last digit needs to be set to 5 and 29 respectively */ + snprintf(attr, attr_len, "mp4a.40.2"); + } else if (st->codecpar->codec_id == AV_CODEC_ID_AC3) { + snprintf(attr, attr_len, "ac-3"); + } else if (st->codecpar->codec_id == AV_CODEC_ID_EAC3) { + snprintf(attr, attr_len, "ec-3"); + } else { + return AVERROR_INVALIDDATA; + } + + return 0; +} diff --git a/libavcodec/hls.h b/libavcodec/hls.h new file mode 100644 index 0000000000..160e31de13 --- /dev/null +++ b/libavcodec/hls.h @@ -0,0 +1,42 @@ +/* + * HLS public API + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * A public API for HLS codec information + */ + +#ifndef AVCODEC_HLS_H +#define AVCODEC_HLS_H + +#include "libavformat/avformat.h" + +/** + * Write a NULL-terminated string representing the codec attributes + * associated with the given AVStream. + * + * @param st stream to extract codec attributes from + * @param attr buffer to write the codec attributes to + * @param attr_len size of the buffer + * @return 0 on success, a negative AVERROR code on failure + */ +int av_hls_codec_attr(AVStream *st, char *attr, size_t attr_len); + +#endif diff --git a/libavformat/avc.c b/libavformat/avc.c index b0ceb1d2d8..f85483271b 100644 --- a/libavformat/avc.c +++ b/libavformat/avc.c @@ -300,7 +300,7 @@ const uint8_t *ff_avc_mp4_find_startcode(const uint8_t *start, return start + res; } -uint8_t *ff_nal_unit_extract_rbsp(const uint8_t *src, uint32_t src_len, +uint8_t *avpriv_nal_unit_extract_rbsp(const uint8_t *src, uint32_t src_len, uint32_t *dst_len, int header_len) { uint8_t *dst; @@ -374,7 +374,7 @@ int ff_avc_decode_sps(H264SPS *sps, const uint8_t *buf, int buf_size) GetBitContext gb; uint8_t *rbsp_buf; - rbsp_buf = ff_nal_unit_extract_rbsp(buf, buf_size, &rbsp_size, 0); + rbsp_buf = avpriv_nal_unit_extract_rbsp(buf, buf_size, &rbsp_size, 0); if (!rbsp_buf) return AVERROR(ENOMEM); diff --git a/libavformat/avc.h b/libavformat/avc.h index 0ce95c194e..0bdaba162b 100644 --- a/libavformat/avc.h +++ b/libavformat/avc.h @@ -63,7 +63,7 @@ int ff_avc_write_annexb_extradata(const uint8_t *in, uint8_t **buf, int *size); const uint8_t *ff_avc_mp4_find_startcode(const uint8_t *start, const uint8_t *end, int nal_length_size); -uint8_t *ff_nal_unit_extract_rbsp(const uint8_t *src, uint32_t src_len, +uint8_t *avpriv_nal_unit_extract_rbsp(const uint8_t *src, uint32_t src_len, uint32_t *dst_len, int header_len); typedef struct { diff --git a/libavformat/hevc.c b/libavformat/hevc.c index ca5187a92e..5ea4280ebe 100644 --- a/libavformat/hevc.c +++ b/libavformat/hevc.c @@ -708,7 +708,7 @@ static int hvcc_add_nal_unit(uint8_t *nal_buf, uint32_t nal_size, uint8_t *rbsp_buf; uint32_t rbsp_size; - rbsp_buf = ff_nal_unit_extract_rbsp(nal_buf, nal_size, &rbsp_size, 2); + rbsp_buf = avpriv_nal_unit_extract_rbsp(nal_buf, nal_size, &rbsp_size, 2); if (!rbsp_buf) { ret = AVERROR(ENOMEM); goto end; diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c index 4ef84c05c1..1c0a35c36d 100644 --- a/libavformat/hlsenc.c +++ b/libavformat/hlsenc.c @@ -40,6 +40,7 @@ #include "libavutil/time_internal.h" #include "libavcodec/defs.h" +#include "libavcodec/hls.h" #include "avformat.h" #include "avio_internal.h" @@ -344,89 +345,17 @@ static void write_codec_attr(AVStream *st, VariantStream *vs) { int codec_strlen = strlen(vs->codec_attr); char attr[32]; + int ret; if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) return; if (vs->attr_status == CODEC_ATTRIBUTE_WILL_NOT_BE_WRITTEN) return; - if (st->codecpar->codec_id == AV_CODEC_ID_H264) { - uint8_t *data = st->codecpar->extradata; - if (data) { - const uint8_t *p; - - if (AV_RB32(data) == 0x01 && (data[4] & 0x1F) == 7) - p = &data[5]; - else if (AV_RB24(data) == 0x01 && (data[3] & 0x1F) == 7) - p = &data[4]; - else if (data[0] == 0x01) /* avcC */ - p = &data[1]; - else - goto fail; - snprintf(attr, sizeof(attr), - "avc1.%02x%02x%02x", p[0], p[1], p[2]); - } else { - goto fail; - } - } else if (st->codecpar->codec_id == AV_CODEC_ID_HEVC) { - uint8_t *data = st->codecpar->extradata; - int profile = AV_PROFILE_UNKNOWN; - int level = AV_LEVEL_UNKNOWN; - - if (st->codecpar->profile != AV_PROFILE_UNKNOWN) - profile = st->codecpar->profile; - if (st->codecpar->level != AV_LEVEL_UNKNOWN) - level = st->codecpar->level; - - /* check the boundary of data which from current position is small than extradata_size */ - while (data && (data - st->codecpar->extradata + 19) < st->codecpar->extradata_size) { - /* get HEVC SPS NAL and seek to profile_tier_level */ - if (!(data[0] | data[1] | data[2]) && data[3] == 1 && ((data[4] & 0x7E) == 0x42)) { - uint8_t *rbsp_buf; - int remain_size = 0; - int rbsp_size = 0; - /* skip start code + nalu header */ - data += 6; - /* process by reference General NAL unit syntax */ - remain_size = st->codecpar->extradata_size - (data - st->codecpar->extradata); - rbsp_buf = ff_nal_unit_extract_rbsp(data, remain_size, &rbsp_size, 0); - if (!rbsp_buf) - return; - if (rbsp_size < 13) { - av_freep(&rbsp_buf); - break; - } - /* skip sps_video_parameter_set_id u(4), - * sps_max_sub_layers_minus1 u(3), - * and sps_temporal_id_nesting_flag u(1) */ - profile = rbsp_buf[1] & 0x1f; - /* skip 8 + 8 + 32 + 4 + 43 + 1 bit */ - level = rbsp_buf[12]; - av_freep(&rbsp_buf); - break; - } - data++; - } - if (st->codecpar->codec_tag == MKTAG('h','v','c','1') && - profile != AV_PROFILE_UNKNOWN && - level != AV_LEVEL_UNKNOWN) { - snprintf(attr, sizeof(attr), "%s.%d.4.L%d.B01", av_fourcc2str(st->codecpar->codec_tag), profile, level); - } else - goto fail; - } else if (st->codecpar->codec_id == AV_CODEC_ID_MP2) { - snprintf(attr, sizeof(attr), "mp4a.40.33"); - } else if (st->codecpar->codec_id == AV_CODEC_ID_MP3) { - snprintf(attr, sizeof(attr), "mp4a.40.34"); - } else if (st->codecpar->codec_id == AV_CODEC_ID_AAC) { - /* TODO : For HE-AAC, HE-AACv2, the last digit needs to be set to 5 and 29 respectively */ - snprintf(attr, sizeof(attr), "mp4a.40.2"); - } else if (st->codecpar->codec_id == AV_CODEC_ID_AC3) { - snprintf(attr, sizeof(attr), "ac-3"); - } else if (st->codecpar->codec_id == AV_CODEC_ID_EAC3) { - snprintf(attr, sizeof(attr), "ec-3"); - } else { - goto fail; - } + ret = av_hls_codec_attr(st, attr, sizeof(attr)); + if (ret < 0) + goto fail; + // Don't write the same attribute multiple times if (!av_stristr(vs->codec_attr, attr)) { snprintf(vs->codec_attr + codec_strlen,