From patchwork Thu Jan 10 02:26:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rodger Combs X-Patchwork-Id: 11691 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 324ED44CC7F for ; Thu, 10 Jan 2019 04:32:40 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1F27068A726; Thu, 10 Jan 2019 04:32:28 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f195.google.com (mail-qt1-f195.google.com [209.85.160.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CCE0068A216 for ; Thu, 10 Jan 2019 04:32:21 +0200 (EET) Received: by mail-qt1-f195.google.com with SMTP id t33so10825901qtt.4 for ; Wed, 09 Jan 2019 18:32:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=iIoIkPaKluKJqIgWti9VqjGZYHDmFr+1zLTMMzDyw48=; b=uKv3ESG7A9Vz6OP1QVGeHsSYMVmF0hPkhNCAZafOxsPYvyqBxVqiGzzxCujpGEHYvK CVqcNDCwxYlcxmoa9yKMVNrJcNTLXLil20CiS2Z3BvQ5hg2P6Dp5PadGG8AP358j+sU+ DQUgTTBAgrfLKCniDPpE7e5QoDXong4YGBHZGp8iFNMQrzIfiE86XddNDWxLDbpXcWe3 D447L6yevT1JT+X+kjyVU5ForFwCTSGhcmc81FOjt+x0nYaioJbvXLd7nINzXSJfKdy9 4fGMUzz7KPcVqu8aHTKxACyjjn2lcMCZxLz+1QpMAw2g8JvalchmGNpXIQI8PRVbTP2A HQqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=iIoIkPaKluKJqIgWti9VqjGZYHDmFr+1zLTMMzDyw48=; b=j4uHCEdqFLEcrDwkoTafi4Jxavx712ciNbDQfEV+ogVOBM+jIXJH7tB/lA4IfsKLby sqhQVRjTHgIzVv+mAGvHz/Il1xAbRZ+Apc7UkdfLb3VRqZDbCfOsnxWxl3RgJdmMLBsS jGclUGanxDKIw8TmYhMhIbJr4Ufsdb/rk4HvDwJ9xTa0LCS0P9gQBJ9Y+aiSv49tDL+x 1EnU7wIaSthN0vNrCKPh0axTsmObtiaQAm0cFKJLTi9wEFZJUbfqG4RE/7o6PYZ8zLCy RUjDhUiM45mHF2J9egdf2rCTGh64DgsaI60MtQJgHXEiNRjNFX0whMxtjRPTSbOmEUnt QWew== X-Gm-Message-State: AJcUukcjuN5RNhy6bFZP2AV2ocdLNpSsIi9xerSpHQZm7sn86m3yLlAD SFY4j3al6imtOLxzc5GgrBnYBW/3 X-Google-Smtp-Source: ALg8bN6Foub1SNH02lK4huOSgrCZOh9JqTe8B7BmG3l3mXzFJLaFzr2DR2z17IznjyMNPPHmYGCLXw== X-Received: by 2002:a0c:d4f9:: with SMTP id y54mr8003096qvh.98.1547087212330; Wed, 09 Jan 2019 18:26:52 -0800 (PST) Received: from localhost.localdomain ([65.216.242.54]) by smtp.gmail.com with ESMTPSA id q17sm43149817qtc.19.2019.01.09.18.26.50 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 09 Jan 2019 18:26:50 -0800 (PST) From: Rodger Combs To: ffmpeg-devel@ffmpeg.org Date: Wed, 9 Jan 2019 21:26:41 -0500 Message-Id: <20190110022641.95676-1-rodger.combs@gmail.com> X-Mailer: git-send-email 2.19.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] lavc: vt_hevc: fix crash if vps_list[0] or sps_list[0] are null X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Instead of assuming id 0 is used, use the same logic as used for PPS, where all available entries in the list are emitted. --- libavcodec/videotoolbox.c | 86 ++++++++++++++++++--------------------- 1 file changed, 40 insertions(+), 46 deletions(-) diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c index da7236f100..fb3501f413 100644 --- a/libavcodec/videotoolbox.c +++ b/libavcodec/videotoolbox.c @@ -176,26 +176,31 @@ CFDataRef ff_videotoolbox_avcc_extradata_create(AVCodecContext *avctx) CFDataRef ff_videotoolbox_hvcc_extradata_create(AVCodecContext *avctx) { HEVCContext *h = avctx->priv_data; - const HEVCVPS *vps = (const HEVCVPS *)h->ps.vps_list[0]->data; - const HEVCSPS *sps = (const HEVCSPS *)h->ps.sps_list[0]->data; - int i, num_pps = 0; + int i, num_vps = 0, num_sps = 0, num_pps = 0; + const HEVCVPS *vps = h->ps.vps; + const HEVCSPS *sps = h->ps.sps; const HEVCPPS *pps = h->ps.pps; PTLCommon ptlc = vps->ptl.general_ptl; VUI vui = sps->vui; uint8_t parallelismType; CFDataRef data = NULL; uint8_t *p; - int vt_extradata_size = 23 + 5 + vps->data_size + 5 + sps->data_size + 3; + int vt_extradata_size = 23 + 3 + 3 + 3; uint8_t *vt_extradata; - for (i = 0; i < HEVC_MAX_PPS_COUNT; i++) { - if (h->ps.pps_list[i]) { - const HEVCPPS *pps = (const HEVCPPS *)h->ps.pps_list[i]->data; - vt_extradata_size += 2 + pps->data_size; - num_pps++; - } +#define COUNT_SIZE_PS(T, t) \ + for (i = 0; i < HEVC_MAX_##T##PS_COUNT; i++) { \ + if (h->ps.t##ps_list[i]) { \ + const HEVC##T##PS *lps = (const HEVC##T##PS *)h->ps.t##ps_list[i]->data; \ + vt_extradata_size += 2 + lps->data_size; \ + num_##t##ps++; \ + } \ } + COUNT_SIZE_PS(V, v) + COUNT_SIZE_PS(S, s) + COUNT_SIZE_PS(P, p) + vt_extradata = av_malloc(vt_extradata_size); if (!vt_extradata) return NULL; @@ -286,44 +291,33 @@ CFDataRef ff_videotoolbox_hvcc_extradata_create(AVCodecContext *avctx) AV_W8(p + 22, 3); p += 23; - /* vps */ - /* - * bit(1) array_completeness; - * unsigned int(1) reserved = 0; - * unsigned int(6) NAL_unit_type; - */ - AV_W8(p, 1 << 7 | - HEVC_NAL_VPS & 0x3f); - /* unsigned int(16) numNalus; */ - AV_WB16(p + 1, 1); - /* unsigned int(16) nalUnitLength; */ - AV_WB16(p + 3, vps->data_size); - /* bit(8*nalUnitLength) nalUnit; */ - memcpy(p + 5, vps->data, vps->data_size); - p += 5 + vps->data_size; - - /* sps */ - AV_W8(p, 1 << 7 | - HEVC_NAL_SPS & 0x3f); - AV_WB16(p + 1, 1); - AV_WB16(p + 3, sps->data_size); - memcpy(p + 5, sps->data, sps->data_size); - p += 5 + sps->data_size; - - /* pps */ - AV_W8(p, 1 << 7 | - HEVC_NAL_PPS & 0x3f); - AV_WB16(p + 1, num_pps); - p += 3; - for (i = 0; i < HEVC_MAX_PPS_COUNT; i++) { - if (h->ps.pps_list[i]) { - const HEVCPPS *pps = (const HEVCPPS *)h->ps.pps_list[i]->data; - AV_WB16(p, pps->data_size); - memcpy(p + 2, pps->data, pps->data_size); - p += 2 + pps->data_size; - } + +#define APPEND_PS(T, t) \ + /* \ + * bit(1) array_completeness; \ + * unsigned int(1) reserved = 0; \ + * unsigned int(6) NAL_unit_type; \ + */ \ + AV_W8(p, 1 << 7 | \ + HEVC_NAL_##T##PS & 0x3f); \ + /* unsigned int(16) numNalus; */ \ + AV_WB16(p + 1, num_##t##ps); \ + p += 3; \ + for (i = 0; i < HEVC_MAX_##T##PS_COUNT; i++) { \ + if (h->ps.t##ps_list[i]) { \ + const HEVC##T##PS *lps = (const HEVC##T##PS *)h->ps.t##ps_list[i]->data; \ + /* unsigned int(16) nalUnitLength; */ \ + AV_WB16(p, lps->data_size); \ + /* bit(8*nalUnitLength) nalUnit; */ \ + memcpy(p + 2, lps->data, lps->data_size); \ + p += 2 + lps->data_size; \ + } \ } + APPEND_PS(V, v) + APPEND_PS(S, s) + APPEND_PS(P, p) + av_assert0(p - vt_extradata == vt_extradata_size); data = CFDataCreate(kCFAllocatorDefault, vt_extradata, vt_extradata_size);