From patchwork Wed Apr 15 21:06:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 18987 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B8A92447ADC for ; Thu, 16 Apr 2020 00:06:29 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8EF8268BB53; Thu, 16 Apr 2020 00:06:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9762F68B5BB for ; Thu, 16 Apr 2020 00:06:22 +0300 (EEST) Received: by mail-wr1-f67.google.com with SMTP id t14so1862683wrw.12 for ; Wed, 15 Apr 2020 14:06:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=gWrKFzhY6WmaTCH22JSUqDDNhlRft+ihbj01VULcmOI=; b=gILE4fyc8vwUXmBJSB6Rx1ws93xtx5l8GPWps6bZhSD3QPpVrnbuSpLF0gA0TKc375 tfZcXNjVCWzXzRXKaSoyZkSJMgRbDVMFhAik+Bjm0/XHKySDWCdcDWRs4zBC/UzzrlHQ w4p/nZ/Yt/p385fzaMlBcSGitT8jIgWHDr7RN3fennyebcJgr5Z741hAJMna6u1O5Mle hH+qztmjEgzRx/lQMW4FqmxGPMDRzwvAbDWmLRwR1TY5Rq+cMSes8Pk0mheo5LpVwzfx s66o79urldTlRCpTS4CFwRALxn3CmjKGjV7bqBfRJsGDppp40xjRa+uLU7wY4gr141P6 sQPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=gWrKFzhY6WmaTCH22JSUqDDNhlRft+ihbj01VULcmOI=; b=XrtxDoff04kROGMvHeJDckmCrjjukEVR2I7j4VFdj03LUW5Polwk1igff7uK+AWVqd +N/yoR2tXLKJRERWktRg64Niytr0oOYf6hW5CHnVe2/LQWzek/HOA8AewqhoHavx4PJb LsBG/5vHjHmynw9be9QyMq4yXlU+7LnZL8LKaUIcjHhh4mBZZtHPWdfVVKD5xjLxy499 1X/Vh0AQNsHEP1JT75kQHjbOtDle9DNgAsilzRwS/7RE390N8SfbtHc8a6YgauwgTJ2t qvgb0lUfDTkfUYDucTxgpEB8ifsAH8cw/njWYP6dzYOfXx2rDfP+EtqMCm0d6t1ePnPs k+cg== X-Gm-Message-State: AGi0PuZALVrteQ9BBkC3IzaeG7s3x64SznbYg8NGWsznJQ08J3dJWrWw HdxNXpXshNRviDKkOwxlCQe3xymU X-Google-Smtp-Source: APiQypKMoVcCsrm7XIugT7z1CdIqvdHY2rsm1u8HwHfxcZPRdtPnQYn2SGtRT6WqV3+2r6U+5r9CcQ== X-Received: by 2002:a5d:6091:: with SMTP id w17mr27862755wrt.382.1586984781836; Wed, 15 Apr 2020 14:06:21 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc1ab57.dynamic.kabel-deutschland.de. [188.193.171.87]) by smtp.gmail.com with ESMTPSA id f2sm24647605wro.59.2020.04.15.14.06.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Apr 2020 14:06:21 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Wed, 15 Apr 2020 23:06:13 +0200 Message-Id: <20200415210614.29152-1-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] fftools/ffmpeg_opt: Check attachment filesize X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The data of an attachment file is put into an AVCodecParameter's extradata. The corresponding size field has type int, yet there was no check for the size to fit into an int. As a consequence, it was possible to create extradata with negative size (by using a big enough max_alloc). Other errors were also possible: If SIZE_MAX < INT64_MAX (e.g. on 32bit systems) then the file size might be truncated before the allocation; and avio_read() takes an int, too, so one would not have read as much as one desired. Furthermore, the extradata is now padded as is required. Signed-off-by: Andreas Rheinhardt --- fftools/ffmpeg_opt.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 93b3d96205..680f0f1dfb 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -2432,12 +2432,14 @@ loop_end: o->attachments[i]); exit_program(1); } - if (!(attachment = av_malloc(len))) { - av_log(NULL, AV_LOG_FATAL, "Attachment %s too large to fit into memory.\n", + if (len > INT_MAX - AV_INPUT_BUFFER_PADDING_SIZE || + !(attachment = av_malloc(len + AV_INPUT_BUFFER_PADDING_SIZE))) { + av_log(NULL, AV_LOG_FATAL, "Attachment %s too large.\n", o->attachments[i]); exit_program(1); } avio_read(pb, attachment, len); + memset(attachment + len, 0, AV_INPUT_BUFFER_PADDING_SIZE); ost = new_attachment_stream(o, oc, -1); ost->stream_copy = 0; From patchwork Wed Apr 15 21:06:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 18988 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 15D4C447ADC for ; Thu, 16 Apr 2020 00:06:53 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F368068BB5F; Thu, 16 Apr 2020 00:06:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 32A5068B545 for ; Thu, 16 Apr 2020 00:06:47 +0300 (EEST) Received: by mail-wr1-f67.google.com with SMTP id t14so1864897wrw.12 for ; Wed, 15 Apr 2020 14:06:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=I2QSlHRCttsPBp08bca6w3HzJ5F45t0C4stC1f/DFLo=; b=s+Ms5ywLKGNAphi0ox9tYHIwexx+96Idm13cU1ZoIgEvTDXO4H9f2ffczuzZsMlWCJ zODrMIBwV81MZ0ZKbHjCPO+InvlzHXh6FKW5VBXeBXQ8dHdmoM8ZEVhP5OZuF5BCf2vl 6f2unvQia4/BNowtaA2JNz/U2GFyLEGl9z67ARI3d0qu1uLmDw3DiWLDBiafwMKTNA6e 80OhvbaM6ZM9kR5PMTxgOkzmelz76avCs1coGezgOnmha+ctra0gwSiayDbab0Got/8T AoHRrCwBO4QCuiNsaT/oFYpyfXY++N4ke36HUOzvAP0HkBhPQU6DtMXGM26X2JP2gYRK fnyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I2QSlHRCttsPBp08bca6w3HzJ5F45t0C4stC1f/DFLo=; b=RlUY3q7ifbbDPzkcyoEbsDWVSmvEw3zkW4D72sYLOZJkfj1jWs4W7SSq7wW2+Ce+Ky Ji/YgOTlkW+BgZ9aOoaEfyMdQbrNVOEsrT+B3w9CNS96FL1eEBWVfzCXb+UbK1tNqOeg Xbi7/0DKYYnd983vfwGOZ2HRUEmBA+kXEdKLTObhbSufkJ1H+qUS+VK6HFkipY+XnyjN 8dsW7duatLsDByZNOQ2RwTG7UcwviqnIDw5Gvodm/cwbIEL5CbxJu2EaDG/KlJZpEBlX OEu3/+JDKjz97Y38DbQNNN116u66JJ215WwQHwtDo2ooRCkHU4wrasPtcpv9J58eYeuo WaRA== X-Gm-Message-State: AGi0Pua6bMkcUOQbpGWdAHyLRINgXzx4BdwB3CO7H8ATP1Z6JPxt17Op clk2gzKgaVqcAYV8RxZV1cyZR5Un X-Google-Smtp-Source: APiQypI4zb79Wi6FJOkqnOWM9hOFBytOnX4tKCpLW9MqFvUjtJKmo3N7MUHmaFlwpndWYWjAuwijwQ== X-Received: by 2002:a5d:420d:: with SMTP id n13mr31300970wrq.204.1586984806344; Wed, 15 Apr 2020 14:06:46 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc1ab57.dynamic.kabel-deutschland.de. [188.193.171.87]) by smtp.gmail.com with ESMTPSA id f2sm24647605wro.59.2020.04.15.14.06.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Apr 2020 14:06:45 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Wed, 15 Apr 2020 23:06:14 +0200 Message-Id: <20200415210614.29152-2-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200415210614.29152-1-andreas.rheinhardt@gmail.com> References: <20200415210614.29152-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] avformat/matroskaenc: Make ebml_num_size() more robust X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Matroska (or actually EBML) uses variable-length numbers where only seven bits of every byte is usable for the length; the other bits encode the length of the variable-length number. So in order to find out how many bytes one needs to encode a given number one can use a loop like while (num >> 7 * bytes) bytes++; the Matroska muxer effectively did this. Yet it has a disadvantage: It is impossible for the result of a single right shift of an unsigned number with most significant bit set to be zero, because one can only shift by 0..(width - 1). On some architectures like x64 it is not even possible to do it with undefined right shifts in which case this leads to an infinite loop. This can be easily avoided by switching to a loop whose condition is (num >>= 7). The maximum value the so modified function can return is 10; any value > 8 is invalid and will now lead to an assert in put_ebml_num() or in start_ebml_master() (or actually in put_ebml_size_unknown()). Signed-off-by: Andreas Rheinhardt --- One can run into this infinite loop by adding an attachment with (int)size < 0 with ffmpeg on git master. If one applied this here without the previous commit, one would run into a well-deserved assert. libavformat/matroskaenc.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index d3256d8f5d..407920172d 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -194,9 +194,11 @@ static void put_ebml_size_unknown(AVIOContext *pb, int bytes) */ static int ebml_num_size(uint64_t num) { - int bytes = 1; - while ((num + 1) >> bytes * 7) + int bytes = 0; + num++; + do { bytes++; + } while (num >>= 7); return bytes; } From patchwork Thu Apr 16 23:54:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 19014 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E7F8C448684 for ; Fri, 17 Apr 2020 02:54:54 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CA90568BCF5; Fri, 17 Apr 2020 02:54:54 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2338D680055 for ; Fri, 17 Apr 2020 02:54:48 +0300 (EEST) Received: by mail-wr1-f68.google.com with SMTP id k1so893319wrx.4 for ; Thu, 16 Apr 2020 16:54:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5t9KgwuBnUokqdNonpa5jP43onn6io5KdGpOwQpblWU=; b=GuCKfA+2LotSaZbEYIw19mpOeozmAt789/1Bz7GJpFOVYshigKhWY5ymaWp+lBdZsj NVRJRBfzog4BNQdOAifLVaHQlxVclatuBG4mkowMN/aHmB+Azde1DnJeOuXOch9xhMhQ d1J6rHVzFgDuHssAbvuMQnxn2lP0tvPvZea9w3fE3U8+LXS0aCE3lwrNo7EHCN9VOjm4 Narn0vZ3O38QXYuk3uPVOyt//fnfnm1MnsfBuJHa/cUxUYQY4ZLi8Bg0g4Os2QB/eoHU 8IoFC3oDcAuh1mWGATyQWFvR8gWkq4yrkuHPVYMDY/QmNetuWaszscwvyKISJBjieL6B 3LGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5t9KgwuBnUokqdNonpa5jP43onn6io5KdGpOwQpblWU=; b=gqFd7/6t1jJ9w4f7Ve+NgoCoLaNo0kuxaGYJBGpP9Ptzg8gSPM0vTkjnib00Aa4OJU AjQhVmuYtmpNv0F/GpCPPPuRpz9mcLLEH6MMuze0U+SY5UINa6O6kh81ppTFjcH+u6OG fSCZpg5fydmK6Wh0teY403QrU+BlJGqd1JxntNnsCCmKxxjVFxzQHsf92sdrefrS5EgW a4nySFowbCrunuM422qVtT+qw22JtAkQZIC6bDnDixXp5xKzD8BxsW3JTC5T7VaXKTdW kHpEglcMxfHQy7JgjYXG8l3KY4D1vbmefoLmzBH7kI46gC6jNb//NzEmaxyBnKuOuEjV /NCg== X-Gm-Message-State: AGi0PuasujZbyXfJPoZSog+ROK3nHwhvbccpGN80lUswCF0NFuSz/1zj sAb2d4JvbxUw1DTiYZqvwKb1+YAc X-Google-Smtp-Source: APiQypJvwec0tOkAm2yVpbHzmqyefkv2IZKP9/y/XBPkfX4scSL/9IimpEHvH4Hx/iloPDwV8l2LEg== X-Received: by 2002:adf:db0a:: with SMTP id s10mr761779wri.361.1587081286843; Thu, 16 Apr 2020 16:54:46 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc1ab57.dynamic.kabel-deutschland.de. [188.193.171.87]) by smtp.gmail.com with ESMTPSA id i17sm5603488wml.23.2020.04.16.16.54.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Apr 2020 16:54:46 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Apr 2020 01:54:33 +0200 Message-Id: <20200416235435.20409-1-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200415210614.29152-1-andreas.rheinhardt@gmail.com> References: <20200415210614.29152-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/5] avformat/matroskaenc: Rename functions to better reflect what they do X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" EBML uses variable length integers both for the EBML IDs as well as for the EBML lengths; Matroska also uses them for the TrackNumber in (Simple)Blocks and for the lengths of laces when EBML lacing is used. When encoding EBML lengths, certain encodings have a special meaning, namely that the element has an unknown length. This is not so when encoding general EBML variable length integers. Yet the functions called ebml_num_size() and put_ebml_num() had this special meaning hardcoded, i.e. they are there to write EBML lengths and not general EBML numbers. So rename them. Signed-off-by: Andreas Rheinhardt --- libavformat/matroskaenc.c | 53 ++++++++++++++++++++------------------- 1 file changed, 27 insertions(+), 26 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 407920172d..7cea813dc0 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -190,30 +190,31 @@ static void put_ebml_size_unknown(AVIOContext *pb, int bytes) } /** - * Calculate how many bytes are needed to represent a given number in EBML. + * Calculate how many bytes are needed to represent the length field + * of an EBML element whose payload has a given length. */ -static int ebml_num_size(uint64_t num) +static int ebml_length_size(uint64_t length) { int bytes = 0; - num++; + length++; do { bytes++; - } while (num >>= 7); + } while (length >>= 7); return bytes; } /** - * Write a number in EBML variable length format. + * Write a length as EBML variable length integer. * * @param bytes The number of bytes that need to be used to write the number. * If zero, the minimal number of bytes will be used. */ -static void put_ebml_num(AVIOContext *pb, uint64_t num, int bytes) +static void put_ebml_length(AVIOContext *pb, uint64_t length, int bytes) { - int i, needed_bytes = ebml_num_size(num); + int i, needed_bytes = ebml_length_size(length); // sizes larger than this are currently undefined in EBML - av_assert0(num < (1ULL << 56) - 1); + av_assert0(length < (1ULL << 56) - 1); if (bytes == 0) bytes = needed_bytes; @@ -221,9 +222,9 @@ static void put_ebml_num(AVIOContext *pb, uint64_t num, int bytes) // the bytes that we ought to use. av_assert0(bytes >= needed_bytes); - num |= 1ULL << bytes * 7; + length |= 1ULL << bytes * 7; for (i = bytes - 1; i >= 0; i--) - avio_w8(pb, (uint8_t)(num >> i * 8)); + avio_w8(pb, (uint8_t)(length >> i * 8)); } /** @@ -232,7 +233,7 @@ static void put_ebml_num(AVIOContext *pb, uint64_t num, int bytes) static void put_ebml_uid(AVIOContext *pb, uint32_t elementid, uint64_t uid) { put_ebml_id(pb, elementid); - put_ebml_num(pb, 8, 0); + put_ebml_length(pb, 8, 0); avio_wb64(pb, uid); } @@ -244,7 +245,7 @@ static void put_ebml_uint(AVIOContext *pb, uint32_t elementid, uint64_t val) bytes++; put_ebml_id(pb, elementid); - put_ebml_num(pb, bytes, 0); + put_ebml_length(pb, bytes, 0); for (i = bytes - 1; i >= 0; i--) avio_w8(pb, (uint8_t)(val >> i * 8)); } @@ -257,7 +258,7 @@ static void put_ebml_sint(AVIOContext *pb, uint32_t elementid, int64_t val) while (tmp>>=8) bytes++; put_ebml_id(pb, elementid); - put_ebml_num(pb, bytes, 0); + put_ebml_length(pb, bytes, 0); for (i = bytes - 1; i >= 0; i--) avio_w8(pb, (uint8_t)(val >> i * 8)); } @@ -265,7 +266,7 @@ static void put_ebml_sint(AVIOContext *pb, uint32_t elementid, int64_t val) static void put_ebml_float(AVIOContext *pb, uint32_t elementid, double val) { put_ebml_id(pb, elementid); - put_ebml_num(pb, 8, 0); + put_ebml_length(pb, 8, 0); avio_wb64(pb, av_double2int(val)); } @@ -273,7 +274,7 @@ static void put_ebml_binary(AVIOContext *pb, uint32_t elementid, const void *buf, int size) { put_ebml_id(pb, elementid); - put_ebml_num(pb, size, 0); + put_ebml_length(pb, size, 0); avio_write(pb, buf, size); } @@ -299,10 +300,10 @@ static void put_ebml_void(AVIOContext *pb, int size) // size if possible, 1 byte otherwise if (size < 10) { size -= 2; - put_ebml_num(pb, size, 0); + put_ebml_length(pb, size, 0); } else { size -= 9; - put_ebml_num(pb, size, 8); + put_ebml_length(pb, size, 8); } ffio_fill(pb, 0, size); } @@ -310,7 +311,7 @@ static void put_ebml_void(AVIOContext *pb, int size) static ebml_master start_ebml_master(AVIOContext *pb, uint32_t elementid, uint64_t expectedsize) { - int bytes = expectedsize ? ebml_num_size(expectedsize) : 8; + int bytes = expectedsize ? ebml_length_size(expectedsize) : 8; put_ebml_id(pb, elementid); put_ebml_size_unknown(pb, bytes); @@ -323,7 +324,7 @@ static void end_ebml_master(AVIOContext *pb, ebml_master master) if (avio_seek(pb, master.pos - master.sizebytes, SEEK_SET) < 0) return; - put_ebml_num(pb, pos - master.pos, master.sizebytes); + put_ebml_length(pb, pos - master.pos, master.sizebytes); avio_seek(pb, pos, SEEK_SET); } @@ -349,7 +350,7 @@ static void end_ebml_master_crc32(AVIOContext *pb, AVIOContext **dyn_cp, put_ebml_id(pb, id); size = avio_get_dyn_buf(*dyn_cp, &buf); - put_ebml_num(pb, size, length_size); + put_ebml_length(pb, size, length_size); if (mkv->write_crc) { skip = 6; /* Skip reserved 6-byte long void element from the dynamic buffer. */ AV_WL32(crc, av_crc(av_crc_get_table(AV_CRC_32_IEEE_LE), UINT32_MAX, buf + skip, size - skip) ^ UINT32_MAX); @@ -376,7 +377,7 @@ static void end_ebml_master_crc32_preliminary(AVIOContext *pb, AVIOContext *dyn_ *pos = avio_tell(pb); put_ebml_id(pb, id); - put_ebml_num(pb, size, 0); + put_ebml_length(pb, size, 0); avio_write(pb, buf, size); } @@ -461,7 +462,7 @@ static int mkv_write_seekhead(AVIOContext *pb, MatroskaMuxContext *mkv, MAX_SEEKENTRY_SIZE); put_ebml_id(dyn_cp, MATROSKA_ID_SEEKID); - put_ebml_num(dyn_cp, ebml_id_size(entry->elementid), 0); + put_ebml_length(dyn_cp, ebml_id_size(entry->elementid), 0); put_ebml_id(dyn_cp, entry->elementid); put_ebml_uint(dyn_cp, MATROSKA_ID_SEEKPOSITION, entry->segmentpos); @@ -1891,7 +1892,7 @@ static int mkv_write_header(AVFormatContext *s) static int mkv_blockgroup_size(int pkt_size) { int size = pkt_size + 4; - size += ebml_num_size(size); + size += ebml_length_size(size); size += 2; // EBML ID for block and block duration size += 9; // max size of block duration incl. length field return size; @@ -2030,7 +2031,7 @@ static int mkv_write_block(AVFormatContext *s, AVIOContext *pb, } put_ebml_id(pb, blockid); - put_ebml_num(pb, size + 4, 0); + put_ebml_length(pb, size + 4, 0); // this assumes stream_index is less than 126 avio_w8(pb, 0x80 | track_number); avio_wb16(pb, ts - mkv->cluster_pts); @@ -2098,7 +2099,7 @@ static int mkv_write_vtt_blocks(AVFormatContext *s, AVIOContext *pb, AVPacket *p blockgroup = start_ebml_master(pb, MATROSKA_ID_BLOCKGROUP, mkv_blockgroup_size(size)); put_ebml_id(pb, MATROSKA_ID_BLOCK); - put_ebml_num(pb, size + 4, 0); + put_ebml_length(pb, size + 4, 0); avio_w8(pb, 0x80 | track->track_num); // this assumes track_num is less than 126 avio_wb16(pb, ts - mkv->cluster_pts); avio_w8(pb, flags); @@ -2443,7 +2444,7 @@ static int mkv_write_trailer(AVFormatContext *s) if (mkv->reserve_cues_space) { size = avio_tell(cues); - length_size = ebml_num_size(size); + length_size = ebml_length_size(size); size += 4 + length_size; if (mkv->reserve_cues_space < size) { av_log(s, AV_LOG_WARNING, From patchwork Thu Apr 16 23:54:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 19015 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id D35F7448684 for ; Fri, 17 Apr 2020 02:54:55 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BB52D68BCFC; Fri, 17 Apr 2020 02:54:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AB10F680055 for ; Fri, 17 Apr 2020 02:54:48 +0300 (EEST) Received: by mail-wr1-f68.google.com with SMTP id h26so871116wrb.7 for ; Thu, 16 Apr 2020 16:54:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1PID7Y3EY0/OiRXDPf7OvVaKsW5cEfC9iKORHXjBVf0=; b=OyJ82TU2YGGoxn9nCu710fWbZuAprgPWaAeEJtNU0r+AGGDriBlDsIRVQ6JggBE4pu 6/w645sVQx5aTkJRCiHW1xzv112NHKsnYw6pjHNfM32DWyAY5RGsNMumVfcRwwq5KGqs igbv2LA/Yad6wAY07oNapjvCYOMK4ime4BoRrA87DYYSMK5Fn7/bdnFh5FWGiF8XU6to kUsXfSfSBKvDloXKvf7Rhb6oPHfpTkbZ9HoZXVLitii2NinBPHmSwyDWbKOOr4FiCVly R4WNtAsdQugM3Y/amgNh09pkD65g7pKZNdL2lsur80p/SLfyFleajUWuKdGLn9Eh98yK eC2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1PID7Y3EY0/OiRXDPf7OvVaKsW5cEfC9iKORHXjBVf0=; b=HJkKPMzDQwVzMMiJXvilsS64wYr7+8BAmVdVDJuV5HI6iiloYbV05J6uikweqSg6D9 uMq+byKQ2ovUNTH65RvlSDEjr3g2edLYSo3NoeeMYij0sfiDx2RwXC2CmeXocRrEJvlA LgV5FJJX6K+YYSOu+FAAzaXV3EzVjJvBzUtl4aPMwOvXLdbbHEXoovugD1bcdOHJn+0v K5lA1SUqyejs79CFZcqXFYDlE1F590I8A+tDhxxzxj5IhOCDbKJJ7W0gepm0fTuB6Cjg df1R8+IpAv0lJuMKMjjUZE4l/JpJRyZQBxdRoWWVa0xriWjTIb97GhSOzz5EZIHch0hZ Yeaw== X-Gm-Message-State: AGi0PuaSPaW0ZM2+VLTSTLhqmr/BygAat5g0ISVsZx+AyoGrFwcHNLi8 M7dpRJn1syXgh0Xw8QqP0rIzLT5B X-Google-Smtp-Source: APiQypIPoDE7P5WLcd7nSYp3Xq8sPfKJ2pCBLbx0sFFFb/63WumiskuTUWyeBmRZp/vylHj/ExM2Iw== X-Received: by 2002:adf:df01:: with SMTP id y1mr740155wrl.401.1587081287758; Thu, 16 Apr 2020 16:54:47 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc1ab57.dynamic.kabel-deutschland.de. [188.193.171.87]) by smtp.gmail.com with ESMTPSA id i17sm5603488wml.23.2020.04.16.16.54.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Apr 2020 16:54:47 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Apr 2020 01:54:34 +0200 Message-Id: <20200416235435.20409-2-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200415210614.29152-1-andreas.rheinhardt@gmail.com> References: <20200415210614.29152-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/5] avformat/matroskaenc: Refactor writing EBML lengths X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This commit factors the ability to write ordinary EBML numbers out of the functions for writing EBML lengths. This is in preparation for future commits. Signed-off-by: Andreas Rheinhardt --- libavformat/matroskaenc.c | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 7cea813dc0..92cd819312 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -190,19 +190,38 @@ static void put_ebml_size_unknown(AVIOContext *pb, int bytes) } /** - * Calculate how many bytes are needed to represent the length field - * of an EBML element whose payload has a given length. + * Returns how many bytes are needed to represent a number + * as EBML variable length integer. */ -static int ebml_length_size(uint64_t length) +static int ebml_num_size(uint64_t num) { int bytes = 0; - length++; do { bytes++; - } while (length >>= 7); + } while (num >>= 7); return bytes; } +/** + * Calculate how many bytes are needed to represent the length field + * of an EBML element whose payload has a given length. + */ +static int ebml_length_size(uint64_t length) +{ + return ebml_num_size(length + 1); +} + +/** + * Write a number as EBML variable length integer on `bytes` bytes. + * `bytes` is taken literally without checking. + */ +static void put_ebml_num(AVIOContext *pb, uint64_t num, int bytes) +{ + num |= 1ULL << bytes * 7; + for (int i = bytes - 1; i >= 0; i--) + avio_w8(pb, (uint8_t)(num >> i * 8)); +} + /** * Write a length as EBML variable length integer. * @@ -211,7 +230,7 @@ static int ebml_length_size(uint64_t length) */ static void put_ebml_length(AVIOContext *pb, uint64_t length, int bytes) { - int i, needed_bytes = ebml_length_size(length); + int needed_bytes = ebml_length_size(length); // sizes larger than this are currently undefined in EBML av_assert0(length < (1ULL << 56) - 1); @@ -221,10 +240,7 @@ static void put_ebml_length(AVIOContext *pb, uint64_t length, int bytes) // The bytes needed to write the given size must not exceed // the bytes that we ought to use. av_assert0(bytes >= needed_bytes); - - length |= 1ULL << bytes * 7; - for (i = bytes - 1; i >= 0; i--) - avio_w8(pb, (uint8_t)(length >> i * 8)); + put_ebml_num(pb, length, bytes); } /** From patchwork Thu Apr 16 23:54:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 19016 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id A563B448684 for ; Fri, 17 Apr 2020 02:54:57 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 911D068BCFF; Fri, 17 Apr 2020 02:54:57 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C9ACA68BCEA for ; Fri, 17 Apr 2020 02:54:49 +0300 (EEST) Received: by mail-wm1-f67.google.com with SMTP id x4so977393wmj.1 for ; Thu, 16 Apr 2020 16:54:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7iBYA/F2iUVHXLVj5V75jek6mP1QST5h80855jnMQTk=; b=hbrw/P7s/gPopEQTMX+qiMXNx2U7xyJgQUjmQ+aRG76mFUOB2hAkc51QimW7xJd6yO hIPwfofuBEXJIf17ZhJrB5eBxaDzpjjY5qiITDa0RlgYxoPid1dIag0fNZk+iKLLfOus KJ4loErEOOQ1hT+mFMv7QcBxg+JPa359qqe3ivaVnkddBOBVoCKdx6OkgzaIBCwWxcL8 qCa6ZpQdtDgWw6hADEWakbie/Jnmy9Ay9YKP7yZemAHwLR2QSglF+ITcKYkP+UdzaMbr lm3J1SpE7Kpustc2r6ajjv5Ps+4pDaCxYboteFBAl3RE1fyaH7Zup0GKMNHLwr8t/wSB xiVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7iBYA/F2iUVHXLVj5V75jek6mP1QST5h80855jnMQTk=; b=T2nCYuJln7nldRPs9BLLFntlYYJbvmr4lli/cvsthDyN9PCdafiDIZrpuRuwOimJqE 5FAP+17O+s3HfFYzrCyQ6xr5fRucIRfdUvCdqEsTbwwj09tHmHMKwyFDSvsTHPfRZmU6 Vb73Ci632HJp06bbH7mEpsfJh1x8vczbWV8BQBLaU5wOekvVd2pOEyKJgOK+EoU71znc Fydiwo/BQiv35Eo52dSoZB45dTJfyl01nZ9qm4ny8DskuNpkxOYY+f2Th652KWNtUpIu PD28vX+uA+Nc8SWWBlcDVtlTcmH5F8rOFngHNx0Ga7eymVrjksgjOuDfa741B8BaGBC+ K74g== X-Gm-Message-State: AGi0PubQFH/wHsXxMgRuMpTPF4l0i9QwD2OFoYF9tPCDbNoZbYYCmpBx bhmBzMUUtpa9lB8BkoLTvGdmSUeD X-Google-Smtp-Source: APiQypI1737lfSThty5Jaw2mBcdZqffLhN0KqRm0NYSTa0EAcf7yfSQ4YmkWSz4btr+QLIvwl1LHVA== X-Received: by 2002:a7b:cc88:: with SMTP id p8mr378363wma.108.1587081288739; Thu, 16 Apr 2020 16:54:48 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc1ab57.dynamic.kabel-deutschland.de. [188.193.171.87]) by smtp.gmail.com with ESMTPSA id i17sm5603488wml.23.2020.04.16.16.54.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Apr 2020 16:54:48 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Apr 2020 01:54:35 +0200 Message-Id: <20200416235435.20409-3-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200415210614.29152-1-andreas.rheinhardt@gmail.com> References: <20200415210614.29152-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/5] avformat/matroskaenc: Remove limit on the number of tracks X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The Matroska file format has practically no limit on the number of tracks (the current limit is 2^56 - 1); yet because they are encoded in a variable length format in (Simple)Blocks this muxer has simply imposed a limit on the number of tracks in order to ensure that they can always be written on one byte in order to simplify the muxing process. This commit removes said limit. Also, zero is an invalid TrackNumber, so disallow this value in the dash_track_number option. Signed-off-by: Andreas Rheinhardt --- libavformat/matroskaenc.c | 34 +++++++++++++--------------------- 1 file changed, 13 insertions(+), 21 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 92cd819312..fcca4c2e87 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -93,6 +93,7 @@ typedef struct mkv_track { int has_cue; uint64_t uid; unsigned track_num; + int track_num_size; int sample_rate; int64_t sample_rate_offset; int64_t last_timestamp; @@ -105,10 +106,6 @@ typedef struct mkv_track { #define MODE_MATROSKAv2 0x01 #define MODE_WEBM 0x02 -/** Maximum number of tracks allowed in a Matroska file (with track numbers in - * range 1 to 126 (inclusive) */ -#define MAX_TRACKS 126 - typedef struct MatroskaMuxContext { const AVClass *class; int mode; @@ -1905,9 +1902,9 @@ static int mkv_write_header(AVFormatContext *s) return 0; } -static int mkv_blockgroup_size(int pkt_size) +static int mkv_blockgroup_size(int pkt_size, int track_num_size) { - int size = pkt_size + 4; + int size = pkt_size + track_num_size + 3; size += ebml_length_size(size); size += 2; // EBML ID for block and block duration size += 9; // max size of block duration incl. length field @@ -2047,9 +2044,8 @@ static int mkv_write_block(AVFormatContext *s, AVIOContext *pb, } put_ebml_id(pb, blockid); - put_ebml_length(pb, size + 4, 0); - // this assumes stream_index is less than 126 - avio_w8(pb, 0x80 | track_number); + put_ebml_length(pb, size + track->track_num_size + 3, 0); + put_ebml_num(pb, track_number, track->track_num_size); avio_wb16(pb, ts - mkv->cluster_pts); avio_w8(pb, (blockid == MATROSKA_ID_SIMPLEBLOCK && keyframe) ? (1 << 7) : 0); avio_write(pb, data + offset, size); @@ -2112,11 +2108,12 @@ static int mkv_write_vtt_blocks(AVFormatContext *s, AVIOContext *pb, AVPacket *p size, pkt->pts, pkt->dts, pkt->duration, avio_tell(pb), mkv->cluster_pos, track->track_num, 1); - blockgroup = start_ebml_master(pb, MATROSKA_ID_BLOCKGROUP, mkv_blockgroup_size(size)); + blockgroup = start_ebml_master(pb, MATROSKA_ID_BLOCKGROUP, + mkv_blockgroup_size(size, track->track_num_size)); put_ebml_id(pb, MATROSKA_ID_BLOCK); - put_ebml_length(pb, size + 4, 0); - avio_w8(pb, 0x80 | track->track_num); // this assumes track_num is less than 126 + put_ebml_length(pb, size + track->track_num_size + 3, 0); + put_ebml_num(pb, track->track_num, track->track_num_size); avio_wb16(pb, ts - mkv->cluster_pts); avio_w8(pb, flags); avio_printf(pb, "%.*s\n%.*s\n%.*s", id_size, id, settings_size, settings, pkt->size, pkt->data); @@ -2290,7 +2287,8 @@ static int mkv_write_packet_internal(AVFormatContext *s, AVPacket *pkt, int add_ duration = mkv_write_vtt_blocks(s, pb, pkt); } else { ebml_master blockgroup = start_ebml_master(pb, MATROSKA_ID_BLOCKGROUP, - mkv_blockgroup_size(pkt->size)); + mkv_blockgroup_size(pkt->size, + track->track_num_size)); #if FF_API_CONVERGENCE_DURATION FF_DISABLE_DEPRECATION_WARNINGS @@ -2652,13 +2650,7 @@ static int mkv_init(struct AVFormatContext *s) nb_tracks++; track->track_num = mkv->is_dash ? mkv->dash_track_number : nb_tracks; - } - - if (nb_tracks > MAX_TRACKS) { - av_log(s, AV_LOG_ERROR, - "%u > "AV_STRINGIFY(MAX_TRACKS)" tracks (excluding attachments)" - " not supported for muxing in Matroska\n", nb_tracks); - return AVERROR(EINVAL); + track->track_num_size = ebml_num_size(track->track_num); } return 0; @@ -2716,7 +2708,7 @@ static const AVOption options[] = { { "cluster_size_limit", "Store at most the provided amount of bytes in a cluster. ", OFFSET(cluster_size_limit), AV_OPT_TYPE_INT , { .i64 = -1 }, -1, INT_MAX, FLAGS }, { "cluster_time_limit", "Store at most the provided number of milliseconds in a cluster.", OFFSET(cluster_time_limit), AV_OPT_TYPE_INT64, { .i64 = -1 }, -1, INT64_MAX, FLAGS }, { "dash", "Create a WebM file conforming to WebM DASH specification", OFFSET(is_dash), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, - { "dash_track_number", "Track number for the DASH stream", OFFSET(dash_track_number), AV_OPT_TYPE_INT, { .i64 = 1 }, 0, 127, FLAGS }, + { "dash_track_number", "Track number for the DASH stream", OFFSET(dash_track_number), AV_OPT_TYPE_INT, { .i64 = 1 }, 1, INT_MAX, FLAGS }, { "live", "Write files assuming it is a live stream.", OFFSET(is_live), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, { "allow_raw_vfw", "allow RAW VFW mode", OFFSET(allow_raw_vfw), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, { "write_crc32", "write a CRC32 element inside every Level 1 element", OFFSET(write_crc), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, FLAGS },