From patchwork Wed Jan 22 19:27:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 17473 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 9533944B208 for ; Wed, 22 Jan 2020 21:27:33 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 78C7368B3A0; Wed, 22 Jan 2020 21:27:33 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 556A7689CCC for ; Wed, 22 Jan 2020 21:27:27 +0200 (EET) Received: by mail-wr1-f65.google.com with SMTP id y17so341371wrh.5 for ; Wed, 22 Jan 2020 11:27:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IN9b1BM0VujMAjjpiVBHDVcZ2ZK28RucpCN8F3QVo48=; b=A72WqumJ1m2Plleqz0IXj8ipvOWcg3ftct78eVhnIN936QAmHj4gzR9xGh+Uf7WwK/ lfjWhSFi5YsJmOlS6hE/i+8N8i6GD7HLI2cLPux7eDf09xd0GhCkWVqH3dVpc6VD51sD V88OJTi2tdz6sPYoX0EA/u1R1fzK9tCPsRHEf0TByDi8SfvA/QWmFhTzqTUefl2pfjyk QT9slHwyNtLY1oiGaPaxO5ZWMlrViPAg3+7oHgPVNn+Dse0WeENjfnaJ8b4s63x8xBLy zV4Dxfgmb2ClbizW/EgYvWeOwlIJeys0rdBakL9jTbTsUqYly1+L2Evwmts7fwNVESZf X9LA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IN9b1BM0VujMAjjpiVBHDVcZ2ZK28RucpCN8F3QVo48=; b=fmi53puyMf8kBBwY/PM1FBkHa42eWSxphmsklSqTQPCXAhiz98gK7MzYFCBpHjSPrt m4UJEd8aSHKuJwf0AsVp5p/u2u88wNGpj4n19I1GPVNzgxic+ntdDOztG0sHk7WtnGZ4 88oHA/Mm/E7BqOhc1bkOPugtzmvvmR0ias2n6AfWCfOAnzB7pGQwoPlHsmMGBXz4cIly nbM8O/ixBTyv+WfGxQwOrI/KkHFWnd4ag6r/6kDhg13sJ1X6SuwLmhrvSVqMdWN/KlLS HdcA+/dZHbHB8HK+zewI+q1MCCMvtNG6neqhadaCnT7adoLw3jQuAqhrBfiOIGt7Jdxs +HlQ== X-Gm-Message-State: APjAAAVZjQQ7sYVWnq2DSBpucKY8iIRkmaFFMFosl2uuVnL6iaa/L7kJ QyQx6eoC/+AxU965sLPbkaUBKODf X-Google-Smtp-Source: APXvYqwuAcrX4HtgEkwi8Ct/FY/VDl/9jErjryQehOenEW74fxvolqKDctoG6XLmDmCmVl/RVogTVQ== X-Received: by 2002:adf:dd4d:: with SMTP id u13mr13601757wrm.394.1579721246510; Wed, 22 Jan 2020 11:27:26 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc08bbf.dynamic.kabel-deutschland.de. [188.192.139.191]) by smtp.gmail.com with ESMTPSA id v3sm58196792wru.32.2020.01.22.11.27.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2020 11:27:25 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 Jan 2020 20:27:15 +0100 Message-Id: <20200122192717.11678-1-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200101005837.11356-1-andreas.rheinhardt@gmail.com> References: <20200101005837.11356-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 26/28] avformat/matroskaenc: Stop reallocating of Cluster buffer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The Matroska muxer uses a dynamic buffer to buffer the content of Clusters before eventually writing them. Up until now, each time a Cluster was written, the dynamic buffer was closed, i.e. freed; now it is only reset, saving allocations of the AVIOContext itself, its opaque as well as most of the reallocations of the buffer. This is advantageous performance-wise, in particular on systems where reallocations are slow (namely Windows). The following table shows the decicyles for writing a frame on Linux (Ubuntu 19.10) and Windows (7) on an x64 Haswell (to /dev/null on Linux, to stdout which is discarded on Windows (the default values of the size and duration of clusters for seekable output have been explicitly set in this case); in all tests, writing CRC-32 values has been disabled in all tests; calls to the muxer's write_packet function in write_packet() in libavformat/mux.c have been timed; each of the following tests has been repeated 50 times): | Windows before | Windows after | Linux before | Linux after diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 5065b2adbd..8bbacf5cd3 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -337,7 +337,7 @@ static int start_ebml_master_crc32(AVIOContext **dyn_cp, MatroskaMuxContext *mkv { int ret; - if ((ret = avio_open_dyn_buf(dyn_cp)) < 0) + if (!*dyn_cp && (ret = avio_open_dyn_buf(dyn_cp)) < 0) return ret; if (mkv->write_crc) @@ -348,13 +348,13 @@ static int start_ebml_master_crc32(AVIOContext **dyn_cp, MatroskaMuxContext *mkv static void end_ebml_master_crc32(AVIOContext *pb, AVIOContext **dyn_cp, MatroskaMuxContext *mkv, uint32_t id, - int length_size) + int length_size, int keep_buffer) { uint8_t *buf, crc[4]; int size, skip = 0; put_ebml_id(pb, id); - size = avio_close_dyn_buf(*dyn_cp, &buf); + size = avio_get_dyn_buf(*dyn_cp, &buf); put_ebml_num(pb, size, length_size); if (mkv->write_crc) { skip = 6; /* Skip reserved 6-byte long void element from the dynamic buffer. */ @@ -363,8 +363,11 @@ static void end_ebml_master_crc32(AVIOContext *pb, AVIOContext **dyn_cp, } avio_write(pb, buf + skip, size - skip); - av_free(buf); - *dyn_cp = NULL; + if (keep_buffer) { + ffio_reset_dyn_buf(*dyn_cp); + } else { + ffio_free_dyn_buf(dyn_cp); + } } /** @@ -449,7 +452,7 @@ static void mkv_add_seekhead_entry(MatroskaMuxContext *mkv, uint32_t elementid, static int mkv_write_seekhead(AVIOContext *pb, MatroskaMuxContext *mkv, int error_on_seek_failure, int64_t destpos) { - AVIOContext *dyn_cp; + AVIOContext *dyn_cp = NULL; mkv_seekhead *seekhead = &mkv->seekhead; int64_t remaining, ret64; int i, ret; @@ -476,7 +479,7 @@ static int mkv_write_seekhead(AVIOContext *pb, MatroskaMuxContext *mkv, put_ebml_uint(dyn_cp, MATROSKA_ID_SEEKPOSITION, entry->segmentpos); end_ebml_master(dyn_cp, seekentry); } - end_ebml_master_crc32(pb, &dyn_cp, mkv, MATROSKA_ID_SEEKHEAD, 0); + end_ebml_master_crc32(pb, &dyn_cp, mkv, MATROSKA_ID_SEEKHEAD, 0, 0); remaining = seekhead->filepos + seekhead->reserved_size - avio_tell(pb); put_ebml_void(pb, remaining); @@ -1416,7 +1419,7 @@ static int mkv_write_tracks(AVFormatContext *s) end_ebml_master_crc32_preliminary(pb, mkv->tracks_bc, MATROSKA_ID_TRACKS, &mkv->tracks_pos); else - end_ebml_master_crc32(pb, &mkv->tracks_bc, mkv, MATROSKA_ID_TRACKS, 0); + end_ebml_master_crc32(pb, &mkv->tracks_bc, mkv, MATROSKA_ID_TRACKS, 0, 0); return 0; } @@ -1424,7 +1427,7 @@ static int mkv_write_tracks(AVFormatContext *s) static int mkv_write_chapters(AVFormatContext *s) { MatroskaMuxContext *mkv = s->priv_data; - AVIOContext *dyn_cp, *pb = s->pb; + AVIOContext *dyn_cp = NULL, *pb = s->pb; ebml_master editionentry; AVRational scale = {1, 1E9}; int i, ret; @@ -1472,7 +1475,7 @@ static int mkv_write_chapters(AVFormatContext *s) end_ebml_master(dyn_cp, chapteratom); } end_ebml_master(dyn_cp, editionentry); - end_ebml_master_crc32(pb, &dyn_cp, mkv, MATROSKA_ID_CHAPTERS, 0); + end_ebml_master_crc32(pb, &dyn_cp, mkv, MATROSKA_ID_CHAPTERS, 0, 0); mkv->wrote_chapters = 1; return 0; @@ -1671,7 +1674,7 @@ static int mkv_write_tags(AVFormatContext *s) end_ebml_master_crc32_preliminary(s->pb, mkv->tags_bc, MATROSKA_ID_TAGS, &mkv->tags_pos); else - end_ebml_master_crc32(s->pb, &mkv->tags_bc, mkv, MATROSKA_ID_TAGS, 0); + end_ebml_master_crc32(s->pb, &mkv->tags_bc, mkv, MATROSKA_ID_TAGS, 0, 0); } return 0; } @@ -1679,7 +1682,7 @@ static int mkv_write_tags(AVFormatContext *s) static int mkv_write_attachments(AVFormatContext *s) { MatroskaMuxContext *mkv = s->priv_data; - AVIOContext *dyn_cp, *pb = s->pb; + AVIOContext *dyn_cp = NULL, *pb = s->pb; AVLFG c; int i, ret; @@ -1767,7 +1770,7 @@ static int mkv_write_attachments(AVFormatContext *s) mkv->attachments->entries[mkv->attachments->num_entries].stream_idx = i; mkv->attachments->entries[mkv->attachments->num_entries++].fileuid = fileuid; } - end_ebml_master_crc32(pb, &dyn_cp, mkv, MATROSKA_ID_ATTACHMENTS, 0); + end_ebml_master_crc32(pb, &dyn_cp, mkv, MATROSKA_ID_ATTACHMENTS, 0, 0); return 0; } @@ -1908,7 +1911,7 @@ static int mkv_write_header(AVFormatContext *s) end_ebml_master_crc32_preliminary(s->pb, mkv->info_bc, MATROSKA_ID_INFO, &mkv->info_pos); else - end_ebml_master_crc32(s->pb, &mkv->info_bc, mkv, MATROSKA_ID_INFO, 0); + end_ebml_master_crc32(s->pb, &mkv->info_bc, mkv, MATROSKA_ID_INFO, 0, 0); pb = s->pb; ret = mkv_write_tracks(s); @@ -2188,7 +2191,7 @@ static void mkv_end_cluster(AVFormatContext *s) { MatroskaMuxContext *mkv = s->priv_data; - end_ebml_master_crc32(s->pb, &mkv->cluster_bc, mkv, MATROSKA_ID_CLUSTER, 0); + end_ebml_master_crc32(s->pb, &mkv->cluster_bc, mkv, MATROSKA_ID_CLUSTER, 0, 1); mkv->cluster_pos = -1; avio_write_marker(s->pb, AV_NOPTS_VALUE, AVIO_DATA_MARKER_FLUSH_POINT); } @@ -2492,7 +2495,8 @@ static int mkv_write_trailer(AVFormatContext *s) } if (mkv->cluster_bc) { - end_ebml_master_crc32(pb, &mkv->cluster_bc, mkv, MATROSKA_ID_CLUSTER, 0); + end_ebml_master_crc32(pb, &mkv->cluster_bc, mkv, + MATROSKA_ID_CLUSTER, 0, 0); } ret = mkv_write_chapters(s); @@ -2506,7 +2510,7 @@ static int mkv_write_trailer(AVFormatContext *s) endpos = avio_tell(pb); if (mkv->cues.num_entries) { - AVIOContext *cues; + AVIOContext *cues = NULL; uint64_t size; int64_t cuespos = endpos; int length_size = 0; @@ -2548,7 +2552,8 @@ static int mkv_write_trailer(AVFormatContext *s) } } mkv_add_seekhead_entry(mkv, MATROSKA_ID_CUES, cuespos); - end_ebml_master_crc32(pb, &cues, mkv, MATROSKA_ID_CUES, length_size); + end_ebml_master_crc32(pb, &cues, mkv, MATROSKA_ID_CUES, + length_size, 0); if (mkv->reserve_cues_space) { if (size < mkv->reserve_cues_space) put_ebml_void(pb, mkv->reserve_cues_space - size); @@ -2564,11 +2569,11 @@ static int mkv_write_trailer(AVFormatContext *s) av_log(s, AV_LOG_DEBUG, "end duration = %" PRIu64 "\n", mkv->duration); avio_seek(mkv->info_bc, mkv->duration_offset, SEEK_SET); put_ebml_float(mkv->info_bc, MATROSKA_ID_DURATION, mkv->duration); - end_ebml_master_crc32(pb, &mkv->info_bc, mkv, MATROSKA_ID_INFO, 0); + end_ebml_master_crc32(pb, &mkv->info_bc, mkv, MATROSKA_ID_INFO, 0, 0); // write tracks master avio_seek(pb, mkv->tracks_pos, SEEK_SET); - end_ebml_master_crc32(pb, &mkv->tracks_bc, mkv, MATROSKA_ID_TRACKS, 0); + end_ebml_master_crc32(pb, &mkv->tracks_bc, mkv, MATROSKA_ID_TRACKS, 0, 0); // update stream durations if (!mkv->is_live) { @@ -2598,7 +2603,7 @@ static int mkv_write_trailer(AVFormatContext *s) } if (mkv->tags_bc && !mkv->is_live) { avio_seek(pb, mkv->tags_pos, SEEK_SET); - end_ebml_master_crc32(pb, &mkv->tags_bc, mkv, MATROSKA_ID_TAGS, 0); + end_ebml_master_crc32(pb, &mkv->tags_bc, mkv, MATROSKA_ID_TAGS, 0, 0); } avio_seek(pb, endpos, SEEK_SET);