From patchwork Mon May 4 18:22:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 19501 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 64A4844AD26 for ; Mon, 4 May 2020 21:23:34 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 534C568BE9C; Mon, 4 May 2020 21:23:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E85D168BE9D for ; Mon, 4 May 2020 21:23:25 +0300 (EEST) Received: by mail-wm1-f65.google.com with SMTP id x4so581275wmj.1 for ; Mon, 04 May 2020 11:23:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PXoZVp7lSAQVWqr21HFY3qkO89F8ivsPqLbjRtUYhn0=; b=lL+DCWSEWYSaSbKUkLBjdw3gDM9hNijDKbEwSG7czfa4CXU5zQ0u0pXo27rlzQ4mD/ Fmcy5Iy2iGKfDkb7wsFUp1ZSw2sAYI8p1hOj2kNIkliCP7G39XYRzUyjMbxrviiAXoNQ q1pVcETrDzbLXAY6eY2qt+81E3EYL7Ihy3tnSQ4+VCtEfgPQeQHzIqRnHt5ihHN1HqMJ awK5nbtuKHWRFCfSs7XaSr2EI0xXI/7QfuNKdK0xuXDfSGw8F3S5ZSQVHBnAqITpwZR/ SXLm9eZKX2bTc4L714wUSQ1KH2Lo6mCqarxSSR7rh3M85YAbBLyC9uHZ53xvABUu5bHe R+Yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PXoZVp7lSAQVWqr21HFY3qkO89F8ivsPqLbjRtUYhn0=; b=Q/V0/6ORmXBIs1hhOsdtYhaA0eMWXnSA7F1inOW9ERBxCMGOojrR8XejiXvLgBzB/8 OJaV+v6Z/btQnthy28RdVP7aC5jgeQ/QUu2pZw34ASHYAPtj8f++VhwuxQ7McKKuIQHt 9+W8OMpBLxGEXmIL1CSf+AhYK5xjBPL86z5hVbsvfaHHotv3905PvP/aPdOIZBnpB1PW /ucXA0bPlOZV8+lLm2gR0otpjocHv116P6IJhN292bQJYfdHlEOS39YpqRDoNoAxtJns iQ3fJjFqGpMNkRice4hC9VgFyr3fx/KB6+h6HtbSBoKqyRxeOy+ttbJM/H7xJnEp5k6O J0aA== X-Gm-Message-State: AGi0PuZ+FmeQWNkHidbvUN4oO9B08q05od2kCRr1PesW/3eEn353bOUH q+uoo6stjhfoShaIp9FKdXpIAMeD X-Google-Smtp-Source: APiQypK5JwcBzc70GqwC0mLZbEV4yYddrAv6gGh4+pQELPBV2RQQs/GjZlJQAoP/qfohracvjKJaqQ== X-Received: by 2002:a7b:c1da:: with SMTP id a26mr15214417wmj.125.1588616604752; Mon, 04 May 2020 11:23:24 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc1ab57.dynamic.kabel-deutschland.de. [188.193.171.87]) by smtp.gmail.com with ESMTPSA id u188sm358595wmg.37.2020.05.04.11.23.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 May 2020 11:23:23 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Mon, 4 May 2020 20:22:46 +0200 Message-Id: <20200504182250.26141-6-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200504182250.26141-1-andreas.rheinhardt@gmail.com> References: <20200504182250.26141-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/10] avformat/nutenc: Avoid intermediate dynamic buffers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" NUT uses variable length fields and in order to write the length fields of its various elements on the lowest amount of bytes possible, the NUT muxer uses dynamic buffers to assemble the elements in memory before writing them. Several of these elements had a small prelude containing the amount of subentries of this element and because this amount is also variable-length coded, the muxer used a dynamic buffer for the entries, then wrote the prelude in the dynamic buffer (destined for the whole element) and then copied the content of the dynamic buffer for the entries to the other dynamic buffer. This commit changes this: Given that this prelude is always very small (<= 50 bytes), it is now written into a separate buffer on the stack; put_packet(), the function that actually outputs the data, now receives both this prelude as well as the dynamic buffer containing the other contents and treats it as if it were one buffer. Furthermore, up until now writing a syncpoint also used a dynamic buffer, although its size is always very small (<= 30 bytes). In this case, the dynamic buffer could be completely eliminated: The whole element is treated as a prelude. Signed-off-by: Andreas Rheinhardt --- libavformat/nutenc.c | 149 ++++++++++++++++++------------------------- 1 file changed, 63 insertions(+), 86 deletions(-) diff --git a/libavformat/nutenc.c b/libavformat/nutenc.c index a75d9282fe..404a265597 100644 --- a/libavformat/nutenc.c +++ b/libavformat/nutenc.c @@ -318,6 +318,7 @@ static void put_s##suffix(type dst, int64_t val) \ } PUT_FUNCTIONS(AVIOContext *, avio_w8,) +PUT_FUNCTIONS(uint8_t **, bytestream_put_byte, _buf) /** * Store a string as vb. @@ -331,11 +332,11 @@ static void put_str(AVIOContext *bc, const char *string) } static void put_packet(NUTContext *nut, AVIOContext *bc, AVIOContext *dyn_bc, - uint64_t startcode) + const uint8_t *prelude, int prelude_size, uint64_t startcode) { uint8_t *dyn_buf = NULL; - int dyn_size = avio_get_dyn_buf(dyn_bc, &dyn_buf); - int forw_ptr = dyn_size + 4; + int dyn_size = dyn_bc ? avio_get_dyn_buf(dyn_bc, &dyn_buf) : 0; + unsigned forw_ptr = prelude_size + dyn_size + 4; if (forw_ptr > 4096) ffio_init_checksum(bc, ff_crc04C11DB7_update, 0); @@ -345,10 +346,14 @@ static void put_packet(NUTContext *nut, AVIOContext *bc, AVIOContext *dyn_bc, avio_wl32(bc, ffio_get_checksum(bc)); ffio_init_checksum(bc, ff_crc04C11DB7_update, 0); - avio_write(bc, dyn_buf, dyn_size); + if (prelude_size) + avio_write(bc, prelude, prelude_size); + if (dyn_bc) { + avio_write(bc, dyn_buf, dyn_size); + ffio_reset_dyn_buf(dyn_bc); + } avio_wl32(bc, ffio_get_checksum(bc)); - ffio_reset_dyn_buf(dyn_bc); } static void write_mainheader(NUTContext *nut, AVIOContext *bc) @@ -496,50 +501,41 @@ static int add_info(AVIOContext *bc, const char *type, const char *value) return 1; } -static int write_globalinfo(NUTContext *nut, AVIOContext *bc) +static void write_globalinfo(NUTContext *nut, AVIOContext *bc, + uint8_t prelude[50], int *prelude_size) { AVFormatContext *s = nut->avf; AVDictionaryEntry *t = NULL; - AVIOContext *dyn_bc; - uint8_t *dyn_buf = NULL; - int count = 0, dyn_size; - int ret = avio_open_dyn_buf(&dyn_bc); - if (ret < 0) - return ret; + uint8_t *ptr = prelude; + int count = 0; ff_standardize_creation_time(s); while ((t = av_dict_get(s->metadata, "", t, AV_DICT_IGNORE_SUFFIX))) - count += add_info(dyn_bc, t->key, t->value); + count += add_info(bc, t->key, t->value); - put_v(bc, 0); //stream_if_plus1 - put_v(bc, 0); //chapter_id - put_v(bc, 0); //timestamp_start - put_v(bc, 0); //length + put_v_buf(&ptr, 0); //stream_if_plus1 + put_v_buf(&ptr, 0); //chapter_id + put_v_buf(&ptr, 0); //timestamp_start + put_v_buf(&ptr, 0); //length - put_v(bc, count); - - dyn_size = avio_close_dyn_buf(dyn_bc, &dyn_buf); - avio_write(bc, dyn_buf, dyn_size); - av_free(dyn_buf); - return 0; + put_v_buf(&ptr, count); + *prelude_size = ptr - prelude; } -static int write_streaminfo(NUTContext *nut, AVIOContext *bc, int stream_id) { +static int write_streaminfo(NUTContext *nut, AVIOContext *bc, int stream_id, + uint8_t prelude[50], int *prelude_size) +{ AVFormatContext *s= nut->avf; AVStream* st = s->streams[stream_id]; AVDictionaryEntry *t = NULL; - AVIOContext *dyn_bc; - uint8_t *dyn_buf=NULL; - int count=0, dyn_size, i; - int ret = avio_open_dyn_buf(&dyn_bc); - if (ret < 0) - return ret; + uint8_t *ptr = prelude; + int count = 0, i; while ((t = av_dict_get(st->metadata, "", t, AV_DICT_IGNORE_SUFFIX))) - count += add_info(dyn_bc, t->key, t->value); + count += add_info(bc, t->key, t->value); for (i=0; ff_nut_dispositions[i].flag; ++i) { if (st->disposition & ff_nut_dispositions[i].flag) - count += add_info(dyn_bc, "Disposition", ff_nut_dispositions[i].str); + count += add_info(bc, "Disposition", ff_nut_dispositions[i].str); } if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) { uint8_t buf[256]; @@ -547,51 +543,40 @@ static int write_streaminfo(NUTContext *nut, AVIOContext *bc, int stream_id) { snprintf(buf, sizeof(buf), "%d/%d", st->r_frame_rate.num, st->r_frame_rate.den); else snprintf(buf, sizeof(buf), "%d/%d", st->avg_frame_rate.num, st->avg_frame_rate.den); - count += add_info(dyn_bc, "r_frame_rate", buf); + count += add_info(bc, "r_frame_rate", buf); } - dyn_size = avio_close_dyn_buf(dyn_bc, &dyn_buf); if (count) { - put_v(bc, stream_id + 1); //stream_id_plus1 - put_v(bc, 0); //chapter_id - put_v(bc, 0); //timestamp_start - put_v(bc, 0); //length + put_v_buf(&ptr, stream_id + 1); //stream_id_plus1 + put_v_buf(&ptr, 0); //chapter_id + put_v_buf(&ptr, 0); //timestamp_start + put_v_buf(&ptr, 0); //length - put_v(bc, count); - - avio_write(bc, dyn_buf, dyn_size); + put_v_buf(&ptr, count); } + *prelude_size = ptr - prelude; - av_free(dyn_buf); return count; } -static int write_chapter(NUTContext *nut, AVIOContext *bc, int id) +static void write_chapter(NUTContext *nut, AVIOContext *bc, int id, + uint8_t prelude[50], int *prelude_size) { - AVIOContext *dyn_bc; - uint8_t *dyn_buf = NULL; AVDictionaryEntry *t = NULL; AVChapter *ch = nut->avf->chapters[id]; - int ret, dyn_size, count = 0; - - ret = avio_open_dyn_buf(&dyn_bc); - if (ret < 0) - return ret; + uint8_t *ptr = prelude; + int count = 0; - put_v(bc, 0); // stream_id_plus1 - put_s(bc, id + 1); // chapter_id - put_tt(nut, nut->chapter[id].time_base, bc, ch->start); // chapter_start - put_v(bc, ch->end - ch->start); // chapter_len + put_v_buf(&ptr, 0); // stream_id_plus1 + put_s_buf(&ptr, id + 1); // chapter_id + put_tt_buf(nut, nut->chapter[id].time_base, &ptr, ch->start); // chapter_start + put_v_buf(&ptr, ch->end - ch->start); // chapter_len while ((t = av_dict_get(ch->metadata, "", t, AV_DICT_IGNORE_SUFFIX))) - count += add_info(dyn_bc, t->key, t->value); + count += add_info(bc, t->key, t->value); - put_v(bc, count); - - dyn_size = avio_close_dyn_buf(dyn_bc, &dyn_buf); - avio_write(bc, dyn_buf, dyn_size); - av_freep(&dyn_buf); - return 0; + put_v_buf(&ptr, count); + *prelude_size = ptr - prelude; } static int write_index(NUTContext *nut, AVIOContext *bc) { @@ -650,7 +635,8 @@ static int write_headers(AVFormatContext *avctx, AVIOContext *bc) { NUTContext *nut = avctx->priv_data; AVIOContext *dyn_bc; - int i, ret; + uint8_t prelude[50]; + int i, prelude_size, ret; ff_metadata_conv_ctx(avctx, ff_nut_metadata_conv, NULL); @@ -658,34 +644,28 @@ static int write_headers(AVFormatContext *avctx, AVIOContext *bc) if (ret < 0) return ret; write_mainheader(nut, dyn_bc); - put_packet(nut, bc, dyn_bc, MAIN_STARTCODE); + put_packet(nut, bc, dyn_bc, NULL, 0, MAIN_STARTCODE); for (i = 0; i < nut->avf->nb_streams; i++) { ret = write_streamheader(avctx, dyn_bc, nut->avf->streams[i], i); if (ret < 0) { goto fail; } - put_packet(nut, bc, dyn_bc, STREAM_STARTCODE); + put_packet(nut, bc, dyn_bc, NULL, 0, STREAM_STARTCODE); } - write_globalinfo(nut, dyn_bc); - put_packet(nut, bc, dyn_bc, INFO_STARTCODE); + write_globalinfo(nut, dyn_bc, prelude, &prelude_size); + put_packet(nut, bc, dyn_bc, prelude, prelude_size, INFO_STARTCODE); for (i = 0; i < nut->avf->nb_streams; i++) { - ret = write_streaminfo(nut, dyn_bc, i); + ret = write_streaminfo(nut, dyn_bc, i, prelude, &prelude_size); if (ret > 0) - put_packet(nut, bc, dyn_bc, INFO_STARTCODE); - else if (ret < 0) { - goto fail; - } + put_packet(nut, bc, dyn_bc, prelude, prelude_size, INFO_STARTCODE); } for (i = 0; i < nut->avf->nb_chapters; i++) { - ret = write_chapter(nut, dyn_bc, i); - if (ret < 0) { - goto fail; - } - put_packet(nut, bc, dyn_bc, INFO_STARTCODE); + write_chapter(nut, dyn_bc, i, prelude, &prelude_size); + put_packet(nut, bc, dyn_bc, prelude, prelude_size, INFO_STARTCODE); } nut->last_syncpoint_pos = INT_MIN; @@ -959,7 +939,7 @@ static int nut_write_packet(AVFormatContext *s, AVPacket *pkt) { NUTContext *nut = s->priv_data; StreamContext *nus = &nut->stream[pkt->stream_index]; - AVIOContext *bc = s->pb, *dyn_bc, *sm_bc = NULL; + AVIOContext *bc = s->pb, *sm_bc = NULL; FrameCode *fc; int64_t coded_pts; int best_length, frame_code, flags, needed_flags, i, header_idx; @@ -1007,6 +987,7 @@ static int nut_write_packet(AVFormatContext *s, AVPacket *pkt) if (store_sp && (!(nut->flags & NUT_PIPE) || nut->last_syncpoint_pos == INT_MIN)) { int64_t sp_pos = INT64_MAX; + uint8_t syncpoint[30], *ptr = syncpoint; ff_nut_reset_ts(nut, *nus->time_base, pkt->dts); for (i = 0; i < s->nb_streams; i++) { @@ -1029,18 +1010,14 @@ static int nut_write_packet(AVFormatContext *s, AVPacket *pkt) } nut->last_syncpoint_pos = avio_tell(bc); - ret = avio_open_dyn_buf(&dyn_bc); - if (ret < 0) - goto fail; - put_tt(nut, nus->time_base, dyn_bc, pkt->dts); - put_v(dyn_bc, sp_pos != INT64_MAX ? (nut->last_syncpoint_pos - sp_pos) >> 4 : 0); + put_tt_buf(nut, nus->time_base, &ptr, pkt->dts); + put_v_buf(&ptr, sp_pos != INT64_MAX ? (nut->last_syncpoint_pos - sp_pos) >> 4 : 0); if (nut->flags & NUT_BROADCAST) { - put_tt(nut, nus->time_base, dyn_bc, + put_tt_buf(nut, nus->time_base, &ptr, av_rescale_q(av_gettime(), AV_TIME_BASE_Q, *nus->time_base)); } - put_packet(nut, bc, dyn_bc, SYNCPOINT_STARTCODE); - ffio_free_dyn_buf(&dyn_bc); + put_packet(nut, bc, NULL, syncpoint, ptr - syncpoint, SYNCPOINT_STARTCODE); if (nut->write_index) { if ((ret = ff_nut_add_sp(nut, nut->last_syncpoint_pos, 0 /*unused*/, pkt->dts)) < 0) @@ -1193,7 +1170,7 @@ static int nut_write_trailer(AVFormatContext *s) if (ret >= 0) { av_assert1(nut->write_index); // sp_count should be 0 if no index is going to be written write_index(nut, dyn_bc); - put_packet(nut, bc, dyn_bc, INDEX_STARTCODE); + put_packet(nut, bc, dyn_bc, NULL, 0, INDEX_STARTCODE); ffio_free_dyn_buf(&dyn_bc); }