From patchwork Sun Jun 18 22:02:51 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marton Balint X-Patchwork-Id: 4025 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.22.4 with SMTP id 4csp643717vsw; Sun, 18 Jun 2017 15:03:10 -0700 (PDT) X-Received: by 10.223.179.65 with SMTP id k1mr6851813wrd.5.1497823390345; Sun, 18 Jun 2017 15:03:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1497823390; cv=none; d=google.com; s=arc-20160816; b=V6tVPqLWNUQkTFGNyifcpGoB9K6liEBBpTau8YFRmShLnAE5t/WZkGcbzFYis5G+DS DTxrdCNpo/U8e5OpVF2buTA4zLBJIHTHEjlBmnHS+B6/jLxWcGqGdqZVbQMR0ltRedY9 TQOYz3Rr0UeoDEcFTg3u3m3rrNMNWTzhmMYS6wGPuj+jJ5b3CSc+3ZF5vCx/GSuGhV21 I5336zBS4kAHfqGDVUX1+tfhGmTNi2wouxkhPP5BahsmPIDxMWFTMpHIx2fU1NzHXk3F 7sKu7Y2+dhhew6X0F14JRut/EXazIR1fAebk1Dh4w4g9pxuoMjHXAsX9PADbUJXQ7MqS 1+Ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to :arc-authentication-results; bh=1Y/ekNjKHTOuxkjJMWy40aFmJjlRKrnW4J0tkL+pt/w=; b=Gvwt5MKtacC5hTwrOZ8cxWu4f+0ldc07MIk8mw+yubbxmdqftyNKJFSALyEJRJqbfn vok7m/S6KkU2pHwO1YM8IlEOXXOEy85NSUzgpc1crB5fQg3gp25ra1ZLY6J9UC9qSqJG 4FCRmm+UbxNXJDsVtZ6ZpxPhZCPEv/DyUT12aQdLs9rSJIgR8JjZAtRaGzJNQHtLzeOn iDLQ2OxUZVbraTGNAtKSrrlwHp/I4hcp4t6eHrrsohOh4F1j+0bEkk3ZoyJernt7qskF QksChGzrmFlUJZOol70ij7ZwTF91j3fyW+knFLpqLkMxYXaccWjQBoniN9xMxwd5MHc9 yZ4Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p68si8326156wrc.206.2017.06.18.15.03.09; Sun, 18 Jun 2017 15:03:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C055368A282; Mon, 19 Jun 2017 01:03:02 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from iq.passwd.hu (iq.passwd.hu [217.27.212.140]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9445868A200 for ; Mon, 19 Jun 2017 01:02:56 +0300 (EEST) Received: from localhost (localhost [127.0.0.1]) by iq.passwd.hu (Postfix) with ESMTP id D4BC8103426; Mon, 19 Jun 2017 00:03:00 +0200 (CEST) X-Virus-Scanned: amavisd-new at passwd.hu Received: from iq.passwd.hu ([127.0.0.1]) by localhost (iq.passwd.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vVbIlWHJeDUQ; Mon, 19 Jun 2017 00:02:59 +0200 (CEST) Received: from bluegene.passwd.hu (localhost [127.0.0.1]) by iq.passwd.hu (Postfix) with ESMTP id 1FEAC103425; Mon, 19 Jun 2017 00:02:59 +0200 (CEST) From: Marton Balint To: ffmpeg-devel@ffmpeg.org Date: Mon, 19 Jun 2017 00:02:51 +0200 Message-Id: <20170618220254.21682-1-cus@passwd.hu> X-Mailer: git-send-email 2.12.3 Subject: [FFmpeg-devel] [PATCH 1/4] avformat/aviobuf: fix flushing write buffers after seeking backward or forward X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Marton Balint MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This patch makes aviobuf work more like traditinal file IO, which is how people think about it. For example, in the past, aviobuf only flushed buffers until the current buffer position, even if more data was written to it previously, and a backward seek was used to reposition the IO context. From now, aviobuf will keep track of the written data, so no explicit seek will be required till the end of the buffer, or till the end of file before flushing. This fixes at least one regression, fate-vsynth3-flv was broken if flush_packets option was set to false, an explicit seek was removed in 4e3cc4bdd8acedbcc703607ed0efbb64bb5c3cc4. Also from now on, if a forward seek in the write buffer were to cause a gap between the already written data and the new file position, a flush will happen. The must_flush varable is also removed, which might have caused needless flushes with multiple seeks whithin the write buffer. Since we know the amount of data written to it, we will know when to flush. Signed-off-by: Marton Balint --- libavformat/avio.h | 42 ++++++++++++++++++++++++++---------------- libavformat/aviobuf.c | 26 ++++++++++++++------------ libavformat/version.h | 2 +- 3 files changed, 41 insertions(+), 29 deletions(-) diff --git a/libavformat/avio.h b/libavformat/avio.h index 525eb7129e..844a5723d3 100644 --- a/libavformat/avio.h +++ b/libavformat/avio.h @@ -168,8 +168,9 @@ typedef struct AVIOContext { const AVClass *av_class; /* - * The following shows the relationship between buffer, buf_ptr, buf_end, buf_size, - * and pos, when reading and when writing (since AVIOContext is used for both): + * The following shows the relationship between buffer, buf_ptr, + * buf_ptr_max, buf_end, buf_size, and pos, when reading and when writing + * (since AVIOContext is used for both): * ********************************************************************************** * READING @@ -196,21 +197,24 @@ typedef struct AVIOContext { * WRITING ********************************************************************************** * - * | buffer_size | - * |-------------------------------| - * | | + * | buffer_size | + * |--------------------------------------| + * | | * - * buffer buf_ptr buf_end - * +-------------------+-----------+ - * |/ / / / / / / / / /| | - * write buffer: | / to be flushed / | | - * |/ / / / / / / / / /| | - * +-------------------+-----------+ + * buf_ptr_max + * buffer (buf_ptr) buf_end + * +-----------------------+--------------+ + * |/ / / / / / / / / / / /| | + * write buffer: | / / to be flushed / / | | + * |/ / / / / / / / / / / /| | + * +-----------------------+--------------+ + * buf_ptr can be in this + * due to a backward seek * - * pos - * +--------------------------+-----------------------------------+ - * output file: | | | - * +--------------------------+-----------------------------------+ + * pos + * +-------------+----------------------------------------------+ + * output file: | | | + * +-------------+----------------------------------------------+ * */ unsigned char *buffer; /**< Start of the buffer. */ @@ -226,7 +230,7 @@ typedef struct AVIOContext { int (*write_packet)(void *opaque, uint8_t *buf, int buf_size); int64_t (*seek)(void *opaque, int64_t offset, int whence); int64_t pos; /**< position in the file of the current buffer */ - int must_flush; /**< true if the next seek should flush */ + int must_flush; /**< unused */ int eof_reached; /**< true if eof reached */ int write_flag; /**< true if open for writing */ int max_packet_size; @@ -329,6 +333,12 @@ typedef struct AVIOContext { int (*short_seek_get)(void *opaque); int64_t written; + + /** + * Maximum reached position before a backward seek in the write buffer, + * used keeping track of already written data for a later flush. + */ + unsigned char *buf_ptr_max; } AVIOContext; /** diff --git a/libavformat/aviobuf.c b/libavformat/aviobuf.c index 1667e9f08b..dcb91570d8 100644 --- a/libavformat/aviobuf.c +++ b/libavformat/aviobuf.c @@ -91,6 +91,7 @@ int ffio_init_context(AVIOContext *s, s->orig_buffer_size = s->buffer_size = buffer_size; s->buf_ptr = buffer; + s->buf_ptr_max = buffer; s->opaque = opaque; s->direct = 0; @@ -100,7 +101,6 @@ int ffio_init_context(AVIOContext *s, s->read_packet = read_packet; s->seek = seek; s->pos = 0; - s->must_flush = 0; s->eof_reached = 0; s->error = 0; s->seekable = seek ? AVIO_SEEKABLE_NORMAL : 0; @@ -171,15 +171,16 @@ static void writeout(AVIOContext *s, const uint8_t *data, int len) static void flush_buffer(AVIOContext *s) { - if (s->write_flag && s->buf_ptr > s->buffer) { - writeout(s, s->buffer, s->buf_ptr - s->buffer); + s->buf_ptr_max = FFMAX(s->buf_ptr, s->buf_ptr_max); + if (s->write_flag && s->buf_ptr_max > s->buffer) { + writeout(s, s->buffer, s->buf_ptr_max - s->buffer); if (s->update_checksum) { s->checksum = s->update_checksum(s->checksum, s->checksum_ptr, - s->buf_ptr - s->checksum_ptr); + s->buf_ptr_max - s->checksum_ptr); s->checksum_ptr = s->buffer; } } - s->buf_ptr = s->buffer; + s->buf_ptr = s->buf_ptr_max = s->buffer; if (!s->write_flag) s->buf_end = s->buffer; } @@ -228,8 +229,10 @@ void avio_write(AVIOContext *s, const unsigned char *buf, int size) void avio_flush(AVIOContext *s) { + int seekback = s->write_flag ? FFMIN(0, s->buf_ptr - s->buf_ptr_max) : 0; flush_buffer(s); - s->must_flush = 0; + if (seekback) + avio_seek(s, seekback, SEEK_CUR); } int64_t avio_seek(AVIOContext *s, int64_t offset, int whence) @@ -269,8 +272,9 @@ int64_t avio_seek(AVIOContext *s, int64_t offset, int whence) short_seek = s->short_seek_threshold; offset1 = offset - pos; // "offset1" is the relative offset from the beginning of s->buffer - if (!s->must_flush && (!s->direct || !s->seek) && - offset1 >= 0 && offset1 <= buffer_size - s->write_flag) { + s->buf_ptr_max = FFMAX(s->buf_ptr_max, s->buf_ptr); + if ((!s->direct || !s->seek) && + offset1 >= 0 && offset1 <= (s->write_flag ? s->buf_ptr_max - s->buffer : buffer_size)) { /* can do the seek inside the buffer */ s->buf_ptr = s->buffer + offset1; } else if ((!(s->seekable & AVIO_SEEKABLE_NORMAL) || @@ -299,7 +303,6 @@ int64_t avio_seek(AVIOContext *s, int64_t offset, int whence) int64_t res; if (s->write_flag) { flush_buffer(s); - s->must_flush = 1; } if (!s->seek) return AVERROR(EPIPE); @@ -308,7 +311,7 @@ int64_t avio_seek(AVIOContext *s, int64_t offset, int whence) s->seek_count ++; if (!s->write_flag) s->buf_end = s->buffer; - s->buf_ptr = s->buffer; + s->buf_ptr = s->buf_ptr_max = s->buffer; s->pos = offset; } s->eof_reached = 0; @@ -994,7 +997,7 @@ int ffio_set_buf_size(AVIOContext *s, int buf_size) s->buffer = buffer; s->orig_buffer_size = s->buffer_size = buf_size; - s->buf_ptr = buffer; + s->buf_ptr = s->buf_ptr_max = buffer; url_resetbuf(s, s->write_flag ? AVIO_FLAG_WRITE : AVIO_FLAG_READ); return 0; } @@ -1052,7 +1055,6 @@ int ffio_rewind_with_probe_data(AVIOContext *s, unsigned char **bufp, int buf_si s->pos = buf_size; s->buf_end = s->buf_ptr + buf_size; s->eof_reached = 0; - s->must_flush = 0; return 0; } diff --git a/libavformat/version.h b/libavformat/version.h index b00a8b0333..1fb8ffb2b9 100644 --- a/libavformat/version.h +++ b/libavformat/version.h @@ -32,7 +32,7 @@ // Major bumping may affect Ticket5467, 5421, 5451(compatibility with Chromium) // Also please add any ticket numbers that you believe might be affected here #define LIBAVFORMAT_VERSION_MAJOR 57 -#define LIBAVFORMAT_VERSION_MINOR 73 +#define LIBAVFORMAT_VERSION_MINOR 74 #define LIBAVFORMAT_VERSION_MICRO 100 #define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \