From patchwork Sun Apr 25 18:01:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Ribble X-Patchwork-Id: 27363 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:5014:0:0:0:0:0 with SMTP id e20csp3339675iob; Sun, 25 Apr 2021 11:33:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzWGTkbuxya4Dm8rN26LsELbX8YQEmXgajD0gcRekEI3+nRqWs8sOyBlfxzh8r9veQDNRhg X-Received: by 2002:a17:906:1986:: with SMTP id g6mr14613347ejd.533.1619375627739; Sun, 25 Apr 2021 11:33:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619375627; cv=none; d=google.com; s=arc-20160816; b=PIBmAhmkrLht62mLLVeLU0bpYsF4NJKtv0w/g/z3VlGNRJbxOfboWrT3ZsDD42OHUx ZBGsZ204kQgLCYAyi78OZSxwf1du0nHbavdBhi3fjbAZXZX4jfwC1iO5swJuDJQ3sXdV vcrgvhmWC3iCA+jNJIXduKg39ER1BASRAe9afESt8ZtrmVJLYsYf6ywdEeYdjDE2QGso W8EAdlVYPWhsS2nb/zq/o9Vg0XxWxw4x5d8+B2DeTk2/dXe8E/Hvo2g1LW6iTp++1fhz OpAoVrYm/fuYyWL6g+kLDYvxpCsp0btVyLj7UFvINwDPEV0wISFtn1q/ABK7jLV4tZVz nDnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=zhkERMXusKvvuqln/BjlFsLyQAiKlOXfIM0vrNVn+Wg=; b=m93OmYZ4tSL0Vt28i7ZYSx7wtmMn616wDX9/HSyoqTxEMEl8c3HMWG5+zZQkrNEFSp tm0PqvegPu5RGimuyB7b5EN0BFIbQAR02SRJ876afb+PpXp+Edxr4AYKg3KCq6IBQm+q 6NFNKsjkBC7RqTQ2VMIEKCwcwxbLZWpNv521RZdpE3TGTLI6DigMOrsh1/DozTgelLde aVKcWRrdGTcMkru1cyppHrS21WP/ocaUyC9akIinl6uUy9+chuhZJRrs9lHFzUHok4Uj YdHbodE7Ll8Bx73gfpn11uga5wNv5tRhsGc9n8EV0715cOA485NcAgitvyQAm5wlJEHu sNPw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@resi.io header.s=google header.b=ILbZatNw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=resi.io Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hp26si11088382ejc.209.2021.04.25.11.33.46; Sun, 25 Apr 2021 11:33:47 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@resi.io header.s=google header.b=ILbZatNw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=resi.io Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 23B03687F49; Sun, 25 Apr 2021 21:33:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f180.google.com (mail-yb1-f180.google.com [209.85.219.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0FFDA680BE0 for ; Sun, 25 Apr 2021 21:33:35 +0300 (EEST) Received: by mail-yb1-f180.google.com with SMTP id y2so59917524ybq.13 for ; Sun, 25 Apr 2021 11:33:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=resi.io; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=CcmwgLj05LzJk1fM48G29/L57oDxHMrvg/TaWUwEgQA=; b=ILbZatNwg1lFEi+SM3uoPPEMgXFmI76irYwh8wIsc75aw1G0Ftj0B5TsJZLW+Q2chQ 66jTUBh/QXtW4kStVG4XPGutVXputDpgjjNIcMC5d72LewQ+OhSaSQfgaYfItk41iUCN shHZDfd44W7ssA8EcSKIMLtW/BI1hXCrFfywXwcM3I0TeHDjox3BNclDJba9NkNrliVc IZKLqQZG9j0GuB6N4OrtI2IZlCyiwv75llzI2+IkiVo0jSK9KlMpCqmuvt8F+8TYZKAg XHxs7ExAoldSWwnd7rZebtgETEMYuwfeabifrX1QDDpzzplKnCtbkf7Twe2EwRz5lSQm pinA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=CcmwgLj05LzJk1fM48G29/L57oDxHMrvg/TaWUwEgQA=; b=rJ9RUv+amJIpTBTHg2sEBKXGaL2wRyAR8Nj8PQLu64hSJKDvpH+lX/gJi8YqH4HKLW qs/YsHUlTTjWq14nTDIYMnBMxfstBZ4NbmKNaPI4U2D2GkhlTG7dWXFx7uglyhvTQ/zS oHlOPNiaIjIeqS+rU7y5mQFF7I5c+xcH4xRt0VVAGwEl8csr20voUraBlxYI+IJaUG6V UuHMOKsPoyCQCxVvCinIRQGnBAe2B/geUfie61tvbEtYI68tR4mrrPF5PpxKvHBeFKpU jkM7IDg7pOSMNw3ey+rtZ0QLNFQsuhryMFgqjwa8ZODErfyDPk6vq6KwHcqIovZG3/Sq XtVQ== X-Gm-Message-State: AOAM5325GlNbj/UM6iauvKMclBny6TckihPABGIBKKvKpWfdS2JrZ+fa uQBXkDW6Fuo7HN81isFAphYvNLHBsjwjAHvC X-Received: by 2002:a4a:e591:: with SMTP id o17mr10219946oov.38.1619373723692; Sun, 25 Apr 2021 11:02:03 -0700 (PDT) Received: from hoth.ribblenet.woot.myvwan.com ([47.184.170.144]) by smtp.gmail.com with ESMTPSA id j5sm280586otq.10.2021.04.25.11.02.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Apr 2021 11:02:03 -0700 (PDT) From: Chris Ribble To: ffmpeg-devel@ffmpeg.org Date: Sun, 25 Apr 2021 13:01:17 -0500 Message-Id: <20210425180117.13093-1-chris.ribble@resi.io> X-Mailer: git-send-email 2.26.3 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avformat/dashenc: reopen new http session for http_persistent X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Chris Ribble Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ZjrqILwo7Shd Fix ticket: 8693 Based on implementation in hlsenc.c Signed-off-by: Chris Ribble --- libavformat/dashenc.c | 67 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 53 insertions(+), 14 deletions(-) diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c index 81a5c2b452..593c416850 100644 --- a/libavformat/dashenc.c +++ b/libavformat/dashenc.c @@ -122,6 +122,7 @@ typedef struct OutputStream { const char *single_file_name; /* file names selected for this particular stream */ const char *init_seg_name; const char *media_seg_name; + uint8_t *temp_buffer; char codec_str[100]; int written_len; @@ -242,12 +243,13 @@ static int dashenc_io_open(AVFormatContext *s, AVIOContext **pb, char *filename, return err; } -static void dashenc_io_close(AVFormatContext *s, AVIOContext **pb, char *filename) { +static int dashenc_io_close(AVFormatContext *s, AVIOContext **pb, char *filename) { DASHContext *c = s->priv_data; int http_base_proto = filename ? ff_is_http_proto(filename) : 0; + int ret = 0; if (!*pb) - return; + return ret; if (!http_base_proto || !c->http_persistent) { ff_format_io_close(s, pb); @@ -257,8 +259,10 @@ static void dashenc_io_close(AVFormatContext *s, AVIOContext **pb, char *filenam av_assert0(http_url_context); avio_flush(*pb); ffurl_shutdown(http_url_context, AVIO_FLAG_WRITE); + ret = ff_http_get_shutdown_status(http_url_context); #endif } + return ret; } static const char *get_format_str(SegmentType segment_type) { @@ -456,33 +460,38 @@ static void set_codec_str(AVFormatContext *s, AVCodecParameters *par, static int flush_dynbuf(DASHContext *c, OutputStream *os, int *range_length) { - uint8_t *buffer; + AVFormatContext *ctx = os->ctx; - if (!os->ctx->pb) { + if (!ctx->pb) { return AVERROR(EINVAL); } // flush - av_write_frame(os->ctx, NULL); - avio_flush(os->ctx->pb); + av_write_frame(ctx, NULL); + avio_flush(ctx->pb); if (!c->single_file) { // write out to file - *range_length = avio_close_dyn_buf(os->ctx->pb, &buffer); - os->ctx->pb = NULL; + *range_length = avio_close_dyn_buf(ctx->pb, &os->temp_buffer); + ctx->pb = NULL; if (os->out) - avio_write(os->out, buffer + os->written_len, *range_length - os->written_len); + avio_write(os->out, os->temp_buffer + os->written_len, *range_length - os->written_len); os->written_len = 0; - av_free(buffer); // re-open buffer - return avio_open_dyn_buf(&os->ctx->pb); + return avio_open_dyn_buf(&ctx->pb); } else { - *range_length = avio_tell(os->ctx->pb) - os->pos; + *range_length = avio_tell(ctx->pb) - os->pos; return 0; } } +static void reflush_dynbuf(OutputStream *os, int *range_length) +{ + // re-open buffer + avio_write(os->out, os->temp_buffer, *range_length);; +} + static void set_http_options(AVDictionary **options, DASHContext *c) { if (c->method) @@ -599,6 +608,7 @@ static int flush_init_segment(AVFormatContext *s, OutputStream *os) int ret, range_length; ret = flush_dynbuf(c, os, &range_length); + av_freep(&os->temp_buffer); if (ret < 0) return ret; @@ -645,6 +655,7 @@ static void dash_free(AVFormatContext *s) av_freep(&os->single_file_name); av_freep(&os->init_seg_name); av_freep(&os->media_seg_name); + av_freep(&os->temp_buffer); } av_freep(&c->streams); @@ -1250,7 +1261,18 @@ static int write_manifest(AVFormatContext *s, int final) avio_printf(out, "\n"); avio_flush(out); - dashenc_io_close(s, &c->mpd_out, temp_filename); + ret = dashenc_io_close(s, &c->mpd_out, temp_filename); + if (ret < 0) { + av_log(s, AV_LOG_WARNING, "upload manifest failed, will retry with a new http session.\n"); + ff_format_io_close(s, &c->mpd_out); + + set_http_options(&opts, c); + dashenc_io_open(s, &c->mpd_out, temp_filename, &opts); + av_dict_free(&opts); + + dashenc_io_close(s, &c->mpd_out, temp_filename); + return ret; + } if (use_rename) { if ((ret = ff_rename(temp_filename, s->url, s)) < 0) @@ -1955,7 +1977,21 @@ static int dash_flush(AVFormatContext *s, int final, int stream) if (c->single_file) { find_index_range(s, os->full_path, os->pos, &index_length); } else { - dashenc_io_close(s, &os->out, os->temp_path); + ret = dashenc_io_close(s, &os->out, os->temp_path); + if (ret < 0) { + AVDictionary *http_opts = NULL; + + av_log(s, AV_LOG_WARNING, "upload segment failed, will retry with a new http session.\n"); + ff_format_io_close(s, &os->out); + + set_http_options(&http_opts, c); + ret = dashenc_io_open(s, &os->out, os->temp_path, &http_opts); + av_dict_free(&http_opts); + + reflush_dynbuf(os, &range_length); + ret = dashenc_io_close(s, &os->out, os->temp_path); + } + av_freep(&os->temp_buffer); if (use_rename) { ret = ff_rename(os->temp_path, os->full_path, os->ctx); @@ -2023,6 +2059,9 @@ static int dash_flush(AVFormatContext *s, int final, int stream) c->nr_of_streams_flushed = 0; } ret = write_manifest(s, final); + if (ret < 0 && c->http_persistent) { + ret = write_manifest(s, final); + } } return ret; }