From patchwork Thu Jan 5 18:07:27 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Soft Works X-Patchwork-Id: 2067 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.89.21 with SMTP id n21csp5467896vsb; Thu, 5 Jan 2017 10:07:41 -0800 (PST) X-Received: by 10.28.23.16 with SMTP id 16mr67339520wmx.134.1483639661737; Thu, 05 Jan 2017 10:07:41 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h70si82532850wme.114.2017.01.05.10.07.41; Thu, 05 Jan 2017 10:07:41 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E4FC968A129; Thu, 5 Jan 2017 20:07:33 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from COL004-OMC3S8.hotmail.com (col004-omc3s8.hotmail.com [65.55.34.146]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EF3A46899EA for ; Thu, 5 Jan 2017 20:07:26 +0200 (EET) Received: from NAM01-BN3-obe.outbound.protection.outlook.com ([65.55.34.135]) by COL004-OMC3S8.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Thu, 5 Jan 2017 10:07:30 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=P5bKcLreEv04oIBlBEgV36kyUnBnx47wNVExPGU7gZI=; b=rf6m4Zfl7vPbmPN+iApwnOFGAySw6oedb0dlYi7QZABMLPtngKJYTLKEC3m5fI9vfC4aJ4FHKxrVsAofefjY5ULJEOVIjQp0NFZrZlI5y3EyNFvtANKqjDwNsrRxtYVe/nvNz4VBsg/JoGlRIUV7KHCfZY4Aif/EOXCcpEOH7VAFVXqw+ixL904BBbNChoQuYFMreggonfHl/0lCxZJw97OGOxKR3He0XPiln4/umzimYbuoA1SuzLAQ4of7j8JuXktTu4nh6LJtVZVdfL/QBBFqEMbkTMMPW1PS5tkEXCOR3Vr8pMngSekiLiy5poKYQBFyjhwRI6c4fmcZxhoJgQ== Received: from SN1NAM01FT028.eop-nam01.prod.protection.outlook.com (10.152.64.51) by SN1NAM01HT058.eop-nam01.prod.protection.outlook.com (10.152.65.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.803.8; Thu, 5 Jan 2017 18:07:27 +0000 Received: from MWHPR13MB1678.namprd13.prod.outlook.com (10.152.64.57) by SN1NAM01FT028.mail.protection.outlook.com (10.152.65.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.803.8 via Frontend Transport; Thu, 5 Jan 2017 18:07:27 +0000 Received: from MWHPR13MB1678.namprd13.prod.outlook.com ([10.171.144.16]) by MWHPR13MB1678.namprd13.prod.outlook.com ([10.171.144.16]) with mapi id 15.01.0829.007; Thu, 5 Jan 2017 18:07:27 +0000 From: Soft Works To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH 2/2] avformat/matroskaenc: Regression fix for invalid MKV headers Thread-Index: AQHSZ36Mqo4b1MUawkqsl4yPYfQxiw== Date: Thu, 5 Jan 2017 18:07:27 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: authentication-results: ffmpeg.org; dkim=none (message not signed) header.d=none; ffmpeg.org; dmarc=none action=none header.from=hotmail.com; x-incomingtopheadermarker: OriginalChecksum:F9726E451C42562FB5CA92BF6712B2DC283940AA507A71C0C2EC344D917361A9; UpperCasedChecksum:CDC6198F6FC02E4E4B61302C2CDF00E987FAE06C8AA3B5869703D1FC5AE10844; SizeAsReceived:7252; Count:37 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [5tFCMg9VHpjposr/icMtJsjNg7NlhZTO] x-incomingheadercount: 37 x-eopattributedmessage: 0 x-microsoft-exchange-diagnostics: 1; SN1NAM01HT058; 7:IwGic5guotD81kUc1vGSgj8wGiMaU+sfijzHs64UfEQ+q8BUO7WyCAcvH7Hpy3djMU0xV1lVIA0UbBRR7as3W6hEnaUM/ofpn5WncjvR+SmgesvEMCtD6W2T1e0uzC0f6y8Uo9vEO8ENX2NNb16YQKrKlxm0rnILpI1V2WkfCYjmma8MytQkU76OJn7s8c/zQxLDr0dzsybT5DMZTg9HzPeYas0OwLUdIJdQfZpsF9+TxL1sgyKO5qMA5z0oE8D7Tu7mtHrBpB+I3fJ77Wv5Yu5xniutZvwlYSqYwHvNCQIawfksnF8hrpNAjQgHriTj9QiHK8AwNl7PT2Ro7IyIQ0m4aBvqmsjSpb+VaduU4YVaZJobSh5lL3d4rrh/BVAMt3XuqBRjUd7PrLz8PpzuXFCx1LxFL8EnrIERt7xL4GNn74n0XG/sjk+4h104g+JruLlAIWoYeB237TAi7Jithg== x-forefront-antispam-report: EFV:NLI; SFV:NSPM; SFS:(10019020)(98900003); DIR:OUT; SFP:1102; SCL:1; SRVR:SN1NAM01HT058; H:MWHPR13MB1678.namprd13.prod.outlook.com; FPR:; SPF:None; LANG:en; x-ms-office365-filtering-correlation-id: df39922e-9b7a-49eb-064d-08d43595b528 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(1601124038)(5061506293)(5061507293)(1603103113)(1603101340)(1601125047)(1701031013); SRVR:SN1NAM01HT058; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(444111334)(444112120)(432015012)(102415395)(82015046); SRVR:SN1NAM01HT058; BCL:0; PCL:0; RULEID:; SRVR:SN1NAM01HT058; x-forefront-prvs: 0178184651 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Jan 2017 18:07:27.1070 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1NAM01HT058 X-OriginalArrivalTime: 05 Jan 2017 18:07:30.0961 (UTC) FILETIME=[94F24010:01D2677E] Subject: [FFmpeg-devel] [PATCH 2/2] avformat/matroskaenc: Regression fix for invalid MKV headers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The following three commits created a regression by writing initially invalid mkv headers: 650e17d88b63b5aca6e0a43483e89e64b0f7d2dd avformat/matroskaenc: write a CRC32 element on Tags 3bcadf822711720ff0f8d14db71ae47cdf97e652 avformat/matroskaenc: write a CRC32 element on Info ee888cfbe777cd2916a3548c750e433ab8f8e6a5 avformat/matroskaenc: postpone writing the Tracks master Symptoms: - You can no longer playback a file that is still processed by ffmpeg, e.g. VLC fails playback - You can no longer stream a file to a client while if is still being processed - Various diagnosing tools show header errors or incomplete headers (e.g. ffprobe, mediainfo, mkvalidator) Note: The symptoms do not apply to completed files or ffmpeg runs that were interrupted with 'q' Cause: The mentioned commits made changes in a way that some header elements are only partially written in mkv_write_header, leaving the header in an invalid state. Only in mkv_write_trailer, these elements are finished correctly, but that does only occur at the end of the process. Regression: Before these commits were applied, mkv headers have always been valid, even before completion of ffmpeg. This has worked reliably over many versions of ffmpeg, to it was an obvious regression. Bugtracker: This issue has been recorded as #5977 which is resolved by this patch Patch: The patch adds a new function 'end_ebml_master_crc32_preliminary' that preliminarily finishes the ebl element without destroying the buffer. The buffer can be used to update the ebml element later during mkv_write_trailer. But most important: mkv_write_header finishes with a valid mkv header again. --- libavformat/matroskaenc.c | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 827d755..27d83a6 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -367,6 +367,28 @@ static void end_ebml_master_crc32(AVIOContext *pb, AVIOContext **dyn_cp, Matrosk *dyn_cp = NULL; } +/** +* Complete ebml master whithout destroying the buffer, allowing for later updates +*/ +static void end_ebml_master_crc32_preliminary(AVIOContext *pb, AVIOContext **dyn_cp, MatroskaMuxContext *mkv, + ebml_master master) +{ + uint8_t *buf, crc[4]; + int size, skip = 0; + + if (pb->seekable) { + + size = avio_get_dyn_buf(*dyn_cp, &buf); + if (mkv->write_crc && mkv->mode != MODE_WEBM) { + skip = 6; /* Skip reserved 6-byte long void element from the dynamic buffer. */ + AV_WL32(crc, av_crc(av_crc_get_table(AV_CRC_32_IEEE_LE), UINT32_MAX, buf + skip, size - skip) ^ UINT32_MAX); + put_ebml_binary(pb, EBML_ID_CRC32, crc, sizeof(crc)); + } + avio_write(pb, buf + skip, size - skip); + end_ebml_master(pb, master); + } +} + static void put_xiph_size(AVIOContext *pb, int size) { ffio_fill(pb, 255, size / 255); @@ -1309,7 +1331,7 @@ static int mkv_write_tracks(AVFormatContext *s) } if (pb->seekable && !mkv->is_live) - put_ebml_void(pb, avio_tell(mkv->tracks_bc)); + end_ebml_master_crc32_preliminary(pb, &mkv->tracks_bc, mkv, mkv->tracks_master); else end_ebml_master_crc32(pb, &mkv->tracks_bc, mkv, mkv->tracks_master); @@ -1554,7 +1576,7 @@ static int mkv_write_tags(AVFormatContext *s) if (mkv->tags.pos) { if (s->pb->seekable && !mkv->is_live) - put_ebml_void(s->pb, avio_tell(mkv->tags_bc)); + end_ebml_master_crc32_preliminary(s->pb, &mkv->tags_bc, mkv, mkv->tags); else end_ebml_master_crc32(s->pb, &mkv->tags_bc, mkv, mkv->tags); } @@ -1811,7 +1833,7 @@ static int mkv_write_header(AVFormatContext *s) } } if (s->pb->seekable && !mkv->is_live) - put_ebml_void(s->pb, avio_tell(pb)); + end_ebml_master_crc32_preliminary(s->pb, &mkv->info_bc, mkv, mkv->info); else end_ebml_master_crc32(s->pb, &mkv->info_bc, mkv, mkv->info); pb = s->pb;