From patchwork Sat Jan 7 17:57:13 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tobias Stoeckmann X-Patchwork-Id: 2097 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.89.21 with SMTP id n21csp6483263vsb; Sat, 7 Jan 2017 16:39:05 -0800 (PST) X-Received: by 10.28.169.135 with SMTP id s129mr1678751wme.24.1483835945727; Sat, 07 Jan 2017 16:39:05 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q18si4495791wrc.324.2017.01.07.16.39.05; Sat, 07 Jan 2017 16:39:05 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 32D6C68A58F; Sun, 8 Jan 2017 02:38:56 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.17.13]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 711EE68A481 for ; Sat, 7 Jan 2017 19:56:13 +0200 (EET) Received: from localhost ([79.234.34.66]) by mrelayeu.kundenserver.de (mreue102 [212.227.15.145]) with ESMTPSA (Nemesis) id 0MGAUR-1cE58W0B0B-00F9xe for ; Sat, 07 Jan 2017 18:56:19 +0100 Date: Sat, 7 Jan 2017 18:57:13 +0100 From: Tobias Stoeckmann To: ffmpeg-devel@ffmpeg.org Message-ID: <20170107175713.GA32549@localhost> MIME-Version: 1.0 Content-Disposition: inline X-Provags-ID: V03:K0:G8LpgoB4OH/R77hEfkVBgiV2AGPpBfOdbO/fGt9nHAF4ZJzlK9a 3aEZkHlLYF572XM9NoHO4UNudesZ7MeOoFEugsf47U2JWNT0utnVi6AtxjlvURGAawoVat/ +WOLMWh1O1ZWOR749nzaG+mVrtyC6bvoTbDKHdgl5j4NpOvdJLQCmBTBg3cpLX2UHC1miZl 2QrsOXbNwxEQQ4uE3FOYA== X-UI-Out-Filterresults: notjunk:1; V01:K0:S/w8O+O2LoY=:QF7dprPvffCbfmyO0YnKUv w+UMBkG69FcSHOfvDymQ+jDkaMkrqhRRcsFb5xhB/vnUspyrtIFnxWJiZWm52kMX+wB7lb19E 63Kj35+ENBCC1dA9gdktftVEiMWdmbCUlZhC+RpITKnd7ZoNdovq9zhf3VG8+Diae+85sRhbo +53r+83mdZyg49/NR9Q/0W9F1FZYPlqUVwpPgajCy7HGgPILldTi54u8bsO/zqrk35Cqk3vwc 7jpZM93pKjTabtPEE32j3/I3HJ+dpIFwQKatSyarQ5itk418/AqdzvjircmPt/1Lk/shGDN7Y t2//Cfifb1oV9FM+r7u/Y63of7Ep2ApN//Mte/8TedgZ+g+jTLUg1loqJvexWL20qIatr08Yw Q9uriCz0MJoyqxx4JKbz1OWMGiMKjPg/xuo4ctxBZKAaLvTuXOtAMH08/VEGg0utFCgZLh/9H +mc5RYsJZmraDW4dLn04uGTT1YNC26BgcpOWD9jkZl+G/lnzy0OUrjNzpGvYR8MZVZU+ts6Nw b7obu17HWCDXax4Ne7pGeObSutSsGXDitF5fCJgflMbDe9aHHK3PIuMnTUUlOgHIjlItNPvne +sEBbixeWydibABnwSrdyht80glvgmt6kKay7dyeZwzzaRdaMwxv0K8x1SS2YGNDiVN2kizqR aJWr3bhShjoAoDgkgXUQYCgb7QG49QQmvgCmJDLdeXRseiwnjc8wzaSEiwK9XTXW+yBg= X-Mailman-Approved-At: Sun, 08 Jan 2017 02:38:54 +0200 Subject: [FFmpeg-devel] [PATCH] Never unmap unallocated space X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The function av_file_map can use mmap to memory-map the content of a file into address space. Just like every other alternative, mmap() could fail due to various reasons, e.g. if not enough address space is available. Unfortunately, av_file_map writes the requested size even on error into the supplied size pointer, which will eventually passed to munmap. As the base pointer will be NULL, the first 'size' bytes of the address space will be unmapped, which will most likely hit a lot of libraries. If that happens, a segmentation fault is most likely to occur. This example can trigger the issue on 32 bit systems. Adjust the seek value if necessary (free memory < seek_size < physical RAM installed): $ uname -m i686 $ dd if=/dev/zero of=large bs=1 count=1 seek=3000000000 $ ffmpeg -f lavfi -i life=filename=large Error occurred in mmap(): Cannot allocate memory Error initializing filter 'life' with args 'filename=large' Segmentation fault $ _ --- I chose to set *size back to 0 in error-cases just because each error case already handles close(fd), too. Adding a new variable and setting *size at the end would have introduced too much noise in this diff. Feel free to adjust. :) --- libavutil/file.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/libavutil/file.c b/libavutil/file.c index 7bdf6cde84..8e12efe3ce 100644 --- a/libavutil/file.c +++ b/libavutil/file.c @@ -87,6 +87,7 @@ int av_file_map(const char *filename, uint8_t **bufptr, size_t *size, err = AVERROR(errno); av_strerror(err, errbuf, sizeof(errbuf)); av_log(&file_log_ctx, AV_LOG_ERROR, "Error occurred in mmap(): %s\n", errbuf); + *size = 0; close(fd); return err; } @@ -98,6 +99,7 @@ int av_file_map(const char *filename, uint8_t **bufptr, size_t *size, mh = CreateFileMapping(fh, NULL, PAGE_READONLY, 0, 0, NULL); if (!mh) { av_log(&file_log_ctx, AV_LOG_ERROR, "Error occurred in CreateFileMapping()\n"); + *size = 0; close(fd); return -1; } @@ -106,6 +108,7 @@ int av_file_map(const char *filename, uint8_t **bufptr, size_t *size, CloseHandle(mh); if (!ptr) { av_log(&file_log_ctx, AV_LOG_ERROR, "Error occurred in MapViewOfFile()\n"); + *size = 0; close(fd); return -1; } @@ -116,6 +119,7 @@ int av_file_map(const char *filename, uint8_t **bufptr, size_t *size, *bufptr = av_malloc(*size); if (!*bufptr) { av_log(&file_log_ctx, AV_LOG_ERROR, "Memory allocation error occurred\n"); + *size = 0; close(fd); return AVERROR(ENOMEM); }