From patchwork Wed Jan 1 22:56:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marton Balint X-Patchwork-Id: 17123 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B761644AA9A for ; Thu, 2 Jan 2020 00:56:34 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A4B2F68AC57; Thu, 2 Jan 2020 00:56:34 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from iq.passwd.hu (iq.passwd.hu [217.27.212.140]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 02A0B68AC48 for ; Thu, 2 Jan 2020 00:56:27 +0200 (EET) Received: from localhost (localhost [127.0.0.1]) by iq.passwd.hu (Postfix) with ESMTP id D7EE1E3313; Wed, 1 Jan 2020 23:56:27 +0100 (CET) X-Virus-Scanned: amavisd-new at passwd.hu Received: from iq.passwd.hu ([127.0.0.1]) by localhost (iq.passwd.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 46UxEeQUoFeK; Wed, 1 Jan 2020 23:56:26 +0100 (CET) Received: from bluegene.passwd.hu (localhost [127.0.0.1]) by iq.passwd.hu (Postfix) with ESMTP id 79FC5E18BC; Wed, 1 Jan 2020 23:56:26 +0100 (CET) From: Marton Balint To: ffmpeg-devel@ffmpeg.org Date: Wed, 1 Jan 2020 23:56:19 +0100 Message-Id: <20200101225619.4103-2-cus@passwd.hu> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20200101225619.4103-1-cus@passwd.hu> References: <20200101225619.4103-1-cus@passwd.hu> Subject: [FFmpeg-devel] [PATCH 2/2] avdevice/xcbgrab: check if frame size fits in INT_MAX X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Marton Balint MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Also fixes a possible overflow and sets stream bitrate. Signed-off-by: Marton Balint --- libavdevice/xcbgrab.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/libavdevice/xcbgrab.c b/libavdevice/xcbgrab.c index 06b486a536..6f6b2dbf15 100644 --- a/libavdevice/xcbgrab.c +++ b/libavdevice/xcbgrab.c @@ -542,6 +542,7 @@ static int create_stream(AVFormatContext *s) AVStream *st = avformat_new_stream(s, NULL); xcb_get_geometry_cookie_t gc; xcb_get_geometry_reply_t *geo; + int64_t frame_size_bits; int ret; if (!st) @@ -580,17 +581,18 @@ static int create_stream(AVFormatContext *s) c->frame_duration = av_rescale_q(1, c->time_base, AV_TIME_BASE_Q); c->time_frame = av_gettime(); - st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; - st->codecpar->codec_id = AV_CODEC_ID_RAWVIDEO; - st->codecpar->width = c->width; - st->codecpar->height = c->height; - ret = pixfmt_from_pixmap_format(s, geo->depth, &st->codecpar->format, &c->bpp); free(geo); if (ret < 0) return ret; - c->frame_size = c->width * c->height * c->bpp / 8; + frame_size_bits = (int64_t)c->width * c->height * c->bpp; + if (frame_size_bits / 8 + AV_INPUT_BUFFER_PADDING_SIZE > INT_MAX) { + av_log(s, AV_LOG_ERROR, "Captured area is too large\n"); + return AVERROR_PATCHWELCOME; + } + c->frame_size = frame_size_bits / 8; + #if CONFIG_LIBXCB_SHM c->shm_pool = av_buffer_pool_init2(c->frame_size + AV_INPUT_BUFFER_PADDING_SIZE, c->conn, allocate_shm_buffer, NULL); @@ -598,6 +600,12 @@ static int create_stream(AVFormatContext *s) return AVERROR(ENOMEM); #endif + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_RAWVIDEO; + st->codecpar->width = c->width; + st->codecpar->height = c->height; + st->codecpar->bit_rate = av_rescale(frame_size_bits, st->avg_frame_rate.num, st->avg_frame_rate.den); + return ret; }