From patchwork Tue Oct 11 16:57:52 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Cadhalpun X-Patchwork-Id: 958 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.66 with SMTP id o63csp26532vsd; Tue, 11 Oct 2016 09:58:04 -0700 (PDT) X-Received: by 10.194.236.228 with SMTP id ux4mr5949645wjc.135.1476205084447; Tue, 11 Oct 2016 09:58:04 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id om8si6002697wjc.45.2016.10.11.09.58.03; Tue, 11 Oct 2016 09:58:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DDE0B689B12; Tue, 11 Oct 2016 19:58:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf0-f65.google.com (mail-lf0-f65.google.com [209.85.215.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 19FC46897B4 for ; Tue, 11 Oct 2016 19:57:55 +0300 (EEST) Received: by mail-lf0-f65.google.com with SMTP id p80so4830046lfp.1 for ; Tue, 11 Oct 2016 09:57:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=from:to:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=9Ww1pAb+wprD9us6OFD94UEe6icPDkzE+xXocyNWifw=; b=BpfnELwMjDuMFx4U+U9Gj5MQNGtDbV5CWapLFKT4v0SriAoiOOCRwcxWeoy7vIvuPo TjcQxf0l6GVjoViERXZ4MXkW6hY+ofHdJAlsTTFkLWXR5dcoiMksxte16JekO3RSJUfH uXqZ3XNwtMbgQFFAwC/HC1wCSObBHxzb7H//7IEndTnhGMPr7MEqgSEiFQuutjiNZM06 8sbErTBmCMjWCit6hvPXgPABuFaYTWNQ5zRHW8l/5d75sHgVvePUwi2OiFIVoxESCcwB Ho+b6sL0cDUFApflOLO9s/fSrmKWHNwi9iclEypw7QB4LVol/BNfTraVzkEOlwy25xdB hk9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=9Ww1pAb+wprD9us6OFD94UEe6icPDkzE+xXocyNWifw=; b=XSsD7OL3vyHKMm7ryVKD7n/y2dkH8ae+t/YwRKL74OqLIUWR5X6Uww/b+HSYFBEaem IORH10gLrqo+uJZA67nojCx+ZicFLgmmfGjnMcrYwQjfKWg6g+OJi3hComznjq9Z1Xki zNxBe6J45TkfPzzYirp3IoZf4MJJ1m9gK91xtFx92VWM9M3oWpgAqcXskwoXff0p57Np d2E5L18UFFQssVE0rOx0uL4TbT9h3IR/JDOlS+DvkX5puVpH0oRBfmBMnycbITOzrE/5 VN4tLsWFADVibX4cIpKx6cO9kZOYE7Cy9reKMKAaZ9GQuM2LxyXXd9gaSZE6iyrVmRVs rA2w== X-Gm-Message-State: AA6/9RmhWOGSUMK/+u4lGSxHZUBTPbUd6skS2uu09TzA+CG9mPnGCa/w6tLGftUn0yP7ZA== X-Received: by 10.25.18.97 with SMTP id h94mr3918744lfi.106.1476205074262; Tue, 11 Oct 2016 09:57:54 -0700 (PDT) Received: from [192.168.2.21] (p5B095A9A.dip0.t-ipconnect.de. [91.9.90.154]) by smtp.googlemail.com with ESMTPSA id 87sm1336906lfs.0.2016.10.11.09.57.53 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Oct 2016 09:57:53 -0700 (PDT) From: Andreas Cadhalpun X-Google-Original-From: Andreas Cadhalpun To: FFmpeg development discussions and patches Message-ID: Date: Tue, 11 Oct 2016 18:57:52 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] libopenjpegenc: recreate image data buffer after encoding frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" openjpeg 2 sets the data pointers of the image components to NULL, causing segfaults if the image is reused. Signed-off-by: Andreas Cadhalpun Signed-off-by: Andreas Cadhalpun --- The relevant openjpeg2 code is: https://sources.debian.net/src/openjpeg2/2.1.2-1/src/lib/openjp2/j2k.c/?hl=10253#L10247 --- libavcodec/libopenjpegenc.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/libavcodec/libopenjpegenc.c b/libavcodec/libopenjpegenc.c index 023fdf4..2f0ab20 100644 --- a/libavcodec/libopenjpegenc.c +++ b/libavcodec/libopenjpegenc.c @@ -776,6 +776,16 @@ static int libopenjpeg_encode_frame(AVCodecContext *avctx, AVPacket *pkt, goto done; } + // openjpeg 2 sets the data pointers of the image components to NULL. + // Thus the image can't be reused. + opj_image_destroy(ctx->image); + ctx->image = mj2_create_image(avctx, &ctx->enc_params); + if (!ctx->image) { + av_log(avctx, AV_LOG_ERROR, "Error creating the mj2 image\n"); + ret = AVERROR(EINVAL); + goto done; + } + av_shrink_packet(pkt, writer.pos); #endif // OPENJPEG_MAJOR_VERSION == 1