From patchwork Thu Sep 8 11:39:55 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sven C. Dack" X-Patchwork-Id: 475 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.134 with SMTP id o128csp779072vsd; Thu, 8 Sep 2016 04:40:07 -0700 (PDT) X-Received: by 10.194.117.104 with SMTP id kd8mr28748362wjb.113.1473334807585; Thu, 08 Sep 2016 04:40:07 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 8si8264447wmu.68.2016.09.08.04.40.05; Thu, 08 Sep 2016 04:40:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@sky.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 09FA2689CBF; Thu, 8 Sep 2016 14:39:53 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from nm36-vm3.bullet.mail.ir2.yahoo.com (nm36-vm3.bullet.mail.ir2.yahoo.com [212.82.97.135]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 989B5689C8F for ; Thu, 8 Sep 2016 14:39:46 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sky.com; s=s1024; t=1473334795; bh=q0BW0cT9vYuMHmZ0jYWGlX+vGi06AC2vsej68ofFTBA=; h=Subject:To:References:From:Date:In-Reply-To:From:Subject; b=gxWkfN4KjZ1QsWRUHpnXkD/IgyZ8oz6qijn5yozkNVsp2p15nOoUBXoXp3SvrEOkvOtJIwp3qN5dFNRr69FggR0U8eTunHvI59w0kBSGrMv0JKU2UCxaSjwcLdqivfN7PWNi74ZKyG85HuZ0Mcm4F1P7cdu3Vu5JanDeKD5GLzo= Received: from [212.82.98.54] by nm36.bullet.mail.ir2.yahoo.com with NNFMP; 08 Sep 2016 11:39:55 -0000 Received: from [46.228.39.68] by tm7.bullet.mail.ir2.yahoo.com with NNFMP; 08 Sep 2016 11:39:55 -0000 Received: from [127.0.0.1] by smtp105.mail.ir2.yahoo.com with NNFMP; 08 Sep 2016 11:39:55 -0000 X-Yahoo-Newman-Id: 946675.14304.bm@smtp105.mail.ir2.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: zHg3JPkVM1koqQ1KTY_1aU3w0_yQjFzYQ1twpZmquxp35Xr HAbmpKtDLqFNvS_ebTQZljeujUVbLaK7ng83cIToYFsxm04oJHE1CxHwRWLf I4EbObp0jX5jyghETXrVzbmLBDhaa8ejIxfLkQQVrzS5wnoAjVYQuLa2g.BG nnujuIJ1yqXd0Lu_x6I_oBzl6MXjEXroDkXYtF_JG3ptJWKLs6nr206.lCsk apuZ2HvJybP..V2155Z8HId17RjefhTVNtu5.wl_56gDJ81czyLQsid8yYy8 5LyLwn.R7zHqWkxo6r0UvqwCLC5CbhlVH18Oba6Huxy_P3b4VJGwPMuDuMq9 iUB_hfJyD2erAUI13geCLnD.P1RVvKIRYdBRJoigAARXvo7oqkfBU0zY.w.D BAzzhVUkMhI9AscEgPZtQkbZ92XZ_JPqJLl3qK2Gk3VaohMynGmEcsh4tL6Q YvSUyVqHnQpzvhrzNBPuO5JaOXILRCb3THky_mZO1FC.wQ5nQunYIWWPZjYh .9pC1N3xO9DDWegFSEULcgi5vNSEGNempyWEvUqvpS4Y- X-Yahoo-SMTP: vPZU.OKswBC.RMkOW9onWi6Zk8BriyC1yj0kMdBuqpLuLjitZVXhzK.Ofd5b To: FFmpeg development discussions and patches References: <66c7556f-6113-46a7-9a68-e0205c07152a@sky.com> From: "Sven C. Dack" Message-ID: <91a6b04a-4244-0157-3e1b-4ed0e5976ac1@sky.com> Date: Thu, 8 Sep 2016 12:39:55 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.2.0 MIME-Version: 1.0 In-Reply-To: Subject: Re: [FFmpeg-devel] Possible incomplete commit "avcodec/nvenc: support RGB input" X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Just a guess, but you are using nv_surface->height, which appears to have been rounded up to 1088 while the actual size is 1080. The 1088 is then passed on to av_image_copy() where it probably tries to read 1920x1088 pixels from the source, but the source might not have that much memory and so it segfaults. If you change it like this: } ... then it's again working for me. Sven On 08/09/16 11:00, Sven C. Dack wrote: > Hallo, > > ich schicke Dir noch einen weitere Backtrace mit etwas mehr Details: > > (gdb) bt > #0 0x00007f19eb8e3a3e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #1 0x0000000000fbc63e in av_image_copy_plane (height=8, bytewidth=7680, > src_linesize=, src=0x7f19ea08f000 "", dst_linesize= out>, dst=) at src/libavutil/imgutils.c:296 > #2 av_image_copy (dst_data=dst_data@entry=0x7ffe7ef4c720, > dst_linesizes=dst_linesizes@entry=0x7ffe7ef4c710, > src_data=src_data@entry=0x402c340, > src_linesizes=src_linesizes@entry=0x402c380, pix_fmt=AV_PIX_FMT_BGR0, width=1920, > height=1088) at src/libavutil/imgutils.c:334 > #3 0x0000000000e37321 in nvenc_copy_frame (avctx=0x395af00, > nv_surface=0x3dff560, nv_surface=0x3dff560, lock_buffer_params=0x7ffe7ef4c740, > lock_buffer_params=0x7ffe7ef4c740, frame=0x402c340) at > src/libavcodec/nvenc.c:1312 > #4 nvenc_upload_frame (nvenc_frame=0x3dff560, frame=0x402c340, > avctx=0x395af00) at src/libavcodec/nvenc.c:1430 > #5 ff_nvenc_encode_frame (avctx=0x395af00, pkt=0x7ffe7ef4dc90, > frame=0x402c340, got_packet=0x7ffe7ef4db4c) at src/libavcodec/nvenc.c:1660 > #6 0x0000000000a8c9c3 in avcodec_encode_video2 (avctx=avctx@entry=0x395af00, > avpkt=avpkt@entry=0x7ffe7ef4dc90, frame=frame@entry=0x402c340, > got_packet_ptr=got_packet_ptr@entry=0x7ffe7ef4db4c) at > src/libavcodec/utils.c:1961 > #7 0x0000000000496448 in do_video_out (s=0x39592c0, ost=ost@entry=0x395ad00, > next_picture=next_picture@entry=0x402c340, sync_ipts=, > sync_ipts@entry=-7.62939453125e-06) at src/ffmpeg.c:1175 > #8 0x0000000000498d7f in reap_filters (flush=flush@entry=0) at src/ffmpeg.c:1366 > #9 0x000000000049a940 in transcode_step () at src/ffmpeg.c:4117 > #10 transcode () at src/ffmpeg.c:4161 > #11 0x000000000047d11c in main (argc=, argv=0x7ffe7ef4e5d8) at > src/ffmpeg.c:4356 > > .. > > #3 0x0000000000e37321 in nvenc_copy_frame (avctx=0x395af00, > nv_surface=0x3dff560, nv_surface=0x3dff560, lock_buffer_params=0x7ffe7ef4c740, > lock_buffer_params=0x7ffe7ef4c740, frame=0x402c340) at > src/libavcodec/nvenc.c:1312 > 1312 av_image_copy(dst_data, dst_linesize, > (gdb) list > 1307 return ret; > 1308 > 1309 if (frame->format == AV_PIX_FMT_YUV420P) > 1310 FFSWAP(uint8_t*, dst_data[1], dst_data[2]); > 1311 > 1312 av_image_copy(dst_data, dst_linesize, > 1313 (const uint8_t**)frame->data, frame->linesize, > frame->format, > 1314 nv_surface->width, nv_surface->height); > 1315 > 1316 return 0; > (gdb) ins dst_data > $10 = {0x2049e0000 "\352", , 0x0, 0x0, 0x0} > (gdb) ins dst_linesize > $11 = {7680, 7680, 7680, 7680} > (gdb) ins frame > $12 = (const AVFrame *) 0x402c340 > (gdb) ins *frame > $13 = {data = {0x7f19e98a6000 "\352", , 0x0, > 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, linesize = {7680, 0, 0, 0, 0, 0, 0, 0}, > extended_data = 0x402c340, width = 1920, height = 1080, nb_samples = 0, format > = 298, > key_frame = 1, pict_type = AV_PICTURE_TYPE_NONE, sample_aspect_ratio = {num > = 0, den = 1}, pts = 0, pkt_pts = 0, pkt_dts = 0, coded_picture_number = 0, > display_picture_number = 0, quality = 0, opaque = 0x0, error = {0, 0, 0, 0, 0, 0, > 0, 0}, repeat_pict = 0, interlaced_frame = 0, top_field_first = 0, > palette_has_changed = 0, reordered_opaque = -9223372036854775808, sample_rate > = 0, channel_layout = 0, buf = {0x402b9c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, > extended_buf = 0x0, nb_extended_buf = 0, side_data = 0x0, nb_side_data = 0, > flags = 0, color_range = AVCOL_RANGE_UNSPECIFIED, color_primaries = > AVCOL_PRI_UNSPECIFIED, color_trc = AVCOL_TRC_UNSPECIFIED, > colorspace = AVCOL_SPC_UNSPECIFIED, chroma_location = > AVCHROMA_LOC_UNSPECIFIED, best_effort_timestamp = 0, pkt_pos = -1, > pkt_duration = 0, metadata = 0x0, decode_error_flags = 0, channels = 0, > pkt_size = 8294400, qscale_table = 0x0, > qstride = 0, qscale_type = 0, qp_table_buf = 0x0, hw_frames_ctx = 0x0} > (gdb) ins nv_surface > $14 = (NvencSurface *) 0x3dff560 > (gdb) ins *nv_surface > $15 = {input_surface = 0x3e0c090, in_ref = 0x0, in_map = {version = 0, > subResourceIndex = 0, inputResource = 0x0, registeredResource = 0x0, > mappedResource = 0x0, mappedBufferFmt = NV_ENC_BUFFER_FORMAT_UNDEFINED, > reserved1 = { > 0 }, reserved2 = {0x0 }}, reg_idx = > 0, width = 1920, height = 1088, pitch = 7680, output_surface = 0x3d97f20, > format = NV_ENC_BUFFER_FORMAT_ARGB, size = 1048576, lockCount = 1} > > > > On 08/09/16 10:30, Timo Rothenpieler wrote: >>>> for fmt in yuv420p nv12 bgr0 rgb0; do >>>> ./ffmpeg -f lavfi -i "testsrc=size=1920x1080:duration=10:rate=30" >>>> -c:v h264_nvenc -global_quality 20 -pix_fmt "$fmt" -y out_"${fmt}".mkv >>>> done >>> You feed to nvenc only rgb? what testsrc only supports. Use testsrc2. >> pix_fmt should make sure it's properly converted, and according to the >> output, it does: >> >> >> Stream #0:0: Video: h264 (h264_nvenc) (Main) (H264 / 0x34363248), >> yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2000 kb/s, 30 fps, 1k >> tbn, 30 tbc >> >> Stream #0:0: Video: h264 (h264_nvenc) (Main) (H264 / 0x34363248), nv12, >> 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2000 kb/s, 30 fps, 1k tbn, 30 tbc >> >> Stream #0:0: Video: h264 (h264_nvenc) (Main) (H264 / 0x34363248), bgr0, >> 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2000 kb/s, 30 fps, 1k tbn, 30 tbc >> >> Stream #0:0: Video: h264 (h264_nvenc) (Main) (H264 / 0x34363248), rgb0, >> 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2000 kb/s, 30 fps, 1k tbn, 30 tbc >> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel --- a/libavcodec/nvenc.c +++ b/libavcodec/nvenc.c @@ -1311,7 +1311,7 @@ static int nvenc_copy_frame(AVCodecContext *avctx, NvencSurface *nv_surface, av_image_copy(dst_data, dst_linesize, (const uint8_t**)frame->data, frame->linesize, frame->format, - nv_surface->width, nv_surface->height); + avctx->width, avctx->height); return 0;