From patchwork Mon Sep 4 17:16:40 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jorge Ramirez-Ortiz X-Patchwork-Id: 4983 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.15.201 with SMTP id 70csp3315085jao; Mon, 4 Sep 2017 10:16:51 -0700 (PDT) X-Received: by 10.223.188.68 with SMTP id a4mr806513wrh.300.1504545411718; Mon, 04 Sep 2017 10:16:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1504545411; cv=none; d=google.com; s=arc-20160816; b=sD2NXmUNFEV8MjVAp7v2SAGOVBddmy0CVy0j/D4otG159klw//dAFhlmn4BJ2lZSor HUdq/gx3RBPXHM7m+vptLhfY7M2lmtZfl3LG/aI+uA066VxYFJzSxTWE9irSYLsfhDOj Rncdw52DMOumdbmLf3tForz5DscYKES9VujrQ7lBVQeAdPDvlJ2kgrLhxL76LSbLDqoD HvczEicBQ8Ttaf1a+sElc8P+qUEsinrctm+7lcwNITD6uGlMduxtLRsXEa/n1L9DJJAM 3FTAc7PBYpfJeNvTxm7BailXiBZgWb+trZ6bb3B6rcSm9I+bbBlgpOqft3d+OqbMv7/O 17MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:content-language:in-reply-to:mime-version :user-agent:date:message-id:references:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=4RrzYK7chy2vfi+pTa1JVH0wbeMMKb3c8MG3Ldpe6HE=; b=R6snQYPBvIzcx2RUj1yvHoNVU/5Mjy2qrKjVjJj5xSnSFQgt6Sem0q8W9HuEJKZg9R zUJSsdVLDMzplX7q77Tw0EJChxuJDQDSV8Ol8oF1xsQbtPep9Pkx5X2hxvYgKEKNmPjv 1L3+q+uNuw0ZXD9hNDDYtKLTcrg1O3URERQf3/VrBaK/oZE+qzRJYhK+/W6gjkRhZl8Z GbRpS0rEB62dgQZk8WZ/63YBb1paDkBehsIJZJ9pKyFsiWGQqhWZ+2S7mtASKKN42ViI cAxEj5TPJ6bqMA2rdkADBl0W1A8tFq8gADdV2cAiPRiyRBwz2pZ1z55nuFykJ7jneR7B ySZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=hdXrFAiz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l15si3802855wrf.136.2017.09.04.10.16.51; Mon, 04 Sep 2017 10:16:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=hdXrFAiz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E67C8689BD7; Mon, 4 Sep 2017 20:16:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f53.google.com (mail-wm0-f53.google.com [74.125.82.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1933B689AE1 for ; Mon, 4 Sep 2017 20:16:40 +0300 (EEST) Received: by mail-wm0-f53.google.com with SMTP id v2so8318297wmf.0 for ; Mon, 04 Sep 2017 10:16:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=subject:from:to:references:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding:content-language; bh=VLEdN+EVOoDNmli6ABJtWeHg2kUP3khpkIIZmYW1X7k=; b=hdXrFAiz8jc9bhhHvNJhmbo6kfIoYpc1yvLLWA8kTzTzwHZ68+ZIEeeOzkI12euimu NhqA61AfC2OB3DM1MY7JaF+dsGPWNHxOPUG2EAwr3CQZGFQDtslx52pOG/nsmCIWP/6L quGD4oo81ANw51pdXHUW0L3gDfqnansvyY3QE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:references:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=VLEdN+EVOoDNmli6ABJtWeHg2kUP3khpkIIZmYW1X7k=; b=ZR2XuvjQAK8BQEhrXGrJS7Z1WdFPNKakz+2ZmATnHMKwrEnVQvE7l2LAFER6Oclodk 8MNeczsVA3Gng1B/vXr2dnh3RVB9KPk3x6QHoZFQ69xIbZINBfqjugyBH+rWXi5b7ww5 5yDfeq0aLUY7UNFg4JF92ePDWyuDwXBSAFbHFNYOVd27ple4CyebtZUeGEzcK4CPdbW+ GWmfc+dwDNN4An7QUO7IW4NusK3Wrm1eRnGcI1hFXFJMG9HZ9Csi+OkZV7oPGitub1rz jX06cLttrQ65sWiuOLXQnOWT+1vq6j1PhOPOdljS+XBuxR/CzF+kC9/WwBTw7gbNiPfy 9Pcg== X-Gm-Message-State: AHPjjUi4XLqbwEjXrA3rS6rxec4p9MeGJHAKwdfQlix847GrBUtTc597 dy6GsFanlcMFg5QQ X-Google-Smtp-Source: ADKCNb7A13TpcSDbLRLueQP8eOXaFYlkNmhkYZu8tPnnP5ghaTVrvOTgKSIFoim6+92VMxYx0lkQWQ== X-Received: by 10.28.15.210 with SMTP id 201mr891967wmp.10.1504545402170; Mon, 04 Sep 2017 10:16:42 -0700 (PDT) Received: from [192.168.1.2] (254.red-79-152-212.dynamicip.rima-tde.net. [79.152.212.254]) by smtp.gmail.com with ESMTPSA id s126sm1122121wmd.43.2017.09.04.10.16.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Sep 2017 10:16:41 -0700 (PDT) From: Jorge Ramirez To: FFmpeg development discussions and patches , Mark Thompson References: <1504383070-27130-1-git-send-email-jorge.ramirez-ortiz@linaro.org> <05cfad94-60a8-f99d-6f55-69bcb70a7c7c@jkqxz.net> <52b1bf27-0244-9855-bbc8-1b3bc4733efc@jkqxz.net> <5ffa1596-2610-262b-b5f2-f305711a3b8e@linaro.org> Message-ID: <94ad3a87-f926-0355-a6c2-2f5204cff0e4@linaro.org> Date: Mon, 4 Sep 2017 19:16:40 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 In-Reply-To: <5ffa1596-2610-262b-b5f2-f305711a3b8e@linaro.org> Content-Language: en-US Subject: Re: [FFmpeg-devel] [PATCHv8] libavcodec: v4l2: add support for v4l2 mem2mem codecs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On 09/04/2017 07:01 PM, Jorge Ramirez wrote: > On 09/04/2017 06:31 PM, Mark Thompson wrote: >> On 04/09/17 17:00, Jorge Ramirez wrote: >>> On 09/03/2017 08:23 PM, Mark Thompson wrote: >>>> On 03/09/17 17:54, Jorge Ramirez wrote: >>>>> On 09/03/2017 02:27 AM, Mark Thompson wrote: >>>>>>> +/* in ffmpeg there is a single thread could be >>>>>>> queueing/dequeuing buffers so a >>>>>>> + * timeout is * required when retrieving a frame in case the >>>>>>> driver has not received >>>>>>> + * enough input * to start generating output. >>>>>>> + * >>>>>>> + * once decoding starts, the timeout should not be hit. >>>>>> This seems like it could introduce a significant delay on startup >>>>>> for no good reason. Can you instead just queue packets until >>>>>> either you run out of input buffers or a nonblocking dequeue >>>>>> succeeds? >>>>>> >>>>>> (I might need to think more about how the semantics of this work.) >>>>>> >>>>> if the decoder needs 4 blocks, the delay is 200ms, if it is 10 >>>>> blocks, that is 500ms which doesn't seem too significant? when I >>>>> test I barely notice the difference with respect to using the h264 >>>>> codec (or any of the others in fact) >>>>> >>>>> the best solution would be to be able to block until the capture >>>>> queue has frames ready but for that we would need another thread >>>>> inputting independently on the other queue...does ffmpeg allow for >>>>> this? separate threads for input and output? >>>> Since the API is nonblocking, you can just return EAGAIN from >>>> receive_frame if there are any free buffers (to request more >>>> input). You would then only block waiting for output if there is >>>> no more input (end of stream) or there aren't any free buffers (so >>>> no more input could be accepted). Ideally there would then be no >>>> timeouts at all except in error cases. >>> sure, can do that as well, not a problem. >>> >>> the encoding API doesnt seem to allow for this though: once it >>> retrieves a valid frame it appears to keep on reading them without >>> inputing others (this causes teh capture queue to block for ever) >>> >>> is this intentional or is it a bug? >> The encode API should be identical to the decode API with >> frames/packets swapped (see docs in avcodec.h). >> >> If you have an lavc-using program which calls receive_packet() >> repeatedly after it has returned EAGAIN and never calls send_packet() >> then that program is wrong. > > thanks for the prompt answer. > > yes I am just using the ffmpeg binary to encode a stream; however once > a valid encoded packet is returned, send_frame is not ever called > again unless I return EAGAIN from v4l2m2m_receive_packet. > But I cant return EAGAIN on receive_packet while I am blocked waiting > for data which will never arrive (since send_frame is not executing) > > seems to me like a bug in ffmeg but I dont like to question baseline > code with obvious bugs (this seems to simple to be real) > > anyway looking at the function do_video_out() the code seems strange > but it explains why my encoding blocks unless I timeout and return > EAGAIN. > > > ret = avcodec_send_frame(enc, in_picture); > if (ret < 0) > goto error; > > while (1) { > ret = avcodec_receive_packet(enc, &pkt); > update_benchmark("encode_video %d.%d", ost->file_index, > ost->index); > if (ret == AVERROR(EAGAIN)) > break; > if (ret < 0) > goto error; > > if (debug_ts) { > av_log(NULL, AV_LOG_INFO, "encoder -> type:video " > "pkt_pts:%s pkt_pts_time:%s pkt_dts:%s > pkt_dts_time:%s\n", > av_ts2str(pkt.pts), av_ts2timestr(pkt.pts, > &enc->time_base), > av_ts2str(pkt.dts), av_ts2timestr(pkt.dts, > &enc->time_base)); > } > > if (pkt.pts == AV_NOPTS_VALUE && > !(enc->codec->capabilities & AV_CODEC_CAP_DELAY)) > pkt.pts = ost->sync_opts; > > av_packet_rescale_ts(&pkt, enc->time_base, > ost->mux_timebase); > > if (debug_ts) { > av_log(NULL, AV_LOG_INFO, "encoder -> type:video " > "pkt_pts:%s pkt_pts_time:%s pkt_dts:%s > pkt_dts_time:%s\n", > av_ts2str(pkt.pts), av_ts2timestr(pkt.pts, > &ost->mux_timebase), > av_ts2str(pkt.dts), av_ts2timestr(pkt.dts, > &ost->mux_timebase)); > } > > frame_size = pkt.size; > output_packet(of, &pkt, ost, 0); > > /* if two pass, output log */ > if (ost->logfile && enc->stats_out) { > fprintf(ost->logfile, "%s", enc->stats_out); > } > } > } > > > so if I queue 20 frames in the output queue and the allow frames to be > dequeued, all of them are dequeued at once and then the code just > blocks waiting for more input... > > > > does the above look ok to you? this patch allows me to remove the timeout on encode.. > > > > >> >> - Mark >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > diff --git a/ffmpeg.c b/ffmpeg.c index ccb6638..57db03a 100644 --- a/ffmpeg.c +++ b/ffmpeg.c @@ -1323,6 +1323,9 @@ static void do_video_out(OutputFile *of, if (ost->logfile && enc->stats_out) { fprintf(ost->logfile, "%s", enc->stats_out); } + + ret = AVERROR(EAGAIN); + break; } } ost->sync_opts++;