From patchwork Thu Mar 24 06:14:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 34946 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:ab0:5fda:0:0:0:0:0 with SMTP id g26csp1059587uaj; Wed, 23 Mar 2022 23:14:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx960vPVPIgG508bLTvBynv3yrPcwTxZ058kjaTJK9ql+Z7dcgWYkm9P2IMQMH/YPVUjdbI X-Received: by 2002:a17:907:7204:b0:6db:ebf4:9988 with SMTP id dr4-20020a170907720400b006dbebf49988mr3818240ejc.709.1648102486847; Wed, 23 Mar 2022 23:14:46 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 13-20020a170906010d00b006df76385bbcsi17118619eje.92.2022.03.23.23.14.46; Wed, 23 Mar 2022 23:14:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@nxp.com header.s=selector2 header.b="njyD/ZN1"; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 93CAB68AF21; Thu, 24 Mar 2022 08:14:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60062.outbound.protection.outlook.com [40.107.6.62]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F0E7868A066 for ; Thu, 24 Mar 2022 08:14:37 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YIb1a9Ks3cayYMeNgpun9IwDsCzZJCSrjTkfOAwXjIuzRlSQoeULr44MPDqUIUnrysbqV2c3qffSA6Jp5KBQSyta4QReFkRSv8/J/Uk1Xx/tgOH5lRLy6cpABvXkM6ICxhjjyvW6McudGIX9TPTs40J5ax8v6igqOTG0U2yKNPswg/LOPE4GFaWLEI+7DjtGLVQgYgLaOtTWMO0J+Xiiw4G07RbMR4rBX63yOjW8xBWDcfFDFWVik06PPZ1TzHYDojy6W2aZ+8SzGXhaYzSU/qebdHBUO54AdMvCfcjEMbLthjp+Z5Di9tXnS6siybtiN/tzm11dVy5teyrSpFXTtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BEouDomXNsgEmIGRSJJMSKd2NzvyBHR4zO/uoq/78qA=; b=NlzGJb2hD826loFhHbsHt5AFR4nt83uUBCVWpTmd32fpWazLHW8epCopu8xMlUEZSbU2hYVprtUcoQcODtxkLFydpvITCJ8BhsD5pSRYAiIYNhEaa6hXOFmIG6zPg5xygh9zquT8xjcPk18r9lMMO98D5exA55P3DsJsMctGQ5xRA1FJ7kJCwQa/a3wbHk8fNjjaGb4dVVTK44TiJu9VseWwFKSENsXvQ+bVnnBEMalH2MS/ab2spt2KjUOAQUKrjiHiQZE6WKwAiGeLC7ldVqmlVexw2zmKkCbznBtbghw6Sq+/zxfVLkgNPlE0mLwWMD1K+kRdZMaq9YS017TbeQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BEouDomXNsgEmIGRSJJMSKd2NzvyBHR4zO/uoq/78qA=; b=njyD/ZN1TQT8IfuZIsuxmubSM9X1Uoz1vUPJZTX+gbKbgEnnhYeiDruudepgFYVK9om0l8ezBoGSqWwSHtJLaATxGbQUbryvL6B0/CBfLo49InlVvKxa+lA7EvkgHyqxrriVT7DRAnIl9HV0SRDoWJGY4ChxrwtssGZGkYCwu9Q= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by VI1PR04MB2960.eurprd04.prod.outlook.com (2603:10a6:802:9::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.17; Thu, 24 Mar 2022 06:14:36 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::c39:69cf:c4ea:967]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::c39:69cf:c4ea:967%5]) with mapi id 15.20.5102.018; Thu, 24 Mar 2022 06:14:36 +0000 From: Ming Qian To: ffmpeg-devel@ffmpeg.org Date: Thu, 24 Mar 2022 14:14:12 +0800 Message-Id: X-Mailer: git-send-email 2.33.0 In-Reply-To: References: X-ClientProxiedBy: SG2PR01CA0134.apcprd01.prod.exchangelabs.com (2603:1096:4:8f::14) To AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9e744304-b56f-4be5-d88f-08da0d5d9153 X-MS-TrafficTypeDiagnostic: VI1PR04MB2960:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hSylldr74rathr8E7ve6AyasLT1wv+QV1R5xk1+8gxpAX2++IkAq9Py1S42kSnJP64Lex1V7TkCR70j0Yv2bUk2ZIzPyejkcJkVjUcasqLVPPA//7bwKPTSELJESlMK0RIFDdAoOXClz8kCRWxMuxOALWlzqCtlFa4H++kFziMEYQK0wrvdMxBBbSNa863P2IMGAH2adWYT0+QAM4QQ7Cgk6cUvFMezHeCcO/xFnPWAZiibN+ErAGIFzrS8XaVG5PyOfUXFCv/mQKPm/AZVDcnY0hjj/ybWjh+zcU7BZigdkdh1vBkBPA45Rl6Ae8sdFrJhnVne1DGHIE/unuGOxRSlhtkhayYPKPWe60fDTRRTEvhhMiWW7J2LvN8qeUAeQ011D5C5X2lQ401XlH7f8eQ271rfXFw7miAdcmQCa9Ct6KBrOSHI1pDlZ4JeVWceAAGPDw+WCt6AnRzYtQX1t0HDD9wNJyd3ErcL9SC3I9Kpr/LnbRzCwXh8RVXxz5Ve4K4NgD/Ar1mA8oPq0ntu7QYn5VUyeDS2JBx5nmenLBMZaOG0lLiEAyZqUVVvSQL98VgCu8lkHx+CZHiwb9C4WL5ymsMtoLvQOdicQRAICBBcWlphzpXAWrxwQb5lPB4K4FfqkklpIkJo7Uh8/651oykjhMUR1z6K4bFBitYj3X3GEZG+/aKepfmA6Il8A0cfHHV35LmKJRXfWtIoNtWcoLNpUPHgxiWyTqgngXirH3uE= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(86362001)(8676002)(66556008)(8936002)(38100700002)(38350700002)(44832011)(66476007)(5660300002)(66946007)(6512007)(6506007)(6666004)(52116002)(83380400001)(36756003)(2906002)(508600001)(316002)(6486002)(2616005)(6916009)(186003)(26005)(70780200001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 04UYtIbHYKTWeePrxcouPpMZvJWQOTued58Wv/9hSYSGCa3bK2RYom49Q5vT+ifp3z8qKjbNnJZtx6DVzqG+i3OvkP9lI1JdrF3Xj3wLv8AsNZJwv0zfUKzhw5VjmuYQ1Mr7vKlTGHzVoSBvylzRGEPIPenfFxJoGFvqWQxTyhUqEbIh7KVhPybgsyQbuIQFvPXZI5hYPEmiwmdVGWyNt3Num3bys18FMTXneEES17YWYnzsQm3TCSpvq9xcYTLHJMhsm7rR2peFB0hZfxBXQmPu10e/WnHj8cYravcNS698b+4W2v7aEkGZ8GRcyBnrQJk4bES5lZ08lEP6mwDMPCO3Ahnc0mTXA5eh21d6MnM5Ax1H8uVHSm/lFKAZ4aJLtRdb0kIGasBgoMMd1nQM507/uyqNEZsbclowa7CfpCybqC9n39mKzLCXO9+CEg7L8mOlbxW3Jbcn/+PfA6yGp4diaehsSypirD6zi0bT4dpBce1bhh6cADXuzjJkZQm4TgL2msPkPvnmbR4KHAar+ei6YSX30AAl+IPBGwsWoORFTJ7bywQ/OccfnYt5RWF4B2yaux/m4NZQUm/+wvB8/x2Ebc99wdCfYhIyvS33TZS0lfm5jYEoMmVBjobEn1gCsu9ml00AxDUc9PIopuPZdJIMf0CqvNJ+ynVchF17Lr9UyFLmky/wcNaCql7+P8USMdd+A2cdGGgt5vI1Y0QiLkz5679kvhhRt+9IWV87iSG0RRSy9CkFmOpSN/QYJhOI+LUKbE5N5QNA1/MhkSohnWutTwp6uV+1EalI7ZQK0CEGiAQsER2UMKX5dHXsz3NL1ccVbqFpYGMtPb3AdjWhU+DFripOglwkwZsMYZV2Tm+z9L+fO8rVMio2Z89xUh1HsL9yKEUBi+sDYQi8VXX8mZ1HVJ58uvFkHZtGhfYZ3H7jr92vU4l07DEKJA7O0WhezUXIT9WdHNY8sauHyu+w7mb4uCVI9XzNSofLhUcNnekBmfsvHKhCbzIXwsfgouTmIK0XAly2xjDRqpuxU46TFncl3aWLj1oHmLLU9jnwrlc7B0fh2uBMXiiZic3TQeXnl2ruDqQnmo8qR8uLDeKLm9FpABMoB+zTZEQqThPQvMBYGyalajSZfR3Fkeh+lVh3rsyV8Y0RnRDjb7WjPcRXy+zT+XXhS81Ftkk0SvXbqCeFRxnJn+R2kMXzbQi4Rda2RXiTsNbOJOWX8tMIerE3NJH2mOom0jhjYsB0DnpV1D/3iz9OVgRjUE/OlHoA+G1S0QeZwk0ouoxwJ8IoPA1xhN3TI453Rp8jdlIxKHCwSzw6BtKQvYOMCB3d7aMz0O2i0HVSE+OtFcsUb5JnIYlrtNvs3Im3anJXm1pkC/ibRJNS1cftVSF8Q8M4Fr+osHLNvY13e/NqQjGWqpmxvBWmFc8YWx83Ic+08yDqZi5NuL5OAL/lzTQmiioWmRo2nYnG X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9e744304-b56f-4be5-d88f-08da0d5d9153 X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Mar 2022 06:14:36.0156 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: reuTd5BcyJjhc9aPsnGNcDqF8di5obBxcbWmPvIu8J5H68ZO2Zk1OWSQ5M/7yy5NwVvTqbblY2m0kwKBj+9bQw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB2960 Subject: [FFmpeg-devel] [PATCH 1/2] avcodec/v4l2_m2m_dec: add a dequeue_timeout parameter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 1P/b5inPuYS2 the dequeue of capture queue will be blocked until decoded frame available or an input buffer is ready to be dequeued. but it may cause death waiting in some case. For example, it has enqueued the first input frame, and then blocks at ff_v4l2_context_dequeue_frame. For some reason, the decoder can't decode the first frame, also it can't return the input buffer. The decoder needs more input data to decode, so the decoder is just waiting. This creates some kind of deadlock. And in linux/Documentation/userspace-api/media/v4l/dev-decoder.rst, there are some descriptor like below: The client must not assume any direct relationship between CAPTURE and OUTPUT buffers and any specific timing of buffers becoming available to dequeue. Specifically: [snip] - a buffer queued to OUTPUT may result in a buffer being produced on CAPTURE later into decode process, and/or after processing further OUTPUT buffers, or be returned out of order, e.g. if display reordering is used, And I meet a similar case, there are 16 output buffers, but only one buffer is queued to driver, then blocks at ff_v4l2_context_dequeue_frame. But the decoder need more frame data, otherwise it won't decode it. So I think the client should keep processing OUTPUT buffers if there are available buffers until the end. To resolve it, I think we can set a reasonable timeout instead of -1. So I add a parameter dequeue_timeout, and it works on my side. Signed-off-by: Ming Qian --- libavcodec/v4l2_m2m.h | 5 ++++- libavcodec/v4l2_m2m_dec.c | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/libavcodec/v4l2_m2m.h b/libavcodec/v4l2_m2m.h index 16e0a6d6b90f..5c7e7c955383 100644 --- a/libavcodec/v4l2_m2m.h +++ b/libavcodec/v4l2_m2m.h @@ -38,7 +38,9 @@ #define V4L_M2M_DEFAULT_OPTS \ { "num_output_buffers", "Number of buffers in the output context",\ - OFFSET(num_output_buffers), AV_OPT_TYPE_INT, { .i64 = 16 }, 6, INT_MAX, FLAGS } + OFFSET(num_output_buffers), AV_OPT_TYPE_INT, { .i64 = 16 }, 6, INT_MAX, FLAGS }, \ + { "dequeue_timeout", "timeout of dequeue in the capture context",\ + OFFSET(dequeue_timeout), AV_OPT_TYPE_INT, { .i64 = 32 }, -1, INT_MAX, FLAGS } typedef struct V4L2m2mContext { char devname[PATH_MAX]; @@ -76,6 +78,7 @@ typedef struct V4L2m2mPriv { int num_output_buffers; int num_capture_buffers; + int dequeue_timeout; } V4L2m2mPriv; /** diff --git a/libavcodec/v4l2_m2m_dec.c b/libavcodec/v4l2_m2m_dec.c index 4c521eba343c..e386e09e3e1e 100644 --- a/libavcodec/v4l2_m2m_dec.c +++ b/libavcodec/v4l2_m2m_dec.c @@ -138,6 +138,7 @@ static int v4l2_receive_frame(AVCodecContext *avctx, AVFrame *frame) V4L2m2mContext *s = ((V4L2m2mPriv*)avctx->priv_data)->context; V4L2Context *const capture = &s->capture; V4L2Context *const output = &s->output; + int timeout = ((V4L2m2mPriv*)avctx->priv_data)->dequeue_timeout; int ret; if (!s->buf_pkt.size) { @@ -172,7 +173,9 @@ static int v4l2_receive_frame(AVCodecContext *avctx, AVFrame *frame) } dequeue: - return ff_v4l2_context_dequeue_frame(capture, frame, -1); + if (s->draining) + timeout = -1; + return ff_v4l2_context_dequeue_frame(capture, frame, timeout); fail: av_packet_unref(&s->buf_pkt); return ret;