From patchwork Fri Jun 30 03:04:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hsia-Jun Li X-Patchwork-Id: 42343 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1e:b0:12b:9ae3:586d with SMTP id c30csp2189669pzh; Thu, 29 Jun 2023 20:04:30 -0700 (PDT) X-Google-Smtp-Source: APBJJlGsYis2kXkogzkyTVaQAjv5KyewI7NL6Q6RUaJLHNqBVi0+GTfr6Qgw0RnOv+HEFlA1B7ik X-Received: by 2002:a05:6512:3192:b0:4f9:56aa:26c2 with SMTP id i18-20020a056512319200b004f956aa26c2mr1200997lfe.57.1688094270690; Thu, 29 Jun 2023 20:04:30 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u16-20020a056402065000b0051dec0e4eccsi631475edx.47.2023.06.29.20.04.29; Thu, 29 Jun 2023 20:04:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@Synaptics.onmicrosoft.com header.s=selector2-Synaptics-onmicrosoft-com header.b=SnaYbCNT; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=synaptics.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5F1CC68C2E0; Fri, 30 Jun 2023 06:04:26 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2054.outbound.protection.outlook.com [40.107.243.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0C865689D6F for ; Fri, 30 Jun 2023 06:04:19 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VQE1aMvAj2N/bYpv1F0QGIwLk739wwPR1Z5DSGPPbEBzVCMDBQYXWT/V51PcatSEgHMcLbjI5iZGRqNPvqm/9s724dEZd5wsxNjtdiU1qRnklyCxGghGC3QydS1H8hhpmCPblknYabcW+T8BydXNFOif3Q/gGrOiw2x7wy8qyBa2pOaCNKhzp8cIeUZELyy+TyeJCvvWZRpwkE1aK7rI/FWpG86XBoFeccnXqxAWq5XEgpTGKVq0thMNbQHrrLZZEveSGkpSlb2iqIm5gcqKKNCFdOMm/q9M+JGQxnON3uQsexV38CsRPrZ/PDyRiBYz3mdIgP58h2GEVUG5jzn0qg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=leMG51TSVbDkuIAYZN+45OuhClo1FqzZvMJlIlB3bhw=; b=DKkkenxNjRXbolsQ31U3HTPdEtdHI6scaaiS6Zd9/ti7elSkCgieDo/1By43HkgHQPDXC7+F3nNYAaEne4OVjHFHhhuRWOhtEt/5vUiye8nPPBQS1DQoufoeUki0fZmpLR1yYnByPjLWp0Y1f3LbcBlKnP1JGk/GVCEo+14dRW5Au6OwDK19Mxysea/M7jT4b2rMsrO8F/h1DlIXDRLqdNX5xR+YHSlceN5bC4YQshasQlsqrhcfrrPhBkbBinfg4PygdfyQVF0Ee/mpV7GCXnPZcbFGUvYAvdJnqDLd7Mbz2yf9S4IGpNPulaCgBWTprJEa7OoWS3KOau17RMSE0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=synaptics.com; dmarc=pass action=none header.from=synaptics.com; dkim=pass header.d=synaptics.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Synaptics.onmicrosoft.com; s=selector2-Synaptics-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=leMG51TSVbDkuIAYZN+45OuhClo1FqzZvMJlIlB3bhw=; b=SnaYbCNTj74lydrzXkQgz5rMFKG7gQN4o4qSu8ekpdEgN254zATzhP+5AL38csdJydD2xeptnoD95HSPJP9qzvu4qq2qeVzrhc2wAVhiAFwdRJzSVorwDQ7EvbsGHkLyUCTdmzpvPOX4rzpDgBfMqb5Z0nkCAFI5c0C+2smBBnU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=synaptics.com; Received: from DM6PR03MB5196.namprd03.prod.outlook.com (2603:10b6:5:24a::19) by SJ0PR03MB5552.namprd03.prod.outlook.com (2603:10b6:a03:288::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.19; Fri, 30 Jun 2023 03:04:14 +0000 Received: from DM6PR03MB5196.namprd03.prod.outlook.com ([fe80::6882:b9c1:2b2d:998d]) by DM6PR03MB5196.namprd03.prod.outlook.com ([fe80::6882:b9c1:2b2d:998d%5]) with mapi id 15.20.6544.019; Fri, 30 Jun 2023 03:04:14 +0000 Message-ID: <5d1fc70a-675a-0d18-87be-e6e00a8fc607@synaptics.com> Date: Fri, 30 Jun 2023 11:04:07 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 From: Hsia-Jun Li Content-Language: en-GB To: ffmpeg-devel@ffmpeg.org X-ClientProxiedBy: SJ0PR13CA0053.namprd13.prod.outlook.com (2603:10b6:a03:2c2::28) To DM6PR03MB5196.namprd03.prod.outlook.com (2603:10b6:5:24a::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR03MB5196:EE_|SJ0PR03MB5552:EE_ X-MS-Office365-Filtering-Correlation-Id: 03fa79bf-a815-4fe9-6996-08db7916af10 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F9LkWZBqwEEikQKrP90Tbpl/i/JSF4PpyChth11rcvRTG22Nc1jQnkK2zC+JMvvU8ZHHiAwfHYxjaGiPZUZIj8CAMcsJL/QrYpK1BtDI//yTvx18wsmD0wbpqroB40ZmOR7hQN/Svh6KWk6ICN9KQu6bpWoPZPswRsnkEoDrdEB4gxBXMGIfTE90ciqsL5cRZiL0pYy+ojMgjVTxKsMdz1l8Ww9A2Oc1oUdgpxdP6AZE+7T+EUP4m3fgctjvBvdCtJCxdqRKdlDqWqWw/oWrQ7fWLyAWqBkFVLru6YQ6ESdBI54gr87EYSHezodLXY6n2hsqkQRR9GPu9o8pqDZD8C4ekLrc8ItjLT6cEpoqdb/4A05IypMT3PccuLGkLrLn/KOdJjatUU5yjTCY6SOZOvig+XHwSrBuC+0qQ6D5B3oDvyaoa/Mgu4DU/l0Bid//Bv4BnR3axN1NbtlYvrvv4Yd23CZyGBJ/VU7g/R6dk2wcLElP4rQFs2CY0n4oyZ+wDz/Xp5gq4DjgN3N8E3PfUpt90cKYpNcUd3hMX14tUyMED9lti9s2WK3ZM7wZVImGk4OI7ac5PBfFtlknuplH4TajtMkd93QlCNxmezeOSof7G9YkFcFyT8InicvmVSwCg6L76B4AhyiNYTghs9gSEITNpP3FIHaNMJ8wwN6iQTLy4oNj7FP/TOYEzQ7glQZGJmbndg9AUNOR3AXFKnE9Lw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR03MB5196.namprd03.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(346002)(39860400002)(136003)(366004)(376002)(396003)(451199021)(83380400001)(30864003)(2616005)(2906002)(38350700002)(38100700002)(36756003)(8936002)(8676002)(5660300002)(86362001)(31686004)(6512007)(41300700001)(66946007)(66556008)(66476007)(6916009)(4326008)(6666004)(316002)(6486002)(33964004)(52116002)(478600001)(186003)(31696002)(6506007)(26005)(70780200001)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?HJiV9jC4yDMDEeXyf9WARa6d79dz?= =?utf-8?q?GK5fTSHz/LbExrV3cEr2AMhM/nFt5pNrJJysOuzKFiHhEdQ57ye4XpqrLkmaxKxbD?= =?utf-8?q?H+lYt3lGiTA/SVsahlK0fSBsJfkrFo2gVzXq3onUFB/7uwgIxNxsr7i53sAGQYaMD?= =?utf-8?q?MYM0/zBgVenkQpvG5kwGnQUeB8RFssI7b1pU9OBYd7Cv90sO6+nc6nEpCehQbS5ws?= =?utf-8?q?TDoh1g2q6Ff58VnVgLbfHwtQ/wmsT+h5iK0mMUU29+Coi0yU2bZILj7es0CRr92Vf?= =?utf-8?q?RgmaC6vUjE9N/12g4pPOkCtq6CXMQO+BHxU7KqinQBQjaT4Xyh9a07fRDpBnjMMUD?= =?utf-8?q?RkRX7D1cT1JTE14Fi1llqrYrC+BzdYOgYHFGA/qygmyvb5Ige0LC/Ks4WRaMx7i6D?= =?utf-8?q?ZiK8xzQMC+7Q7rVpakp6MaYjuAN5dCR5wsWl9q2ZE/C9Ygpv30n3fTec5BP9C1mAr?= =?utf-8?q?QNPUrGqZDRpGIFfdXq834OVZmg+4+uwpbdoxHJuMyBkAiautbIV6S3B/Fx++U6w+k?= =?utf-8?q?sYfRJgU6xbQw3V8CtuM8gC0VkgQsW2UzypjJP8xHC+t2jE9BMgIf/dTa0g7sCYjnE?= =?utf-8?q?DXEPakzoC6aVe53Xnz1NW/XQVZblebzjYlarfxDMIgIfC2dWkkDh2uDQECDk0yLgS?= =?utf-8?q?+/L+j9dz56Hke55rwCRMa8Kmo9PWYUQzUlRs8ZPnaT+0K9guwvim0Hmk+ulAblmJ/?= =?utf-8?q?5kUTkFCMfby91f3kKzp7nWZe/dnEMGhhSeH0mSIbiiMkVRErwh6OoOd4ntd6veGTs?= =?utf-8?q?j6PUyCykNRBRq1z5Nx+yodIkf8DCgai5Y/3msU9CSeF8XM2OX39mMHCtEGk6V7+IY?= =?utf-8?q?JqR/cZp25q/cLJLft2giC0AXHW3gYn5b7bJBETn4P2lRkHKWcnNYlqDrr+gnzynDV?= =?utf-8?q?+mUjcs3+cTz0w9oGrv9W0XPE6P+yxzk4Jayuzx2CZlnppsJimPGu0hB7SGsk85mw5?= =?utf-8?q?7ui6T0vi1AGUutY1x3tWSatACqoQXPI/zBW336O1BK7pKsccQZL0KYr59+nHVev0B?= =?utf-8?q?UoJyF8r0oAWzwYcf2XmD2hW0YpQ1/lBaj4qL5gmmZoV5qzw24EA0eEL+/cPXOrSnZ?= =?utf-8?q?xMLxDOEwGzOKBJM7r/pHLuoSkVZpc8LW0x/Rz+WgwYkBTm6gde1iIdcFtV4/Pc5mV?= =?utf-8?q?/lBk32LKc0TzLy5hzPFKRsKQeo7RA3KvjYHHaq9n3XZbE4xka1/goiM13eCbwvGIA?= =?utf-8?q?NLJ1ebZmFA4Y+d8GDfGFY3Geu/exW/+7LOE37Ga4/EPQjeNSk8xDP3BEitZlzaaQ1?= =?utf-8?q?wsFgMBOc8/b4t0chB2jpyntKTilGcANhq5NXmcQTLCgsG3gBw97sBdDyub+bDMA6d?= =?utf-8?q?5oeyrhACRfeN/uL5eWbyQY9iAdfmoWJrZ1ji5xIBAb4c8o6Z8dCz92YNkYZNrmGQg?= =?utf-8?q?TJEouFY79Yz3uSp+2HA7G8X+3WFEKCMmhHp7Avk3VtbwLQbFm8oKDIPVQvfspa4lc?= =?utf-8?q?S6mkSTBdd7ggBCnf495oOkf4tkmdoTcvFzV33WlM5RyA/NRGfb+YaLOuau/IZJ+3Y?= =?utf-8?q?fPv649H5cp9o?= X-OriginatorOrg: synaptics.com X-MS-Exchange-CrossTenant-Network-Message-Id: 03fa79bf-a815-4fe9-6996-08db7916af10 X-MS-Exchange-CrossTenant-AuthSource: DM6PR03MB5196.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jun 2023 03:04:14.0339 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 335d1fbc-2124-4173-9863-17e7051a2a0e X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jXg0/W37Z/CnkAvsaVVQQ9acFuDGeqpextOGda4dNcDc1IOeBF03+ej6yNuI+YC9tBIW+n1fhZWWFHrT97AEPQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB5552 X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: [FFmpeg-devel] [PATCH] avcodec/v4l2_m2m_dec: resolve resolution change X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: jramirez@baylibre.com Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: QMpY3TFCX7Rk It shouldn't allocate buffer before the resolution change event appeared in decoder setup. And it should not apply to new resolution before the buffer from the previous sequence is dequeued. Change-Id: Id04550b0f17e1501b670a3bcbdd860d5836259bf Signed-off-by: Hsia-Jun(Randy) Li ---  libavcodec/v4l2_context.c | 89 ++++++++++++++++++++++++++-------------  libavcodec/v4l2_context.h |  6 +++  libavcodec/v4l2_m2m_dec.c | 14 +++---  3 files changed, 73 insertions(+), 36 deletions(-)      capture->av_pix_fmt = avctx->pix_fmt; -- 2.17.1 diff --git a/libavcodec/v4l2_context.c b/libavcodec/v4l2_context.c index a40be94690..87771601a9 100644 --- a/libavcodec/v4l2_context.c +++ b/libavcodec/v4l2_context.c @@ -169,30 +169,17 @@ static int v4l2_start_decode(V4L2Context *ctx)  }  /** - * handle resolution change event and end of stream event + * handle resolution change event   * returns 1 if reinit was successful, negative if it failed   * returns 0 if reinit was not executed   */ -static int v4l2_handle_event(V4L2Context *ctx) +static int v4l2_handle_dyn_res_change(V4L2Context *ctx)  {      V4L2m2mContext *s = ctx_to_m2mctx(ctx);      struct v4l2_format cap_fmt = s->capture.format; -    struct v4l2_event evt = { 0 };      int ret; -    ret = ioctl(s->fd, VIDIOC_DQEVENT, &evt); -    if (ret < 0) { -        av_log(logger(ctx), AV_LOG_ERROR, "%s VIDIOC_DQEVENT\n", ctx->name); -        return 0; -    } - -    if (evt.type == V4L2_EVENT_EOS) { -        ctx->done = 1; -        return 0; -    } - -    if (evt.type != V4L2_EVENT_SOURCE_CHANGE) -        return 0; +    cap_fmt.type = s->capture.type;      ret = ioctl(s->fd, VIDIOC_G_FMT, &cap_fmt);      if (ret) { @@ -201,11 +188,13 @@ static int v4l2_handle_event(V4L2Context *ctx)      }      if (v4l2_resolution_changed(&s->capture, &cap_fmt)) { +        s->capture.format.fmt.pix_mp.pixelformat = cap_fmt.fmt.pix_mp.pixelformat;          s->capture.height = v4l2_get_height(&cap_fmt);          s->capture.width = v4l2_get_width(&cap_fmt);          s->capture.sample_aspect_ratio = v4l2_get_sar(&s->capture);      } else {          v4l2_start_decode(ctx); +        ctx->pending_res_change = 0;          return 0;      } @@ -222,10 +211,41 @@ static int v4l2_handle_event(V4L2Context *ctx)          return AVERROR(EINVAL);      } +    ctx->pending_res_change = 0;      /* reinit executed */      return 1;  } +/** + * capture resolution change event and end of stream event + * returns 1 or negative if it failed + * returns 0 if nothing went wrong + */ +static int v4l2_handle_event(V4L2Context *ctx) +{ +    V4L2m2mContext *s = ctx_to_m2mctx(ctx); +    struct v4l2_event evt = { 0 }; +    int ret; + +    ret = ioctl(s->fd, VIDIOC_DQEVENT, &evt); +    if (ret < 0) { +        av_log(logger(ctx), AV_LOG_ERROR, "%s VIDIOC_DQEVENT\n", ctx->name); +        return errno; +    } + +    if (evt.type == V4L2_EVENT_EOS) { +        ctx->done = 1; +        return 0; +    } + +    if (evt.type != V4L2_EVENT_SOURCE_CHANGE) +        return AVERROR(EINVAL); + +    ctx->pending_res_change = 1; + +    return 0; +} +  static int v4l2_stop_decode(V4L2Context *ctx)  {      struct v4l2_decoder_cmd cmd = { @@ -342,16 +362,19 @@ start:      /* 1. handle resolution changes */      if (pfd.revents & POLLPRI) {          ret = v4l2_handle_event(ctx); -        if (ret < 0) { -            /* if re-init failed, abort */ -            ctx->done = 1; -            return NULL; -        }          if (ret) { -            /* if re-init was successful drop the buffer (if there was one) -             * since we had to reconfigure capture (unmap all buffers) -             */ +            /* if event handler failed, abort */ +            ctx->done = 1;              return NULL; +        } else if (!V4L2_TYPE_IS_OUTPUT(ctx->type)) { +            if (!ctx->streamon) +                ret = v4l2_handle_dyn_res_change(ctx); +                if (ret == 1) +                    return NULL; +        } else { +            /* Poll the device again, we want the buffer with the flag +             * that answer to the event */ +            return v4l2_dequeue_v4l2buf(ctx, timeout);          }      } @@ -391,17 +414,23 @@ dequeue:              return NULL;          } -        if (ctx_to_m2mctx(ctx)->draining && !V4L2_TYPE_IS_OUTPUT(ctx->type)) { +        if (!V4L2_TYPE_IS_OUTPUT(ctx->type)) {              int bytesused = V4L2_TYPE_IS_MULTIPLANAR(buf.type) ?                              buf.m.planes[0].bytesused : buf.bytesused; + +#ifdef V4L2_BUF_FLAG_LAST +            if (buf.flags & V4L2_BUF_FLAG_LAST) { +                    if (ctx_to_m2mctx(ctx)->draining) +                        ctx->done = 1; +                    if (ctx->pending_res_change) +                        ret = v4l2_handle_dyn_res_change(ctx); +            } +#endif              if (bytesused == 0) { -                ctx->done = 1; +                if (ctx_to_m2mctx(ctx)->draining) +                    ctx->done = 1;                  return NULL;              } -#ifdef V4L2_BUF_FLAG_LAST -            if (buf.flags & V4L2_BUF_FLAG_LAST) -                ctx->done = 1; -#endif          }          avbuf = &ctx->buffers[buf.index]; diff --git a/libavcodec/v4l2_context.h b/libavcodec/v4l2_context.h index 6f7460c89a..b30457c5ad 100644 --- a/libavcodec/v4l2_context.h +++ b/libavcodec/v4l2_context.h @@ -87,6 +87,12 @@ typedef struct V4L2Context {       */      int streamon; +    /** +     * Pending resolution change event to handle. Only context for CAPTURE +     * queue could set this flag. +     */ +    int pending_res_change; +      /**       *  Either no more buffers available or an unrecoverable error was notified       *  by the V4L2 kernel driver: once set the context has to be exited. diff --git a/libavcodec/v4l2_m2m_dec.c b/libavcodec/v4l2_m2m_dec.c index 4944d08511..3c38fde700 100644 --- a/libavcodec/v4l2_m2m_dec.c +++ b/libavcodec/v4l2_m2m_dec.c @@ -40,6 +40,7 @@ static int v4l2_try_start(AVCodecContext *avctx)      V4L2Context *const capture = &s->capture;      V4L2Context *const output = &s->output;      struct v4l2_selection selection = { 0 }; +    AVFrame frame;      int ret;      /* 1. start the output process */ @@ -54,15 +55,16 @@ static int v4l2_try_start(AVCodecContext *avctx)      if (capture->streamon)          return 0; -    /* 2. get the capture format */ -    capture->format.type = capture->type; -    ret = ioctl(s->fd, VIDIOC_G_FMT, &capture->format); +    /* TODO wait event here */ +    ret = ff_v4l2_context_dequeue_frame(capture, &frame, 10);      if (ret) { -        av_log(avctx, AV_LOG_WARNING, "VIDIOC_G_FMT ioctl\n"); -        return ret; +        if (ret == AVERROR(EAGAIN)) +            ret = 0; +        else +            return ret;      } -    /* 2.1 update the AVCodecContext */ +    /* 2 update the AVCodecContext */      avctx->pix_fmt = ff_v4l2_format_v4l2_to_avfmt(capture->format.fmt.pix_mp.pixelformat, AV_CODEC_ID_RAWVIDEO);