From patchwork Sat Mar 25 19:15:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 40820 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a30:b0:df:834d:2c1a with SMTP id t48csp225337pzh; Sat, 25 Mar 2023 12:16:23 -0700 (PDT) X-Google-Smtp-Source: AKy350aYjaXmh1WmdLavLuX+jpJdwL0xepuMZljKNYzTSeUFXp7KgA5nvj3CIJU57Y4bY2IeE8hI X-Received: by 2002:aa7:cfce:0:b0:502:2386:d22 with SMTP id r14-20020aa7cfce000000b0050223860d22mr5230436edy.12.1679771782912; Sat, 25 Mar 2023 12:16:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679771782; cv=none; d=google.com; s=arc-20160816; b=N3EK24+gb2Db+MV7AAwKc6nrmTRyKIkoQgOaZiNWTqjaaKkJE6cdDFAckipvJSObOY czaD4nWbpQkLLACoOStcQpR8WtfHjIGLbAICkjFeddpgkH3nWs7xfS74GqtEoRpHxMdg ZdKahuPKZOw1Qj/Pm9M7KsqtR0AnZyVo/Zf7sc3pTv/t/oNlewiXCj4Sha4VO2loQxbN DmHKkJRHO+hoQRpObfVTP/d/C42rBN5Z/3CttnbWLGhzBeDh4c+DllptNReOo1UJEVi4 vqrAV7L/fwr0qTIg082YkIjYD9VzJHPAsWLpaqPEd5hS2IeLGqjS3Y/joMOri2vRUq2h T1Rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=lxyw4XfTCedPFmJ32cT/I2XBX8ccwmVpMOmw5vCibPo=; b=p9/HQk+ddXbE7qWTF4kRnm/vMjKlYfrACh6Pi2Hu/8KzURn08mJB8vVBDlCVqzcLP/ 3ZH7IcpV+Z1SpW6j7lDGnRFJZ725AenmuugzPtmTH6eTu+gAvd/pXJYOluPbAtWHUJuz ZRzTdrdmiqRZRrueIQENlUR7p2UPi31uWywG6UKFdlQ1YxMMtMHvaylTJJDBAGEmqVSV zMFgvJOfhQL4en+2ezJNVKw2amE7w/Xt/TUlgJrQqyOWk7valbij72o3EPzTIRPRN8eH WMf26fa6cj9BtSj67EGQ9e6Vsw+beXWS5zhkxFoCITpTCC4kGQjmkxFv9d6fGL0TBQlh ea5Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id bc18-20020a056402205200b004ab9fed6acesi23961609edb.384.2023.03.25.12.16.22; Sat, 25 Mar 2023 12:16:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4F85D68C976; Sat, 25 Mar 2023 21:16:10 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7A4C668C810 for ; Sat, 25 Mar 2023 21:16:02 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 37538240178 for ; Sat, 25 Mar 2023 20:16:02 +0100 (CET) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id cKpPRRGGjXvn for ; Sat, 25 Mar 2023 20:16:01 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id DF2EE2404F8 for ; Sat, 25 Mar 2023 20:16:00 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 7D0BC3A0404 for ; Sat, 25 Mar 2023 20:15:54 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 25 Mar 2023 20:15:10 +0100 Message-Id: <20230325191529.10578-4-anton@khirnov.net> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230325191529.10578-1-anton@khirnov.net> References: <20230325191529.10578-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 04/23] fftools/sync_queue: document overall design X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: lbmVt1/QSa0g --- fftools/sync_queue.c | 35 +++++++++++++++++++++++++++++++++++ fftools/sync_queue.h | 5 +++++ 2 files changed, 40 insertions(+) diff --git a/fftools/sync_queue.c b/fftools/sync_queue.c index 7c348af300..0aee4ef5ff 100644 --- a/fftools/sync_queue.c +++ b/fftools/sync_queue.c @@ -28,6 +28,41 @@ #include "objpool.h" #include "sync_queue.h" +/* + * How this works: + * -------------- + * time: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 + * ------------------------------------------------------------------- + * | | | | | | | | | | | | | | + * | ┌───┐┌────────┐┌───┐┌─────────────┐ + * stream 0| │d=1││ d=2 ││d=1││ d=3 │ + * | └───┘└────────┘└───┘└─────────────┘ + * ┌───┐ ┌───────────────────────┐ + * stream 1│d=1│ │ d=5 │ + * └───┘ └───────────────────────┘ + * | ┌───┐┌───┐┌───┐┌───┐ + * stream 2| │d=1││d=1││d=1││d=1│ <- stream 2 is the head stream of the queue + * | └───┘└───┘└───┘└───┘ + * ^ ^ + * [stream 2 tail] [stream 2 head] + * + * We have N streams (N=3 in the diagram), each stream is a FIFO. The *tail* of + * each FIFO is the frame with smallest end time, the *head* is the frame with + * the largest end time. Frames submitted to the queue with sq_send() are placed + * after the head, frames returned to the caller with sq_receive() are taken + * from the tail. + * + * The head stream of the whole queue (SyncQueue.head_stream) is the limiting + * stream with the *smallest* head timestamp, i.e. the stream whose source lags + * furthest behind all other streams. It determines which frames can be output + * from the queue. + * + * In the diagram, the head stream is 2, because it head time is t=5, while + * streams 0 and 1 end at t=8 and t=9 respectively. All frames that _end_ at + * or before t=5 can be output, i.e. the first 3 frames from stream 0, first + * frame from stream 1, and all 4 frames from stream 2. + */ + typedef struct SyncQueueStream { AVFifo *fifo; AVRational tb; diff --git a/fftools/sync_queue.h b/fftools/sync_queue.h index 17d282c38c..9659ee5d50 100644 --- a/fftools/sync_queue.h +++ b/fftools/sync_queue.h @@ -38,6 +38,11 @@ typedef union SyncQueueFrame { #define SQFRAME(frame) ((SyncQueueFrame){ .f = (frame) }) #define SQPKT(pkt) ((SyncQueueFrame){ .p = (pkt) }) +/** + * A sync queue provides timestamp synchronization between multiple streams. + * Some of these streams are marked as "limiting", then the queue ensures no + * stream gets ahead of any of the limiting streams. + */ typedef struct SyncQueue SyncQueue; /**