From patchwork Fri May 5 19:09:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41499 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp685956pzb; Fri, 5 May 2023 11:14:28 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6o7UsadkhOtAdF/M2y4deSRt1SvWtCOFPhdSdly1SlB9KJue5REMQU4ZbN01LSNSLgTdF1 X-Received: by 2002:a05:6402:4d9:b0:50b:c8bf:67b1 with SMTP id n25-20020a05640204d900b0050bc8bf67b1mr2275832edw.10.1683310468565; Fri, 05 May 2023 11:14:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683310468; cv=none; d=google.com; s=arc-20160816; b=cLRSmHySgSqbOTkk7qsSO1F7RkDT2ZIyrW51f3vlnHiOJEidSTqYjsfOEQVe8VBvkb PkmT44XjiE8i1HEc1zqBuxR7ZLHZNd2UEdMvWZe0ZYzfRgox1i2DNQkI1oE9PGYvwGj8 IbWGJhAqtLSk5V/JjKOawJhC49gBWZO/0o5hFEmfmCSk19lzRK2/ocXZh3CA7pxoo3br Nrrq49v6HsAVsUJvkDeykgCtEKJtBqen0lgV2G05zonJWn4krMCUDwVabNXQcwGoRB5s wRxEU5e9xmr77I/DDFgyPrkaeGTgJRjPZrSpGtFw26RMB0SXaEmc88IOFOFDX76WbOfS ZKHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=uVIMZrlwzOov84q7mWtcL5k6rqUaJvoojSkCsz4keoI=; b=M+w0s1rtosBG3Ydbbd/DM+LSxSDaxzW6nVYo/GsWO54gzUB45Y4phgqz3fMpwgcWkt cJwyHAk6+XAswEwIcfYb8xi99WNIpWNmuXqcfQkP+7mdELW2bbYUgPjl1idNY5Xxx5eM O2E7N8Po6MFZiYhcqjCpF20FTQr7NmGRYOSUSA9fEzZQ4LgAKGWQX3xiNuuxC5e1SfxU xyM0xcfiPf8PXYY9LyyRYO1Jr8jcxymi/tadC0KGMYSmjIzSpm8h0ksW5bMWFboukjYd E+v3RwnZZVN0Ya6UPfPAY0muQlxSzrtyMd2b4l1C+vpit3CinAGVeA9oM03/dKS2UNhl 54GA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=CJdGHbxu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b17-20020aa7d491000000b005089ebf6cb7si4927324edr.252.2023.05.05.11.14.04; Fri, 05 May 2023 11:14:28 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=CJdGHbxu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 915AC68C15F; Fri, 5 May 2023 21:13:32 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6957E68C176 for ; Fri, 5 May 2023 21:13:24 +0300 (EEST) Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-74e12e93384so105054085a.3 for ; Fri, 05 May 2023 11:13:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683310403; x=1685902403; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=dHdpTVoMhyj4hLIW3Nkvvd7cp7Pxmqk2Fi3QmMqSDvk=; b=CJdGHbxueQ2w0ZjNAADQFNPZJNx3fNjcYAL5siPWFcKjSFOMleN68ciNXxgq3zx7Jy Uqj45J1BIgfGEJGWu3Sr7NEaveMDvRrppdbb4EOcMc214AK48Ia11MeJmmRXINWDW+Qr epM4XmLuyvxqHYUCBPcBifdtE+g0KPl+CVDPnfJJSrZjLeGLzXOoiEzskWEwqArvlh9p vcYW5ejAsdjVuQKNnvZivk6OaUXmnEe35JQ4gjEDThMO1pLpxy1DCr0hZ4+1IM8iFbL6 ewqbzEg191BQy1UuYoJIsKKpfsVvuPMo4oMcs4hpGpG6bVKxNMUuTRgtXXMzysTWmuPW Ea0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683310403; x=1685902403; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dHdpTVoMhyj4hLIW3Nkvvd7cp7Pxmqk2Fi3QmMqSDvk=; b=IdHrLcw67amHmnc2i+HqE2dr0CMVjMg8oGkGFnpLLfVGtL88+tHxTSxLyIj7RAiYDA WbuyPSjOr6LOuwz3+8tUT41ZuJCDSbZljFNfEVzpIHLIZkmZ9kD617/cZqgdG18nH0TS d1xAmvEAr7/TER14ra7Tg6Vbqqvdt0swjSiw0Xbijhb0o/LGIltMWeOCF/8YZLcKdOnD gHv/lmrlesLxfyUankfJ3JwmHPyKz6irb3hll+WwsfFuXhQEFjBglmzzVMCPMWwdRMoN 6yS4YmWqcSpQbr8ZeGSKLs2p+qdwdzbFXDfUYEvsNQbD/9ztSd10d4YVqlB3b3mPJOnS O9EA== X-Gm-Message-State: AC+VfDwaUevZcYtZ43q3RnX1ROajIvR2ehT2cWyqKXrb0Vts5EGNkWnE extXIGQDNUbDdT5Y3/1tBlcfeJtzT6re8JervvU= X-Received: by 2002:a05:6214:301a:b0:5e8:11a0:ae17 with SMTP id ke26-20020a056214301a00b005e811a0ae17mr2745084qvb.43.1683310402985; Fri, 05 May 2023 11:13:22 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id d1-20020a05620a166100b0074e26158cf2sm747289qko.93.2023.05.05.11.13.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2023 11:13:22 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 May 2023 15:09:05 -0400 Message-Id: <1683313747-3775-5-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683313747-3775-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v6 4/6] tinterlace: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: UnLiQz8HhsYh Because the interlacing filter halves the effective framerate, we need to ensure that no CEA-708 data is lost as frames are merged. Make use of the new ccfifo mechanism to ensure that caption data is properly preserved as frames pass through the filter. Thanks to Thomas Mundt for review and noticing a couple of missed codepaths for injection on output. Thanks to Lance Wang for pointing out a memory leak. Signed-off-by: Devin Heitmueller --- libavfilter/tinterlace.h | 2 ++ libavfilter/vf_tinterlace.c | 11 +++++++++++ 2 files changed, 13 insertions(+) diff --git a/libavfilter/tinterlace.h b/libavfilter/tinterlace.h index 37b6c10..9f5ce7e 100644 --- a/libavfilter/tinterlace.h +++ b/libavfilter/tinterlace.h @@ -32,6 +32,7 @@ #include "libavutil/pixdesc.h" #include "drawutils.h" #include "avfilter.h" +#include "ccfifo.h" #define TINTERLACE_FLAG_VLPF 01 #define TINTERLACE_FLAG_CVLPF 2 @@ -77,6 +78,7 @@ typedef struct TInterlaceContext { const AVPixFmtDescriptor *csp; void (*lowpass_line)(uint8_t *dstp, ptrdiff_t width, const uint8_t *srcp, ptrdiff_t mref, ptrdiff_t pref, int clip_max); + AVCCFifo *cc_fifo; } TInterlaceContext; void ff_tinterlace_init_x86(TInterlaceContext *interlace); diff --git a/libavfilter/vf_tinterlace.c b/libavfilter/vf_tinterlace.c index 0326292..8716a94 100644 --- a/libavfilter/vf_tinterlace.c +++ b/libavfilter/vf_tinterlace.c @@ -203,6 +203,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&tinterlace->next); av_freep(&tinterlace->black_data[0][0]); av_freep(&tinterlace->black_data[1][0]); + ff_ccfifo_freep(&tinterlace->cc_fifo); } static int config_out_props(AVFilterLink *outlink) @@ -291,6 +292,11 @@ static int config_out_props(AVFilterLink *outlink) #endif } + if (!(tinterlace->cc_fifo = ff_ccfifo_alloc(outlink->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } + av_log(ctx, AV_LOG_VERBOSE, "mode:%d filter:%s h:%d -> h:%d\n", tinterlace->mode, (tinterlace->flags & TINTERLACE_FLAG_CVLPF) ? "complex" : (tinterlace->flags & TINTERLACE_FLAG_VLPF) ? "linear" : "off", @@ -375,6 +381,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) tinterlace->cur = tinterlace->next; tinterlace->next = picref; + ff_ccfifo_extract(tinterlace->cc_fifo, picref); + cur = tinterlace->cur; next = tinterlace->next; /* we need at least two frames */ @@ -451,6 +459,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); out->pts /= 2; // adjust pts to new framerate + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; } @@ -486,6 +495,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = cur->pts*2; out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); if ((ret = ff_filter_frame(outlink, out)) < 0) return ret; @@ -521,6 +531,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); out->duration = av_rescale_q(1, av_inv_q(outlink->frame_rate), outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret;