From patchwork Thu May 4 22:02:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41474 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp12913pzb; Thu, 4 May 2023 14:07:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5wzfhym73dxKqL669JGxx70bICAceJYPARlBnjzgVsCmowtJD75IYgT1y7es31hwxU9z4o X-Received: by 2002:a05:6402:1504:b0:50b:cf07:ad0 with SMTP id f4-20020a056402150400b0050bcf070ad0mr2362393edw.37.1683234473591; Thu, 04 May 2023 14:07:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683234473; cv=none; d=google.com; s=arc-20160816; b=i+VNfPHkThzoHgW+v5zMEAMF+s2r4UPjnTnR8MqTkKaUsnj5yB+Gs7vKm5V/zFKjat zudYCD7fAhZrtvlj97L8wD+XJ+qVWKslylnFLubQMkkISZkTuaEPLOFQtbYpBpmQ+so9 fdtaunhlmuaWWQ3pe9D9g5byV118mGIBhwfOINIxEpRFdUiP1wpWwyuDWuFYDZcn3opG 0li/5cC0kiYjHocwC7K38ZVk15XAtmZ94RkU7mVj3S1xqx5MWyx7rHLi617dfvuGNIeH uIx2zDsVfSjMSVSO6JmVkbiaj2ELZJET33TyrkrWPAb5k3lWkzA3pIVi+LfI2x5qmAAN nb8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=uVIMZrlwzOov84q7mWtcL5k6rqUaJvoojSkCsz4keoI=; b=N7xEgkWQrwevpUOtxbyhCj2uiFR7r2cPIAsmLsefciU6cIFBj86nAdt4/WFz2Z1qHL b+hJHQxSHewxQvNewM8W22z4rKhMTP5Ldnsd5co5DEc8OLSIsJsPyfS0OiTvM9z8MnYy 7vMw46S1x4b5GgFSq0BKQsZ4Gv+q9Z7RwLz9ZxIn3lKJB/by5b5N1vs5UozzXyqxcuqG aXBtfnq8EQJejFJjibkNveRF3Ii5Ihd1M15mAMuxc0TOkjcDD8E+RRGV0cvG7QzjgBPF XIsZodkBnny7O177GXNvoqJBsWCmIdRLcgBHOPP7HK6DZ6xahLTyJND8UJxngCvNpn5H cuAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=B4JBCh7p; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d11-20020a50fb0b000000b00504ae1b658dsi3601374edq.490.2023.05.04.14.07.41; Thu, 04 May 2023 14:07:53 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=B4JBCh7p; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 260FB68C12D; Fri, 5 May 2023 00:06:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B149268C102 for ; Fri, 5 May 2023 00:06:40 +0300 (EEST) Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-74e07c2ee30so48656485a.1 for ; Thu, 04 May 2023 14:06:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683234399; x=1685826399; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=dHdpTVoMhyj4hLIW3Nkvvd7cp7Pxmqk2Fi3QmMqSDvk=; b=B4JBCh7phvuM2T7b2egmibrDoFO0apdJEjUJWyfWRqRJNG2hYMayxKt6ZtcGd+M515 Lqa8zjzpGdcgXjsSzcvLH7Qy7ZzBnNlwg59rKs/xvcEUwvgRPPJ9DLNjDKMaFgKdGIoU 9WeyetxBpWhJBdgpDuPAXI0JJK+wtnumaxIfzHoyRhG/2ETQ2p627R90lGz9/HCA83bl 58BK+NrA3xhoIelrL+KpFXN+0uuu66SQsxWFgyNSGX+O3xtzOgMUd7CYBBGyrTqYYU1B RxNMdTvqvBwPzDETcy5GASi0JGbuaIV5oDqyWEhy1TC/utbZdozMl0dxU2qo1Rho1j7C jqdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683234399; x=1685826399; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dHdpTVoMhyj4hLIW3Nkvvd7cp7Pxmqk2Fi3QmMqSDvk=; b=kzvOGti6+2c/xglXo3GQVtBIBVYD1ec1fVQAmlmSlX0i7cv2CD0T/aMQWuyOEK/eHE ktRyeNFrtlfHqsWGvxR0kBB6huNren+wPfpAetDB06Z5ubuvH/4KkxJjdjhBpEQs0Egu jdKUEGRzgoiIMzBEFlOrER2F9ROcZZqfKvq7WqqhL/RZ382ymiRxUH911Fs0GfidSa8S EEMe4DO7ItEMaGhU0Pr6J6zaCe93ospdPmrN0NjyDB31KFzRhTuhm0q7zF4WZrBxj05R HEcRR71CoLABCwQYfl9syNbPbDX36SNbsMDv1ynYDELR9ho2aIIu/DPkZQu6DVR5+eTE 8QfQ== X-Gm-Message-State: AC+VfDyZ/M+QyG0s4wa85HmQHsCws6M1Qzrw66xozWKho+W6b24RJeo4 npv3mYUagz8tTHwdsEhPxBRm25ulEPZCK810WlQ= X-Received: by 2002:a05:6214:2689:b0:5cc:277c:b5e with SMTP id gm9-20020a056214268900b005cc277c0b5emr17533780qvb.33.1683234398935; Thu, 04 May 2023 14:06:38 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id 4-20020a05620a070400b0074e21c3bc8asm67996qkc.126.2023.05.04.14.06.38 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 May 2023 14:06:38 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 May 2023 18:02:18 -0400 Message-Id: <1683237740-32743-5-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v5 4/6] tinterlace: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: JCL3ctCYjR6E Because the interlacing filter halves the effective framerate, we need to ensure that no CEA-708 data is lost as frames are merged. Make use of the new ccfifo mechanism to ensure that caption data is properly preserved as frames pass through the filter. Thanks to Thomas Mundt for review and noticing a couple of missed codepaths for injection on output. Thanks to Lance Wang for pointing out a memory leak. Signed-off-by: Devin Heitmueller --- libavfilter/tinterlace.h | 2 ++ libavfilter/vf_tinterlace.c | 11 +++++++++++ 2 files changed, 13 insertions(+) diff --git a/libavfilter/tinterlace.h b/libavfilter/tinterlace.h index 37b6c10..9f5ce7e 100644 --- a/libavfilter/tinterlace.h +++ b/libavfilter/tinterlace.h @@ -32,6 +32,7 @@ #include "libavutil/pixdesc.h" #include "drawutils.h" #include "avfilter.h" +#include "ccfifo.h" #define TINTERLACE_FLAG_VLPF 01 #define TINTERLACE_FLAG_CVLPF 2 @@ -77,6 +78,7 @@ typedef struct TInterlaceContext { const AVPixFmtDescriptor *csp; void (*lowpass_line)(uint8_t *dstp, ptrdiff_t width, const uint8_t *srcp, ptrdiff_t mref, ptrdiff_t pref, int clip_max); + AVCCFifo *cc_fifo; } TInterlaceContext; void ff_tinterlace_init_x86(TInterlaceContext *interlace); diff --git a/libavfilter/vf_tinterlace.c b/libavfilter/vf_tinterlace.c index 0326292..8716a94 100644 --- a/libavfilter/vf_tinterlace.c +++ b/libavfilter/vf_tinterlace.c @@ -203,6 +203,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&tinterlace->next); av_freep(&tinterlace->black_data[0][0]); av_freep(&tinterlace->black_data[1][0]); + ff_ccfifo_freep(&tinterlace->cc_fifo); } static int config_out_props(AVFilterLink *outlink) @@ -291,6 +292,11 @@ static int config_out_props(AVFilterLink *outlink) #endif } + if (!(tinterlace->cc_fifo = ff_ccfifo_alloc(outlink->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } + av_log(ctx, AV_LOG_VERBOSE, "mode:%d filter:%s h:%d -> h:%d\n", tinterlace->mode, (tinterlace->flags & TINTERLACE_FLAG_CVLPF) ? "complex" : (tinterlace->flags & TINTERLACE_FLAG_VLPF) ? "linear" : "off", @@ -375,6 +381,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) tinterlace->cur = tinterlace->next; tinterlace->next = picref; + ff_ccfifo_extract(tinterlace->cc_fifo, picref); + cur = tinterlace->cur; next = tinterlace->next; /* we need at least two frames */ @@ -451,6 +459,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) if (!out) return AVERROR(ENOMEM); out->pts /= 2; // adjust pts to new framerate + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret; } @@ -486,6 +495,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = cur->pts*2; out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); if ((ret = ff_filter_frame(outlink, out)) < 0) return ret; @@ -521,6 +531,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->pts = av_rescale_q(out->pts, tinterlace->preout_time_base, outlink->time_base); out->duration = av_rescale_q(1, av_inv_q(outlink->frame_rate), outlink->time_base); + ff_ccfifo_inject(tinterlace->cc_fifo, out); ret = ff_filter_frame(outlink, out); return ret;