From patchwork Thu May 4 22:02:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 41473 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp12778pzb; Thu, 4 May 2023 14:07:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7NaghDZeKv1w/QLsw1hYc5RcIx4uLu8lbotZ8JS+JxG5c6wSRxhi3SaXff9fzTSiBlgdMP X-Received: by 2002:a17:907:868d:b0:948:d1af:3a11 with SMTP id qa13-20020a170907868d00b00948d1af3a11mr169608ejc.50.1683234457530; Thu, 04 May 2023 14:07:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683234457; cv=none; d=google.com; s=arc-20160816; b=F0OFKlBwXsD6MOLWqKiyNwzky0mj4Jd+0VeLPD+I5JLD2McrCkbHJMgzXrcA4LeIja TvAsbWhTRszSfAHzqF6nbXjIH+OGKlb7XVF5wbQvLA2LspvIbzPugSs/QmNnUVe1cY8q PYGVhoGlMCVpRDZ+no0NP4p8TekeZ914wzOnqPW6/ECdO5RrzrLZZgrKimPtXboyg2D6 LUUObEuG8hGRYvSU2V05ogbm+8diMsAmGkc0rlWAwTSmRcOcRHvSf5/K4eSNi0lez8r6 dYULbBs8FwkHOwwMS1ysvC+P/1j5k8PbFxPT0WN7znH+HdPwsueaC0EBJ/+5DQyywoFt fFoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=bcDlT4bS2F//ovCPqsEsS4Ld/G2EA1Xjznx7V/m2BfU=; b=t1pCHoXguyi15CFAZCJH+G+Kvc6foabClvUMr9LNk4EAUZ5auBDP9n76UkEOZn4xUp dwsR52Xz8bISIyWlbRPmRoPwTxZksh1csAjmv+uDAD56s/K7G/YmhWUUsit5MOtWjbBp qpaY8QabWiuDK0ZCL25iIXUl2uppTHDeapZBJ/7hXJiWT5EcWp4UNgVtF8hPra5gtnEh HH72scx4j8QfIGJFW298OimEvm74kVdZ2a7DWpKW6rg7B/yFcbLRDjFwbSlaCHBEN65x 95Oq1Y9fxcTJe8KRUzgdinQtRTgySPF0MlqBKmidz9yNkB14ppRxJgbuVa7x3ccTdRG7 npvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=yfucjDtL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id os2-20020a170906af6200b0094f6bd17e83si35454ejb.449.2023.05.04.14.07.31; Thu, 04 May 2023 14:07:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=yfucjDtL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3AAD968C126; Fri, 5 May 2023 00:06:48 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f179.google.com (mail-qt1-f179.google.com [209.85.160.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BD99D68B36D for ; Fri, 5 May 2023 00:06:39 +0300 (EEST) Received: by mail-qt1-f179.google.com with SMTP id d75a77b69052e-3f364b1149eso4626801cf.0 for ; Thu, 04 May 2023 14:06:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1683234398; x=1685826398; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=1icKU5ZG9Ueq6yYspjCh0KLlMAJMos3aHnhNpRwpoGA=; b=yfucjDtLCQR/kKwtGM/wXpFQoRLDrc8GDVDyKt8scsKhRLxvO2OBkYiCFRLgetFIXl S7RFB1ZOGbH9LCxtWcOKoaE4+eTbqYvwleae5k2lBY1ANggk6a/8CbESp/7Sous9AEtH sfmq5nv7dpeB8beimY+cNpA66UMwGSTKmbK00HuiJrkXm4uqK8Hyq7Kbvk/vDCnYNERj B3pg+j9ePXXmm5hLBFA+cQDYtS6CERlWJxzjAreMK98KWoo87VfMb6b0Bunjgo/f944B yAAe5f7iEBi8+oOui1w5k1esRNZC8Sn3fYYEm1aEBho4ixEVxt+wUlTBWM1fMDi6RLNs /PbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683234398; x=1685826398; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1icKU5ZG9Ueq6yYspjCh0KLlMAJMos3aHnhNpRwpoGA=; b=NWwidrEOmILOPGo7y1v0iRPo2GF8na6JpG7QmUHwI0chXB4LgTyPLi+sIhukUPQK54 SxUXUBKI7vYmZOdT380gnZbWXfu4tm6U1PjSxnE88y3LXdpGvnAJEvXs8QyWjXBNbFAD U3yekgOwQCEbw4rWx1qVKswlgVtNaPwcuodeqK4PwUFnaDk/XZRhQiJ84MMW8vbR4TY+ XgGl4tn/8DjBjiMgqY/WTa1BIiGojaTUTt6jFaJsAW6Spe6FXeYjU8pJ/gIPp6jbIcq5 95chdjPn5R/caAFnNXNKx+hTtDFXL4sWRejowP2UscIR2sg0qKnUHDwe/gyPEKoF2iI9 w4yA== X-Gm-Message-State: AC+VfDxZoz3vq4RKi4WkK1JipIBc3Gjzp+vH1n0Aeu7ahhWOfnfaofhp FI0Ryug4I+DZpMYY05agaQXdyJzvNMv+Du73Tvk= X-Received: by 2002:a05:622a:58c:b0:3ed:a2f5:f27c with SMTP id c12-20020a05622a058c00b003eda2f5f27cmr6342989qtb.12.1683234397629; Thu, 04 May 2023 14:06:37 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id 4-20020a05620a070400b0074e21c3bc8asm67996qkc.126.2023.05.04.14.06.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 May 2023 14:06:37 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 May 2023 18:02:17 -0400 Message-Id: <1683237740-32743-4-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> References: <1683237740-32743-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v5 3/6] yadif: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PP7YbSlo5rxs Various deinterlacing modes have the effect of doubling the framerate, and we need to ensure that the caption data isn't duplicated (or else you get double captions on-screen). Use the new ccfifo mechanism for yadif (and yadif_cuda and bwdif since they use the same yadif core) so that CEA-708 data is properly preserved through this filter. Signed-off-by: Devin Heitmueller --- libavfilter/vf_bwdif.c | 8 ++++++++ libavfilter/vf_yadif.c | 8 ++++++++ libavfilter/vf_yadif_cuda.c | 9 +++++++++ libavfilter/yadif.h | 2 ++ libavfilter/yadif_common.c | 5 +++++ 5 files changed, 32 insertions(+) diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index 34e8c5e..51e1e02 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -297,6 +297,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + ff_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -332,6 +333,13 @@ static int config_props(AVFilterLink *link) if(yadif->mode&1) link->frame_rate = av_mul_q(link->src->inputs[0]->frame_rate, (AVRational){2,1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(yadif->cc_fifo = ff_ccfifo_alloc(link->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } if (link->w < 3 || link->h < 4) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n"); diff --git a/libavfilter/vf_yadif.c b/libavfilter/vf_yadif.c index 1be02de..f77f811 100644 --- a/libavfilter/vf_yadif.c +++ b/libavfilter/vf_yadif.c @@ -261,6 +261,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + ff_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -293,6 +294,13 @@ static int config_output(AVFilterLink *outlink) if(s->mode & 1) outlink->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = ff_ccfifo_alloc(outlink->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + return AVERROR(ENOMEM); + } if (outlink->w < 3 || outlink->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/vf_yadif_cuda.c b/libavfilter/vf_yadif_cuda.c index 685b8a2..f3f0b56 100644 --- a/libavfilter/vf_yadif_cuda.c +++ b/libavfilter/vf_yadif_cuda.c @@ -205,6 +205,7 @@ static av_cold void deint_cuda_uninit(AVFilterContext *ctx) av_frame_free(&y->prev); av_frame_free(&y->cur); av_frame_free(&y->next); + ff_cc_fifo_freep(&y->cc_fifo); av_buffer_unref(&s->device_ref); s->hwctx = NULL; @@ -291,6 +292,14 @@ static int config_output(AVFilterLink *link) if(y->mode & 1) link->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = ff_cc_fifo_alloc(link->frame_rate, ctx))) { + av_log(ctx, AV_LOG_ERROR, "Failure to setup CC FIFO queue\n"); + ret = AVERROR(ENOMEM); + goto exit; + } if (link->w < 3 || link->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/yadif.h b/libavfilter/yadif.h index c928911..1077576 100644 --- a/libavfilter/yadif.h +++ b/libavfilter/yadif.h @@ -22,6 +22,7 @@ #include "libavutil/opt.h" #include "libavutil/pixdesc.h" #include "avfilter.h" +#include "ccfifo.h" enum YADIFMode { YADIF_MODE_SEND_FRAME = 0, ///< send 1 frame for each frame @@ -76,6 +77,7 @@ typedef struct YADIFContext { int eof; uint8_t *temp_line; int temp_line_size; + AVCCFifo *cc_fifo; /* * An algorithm that treats first and/or last fields in a sequence diff --git a/libavfilter/yadif_common.c b/libavfilter/yadif_common.c index a10cf7a..676853e 100644 --- a/libavfilter/yadif_common.c +++ b/libavfilter/yadif_common.c @@ -60,6 +60,8 @@ static int return_frame(AVFilterContext *ctx, int is_second) yadif->out->pts = AV_NOPTS_VALUE; } } + + ff_ccfifo_inject(yadif->cc_fifo, yadif->out); ret = ff_filter_frame(ctx->outputs[0], yadif->out); yadif->frame_pending = (yadif->mode&1) && !is_second; @@ -96,6 +98,8 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) av_assert0(frame); + ff_ccfifo_extract(yadif->cc_fifo, frame); + if (yadif->frame_pending) return_frame(ctx, 1); @@ -137,6 +141,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) if (!yadif->out) return AVERROR(ENOMEM); + ff_ccfifo_inject(yadif->cc_fifo, yadif->out); av_frame_free(&yadif->prev); if (yadif->out->pts != AV_NOPTS_VALUE) yadif->out->pts *= 2;