From patchwork Fri Mar 17 20:09:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40716 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp15331pzb; Fri, 17 Mar 2023 13:11:11 -0700 (PDT) X-Google-Smtp-Source: AK7set+PJOR0dJtUhimUAey6LJbY9XuAFCk5x7q8gsvpV+SEkMXjCtuL9S8KpqTw7ppkIQZ49NDf X-Received: by 2002:a17:906:36d4:b0:932:d2bb:507 with SMTP id b20-20020a17090636d400b00932d2bb0507mr688005ejc.24.1679083870944; Fri, 17 Mar 2023 13:11:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679083870; cv=none; d=google.com; s=arc-20160816; b=hyYHS314PD6bw9i5TvzfdbM6tNzw0Xe2NbTXhesiZoFg83YdHdPlc2b2MaOjpMLM7V nMwNEQz7FBHt0MX2aD+a0Wyo2AeI5RcCzZ2cVEw03dxJqI/zodr8+nmKwGENBMu+RBKU JmyEO73iCW/1D/VS2CQKNkZrxbSdsF1Weh8XQsXQsKnQNkTUwhzRGJAEW40bmz34ak7q rPmvPTj+gUTMHn8EItG6pFroSJEn9mjlLLdobGAwyRIcrjaHE/tvXA4bqIM6DGnaMmrD quLhgdcshW6vgjwfeiaAnnwD/iXB/LEzhmzg41g2cHxnAhUik3el6068Tvd6oF9Mue2/ 6vXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=66S4oqhr3CX2BaIXqPjuQpjJH5nc87h5h1njzf+OkqE=; b=xr70pK3oZmPNtmjR9N+hJbkLfzo9Jt7E6XV0MkeEudwVO1vcjxCEIz2aJoevNhZI7f VvivKPF9kv8eBFyUtVdrttF8Nw8ALTm6BRlbML4pM10oz/3rNsChxgQd9v6ElkmrL5tU 6AeVtFeQKMWWup/1buwYyAe+qbrhcU9cFt6z9Lf7oeX7D5aUWCDE69MvXSHvOF8JKJTG mpQgywPqnAXRocqQ5b5YcIGArR6KZvqsIZScEPpA/8ToCK0eFyB615ICeC34Yo2ttihH 5dbDy73AsRCRaQhoEel+f7pqm830PqhZEfyHlFGFhXmCclovYtEnOMlJGHAZAYsZg+Jg 2s0Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=Yj0akOxZ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y24-20020a170906071800b009317f50118fsi2633434ejb.621.2023.03.17.13.10.45; Fri, 17 Mar 2023 13:11:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=Yj0akOxZ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5B94668C26F; Fri, 17 Mar 2023 22:09:58 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A8D3C68C24F for ; Fri, 17 Mar 2023 22:09:49 +0200 (EET) Received: by mail-qt1-f173.google.com with SMTP id i24so6958270qtm.6 for ; Fri, 17 Mar 2023 13:09:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679083788; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XEjn1HFG6pYOq83kHK5HbwpVqXMIOBSsp9h3emu4OCY=; b=Yj0akOxZ4sI7Q5CG8srKrDybxKrW6rrlrZrrUtJcegGMgkI4u4xvaezC6ok3BSiez6 xDqaugDb6PZwroRHpsDeP3RncDc7GJrpEeT9QkmRKAQrkBjz36IS0G7O5r+pUYYCZnrc 5HXIHWDk3ic8bPxiBihocPEBR8eg3sS5Vfbq8Kip4qWYa+ZwRlDF2nxY79pj001keZD9 HpJv/iLSqx/aEsQ1MNPCB1QSp8mS+7944Vf/pXNxouwU3kq2xaipm/uzE4tSciz0noaK o6fZgs6SPrWoexnk9RRoYuOTikB4kt4rjbBeWJMxGVLqglLeIlKxm6yZlPjGhK88uZMF lMog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679083788; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XEjn1HFG6pYOq83kHK5HbwpVqXMIOBSsp9h3emu4OCY=; b=NQfSiiKCn3V674C2GOPxIg1ARXdIa2kUkKw+2M1TONXTkWn7WyIxkq37LpONHoDZts o16Cn5DCDcuyBMs/G08TIS+tsQ6PMKmiWHRHjDbucLNMo+WwiYs26raVe2p8dXtI+Al1 AbfC10vdKSBuHhe/hPQye2ruLl5FxHJROL/RxZmK2EQpQBN7RlkSUJgdVOPpcZsPqShb Acvl4EsE7nNjfXdJGtYZSMABmaDcdBlfOzDzpaAVvPSA6DZyymoy8HcSAKOpEXxJP1rX ovpjYwnzemsv/4D/qFva0WelOPB3L/g49EhsaFfYmRH8uWrENDjPqH2YAKpnhNNiHqEJ oYuw== X-Gm-Message-State: AO0yUKX59jSAjxsv8dXmcQ7uE42PyTaypoNOsbvYk0fj0VTdop1LzEgk KRmNNaw5cHGarsO5OJGoWxnMQL8DJB7YzpuaHIo= X-Received: by 2002:a05:622a:4e:b0:3b9:b112:e86b with SMTP id y14-20020a05622a004e00b003b9b112e86bmr17347253qtw.13.1679083788285; Fri, 17 Mar 2023 13:09:48 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id j5-20020ac86645000000b003bd0e7ff466sm2065685qtp.7.2023.03.17.13.09.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 13:09:47 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 17 Mar 2023 16:09:39 -0400 Message-Id: <20230317200941.3936-4-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230317200941.3936-1-dheitmueller@ltnglobal.com> References: <20230317200941.3936-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [RFC PATCH 3/5] yadif: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ngcJjOc1ZcFp Various deinterlacing modes have the effect of doubling the framerate, and we need to ensure that the caption data isn't duplicated (or else you get double captions on-screen). Use the new ccfifo mechanism for yadif (and yadif_cuda and bwdif since they use the same yadif core) so that CEA-708 data is properly preserved through this filter. Signed-off-by: Devin Heitmueller --- libavfilter/vf_bwdif.c | 7 +++++++ libavfilter/vf_yadif.c | 6 ++++++ libavfilter/vf_yadif_cuda.c | 8 ++++++++ libavfilter/yadif.h | 2 ++ libavfilter/yadif_common.c | 5 +++++ 5 files changed, 28 insertions(+) diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index 65c617ebb3..de937d83cf 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -297,6 +297,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + av_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -332,6 +333,12 @@ static int config_props(AVFilterLink *link) if(yadif->mode&1) link->frame_rate = av_mul_q(link->src->inputs[0]->frame_rate, (AVRational){2,1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(yadif->cc_fifo = av_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + if (link->w < 3 || link->h < 4) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n"); diff --git a/libavfilter/vf_yadif.c b/libavfilter/vf_yadif.c index 1be02de1a9..b51d21f6ff 100644 --- a/libavfilter/vf_yadif.c +++ b/libavfilter/vf_yadif.c @@ -261,6 +261,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + av_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -293,6 +294,11 @@ static int config_output(AVFilterLink *outlink) if(s->mode & 1) outlink->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = av_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (outlink->w < 3 || outlink->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/vf_yadif_cuda.c b/libavfilter/vf_yadif_cuda.c index 685b8a2035..d96ec391a0 100644 --- a/libavfilter/vf_yadif_cuda.c +++ b/libavfilter/vf_yadif_cuda.c @@ -206,6 +206,9 @@ static av_cold void deint_cuda_uninit(AVFilterContext *ctx) av_frame_free(&y->cur); av_frame_free(&y->next); + if (yadif->cc_fifo) + av_cc_fifo_free(yadif->cc_fifo); + av_buffer_unref(&s->device_ref); s->hwctx = NULL; av_buffer_unref(&s->input_frames_ref); @@ -291,6 +294,11 @@ static int config_output(AVFilterLink *link) if(y->mode & 1) link->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = av_cc_fifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (link->w < 3 || link->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/yadif.h b/libavfilter/yadif.h index c928911b35..ccd4304860 100644 --- a/libavfilter/yadif.h +++ b/libavfilter/yadif.h @@ -21,6 +21,7 @@ #include "libavutil/opt.h" #include "libavutil/pixdesc.h" +#include "libavutil/ccfifo.h" #include "avfilter.h" enum YADIFMode { @@ -76,6 +77,7 @@ typedef struct YADIFContext { int eof; uint8_t *temp_line; int temp_line_size; + AVCCFifo *cc_fifo; /* * An algorithm that treats first and/or last fields in a sequence diff --git a/libavfilter/yadif_common.c b/libavfilter/yadif_common.c index a10cf7a17f..f95b8c10d3 100644 --- a/libavfilter/yadif_common.c +++ b/libavfilter/yadif_common.c @@ -43,6 +43,7 @@ static int return_frame(AVFilterContext *ctx, int is_second) return AVERROR(ENOMEM); av_frame_copy_props(yadif->out, yadif->cur); + av_ccfifo_inject(yadif->cc_fifo, yadif->out); yadif->out->interlaced_frame = 0; if (yadif->current_field == YADIF_FIELD_BACK_END) yadif->current_field = YADIF_FIELD_END; @@ -96,6 +97,8 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) av_assert0(frame); + av_ccfifo_extract(yadif->cc_fifo, frame); + if (yadif->frame_pending) return_frame(ctx, 1); @@ -137,6 +140,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) if (!yadif->out) return AVERROR(ENOMEM); + av_ccfifo_inject(yadif->cc_fifo, yadif->out); av_frame_free(&yadif->prev); if (yadif->out->pts != AV_NOPTS_VALUE) yadif->out->pts *= 2; @@ -148,6 +152,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) return AVERROR(ENOMEM); av_frame_copy_props(yadif->out, yadif->cur); + av_ccfifo_inject(yadif->cc_fifo, yadif->out); yadif->out->interlaced_frame = 0; if (yadif->out->pts != AV_NOPTS_VALUE)