From patchwork Mon Mar 27 16:47:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40878 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a30:b0:df:834d:2c1a with SMTP id t48csp1485394pzh; Mon, 27 Mar 2023 09:47:48 -0700 (PDT) X-Google-Smtp-Source: AKy350bBthz5JNfR6jCK5qdaDweUQ5e/BD5e9AmIwyRF3qGFbA+PijIkKv75JypK+EG1Prt3D1Q9 X-Received: by 2002:a17:907:3f97:b0:93f:505b:9742 with SMTP id hr23-20020a1709073f9700b0093f505b9742mr10405297ejc.65.1679935668760; Mon, 27 Mar 2023 09:47:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679935668; cv=none; d=google.com; s=arc-20160816; b=H4IA/eAd89jDLHBAVSlZ7j55W+AMuO8Nqm+ifL4XBe78x/kAlh4UZCzWQ6lGBcEuHc to+zBJp2F1bAQyFrgOFI7LLBy5PW4xPA/wjyXClJTmNx6gI4RSolfKHcR+38WLFPvhLE iWtS6YF38B+hqRmvpyfIEdkoqkrjPPpMQvA/ziJ0P6I1xcfMR3kOgDdMlBvlD6bQOk/S ZT3ye6Ta/t6nRN7RW1eM4ZhfntKDd7fQM6BEGy5QltGudooJw2eGA7KEu/YmWZW2Lo+7 XTJnwXgUeCfxzlsNx9cbeM9gXhAINgCbCzMMtqQzxhAY84mfXErDxpKBY+20GDDmge6x KquQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=LleGLIeaVuphbWJAHVTqE1glkwbDP0agSYSnJDoAGu0=; b=dbBdXy5J9XPS13X+Zn+0gD+5fVZk7PwrzL4l2bJxW+0eW+t3ckegkD22FNoCvyngbn 4bE/rszfICFUL8Py1CKEjuvIJkPmblgKa2xGVOTze5vOdgTBssiW/HR/WTz+ypwAQm2x 9jkH0KchY5bCBeTiLOz1TdwPtHsyTHFY166ia9oOBEqFdWaFbtQ//BJYYHHkXPUJGLnd bEIOPrFWQlOjvDW5iYIIoA++TG6fWEhs685+h8KYIPSVRn6hqdmtCLNIknnaICfWvdBN SAzltE0LLP44ZgPfX7Fqt/0/EPo4v4BxHtV9Oa7WLH6b1K9eCsNN/nK7F0AGKMEAt7wh JTAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=6CxYDycz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id xo21-20020a170907bb9500b009332ffa5891si14089575ejc.136.2023.03.27.09.47.48; Mon, 27 Mar 2023 09:47:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=6CxYDycz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9CC9068C99F; Mon, 27 Mar 2023 19:47:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E008A68C99C for ; Mon, 27 Mar 2023 19:47:11 +0300 (EEST) Received: by mail-qv1-f41.google.com with SMTP id q88so7085347qvq.13 for ; Mon, 27 Mar 2023 09:47:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1679935630; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rpYUnmwUkr8rWM0d+Fcnd4ripFUMXxsU8NwTIFpmeU8=; b=6CxYDyczTgnElQKk+JHpRxmaGeOf7p89MIn12ZikUddqSCyoFAc2FuJM/XGB685PWB 1DNhVMzJGkaRpML7QihWSZqNUHHRpIC5X979TZMn0RQ8Z3XD8oUeWrkTg5k55J077+kc yX/5wzm1bbYMxaT1mJOlJaMM+cczX1G9GTu8oTRism7xG/NRhPXU63OPmL0XQIaZRAk+ Z0FpAnW5WGKpp5uevSGpDQekHTYUidfJHIU/Q7Nbs2HKp13xVngZeNpAGJ8AEE3eHbaV bVhNWbU46Qt3KLosrF/0UUQNv6CVaN55+a3hv9r0ppu74IO2AzKq+lAX/n4EWSK2YJyV MaVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679935630; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rpYUnmwUkr8rWM0d+Fcnd4ripFUMXxsU8NwTIFpmeU8=; b=VfOv1MxtK0/dKGUiqPDeHc0IMWi3IIiGuWNAT4rRZUjWaMDnVuk//gstBIiCGidYER KjaL+hSyjllA2YsoCT1kO2VW2QC/0G8TbpMpmAqsif/OWsEkUDRJjFC1Cmjq0OwhLU/b JoyYkSj1Chde3oRDM+wn/GY0UnopyX50VyuY7+MBsJ2ZXfDHq3xtiexOeynbH28jpqXL +DmuVOAPmnxJvs5dK9InQ1kbJQdPGxHkwVqc7kwJGPCtkL1KQ4YEhvBNoqefitqHfIZj 00GCp7UqIfU8vFNCmQlmVXhRCT/o6T9aSg1A2UPJRhYOMYDNuY3EhhktJY2RElklXNjR VyXQ== X-Gm-Message-State: AAQBX9dp5uV+maN9qDb/ZYtMSIIsilRC8lRGj0nPMz9x0TLYEbDrc5Zq KhH//mdLXP8Y1AOObZWBsWZBvNO2t/7UMGFHG+Q= X-Received: by 2002:a05:6214:2428:b0:56e:fe7b:8bc2 with SMTP id gy8-20020a056214242800b0056efe7b8bc2mr24070150qvb.32.1679935630483; Mon, 27 Mar 2023 09:47:10 -0700 (PDT) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id lf10-20020a0562142cca00b005dd8b9345e9sm3093100qvb.129.2023.03.27.09.47.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Mar 2023 09:47:10 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 Mar 2023 12:47:02 -0400 Message-Id: <20230327164704.12962-4-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230327164704.12962-1-dheitmueller@ltnglobal.com> References: <20230327164704.12962-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/5] yadif: Properly preserve CEA-708 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 8pkYYrv66DkA Various deinterlacing modes have the effect of doubling the framerate, and we need to ensure that the caption data isn't duplicated (or else you get double captions on-screen). Use the new ccfifo mechanism for yadif (and yadif_cuda and bwdif since they use the same yadif core) so that CEA-708 data is properly preserved through this filter. Signed-off-by: Devin Heitmueller --- libavfilter/vf_bwdif.c | 7 +++++++ libavfilter/vf_yadif.c | 6 ++++++ libavfilter/vf_yadif_cuda.c | 8 ++++++++ libavfilter/yadif.h | 2 ++ libavfilter/yadif_common.c | 5 +++++ 5 files changed, 28 insertions(+) diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index 34e8c5e234..c625b68914 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -297,6 +297,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + av_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -332,6 +333,12 @@ static int config_props(AVFilterLink *link) if(yadif->mode&1) link->frame_rate = av_mul_q(link->src->inputs[0]->frame_rate, (AVRational){2,1}); + else + link->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(yadif->cc_fifo = av_ccfifo_alloc(&link->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); + if (link->w < 3 || link->h < 4) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n"); diff --git a/libavfilter/vf_yadif.c b/libavfilter/vf_yadif.c index 1be02de1a9..b51d21f6ff 100644 --- a/libavfilter/vf_yadif.c +++ b/libavfilter/vf_yadif.c @@ -261,6 +261,7 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&yadif->prev); av_frame_free(&yadif->cur ); av_frame_free(&yadif->next); + av_ccfifo_freep(&yadif->cc_fifo); } static const enum AVPixelFormat pix_fmts[] = { @@ -293,6 +294,11 @@ static int config_output(AVFilterLink *outlink) if(s->mode & 1) outlink->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = av_ccfifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (outlink->w < 3 || outlink->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/vf_yadif_cuda.c b/libavfilter/vf_yadif_cuda.c index 685b8a2035..d96ec391a0 100644 --- a/libavfilter/vf_yadif_cuda.c +++ b/libavfilter/vf_yadif_cuda.c @@ -206,6 +206,9 @@ static av_cold void deint_cuda_uninit(AVFilterContext *ctx) av_frame_free(&y->cur); av_frame_free(&y->next); + if (yadif->cc_fifo) + av_cc_fifo_free(yadif->cc_fifo); + av_buffer_unref(&s->device_ref); s->hwctx = NULL; av_buffer_unref(&s->input_frames_ref); @@ -291,6 +294,11 @@ static int config_output(AVFilterLink *link) if(y->mode & 1) link->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate, (AVRational){2, 1}); + else + outlink->frame_rate = ctx->inputs[0]->frame_rate; + + if (!(s->cc_fifo = av_cc_fifo_alloc(&outlink->frame_rate, ctx))) + av_log(ctx, AV_LOG_VERBOSE, "Failure to setup CC FIFO queue. Captions will be passed through\n"); if (link->w < 3 || link->h < 3) { av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n"); diff --git a/libavfilter/yadif.h b/libavfilter/yadif.h index c928911b35..ccd4304860 100644 --- a/libavfilter/yadif.h +++ b/libavfilter/yadif.h @@ -21,6 +21,7 @@ #include "libavutil/opt.h" #include "libavutil/pixdesc.h" +#include "libavutil/ccfifo.h" #include "avfilter.h" enum YADIFMode { @@ -76,6 +77,7 @@ typedef struct YADIFContext { int eof; uint8_t *temp_line; int temp_line_size; + AVCCFifo *cc_fifo; /* * An algorithm that treats first and/or last fields in a sequence diff --git a/libavfilter/yadif_common.c b/libavfilter/yadif_common.c index a10cf7a17f..24cc72ab5f 100644 --- a/libavfilter/yadif_common.c +++ b/libavfilter/yadif_common.c @@ -60,6 +60,8 @@ static int return_frame(AVFilterContext *ctx, int is_second) yadif->out->pts = AV_NOPTS_VALUE; } } + + av_ccfifo_inject(yadif->cc_fifo, yadif->out); ret = ff_filter_frame(ctx->outputs[0], yadif->out); yadif->frame_pending = (yadif->mode&1) && !is_second; @@ -96,6 +98,8 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) av_assert0(frame); + av_ccfifo_extract(yadif->cc_fifo, frame); + if (yadif->frame_pending) return_frame(ctx, 1); @@ -137,6 +141,7 @@ int ff_yadif_filter_frame(AVFilterLink *link, AVFrame *frame) if (!yadif->out) return AVERROR(ENOMEM); + av_ccfifo_inject(yadif->cc_fifo, yadif->out); av_frame_free(&yadif->prev); if (yadif->out->pts != AV_NOPTS_VALUE) yadif->out->pts *= 2;