From patchwork Mon Mar 11 00:18:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marth64 X-Patchwork-Id: 46942 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dc95:b0:1a1:738b:6bc0 with SMTP id ky21csp944163pzb; Sun, 10 Mar 2024 17:19:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUK1FhLLNOxsFBL9ZVaLZFJk3AuvPQ2YoDKt7idJYy8GHwDdEi/mZoTzEnfyyHMTUpz4gHaAj3rjjwAHJojgza6uGL+drbSQXQYRg== X-Google-Smtp-Source: AGHT+IEOf30DDIsVv6oBkORc+zLdKZGgpUtepmTU79A5NCKoh3TZBsKN6VmZ0ue49Z0ruIUtXHpJ X-Received: by 2002:a05:651c:313:b0:2d2:abcf:1fd6 with SMTP id a19-20020a05651c031300b002d2abcf1fd6mr3153434ljp.4.1710116351804; Sun, 10 Mar 2024 17:19:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710116351; cv=none; d=google.com; s=arc-20160816; b=nxX7dr5Svp3Xtvq0swScguDz5PNQ5xx5dTqupbjKYxwwR4WdLGIGBoH9qWo5QaVtqc vTayfqs5rToX2LLLYpnTEb4bvom8NcIDnHTipV8gndsHEvKka/ugVHTmr/412DX7x8W1 e5Z2PRvZIq50czhwhHk1tAzI/zfPkRqgQ+NwvFB0z1RLvCrrQ/dDSKhvUhKBYt89JXHL 7kTtNW3Mlzm7xywvhbOEQH6gMQ257klhzO2tax98bm6xnFNyDzl/fejtJJNh1796f5m9 OyZF+gBcO7pmQttAjMyVfm4OUgiT7xXfaoIz94JqzQPpCi5yS39i9GCAyOitEFo1SE3l UpgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=0Bhydw70WabC4NLZ4lSaq+5IcKLsO1dBCugtjOcjToQ=; fh=PlWMzmI9LD2qGS7ipLrQl8z0iaQTLQLHzoGuXcBzpCg=; b=nfppWwLOBlQzw4/KTt8NqKRreelWvU0SCDnBmrpPpzUo++0Duw0BwjjDFD0tr5P3UP CwfsaBiGv1MwsZ7VtdwYt7ujxNAvKmDVnV0OW7ccIJ9ovK8rwv0MMB9PGt4S0akBiGzz HB/XpMpxxTnRLrA5aX9Wq5M+dFJZ6pfJBxsko2LxlvfQW12zHRAVX5fZGyyG38Vv+9/F FNGrWpchlM2Wznr0ERXX3sCs88MTrDt6iLxtW2GFbqnJgOfPawsIRzmcBwu7EttQlDew ACbSdwwfb3oSFvJyPB3L8TzWlP5ROCOQgRe4+OM6z7bADytQxn/GcvAyGaYns7XKXa91 IcEg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@proxyid.net header.s=google header.b="IaW/jwkc"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t8-20020a1709063e4800b00a442bf21c32si1921492eji.653.2024.03.10.17.19.11; Sun, 10 Mar 2024 17:19:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@proxyid.net header.s=google header.b="IaW/jwkc"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 23A7A68CE20; Mon, 11 Mar 2024 02:19:08 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ua1-f99.google.com (mail-ua1-f99.google.com [209.85.222.99]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5D71D68C633 for ; Mon, 11 Mar 2024 02:19:01 +0200 (EET) Received: by mail-ua1-f99.google.com with SMTP id a1e0cc1a2514c-7db123701bcso1573954241.2 for ; Sun, 10 Mar 2024 17:19:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proxyid.net; s=google; t=1710116339; x=1710721139; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=eo5Mriggf+guiBb9uHgU8+jBjjNmYrZQFU4Q79GqZRY=; b=IaW/jwkcvInyy8e9GkmIJRQThmNtzDfGi+E226xasoOucPr6AEIBYMhNqR8XOZYRCP G93Go8fCxvkE4DnAfRSW/4d+lHcyd8Fi7+LQIIsySaPjU8060KG1lDqxJbJKi12DNbrS VSReCCbeaQ1+hHISdyVRX1by9n+NGbb+P+Pm567k22w5PtbbLu1dvOSZrDcCIsvpj8gM lvKURnbWYUNCS5rIM6ODfsI2s/4T3XsZSWVXqSe76bzzHqFp1ZhDVJmeW5BRkmyuT9hJ rel2fIDQ0V321VpPkX77UwXH7Ngl64jlGbuGFCOQhNFZ/AXv8py28OuFDcCSR3Cw8Wta CRwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710116339; x=1710721139; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=eo5Mriggf+guiBb9uHgU8+jBjjNmYrZQFU4Q79GqZRY=; b=IG2vLvVcs649212y6A9O1xNTzl1yTO8N6zNi5FofETdXdX9Z4sLZZGG0mwkpOCBjK4 6Thuhl7fGLc6aRIogsCeIwpIdlG4XK3JNFr7EYX4TZa/06zQhxqN+kVuPHYm/NyI3X7q TSilTI2Ov5uqXUHR2bS8eOP+z5zhilwlmD+H7y9WDM+gf1DcpESaPK6vT74Ipld8+BmJ KAgOTSqZXgoI9iOQx3TQR40BWY5QVl+kF8eedX3Fy4SOgWzuRVLuRKI3uslvkM3P/ThR mOytXrerX6MBaVgNIwYF547QKgaSsoD7Tl6tNkfPNP17/YBKUU5eXmYrCX6TB8UcQGCh xAgA== X-Gm-Message-State: AOJu0Yz5WzHPSwDDBc38zfL01ftLz9OQ6VRBftLPVlgEFx70vtM0u8sF iFdmL435HFrJ8O0LuHcqXVPz3Oc4X3sNhbznEjUfBhTHOPFW38f+58lBY+JX3rA6xo3fOPCwt4d YWrpY0Qhxt0HLQFQ0ncnFO0Axz/LEOcvu81jS/ULl X-Received: by 2002:a05:6122:852:b0:4d3:d135:b035 with SMTP id 18-20020a056122085200b004d3d135b035mr1525664vkk.3.1710116339087; Sun, 10 Mar 2024 17:18:59 -0700 (PDT) Received: from wsx-cc1-001.. (c-76-141-249-38.hsd1.il.comcast.net. [76.141.249.38]) by smtp-relay.gmail.com with ESMTPS id gm8-20020a056214268800b00690172a6737sm186911qvb.62.2024.03.10.17.18.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 10 Mar 2024 17:18:59 -0700 (PDT) X-Relaying-Domain: proxyid.net From: Marth64 To: ffmpeg-devel@ffmpeg.org Date: Sun, 10 Mar 2024 19:18:56 -0500 Message-Id: <20240311001856.128390-1-marth64@proxyid.net> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/mpeg12dec: extract only one type of CC substream X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Marth64 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: wTNGEbWKZJAv In MPEG-2 user data, there can be different types of Closed Captions formats embedded (A53, SCTE-20, or DVD). The current behavior of the CC extraction code in the MPEG-2 decoder is to not be aware of multiple formats if multiple exist, therefore allowing one format to overwrite the other during the extraction process since the CC extraction shares one output buffer for the normalized bytes. This causes sources that have two CC formats to produce flawed output. There exist real-world samples which contain both A53 and SCTE-20 captions in the same MPEG-2 stream, and that manifest this problem. Example of symptom: THANK YOU (expected) --> THTHANANK K YOYOUU (actual) The solution is to pick only the first CC substream observed with valid bytes, and ignore the other types. Additionally, provide an option for users to manually "force" a type in the event that this matters for a particular source. In tandem, I am working on an improvement to src_movie to allow passing decoder options (as src_movie via lavfi is the "de facto" way to extract CCs right now). This way, users can realize the newly added option. Signed-off-by: Marth64 --- libavcodec/mpeg12dec.c | 49 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 46 insertions(+), 3 deletions(-) diff --git a/libavcodec/mpeg12dec.c b/libavcodec/mpeg12dec.c index 3a2f17e508..a42e1c661f 100644 --- a/libavcodec/mpeg12dec.c +++ b/libavcodec/mpeg12dec.c @@ -62,6 +62,13 @@ #define A53_MAX_CC_COUNT 2000 +enum Mpeg2ClosedCaptionsFormat { + CC_FORMAT_AUTO, + CC_FORMAT_A53_PART4, + CC_FORMAT_SCTE20, + CC_FORMAT_DVD +}; + typedef struct Mpeg1Context { MpegEncContext mpeg_enc_ctx; int mpeg_enc_ctx_allocated; /* true if decoding context allocated */ @@ -70,6 +77,7 @@ typedef struct Mpeg1Context { AVStereo3D stereo3d; int has_stereo3d; AVBufferRef *a53_buf_ref; + enum Mpeg2ClosedCaptionsFormat cc_format; uint8_t afd; int has_afd; int slice_count; @@ -1908,7 +1916,8 @@ static int mpeg_decode_a53_cc(AVCodecContext *avctx, { Mpeg1Context *s1 = avctx->priv_data; - if (buf_size >= 6 && + if ((!s1->cc_format || s1->cc_format == CC_FORMAT_A53_PART4) && + buf_size >= 6 && p[0] == 'G' && p[1] == 'A' && p[2] == '9' && p[3] == '4' && p[4] == 3 && (p[5] & 0x40)) { /* extract A53 Part 4 CC data */ @@ -1927,9 +1936,15 @@ static int mpeg_decode_a53_cc(AVCodecContext *avctx, memcpy(s1->a53_buf_ref->data + old_size, p + 7, cc_count * UINT64_C(3)); avctx->properties |= FF_CODEC_PROPERTY_CLOSED_CAPTIONS; + + if (!s1->cc_format) { + s1->cc_format = CC_FORMAT_A53_PART4; + av_log(avctx, AV_LOG_DEBUG, "CC: first seen substream is A53 format\n"); + } } return 1; - } else if (buf_size >= 2 && + } else if ((!s1->cc_format || s1->cc_format == CC_FORMAT_SCTE20) && + buf_size >= 2 && p[0] == 0x03 && (p[1]&0x7f) == 0x01) { /* extract SCTE-20 CC data */ GetBitContext gb; @@ -1974,9 +1989,15 @@ static int mpeg_decode_a53_cc(AVCodecContext *avctx, } } avctx->properties |= FF_CODEC_PROPERTY_CLOSED_CAPTIONS; + + if (!s1->cc_format) { + s1->cc_format = CC_FORMAT_SCTE20; + av_log(avctx, AV_LOG_DEBUG, "CC: first seen substream is SCTE-20 format\n"); + } } return 1; - } else if (buf_size >= 11 && + } else if ((!s1->cc_format || s1->cc_format == CC_FORMAT_DVD) && + buf_size >= 11 && p[0] == 'C' && p[1] == 'C' && p[2] == 0x01 && p[3] == 0xf8) { /* extract DVD CC data * @@ -2034,6 +2055,11 @@ static int mpeg_decode_a53_cc(AVCodecContext *avctx, } } avctx->properties |= FF_CODEC_PROPERTY_CLOSED_CAPTIONS; + + if (!s1->cc_format) { + s1->cc_format = CC_FORMAT_DVD; + av_log(avctx, AV_LOG_DEBUG, "CC: first seen substream is DVD format\n"); + } } return 1; } @@ -2598,11 +2624,28 @@ const FFCodec ff_mpeg1video_decoder = { }, }; +#define M2V_OFFSET(x) offsetof(Mpeg1Context, x) +#define M2V_PARAM AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVOption mpeg2video_options[] = { + { "cc_format", "extract a specific Closed Captions format (0=auto)", M2V_OFFSET(cc_format), AV_OPT_TYPE_INT, { .i64 = 0 }, CC_FORMAT_AUTO, CC_FORMAT_DVD, M2V_PARAM }, + { NULL } +}; + +static const AVClass mpeg2video_class = { + .class_name = "MPEG-2 video", + .item_name = av_default_item_name, + .option = mpeg2video_options, + .version = LIBAVUTIL_VERSION_INT, + .category = AV_CLASS_CATEGORY_DECODER, +}; + const FFCodec ff_mpeg2video_decoder = { .p.name = "mpeg2video", CODEC_LONG_NAME("MPEG-2 video"), .p.type = AVMEDIA_TYPE_VIDEO, .p.id = AV_CODEC_ID_MPEG2VIDEO, + .p.priv_class = &mpeg2video_class, .priv_data_size = sizeof(Mpeg1Context), .init = mpeg_decode_init, .close = mpeg_decode_end,