From patchwork Fri Jan 22 11:20:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 25105 Delivered-To: andriy.gelman@gmail.com Received: by 2002:a25:ab61:0:0:0:0:0 with SMTP id u88csp1082978ybi; Fri, 22 Jan 2021 03:21:19 -0800 (PST) X-Google-Smtp-Source: ABdhPJxa2/JToeU5G4vFWzSfUoG3DgOQcBtgdU10Ccxn4UB3yPAfUqL4JRcmimbkHB1B7EOZFHIk X-Received: by 2002:a05:6402:6c4:: with SMTP id n4mr2848878edy.257.1611314479591; Fri, 22 Jan 2021 03:21:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611314479; cv=none; d=google.com; s=arc-20160816; b=PeKbR/NpEvT+cnwGkVioIU/8QEW1bRJhwE/VDaVGmdWmIapstUQhCLsFnjF/6GUNCK NdDkaTL8jDpWXCH13v+UuwxW5mLEfXuuGtlrhE6Tvo8rX26uT3+exEPussd1wAQmddBj RatUzP3A64VZTDQIooFZjB9zOxej9y9rp1i49cqNAP7Gqzo7PZUc3IamEUvsjRRWM0t3 LvpCp7lRbUwSoFn7ILaXF7/ao2E8fnxGidR9A/KVrn68yqOYCLoTHT9bJdGEn8jKCIj7 Ku9wuCX1M24OHJPTMEd1ccaG+MIZqe94kDGGRcXlhZbpzfiMkgMMpHz3C3j+jDQS8SuH U2Eg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=CAz4K8Q+Hn1YzWYYBfgVyX/3eNuqOhcQignYMOs3Sug=; b=O21Zg/ulLqLMIrNky0JKW1akDsDX0YMKZS28HaFjWsfBdJy1bYox391sSXRzozOTff SReax3VW+nsHO52rINyxKSCAI9bSoHxmfmwa9788zQ2nyO1y94nV10kIr4hxv/8p0jKB ImH/UQRcrS/kl+WtnLX1RDxgEY8nczSzC4dPXoVqmFz5Hw+CdyBP1tb/s8NXQdo469+b KaeibWhtnaHi6xaOHVOJOPS6liNdvCQ6OPgrKSdGiIb88L5jBAw1OIWEWUwEe+mUNJWz yTgHWFjFICjcM7NVe/G/8MSfQyoJRtommFnd4rEZbSuweb29S9EdyMKhdaiLJ7xzDK2c TzyA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=OSVrVLFI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g7si3095435edr.423.2021.01.22.03.21.19; Fri, 22 Jan 2021 03:21:19 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=OSVrVLFI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AD1D768A370; Fri, 22 Jan 2021 13:21:09 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f54.google.com (mail-lf1-f54.google.com [209.85.167.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 06D5668A27A for ; Fri, 22 Jan 2021 13:21:03 +0200 (EET) Received: by mail-lf1-f54.google.com with SMTP id v67so7049212lfa.0 for ; Fri, 22 Jan 2021 03:21:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=m1yZwq9IlDE2+HFILR3smsGgsMx/AUqmBDDXX8hBKNw=; b=OSVrVLFI197QasoPVb7Hnl8ocdU6GsLatFGQdSgUI5LzQmiruO0jDCLEgx5vtuX50t jR+N8KZh8G99jp8rKU+yYOR2Yobc0BfqBDBaPe/6O8QIyWrWIvfTPaOqZKtl0HJNoX1b yn95DtSHetzGG9ejVpEwOY8VsQWCz5cA0qlJz63KrKlcCQv7Cdoo2KTXkrv7nfkux02R HYo2GY48AR11IsQNmjBRS9nn5X0b8lvWWMLQkiLJad0Yo2dqMvSASVE5IsB9x3myJJA0 VO3RW7y+ww1XZP6lI9pocypKaUgApTqj+BuNcIaCLdlmGJgDrL1/PWUUIlr52WSH2scL GmTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m1yZwq9IlDE2+HFILR3smsGgsMx/AUqmBDDXX8hBKNw=; b=tPFnjgJoXGBSEatUXvainjWbc4mO9t7BvUHLZTeo6QnbLZmp1YgP+Ql2pRZbQEkC/9 U944rqww1WYBkcpmIB+PbvTmz/LwomkuA595/vpX2H7NOqkFtRzlE6zN+LNZYgyWb3ht b5PYgcgCtGUSUjhUcNWAEHdkFJRPesR/DJq+/L9ydHbsMebWDMoRrOhT6Yfthnq0banU jIZa5mZqh0c7kOnWgYx8ZFGnk0jFbVwSiijuiSi9QuL1C0XPfOQRD6/fFg5lTeJmFnRa MROnY4C0X1LOMnUgyNc2+HAEVJZZ6sMsUxgRSzrg7G5b5lbjkg/5LrRKv2r8WySQbWUW uKFQ== X-Gm-Message-State: AOAM532gV/EvK5+XpzjhN4l0a71jhMCxyHxDU/IwpWui3vuz1tQyjmJE eVm3TWP9D6OmqCc5hU6mzO2hLYyuJVs= X-Received: by 2002:a05:6512:1310:: with SMTP id x16mr774775lfu.19.1611314462416; Fri, 22 Jan 2021 03:21:02 -0800 (PST) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id v24sm919451ljc.56.2021.01.22.03.21.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 03:21:01 -0800 (PST) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Fri, 22 Jan 2021 13:20:55 +0200 Message-Id: <20210122112058.49068-2-jeebjp@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122112058.49068-1-jeebjp@gmail.com> References: <20210122103725.24370-1-jeebjp@gmail.com> <20210122112058.49068-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 1/4] avutil/{avstring, bprint}: add XML escaping from ffprobe to avutil X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zmuT6knN2+NQ Content-Length: 4523 From: Stefano Sabatini --- libavutil/avstring.h | 7 ++++--- libavutil/bprint.c | 15 +++++++++++++++ tools/ffescape.c | 7 ++++--- 3 files changed, 23 insertions(+), 6 deletions(-) diff --git a/libavutil/avstring.h b/libavutil/avstring.h index ee225585b3..189b4726a5 100644 --- a/libavutil/avstring.h +++ b/libavutil/avstring.h @@ -321,9 +321,10 @@ int av_match_name(const char *name, const char *names); char *av_append_path_component(const char *path, const char *component); enum AVEscapeMode { - AV_ESCAPE_MODE_AUTO, ///< Use auto-selected escaping mode. - AV_ESCAPE_MODE_BACKSLASH, ///< Use backslash escaping. - AV_ESCAPE_MODE_QUOTE, ///< Use single-quote escaping. + AV_ESCAPE_MODE_AUTO, ///< Use auto-selected escaping mode. + AV_ESCAPE_MODE_BACKSLASH, ///< Use backslash escaping. + AV_ESCAPE_MODE_QUOTE, ///< Use single-quote escaping. + AV_ESCAPE_MODE_XML_CHAR_DATA, ///< Use XML non-markup character data escaping. }; /** diff --git a/libavutil/bprint.c b/libavutil/bprint.c index 2f059c5ba6..7cdbb75095 100644 --- a/libavutil/bprint.c +++ b/libavutil/bprint.c @@ -283,6 +283,21 @@ void av_bprint_escape(AVBPrint *dstbuf, const char *src, const char *special_cha av_bprint_chars(dstbuf, '\'', 1); break; + case AV_ESCAPE_MODE_XML_CHAR_DATA: + /* escape XML non-markup character data as per 2.4 */ + /* [^<&]* - ([^<&]* ']]>' [^<&]*) */ + for (; *src; src++) { + switch (*src) { + case '&' : av_bprintf(dstbuf, "%s", "&"); break; + case '<' : av_bprintf(dstbuf, "%s", "<"); break; + case '>' : av_bprintf(dstbuf, "%s", ">"); break; + case '"' : av_bprintf(dstbuf, "%s", """); break; + case '\'': av_bprintf(dstbuf, "%s", "'"); break; + default: av_bprint_chars(dstbuf, *src, 1); + } + } + break; + /* case AV_ESCAPE_MODE_BACKSLASH or unknown mode */ default: /* \-escape characters */ diff --git a/tools/ffescape.c b/tools/ffescape.c index 0530d28c6d..e18f1edaf9 100644 --- a/tools/ffescape.c +++ b/tools/ffescape.c @@ -101,9 +101,10 @@ int main(int argc, char **argv) break; } case 'm': - if (!strcmp(optarg, "auto")) escape_mode = AV_ESCAPE_MODE_AUTO; - else if (!strcmp(optarg, "backslash")) escape_mode = AV_ESCAPE_MODE_BACKSLASH; - else if (!strcmp(optarg, "quote")) escape_mode = AV_ESCAPE_MODE_QUOTE; + if (!strcmp(optarg, "auto")) escape_mode = AV_ESCAPE_MODE_AUTO; + else if (!strcmp(optarg, "backslash")) escape_mode = AV_ESCAPE_MODE_BACKSLASH; + else if (!strcmp(optarg, "quote")) escape_mode = AV_ESCAPE_MODE_QUOTE; + else if (!strcmp(optarg, "xml_char_data")) escape_mode = AV_ESCAPE_MODE_XML_CHAR_DATA; else { av_log(NULL, AV_LOG_ERROR, "Invalid value '%s' for option -m, " From patchwork Fri Jan 22 11:20:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 25112 Delivered-To: andriy.gelman@gmail.com Received: by 2002:a25:ab61:0:0:0:0:0 with SMTP id u88csp1083072ybi; Fri, 22 Jan 2021 03:21:29 -0800 (PST) X-Google-Smtp-Source: ABdhPJzHcVd6CpDZu4IweigvYpHIKPsBWOJdamakkNeOKQG4J0JdV5sCu2Q+WpIW4N0zCsfEwy7n X-Received: by 2002:a17:907:e9e:: with SMTP id ho30mr2716601ejc.529.1611314489228; Fri, 22 Jan 2021 03:21:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611314489; cv=none; d=google.com; s=arc-20160816; b=E3MyXmOnUyz2zBBuUHAka9XofESkun0v4Ae7f60gIEetr/4BtCE5eGuXx/DezyEGBO t1wBXSiwcy7erLcRS81hy1z4tu2nzVS9sbXex1cTNlE5mrBsgdkjCta+KvK6W1yB1VYI P1UTkC0klz8FD8QZ1aHjKOKK4bqic2QORCzObiIiw7I5NBI0l+ds90rqMaRahB3nGgrS y/1O+5HnFeZdqeXW925MvocoWkRPliz9bapCWthFfE+Ugdch+6kv+tw/Ccl5zy94tADu FCr6n/0Pg/0Demo5STy1HH0LzxZfdc0WXWWGASpWI0cBqNFxP/4hQnVZ2ZCN2hj42qY9 LsYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=hLv2hw8f4OqCc+jG4cw5eSGJW2XizP70JCqurlxtJgs=; b=wKZ335PjAuSjFC7L8Cr5bQBNZSDOG55StuTMgQqKuJ7FreMkdJckPnbO/kUzIef6aV YDI/OFvpUL1Vx/pLDe7a/Y/8HMQqaYiV38So8szwWgDMDOyQPwMXF9LaQyRkPGn0MXZo nxBWBkTWFYmHrUUUXErJXWFwNMoPL2m3s1UruiqwS0lxO/k1pqOeCqzO2buhAtNKE6H+ DNVoLMQJ6SO2N5KrwUj7B0W1hv3JSr3gL5E5B+k21Mc/pB0+UpYl58/K8iocrx40hxnM DCN00sGt35Lq6QaqksaM92cmkYCZfDQzbPoxHLHGIUPU/HpcXQtHX7LToT4LDHUv+tvo 6LtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=LP2TWUtZ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e6si2463472edz.361.2021.01.22.03.21.28; Fri, 22 Jan 2021 03:21:29 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=LP2TWUtZ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9899968A466; Fri, 22 Jan 2021 13:21:11 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f181.google.com (mail-lj1-f181.google.com [209.85.208.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C7A4268A329 for ; Fri, 22 Jan 2021 13:21:03 +0200 (EET) Received: by mail-lj1-f181.google.com with SMTP id e7so6067724ljg.10 for ; Fri, 22 Jan 2021 03:21:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=0tqesKxf9Q+k2/HqKtigHhunyL1BUuz0/niwUhS0CRo=; b=LP2TWUtZSNEzxVruncY8pgsceJaSjemb1930Jeo4CpfMVu8hkZHFh7S66BE/xxPZS6 id8Z2ij09ptdvOAly8RHMCX4XQzWaif2nv05zzs+FwMxAFSwYDR3GT4NghRVpPy5hI4a I443J435IJiIpranalKCfNS6xPiG0RsMIyzrrwNeaHCn6FZ/uVqt3WVw5PI0NwBIojeh VOHUEFkAjTEtcSbA3yfsHuM/DyF8fL+O1WDZmlY0KQHgXTwtci9vTQMzLvviA8cZQF7v sB2WqJV2qjkBa3knvjyLMY7wVniEsyBvFnGDwJOPOH5tWJGxQdyStzNmOKOw8J1YCwlB PhoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0tqesKxf9Q+k2/HqKtigHhunyL1BUuz0/niwUhS0CRo=; b=GrwIPi0mCaHns5PYNuMsKktod1DSN2C6WYHhl2OaHP+Bv5n4higRKHujRSHwraR5jk Zlpq21rFD6+M3/gGQ5r5I794ogcsEL33lwWlvwjbMSh+wHU3C+HNKxvxQYjKeTXM2SH2 vg3bpmU5x0n6WtnW/vKK4HBg7+a/a+ZXjm4S+T/nAiPoM5NG/Fl7bHU16K8WS0/yRzBq xwoti1IhUuTuAN1xIRYtQoBCu9GJOR951JXtt3bpPfkvUHUVdmZHNX5gP0bQjNZywBWN 7n5yk4OUx0jIeRfY8JCl78J8AogAqEwG7ZTJjeoQwNaJZglo5IvdIUfG/ldKrLNw3TG3 Zu2g== X-Gm-Message-State: AOAM531UP0bLBotCBefgMbN3gUr/hqL+BF4eA+5mCY7rwiB5ScKseLgM gIAw7L/yrO//FS4QTAtwzwzpsgGMkQ4= X-Received: by 2002:a2e:9bce:: with SMTP id w14mr2136257ljj.120.1611314463263; Fri, 22 Jan 2021 03:21:03 -0800 (PST) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id v24sm919451ljc.56.2021.01.22.03.21.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 03:21:02 -0800 (PST) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Fri, 22 Jan 2021 13:20:56 +0200 Message-Id: <20210122112058.49068-3-jeebjp@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122112058.49068-1-jeebjp@gmail.com> References: <20210122103725.24370-1-jeebjp@gmail.com> <20210122112058.49068-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 2/4] avutil/{avstring, bprint}: add XML attribute value escape modes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3wDi6y48NLWm Content-Length: 6177 From: Jan Ekström Signed-off-by: Jan Ekström --- libavutil/avstring.h | 10 ++++++---- libavutil/bprint.c | 26 ++++++++++++++++++++++++++ tools/ffescape.c | 10 ++++++---- 3 files changed, 38 insertions(+), 8 deletions(-) diff --git a/libavutil/avstring.h b/libavutil/avstring.h index 189b4726a5..c83aa9cf57 100644 --- a/libavutil/avstring.h +++ b/libavutil/avstring.h @@ -321,10 +321,12 @@ int av_match_name(const char *name, const char *names); char *av_append_path_component(const char *path, const char *component); enum AVEscapeMode { - AV_ESCAPE_MODE_AUTO, ///< Use auto-selected escaping mode. - AV_ESCAPE_MODE_BACKSLASH, ///< Use backslash escaping. - AV_ESCAPE_MODE_QUOTE, ///< Use single-quote escaping. - AV_ESCAPE_MODE_XML_CHAR_DATA, ///< Use XML non-markup character data escaping. + AV_ESCAPE_MODE_AUTO, ///< Use auto-selected escaping mode. + AV_ESCAPE_MODE_BACKSLASH, ///< Use backslash escaping. + AV_ESCAPE_MODE_QUOTE, ///< Use single-quote escaping. + AV_ESCAPE_MODE_XML_CHAR_DATA, ///< Use XML non-markup character data escaping. + AV_ESCAPE_MODE_XML_ATT_VALUE_SINGLE_QUOTED, ///< Use XML single quoted attribute value escaping. + AV_ESCAPE_MODE_XML_ATT_VALUE_DOUBLE_QUOTED, ///< Use XML double quoted attribute value escaping. }; /** diff --git a/libavutil/bprint.c b/libavutil/bprint.c index 7cdbb75095..c42ccdeee5 100644 --- a/libavutil/bprint.c +++ b/libavutil/bprint.c @@ -298,6 +298,32 @@ void av_bprint_escape(AVBPrint *dstbuf, const char *src, const char *special_cha } break; + case AV_ESCAPE_MODE_XML_ATT_VALUE_SINGLE_QUOTED: + /* escape XML single quoted attribute values as per 2.3 */ + /* "'" ([^<&'] | Reference)* "'" */ + for (; *src; src++) { + switch (*src) { + case '&' : av_bprintf(dstbuf, "%s", "&"); break; + case '<' : av_bprintf(dstbuf, "%s", "<"); break; + case '\'': av_bprintf(dstbuf, "%s", "'"); break; + default: av_bprint_chars(dstbuf, *src, 1); + } + } + break; + + case AV_ESCAPE_MODE_XML_ATT_VALUE_DOUBLE_QUOTED: + /* escape XML double quoted attribute values as per 2.3 */ + /* '"' ([^<&"] | Reference)* '"' */ + for (; *src; src++) { + switch (*src) { + case '&' : av_bprintf(dstbuf, "%s", "&"); break; + case '<' : av_bprintf(dstbuf, "%s", "<"); break; + case '"' : av_bprintf(dstbuf, "%s", """); break; + default: av_bprint_chars(dstbuf, *src, 1); + } + } + break; + /* case AV_ESCAPE_MODE_BACKSLASH or unknown mode */ default: /* \-escape characters */ diff --git a/tools/ffescape.c b/tools/ffescape.c index e18f1edaf9..c827e0b301 100644 --- a/tools/ffescape.c +++ b/tools/ffescape.c @@ -101,10 +101,12 @@ int main(int argc, char **argv) break; } case 'm': - if (!strcmp(optarg, "auto")) escape_mode = AV_ESCAPE_MODE_AUTO; - else if (!strcmp(optarg, "backslash")) escape_mode = AV_ESCAPE_MODE_BACKSLASH; - else if (!strcmp(optarg, "quote")) escape_mode = AV_ESCAPE_MODE_QUOTE; - else if (!strcmp(optarg, "xml_char_data")) escape_mode = AV_ESCAPE_MODE_XML_CHAR_DATA; + if (!strcmp(optarg, "auto")) escape_mode = AV_ESCAPE_MODE_AUTO; + else if (!strcmp(optarg, "backslash")) escape_mode = AV_ESCAPE_MODE_BACKSLASH; + else if (!strcmp(optarg, "quote")) escape_mode = AV_ESCAPE_MODE_QUOTE; + else if (!strcmp(optarg, "xml_char_data")) escape_mode = AV_ESCAPE_MODE_XML_CHAR_DATA; + else if (!strcmp(optarg, "xml_att_value_single_quoted")) escape_mode = AV_ESCAPE_MODE_XML_ATT_VALUE_SINGLE_QUOTED; + else if (!strcmp(optarg, "xml_att_value_double_quoted")) escape_mode = AV_ESCAPE_MODE_XML_ATT_VALUE_DOUBLE_QUOTED; else { av_log(NULL, AV_LOG_ERROR, "Invalid value '%s' for option -m, " From patchwork Fri Jan 22 11:20:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 25111 Delivered-To: andriy.gelman@gmail.com Received: by 2002:a25:ab61:0:0:0:0:0 with SMTP id u88csp1083169ybi; Fri, 22 Jan 2021 03:21:42 -0800 (PST) X-Google-Smtp-Source: ABdhPJzsS1cw0Hxti9X/qb/46oYtS6Akn/lD2gD/n9UoWcHjO18/J9wG8YXKm1HcZ1Kh/4OM2rgy X-Received: by 2002:a17:907:9483:: with SMTP id dm3mr2623571ejc.120.1611314502627; Fri, 22 Jan 2021 03:21:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611314502; cv=none; d=google.com; s=arc-20160816; b=gMJAnDAYhrjMvQQ9S9W1hE+fv6QY7gtIXrzppq+nFojfJP9yEHnjQAvG7tSh5WEyN+ eeO0L6QEnEFE8ejTv1olQEwmpzV8iGVlqAZJ1Zb3IM9L5GmR8/MFJlkefIz9w2rqnWBW tdp0qR/XhW5QwWtWkqd7eWw7ljislqCVxqFikh5dy5wswE6KU+BI0Sz5kTPkZIXNzn0N WZVIOmSBfMfCrKP9/MJjOIRVoOW0dKtUHOP+JaPRgCB0McN0fMnIT3kt8B1jTU7K3EbD HstazXDhBo/u2s3UM9kgCCu/T/fAvLjOuqlqEbsVCLEQ1lAp3UwSYULluQcg8v4D+fqK CICg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=8OW33ikSsV3YYfGV/KeUfJyI5O8iv7JOD1zdsRH2+Po=; b=p2JGk113lLPpiXkfqwakBtYTkvXQeBNMMoaQpBCsv/1vrckKcXU4gYiBt2OpmCA3BN azNzhA4VmjCdcOkOZtyux+XHRojcDmN3uNRLOYLhhsqMdHaZMK4kZn3MMpFDIIsCSczE Fx6ygaklZq48KEyHYCC2tsSnvJ73L9Tt3+oqCyEPsKNleABemsZjff2G4JO8ZJq92OQk 2rYU7zF3zI2PffH8CSjZ0uJ0hrKy5D7k5dZ8chlPQUU8OZ1uk7hFygqHvNJIeuBUPHuk 9dgzzd0B9V67itL3U8aqIq1Wrt1dtGSK7Y+BUB7NLr3JDRGtJvV42z/qr4pywiqXVLMp lanw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=UHKuqO6M; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i2si829186ejp.556.2021.01.22.03.21.42; Fri, 22 Jan 2021 03:21:42 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=UHKuqO6M; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 00C8B68A4F6; Fri, 22 Jan 2021 13:21:13 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B595968A3A3 for ; Fri, 22 Jan 2021 13:21:04 +0200 (EET) Received: by mail-lf1-f43.google.com with SMTP id p21so1926444lfu.11 for ; Fri, 22 Jan 2021 03:21:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=d+GsYG0luftTFVKPAOu8CheXsfq4BSaskskUyq1RYpg=; b=UHKuqO6MfW3iIX9eSbfeKITeAJu1Z/630lsSLUMwNSrlGgN8b7IyGqh0TLX/ENaeA1 ljF5yB2cNkf9g+UdZFRTLa2EU72VkDMQGGka6Z3/+JLzFzJDmz9mtImy0NuvImydLHMB sJPqqd6x8M0LpXAZHyF5+6K0mCj3kDtsdD31+1qPvTib3Ya36ed/lU0w6qbt5IG+TolL 5beKReDcsABv8QofS0f6MTsYnl9DTilLGlhBjaPizPs7mx73q+ADtPWnrWQroQ+OTujt RP71eZX7rGzKC/pKEbQbze1hSo6HoymIkaFYLo1FsWe1vYp7em/p/F/zMdMR1OFS9hz9 kXfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d+GsYG0luftTFVKPAOu8CheXsfq4BSaskskUyq1RYpg=; b=iExBAAcNTOWpPvD7oAhiCP3f1ryU0gGsfnVvEp5xJj4BEf8B0vEdDpOkYAOGUcXlrx AbJFD0XMUFCnt0rbQzo7jBZq+d4tcIIYbW6anZZTfgKTIBsKLDCBCRWnDLF9khRwY8dW +OYscP/3znYQwuDhOQjY9GBg6A3ib5CDjeO7yZWkmNUNVN4lz4TZ/D4ToBHtl7u+Fi1h CWpEwYOa3xKnyxarA9CNglW9rEIjQKzuscaRauZfliHUR2bgd3+7aVUMYv1LzSlZgc0w gR80NhYXc0p3wI2lw5GtDYDqby68fzQ8CAxJ7iR/ipEYnKGlOMKJOAST65PDl6deUvUR zFyg== X-Gm-Message-State: AOAM531ioG3/yQPi9aD6OV4zKSfTu+3Fs8+rd4QmWNr0u+Jp3qUngt8c cPpohFTPyskUHIouRunSdvqbRG8PsUI= X-Received: by 2002:a05:6512:3397:: with SMTP id h23mr367670lfg.103.1611314464161; Fri, 22 Jan 2021 03:21:04 -0800 (PST) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id v24sm919451ljc.56.2021.01.22.03.21.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 03:21:03 -0800 (PST) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Fri, 22 Jan 2021 13:20:57 +0200 Message-Id: <20210122112058.49068-4-jeebjp@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122112058.49068-1-jeebjp@gmail.com> References: <20210122103725.24370-1-jeebjp@gmail.com> <20210122112058.49068-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 3/4] ffprobe: switch to av_bprint_escape for XML escaping X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 1RYDOG7sXwRd Content-Length: 4548 From: Jan Ekström Additionally update the result of the ffprobe XML writing test. Signed-off-by: Jan Ekström --- fftools/ffprobe.c | 32 +++++++++++--------------------- tests/ref/fate/ffprobe_xml | 2 +- 2 files changed, 12 insertions(+), 22 deletions(-) diff --git a/fftools/ffprobe.c b/fftools/ffprobe.c index 3453aa09ff..564c062331 100644 --- a/fftools/ffprobe.c +++ b/fftools/ffprobe.c @@ -1672,24 +1672,6 @@ static av_cold int xml_init(WriterContext *wctx) return 0; } -static const char *xml_escape_str(AVBPrint *dst, const char *src, void *log_ctx) -{ - const char *p; - - for (p = src; *p; p++) { - switch (*p) { - case '&' : av_bprintf(dst, "%s", "&"); break; - case '<' : av_bprintf(dst, "%s", "<"); break; - case '>' : av_bprintf(dst, "%s", ">"); break; - case '"' : av_bprintf(dst, "%s", """); break; - case '\'': av_bprintf(dst, "%s", "'"); break; - default: av_bprint_chars(dst, *p, 1); - } - } - - return dst->str; -} - #define XML_INDENT() printf("%*c", xml->indent_level * 4, ' ') static void xml_print_section_header(WriterContext *wctx) @@ -1761,14 +1743,22 @@ static void xml_print_str(WriterContext *wctx, const char *key, const char *valu if (section->flags & SECTION_FLAG_HAS_VARIABLE_FIELDS) { XML_INDENT(); + av_bprint_escape(&buf, key, NULL, + AV_ESCAPE_MODE_XML_ATT_VALUE_DOUBLE_QUOTED, 0); printf("<%s key=\"%s\"", - section->element_name, xml_escape_str(&buf, key, wctx)); + section->element_name, buf.str); av_bprint_clear(&buf); - printf(" value=\"%s\"/>\n", xml_escape_str(&buf, value, wctx)); + + av_bprint_escape(&buf, value, NULL, + AV_ESCAPE_MODE_XML_ATT_VALUE_DOUBLE_QUOTED, 0); + printf(" value=\"%s\"/>\n", buf.str); } else { if (wctx->nb_item[wctx->level]) printf(" "); - printf("%s=\"%s\"", key, xml_escape_str(&buf, value, wctx)); + + av_bprint_escape(&buf, value, NULL, + AV_ESCAPE_MODE_XML_ATT_VALUE_DOUBLE_QUOTED, 0); + printf("%s=\"%s\"", key, buf.str); } av_bprint_finalize(&buf, NULL); diff --git a/tests/ref/fate/ffprobe_xml b/tests/ref/fate/ffprobe_xml index ccea2874b2..78e6af7a33 100644 --- a/tests/ref/fate/ffprobe_xml +++ b/tests/ref/fate/ffprobe_xml @@ -51,7 +51,7 @@ - + From patchwork Fri Jan 22 11:20:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 25109 Delivered-To: andriy.gelman@gmail.com Received: by 2002:a25:ab61:0:0:0:0:0 with SMTP id u88csp1083227ybi; Fri, 22 Jan 2021 03:21:48 -0800 (PST) X-Google-Smtp-Source: ABdhPJy9nkRXFxPwp97A2jIL0bFDCbP7MhKrN4ZfwpMh/DD+R8xLofgvVA1vNqXYCOYFGut6gX27 X-Received: by 2002:a17:906:6d49:: with SMTP id a9mr2768926ejt.32.1611314508636; Fri, 22 Jan 2021 03:21:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611314508; cv=none; d=google.com; s=arc-20160816; b=px6q/NZohbI2LrEuIXc7B5iCM7SK2LyF2QwE69sFwHnGgMiKma0YXmLNgu7m6aOn7x rCp14Eeg8DT824kNjv2/+KBQUWARxitiktmNGEeM9+hqCGKh7CVNkKsWowtCuOGE2+fB EDtY4PKpDW61xGvHxPhLyufTcOtDpiT7oUJUisdohc0TbqMFtONY5wbjiqCjV07AbW/6 UPY/QGIegx+cv7HmOgMqGMJPWGv5bTpiMEccBZvjn5ndHeh7tPNkdOFgDD/SaenCmN71 MWsqJoDI8ZVCkmLSorUl41UcsIoB3t7NYtvtwjNLmEb9bj5E5DlIbl3dz+l+e/WOOS/b IWdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=i2JDYyVtaQXRi1LCKlWrkeQ4GqyI2nFSFxOaqJ8Wqn4=; b=Yxm8q+DgZqbFqDUyXHjLjBd6MISfRwnGqDyeWqDAdDsGHQCzOsco09HncmUxfgYG1C KKdO1EvbmqckizZaV4sFqbXPx5/HKgRPfcCkNS8Ycow/Cr8YRPYeJfqz4WlpJb9Uv8pg NM5oUa8jZfPQLMtz4xrRuPSlMp7QXLJa9XoVsSlr9aRjl+PuO+yuBHPtymKTzdNNGera sQNqF/8Rj0hwlCH7civ05m100up9AHVH4phzq/MKraV235y8muA8yH4HaGkqjE/bDNeX DUiVzqH82XE0RF6MZSAmyoiASXsNn5bP0U6l7QUjN6mtxAkZz/u9WosTuNxfzdghMeqU ggpA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=X87aUzEN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n5si3536677eda.301.2021.01.22.03.21.48; Fri, 22 Jan 2021 03:21:48 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=X87aUzEN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7A1C568A55B; Fri, 22 Jan 2021 13:21:14 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f179.google.com (mail-lj1-f179.google.com [209.85.208.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EF12E68A3B2 for ; Fri, 22 Jan 2021 13:21:05 +0200 (EET) Received: by mail-lj1-f179.google.com with SMTP id l12so3556335ljc.3 for ; Fri, 22 Jan 2021 03:21:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=9O9XMB3MQPgzSi0KLQWTc5sfH3ZpcAFiA6rxguhpZd8=; b=X87aUzENWs+gDG3CHghNpc7X/wTwkwGYNwQBQqjsUmMnK3Txk14hHrGV27U2W+VngC PAZANhSeNydWNvO5lF7q8JrMj5ql2MXTO12FccDWeXidN+c3r8Q11sQtDgStI1NVShmi 3rZ054TvM+Pwbg0+Oe1bDnqBjZrsaFhGAXDDcCeoo2QQRuO1mDwLTaFByNC3MnzGq3/k 8N8tOR4WEN+7iCQz2cWR9auARusYXGnycvvZ4S8IDyhltkVOeQgip3yCS4VkPFW7ukxT xPXW3waN7luK0pP4c9JJOKq7X4BlDSvr3jNgsOt8XeVH2frjdsRf/H0uwhfFBk85L1R0 a+9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9O9XMB3MQPgzSi0KLQWTc5sfH3ZpcAFiA6rxguhpZd8=; b=mck+1n2bFII5AXK5Tiru8UcS5Q92J0BPTJI5VwkaSt0bnOHsFSb2u8e+mVfszVueG+ NR6TH2wPy3dj9t3Um784NtWyn0wMLq8mv2c1aZH8mpw4nMAbBxmvRoYakwJVUwukK6tF nA0JFjw49+Md/5B9tdBh4RKT39/bpxV0DKg0TWghcODkqTrP3yc2fO8hcmW/TGKeaJBY qGQsyl2vEjfpuYP2zVuZi0YpVBZikgCI2cUD8RbfwPRswRkIaQeOLwXm33kPeoB4cbNE Y2RtJ5Vb6z6pYGY1Z/01Xy5arCUQJm9OOdqi1mNxiFdO+Bv+sfA5vmKuYi/ZW4ktij0v OWMw== X-Gm-Message-State: AOAM53160VbYALbgbJTk0ZrqhC17EZYsIXm6eMvSk8A+aaZonvQN/CXY wmeMkRo60Evbo1MDVc7iJO2en0rUNg0= X-Received: by 2002:a2e:5005:: with SMTP id e5mr744956ljb.232.1611314465191; Fri, 22 Jan 2021 03:21:05 -0800 (PST) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id v24sm919451ljc.56.2021.01.22.03.21.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 03:21:04 -0800 (PST) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Fri, 22 Jan 2021 13:20:58 +0200 Message-Id: <20210122112058.49068-5-jeebjp@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122112058.49068-1-jeebjp@gmail.com> References: <20210122103725.24370-1-jeebjp@gmail.com> <20210122112058.49068-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 4/4] {avcodec, avformat}: add TTML encoder and muxer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: jWrzRIgshdpr Content-Length: 32158 From: Jan Ekström Enables encoding of other subtitle formats into TTML and writing them out as such documents. Signed-off-by: Jan Ekström --- Changelog | 1 + doc/general_contents.texi | 1 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/ttmlenc.c | 179 +++++++++++++++++++++++++++++++++++++ libavcodec/version.h | 4 +- libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/ttmlenc.c | 166 ++++++++++++++++++++++++++++++++++ libavformat/version.h | 4 +- tests/fate/subtitles.mak | 3 + tests/ref/fate/sub-ttmlenc | 122 +++++++++++++++++++++++++ 12 files changed, 480 insertions(+), 4 deletions(-) create mode 100644 libavcodec/ttmlenc.c create mode 100644 libavformat/ttmlenc.c create mode 100644 tests/ref/fate/sub-ttmlenc diff --git a/Changelog b/Changelog index 0b27c15122..9a6aeb4128 100644 --- a/Changelog +++ b/Changelog @@ -56,6 +56,7 @@ version : - shufflepixels filter - tmidequalizer filter - estdif filter +- TTML subtitle encoder and muxer version 4.3: diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 443e8ed8d1..d799382f84 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -1334,6 +1334,7 @@ performance on systems without hardware floating point support). @item SubViewer v1 @tab @tab X @tab @tab X @item SubViewer @tab @tab X @tab @tab X @item TED Talks captions @tab @tab X @tab @tab X +@item TTML @tab X @tab @tab X @tab @item VobSub (IDX+SUB) @tab @tab X @tab @tab X @item VPlayer @tab @tab X @tab @tab X @item WebVTT @tab X @tab X @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 5ce3ee0ec9..d26e0264de 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -669,6 +669,7 @@ OBJS-$(CONFIG_TSCC_DECODER) += tscc.o msrledec.o OBJS-$(CONFIG_TSCC2_DECODER) += tscc2.o OBJS-$(CONFIG_TTA_DECODER) += tta.o ttadata.o ttadsp.o OBJS-$(CONFIG_TTA_ENCODER) += ttaenc.o ttaencdsp.o ttadata.o +OBJS-$(CONFIG_TTML_ENCODER) += ttmlenc.o ass_split.o OBJS-$(CONFIG_TWINVQ_DECODER) += twinvqdec.o twinvq.o metasound_data.o OBJS-$(CONFIG_TXD_DECODER) += txd.o OBJS-$(CONFIG_ULTI_DECODER) += ulti.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index f00d524747..81d20c44ec 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -686,6 +686,7 @@ extern AVCodec ff_subviewer_decoder; extern AVCodec ff_subviewer1_decoder; extern AVCodec ff_text_encoder; extern AVCodec ff_text_decoder; +extern AVCodec ff_ttml_encoder; extern AVCodec ff_vplayer_decoder; extern AVCodec ff_webvtt_encoder; extern AVCodec ff_webvtt_decoder; diff --git a/libavcodec/ttmlenc.c b/libavcodec/ttmlenc.c new file mode 100644 index 0000000000..a9b1411467 --- /dev/null +++ b/libavcodec/ttmlenc.c @@ -0,0 +1,179 @@ +/* + * TTML subtitle encoder + * Copyright (c) 2020 24i + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * TTML subtitle encoder + * @see https://www.w3.org/TR/ttml1/ + * @see https://www.w3.org/TR/ttml2/ + * @see https://www.w3.org/TR/ttml-imsc/rec + */ + +#include "avcodec.h" +#include "libavutil/avstring.h" +#include "libavutil/bprint.h" +#include "ass_split.h" +#include "ass.h" + +typedef struct { + AVCodecContext *avctx; + ASSSplitContext *ass_ctx; + AVBPrint buffer; +} TTMLContext; + +static void ttml_text_cb(void *priv, const char *text, int len) +{ + TTMLContext *s = priv; + AVBPrint cur_line = { 0 }; + AVBPrint *buffer = &s->buffer; + + av_bprint_init(&cur_line, len, AV_BPRINT_SIZE_UNLIMITED); + + av_bprint_append_data(&cur_line, text, len); + if (!av_bprint_is_complete(&cur_line)) { + av_log(s->avctx, AV_LOG_ERROR, + "Failed to move the current subtitle dialog to AVBPrint!\n"); + av_bprint_finalize(&cur_line, NULL); + return; + } + + + av_bprint_escape(buffer, cur_line.str, NULL, AV_ESCAPE_MODE_XML_CHAR_DATA, + 0); + + av_bprint_finalize(&cur_line, NULL); +} + +static void ttml_new_line_cb(void *priv, int forced) +{ + TTMLContext *s = priv; + + av_bprintf(&s->buffer, "
"); +} + +static const ASSCodesCallbacks ttml_callbacks = { + .text = ttml_text_cb, + .new_line = ttml_new_line_cb, +}; + +static int ttml_encode_frame(AVCodecContext *avctx, uint8_t *buf, + int bufsize, const AVSubtitle *sub) +{ + TTMLContext *s = avctx->priv_data; + ASSDialog *dialog; + int i; + + av_bprint_clear(&s->buffer); + + for (i=0; inum_rects; i++) { + const char *ass = sub->rects[i]->ass; + + if (sub->rects[i]->type != SUBTITLE_ASS) { + av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n"); + return AVERROR(EINVAL); + } + +#if FF_API_ASS_TIMING + if (!strncmp(ass, "Dialogue: ", 10)) { + int num; + dialog = ff_ass_split_dialog(s->ass_ctx, ass, 0, &num); + + for (; dialog && num--; dialog++) { + ff_ass_split_override_codes(&ttml_callbacks, s, dialog->text); + } + } else { +#endif + dialog = ff_ass_split_dialog2(s->ass_ctx, ass); + if (!dialog) + return AVERROR(ENOMEM); + + ff_ass_split_override_codes(&ttml_callbacks, s, dialog->text); + ff_ass_free_dialog(&dialog); +#if FF_API_ASS_TIMING + } +#endif + } + + if (!av_bprint_is_complete(&s->buffer)) + return AVERROR(ENOMEM); + if (!s->buffer.len) + return 0; + + // force null-termination, so in case our destination buffer is + // too small, the return value is larger than bufsize minus null. + if (av_strlcpy(buf, s->buffer.str, bufsize) > bufsize - 1) { + av_log(avctx, AV_LOG_ERROR, "Buffer too small for TTML event.\n"); + return AVERROR_BUFFER_TOO_SMALL; + } + + return s->buffer.len; +} + +static av_cold int ttml_encode_close(AVCodecContext *avctx) +{ + TTMLContext *s = avctx->priv_data; + + ff_ass_split_free(s->ass_ctx); + + av_bprint_finalize(&s->buffer, NULL); + + return 0; +} + +static av_cold int ttml_encode_init(AVCodecContext *avctx) +{ + int ret = AVERROR_BUG; + TTMLContext *s = avctx->priv_data; + + s->avctx = avctx; + + if (!(s->ass_ctx = ff_ass_split(avctx->subtitle_header))) { + ret = AVERROR_INVALIDDATA; + goto failure; + } + + if (!(avctx->extradata = av_malloc(4 + AV_INPUT_BUFFER_PADDING_SIZE))) { + ret = AVERROR(ENOMEM); + goto failure; + } + avctx->extradata_size = 4; + + av_bprint_init(&s->buffer, 0, AV_BPRINT_SIZE_UNLIMITED); + + return 0; + +failure: + ff_ass_split_free(s->ass_ctx); + av_bprint_finalize(&s->buffer, NULL); + + return ret; +} + +AVCodec ff_ttml_encoder = { + .name = "ttml", + .long_name = NULL_IF_CONFIG_SMALL("TTML subtitle"), + .type = AVMEDIA_TYPE_SUBTITLE, + .id = AV_CODEC_ID_TTML, + .priv_data_size = sizeof(TTMLContext), + .init = ttml_encode_init, + .encode_sub = ttml_encode_frame, + .close = ttml_encode_close, +}; diff --git a/libavcodec/version.h b/libavcodec/version.h index 9f80caa9e0..807062af7f 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,8 +28,8 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 58 -#define LIBAVCODEC_VERSION_MINOR 117 -#define LIBAVCODEC_VERSION_MICRO 101 +#define LIBAVCODEC_VERSION_MINOR 118 +#define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \ diff --git a/libavformat/Makefile b/libavformat/Makefile index 3a8fbcbe5f..f39a613901 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -543,6 +543,7 @@ OBJS-$(CONFIG_TRUEHD_DEMUXER) += rawdec.o mlpdec.o OBJS-$(CONFIG_TRUEHD_MUXER) += rawenc.o OBJS-$(CONFIG_TTA_DEMUXER) += tta.o apetag.o img2.o OBJS-$(CONFIG_TTA_MUXER) += ttaenc.o apetag.o img2.o +OBJS-$(CONFIG_TTML_MUXER) += ttmlenc.o OBJS-$(CONFIG_TTY_DEMUXER) += tty.o sauce.o OBJS-$(CONFIG_TY_DEMUXER) += ty.o OBJS-$(CONFIG_TXD_DEMUXER) += txd.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 6990af55f4..f4454e8362 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -442,6 +442,7 @@ extern AVInputFormat ff_truehd_demuxer; extern AVOutputFormat ff_truehd_muxer; extern AVInputFormat ff_tta_demuxer; extern AVOutputFormat ff_tta_muxer; +extern AVOutputFormat ff_ttml_muxer; extern AVInputFormat ff_txd_demuxer; extern AVInputFormat ff_tty_demuxer; extern AVInputFormat ff_ty_demuxer; diff --git a/libavformat/ttmlenc.c b/libavformat/ttmlenc.c new file mode 100644 index 0000000000..826f110f86 --- /dev/null +++ b/libavformat/ttmlenc.c @@ -0,0 +1,166 @@ +/* + * TTML subtitle muxer + * Copyright (c) 2020 24i + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * TTML subtitle muxer + * @see https://www.w3.org/TR/ttml1/ + * @see https://www.w3.org/TR/ttml2/ + * @see https://www.w3.org/TR/ttml-imsc/rec + */ + +#include "avformat.h" +#include "internal.h" + +enum TTMLPacketType { + PACKET_TYPE_PARAGRAPH, + PACKET_TYPE_DOCUMENT, +}; + +typedef struct TTMLMuxContext { + enum TTMLPacketType input_type; + unsigned int document_written; +} TTMLMuxContext; + +static const char ttml_header_text[] = +"\n" +"\n" +" \n" +"
\n"; + +static const char ttml_footer_text[] = +"
\n" +" \n" +"\n"; + +static void ttml_write_time(AVIOContext *pb, const char tag[], + int64_t millisec) +{ + int64_t sec, min, hour; + sec = millisec / 1000; + millisec -= 1000 * sec; + min = sec / 60; + sec -= 60 * min; + hour = min / 60; + min -= 60 * hour; + + avio_printf(pb, "%s=\"%02"PRId64":%02"PRId64":%02"PRId64".%03"PRId64"\"", + tag, hour, min, sec, millisec); +} + +static int ttml_write_header(AVFormatContext *ctx) +{ + TTMLMuxContext *ttml_ctx = ctx->priv_data; + ttml_ctx->document_written = 0; + + if (ctx->nb_streams != 1 || + ctx->streams[0]->codecpar->codec_id != AV_CODEC_ID_TTML) { + av_log(ctx, AV_LOG_ERROR, "Exactly one TTML stream is required!\n"); + return AVERROR(EINVAL); + } + + { + AVStream *st = ctx->streams[0]; + AVIOContext *pb = ctx->pb; + + AVDictionaryEntry *lang = av_dict_get(st->metadata, "language", NULL, + 0); + const char *printed_lang = (lang && lang->value) ? lang->value : ""; + + // Not perfect, but decide whether the packet is a document or not + // by the existence of extradata. + ttml_ctx->input_type = st->codecpar->extradata ? + PACKET_TYPE_PARAGRAPH : + PACKET_TYPE_DOCUMENT; + + avpriv_set_pts_info(st, 64, 1, 1000); + + if (ttml_ctx->input_type == PACKET_TYPE_PARAGRAPH) + avio_printf(pb, ttml_header_text, printed_lang); + } + + return 0; +} + +static int ttml_write_packet(AVFormatContext *ctx, AVPacket *pkt) +{ + TTMLMuxContext *ttml_ctx = ctx->priv_data; + AVIOContext *pb = ctx->pb; + + switch (ttml_ctx->input_type) { + case PACKET_TYPE_PARAGRAPH: + // write out a paragraph element with the given contents. + avio_printf(pb, " pts); + avio_w8(pb, '\n'); + ttml_write_time(pb, " end", pkt->pts + pkt->duration); + avio_printf(pb, ">"); + avio_write(pb, pkt->data, pkt->size); + avio_printf(pb, "

\n"); + break; + case PACKET_TYPE_DOCUMENT: + // dump the given document out as-is. + if (ttml_ctx->document_written) { + av_log(ctx, AV_LOG_ERROR, + "Attempting to write multiple TTML documents into a " + "single document! The XML specification forbids this " + "as there has to be a single root tag.\n"); + return AVERROR(EINVAL); + } + avio_write(pb, pkt->data, pkt->size); + ttml_ctx->document_written = 1; + break; + default: + av_log(ctx, AV_LOG_ERROR, "Invalid TTML input packet type!\n"); + return AVERROR(EINVAL); + } + + return 0; +} + +static int ttml_write_trailer(AVFormatContext *ctx) +{ + TTMLMuxContext *ttml_ctx = ctx->priv_data; + AVIOContext *pb = ctx->pb; + + if (ttml_ctx->input_type == PACKET_TYPE_PARAGRAPH) + avio_printf(pb, ttml_footer_text); + + return 0; +} + +AVOutputFormat ff_ttml_muxer = { + .name = "ttml", + .long_name = NULL_IF_CONFIG_SMALL("TTML subtitle"), + .extensions = "ttml", + .mime_type = "text/ttml", + .priv_data_size = sizeof(TTMLMuxContext), + .flags = AVFMT_GLOBALHEADER | AVFMT_VARIABLE_FPS | + AVFMT_TS_NONSTRICT, + .subtitle_codec = AV_CODEC_ID_TTML, + .write_header = ttml_write_header, + .write_packet = ttml_write_packet, + .write_trailer = ttml_write_trailer, +}; diff --git a/libavformat/version.h b/libavformat/version.h index a2b5901e74..b12ce3a843 100644 --- a/libavformat/version.h +++ b/libavformat/version.h @@ -32,8 +32,8 @@ // Major bumping may affect Ticket5467, 5421, 5451(compatibility with Chromium) // Also please add any ticket numbers that you believe might be affected here #define LIBAVFORMAT_VERSION_MAJOR 58 -#define LIBAVFORMAT_VERSION_MINOR 65 -#define LIBAVFORMAT_VERSION_MICRO 101 +#define LIBAVFORMAT_VERSION_MINOR 66 +#define LIBAVFORMAT_VERSION_MICRO 100 #define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \ LIBAVFORMAT_VERSION_MINOR, \ diff --git a/tests/fate/subtitles.mak b/tests/fate/subtitles.mak index 6323d0f93d..ee65afe35b 100644 --- a/tests/fate/subtitles.mak +++ b/tests/fate/subtitles.mak @@ -106,6 +106,9 @@ fate-sub-scc: CMD = fmtstdout ass -ss 57 -i $(TARGET_SAMPLES)/sub/witch.scc FATE_SUBTITLES-$(call ALLYES, MPEGTS_DEMUXER DVBSUB_DECODER DVBSUB_ENCODER) += fate-sub-dvb fate-sub-dvb: CMD = framecrc -i $(TARGET_SAMPLES)/sub/dvbsubtest_filter.ts -map s:0 -c dvbsub +FATE_SUBTITLES-$(call ALLYES, FILE_PROTOCOL PIPE_PROTOCOL SRT_DEMUXER SUBRIP_DECODER TTML_ENCODER TTML_MUXER) += fate-sub-ttmlenc +fate-sub-ttmlenc: CMD = fmtstdout ttml -i $(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt + FATE_SUBTITLES-$(call ENCMUX, ASS, ASS) += $(FATE_SUBTITLES_ASS-yes) FATE_SUBTITLES += $(FATE_SUBTITLES-yes) diff --git a/tests/ref/fate/sub-ttmlenc b/tests/ref/fate/sub-ttmlenc new file mode 100644 index 0000000000..624f37d092 --- /dev/null +++ b/tests/ref/fate/sub-ttmlenc @@ -0,0 +1,122 @@ + + + +
+

Don't show this text it may be used to insert hidden data

+

SubRip subtitles capability tester 1.3o by ale5000
Use VLC 1.1 or higher as reference for most things and MPC Home Cinema for others
This text should be blue
This text should be red
This text should be black
If you see this with the normal font, the player don't (fully) support font face

+

Hidden

+

This text should be small
This text should be normal
This text should be big

+

This should be an E with an accent: È
日本語
This text should be bold, italics and underline
This text should be small and green
This text should be small and red
This text should be big and brown

+

This line should be bold
This line should be italics
This line should be underline
This line should be strikethrough
Both lines
should be underline

+

>
It would be a good thing to
hide invalid html tags that are closed and show the text in them
but show un-closed invalid html tags
Show not opened tags
<

+

and also
hide invalid html tags with parameters that are closed and show the text in them
but show un-closed invalid html tags
This text should be showed underlined without problems also: 2<3,5>1,4<6
This shouldn't be underlined

+

This text should be in the normal position...

+

This text should NOT be in the normal position

+

Implementation is the same of the ASS tag
This text should be at the
top and horizontally centered

+

This text should be at the
middle and horizontally centered

+

This text should be at the
bottom and horizontally centered

+

This text should be at the
top and horizontally at the left

+

This text should be at the
middle and horizontally at the left
(The second position must be ignored)

+

This text should be at the
bottom and horizontally at the left

+

This text should be at the
top and horizontally at the right

+

This text should be at the
middle and horizontally at the right

+

This text should be at the
bottom and horizontally at the right

+

This could be the most difficult thing to implement

+

First text

+

Second, it shouldn't overlap first

+

Third, it should replace second

+

Fourth, it shouldn't overlap first and third

+

Fifth, it should replace third

+

Sixth, it shouldn't be
showed overlapped

+

TEXT 1 (bottom)

+

text 2

+

Hide these tags:
also hide these tags:
but show this: {normal text}

+


\ N is a forced line break
\ h is a hard space
Normal spaces at the start and at the end of the line are trimmed while hard spaces are not trimmed.
The\hline\hwill\hnever\hbreak\hautomatically\hright\hbefore\hor\hafter\ha\hhard\hspace.\h:-D

+


\h\h\h\h\hA (05 hard spaces followed by a letter)
A (Normal spaces followed by a letter)
A (No hard spaces followed by a letter)

+

\h\h\h\h\hA (05 hard spaces followed by a letter)
A (Normal spaces followed by a letter)
A (No hard spaces followed by a letter)
Show this: \TEST and this: \-)

+


A letter followed by 05 hard spaces: A\h\h\h\h\h
A letter followed by normal spaces: A
A letter followed by no hard spaces: A
05 hard spaces between letters: A\h\h\h\h\hA
5 normal spaces between letters: A A

^--Forced line break

+

Both line should be strikethrough,
yes.
Correctly closed tags
should be hidden.

+

It shouldn't be strikethrough,
not opened tag showed as text.
Not opened tag showed as text.

+

Three lines should be strikethrough,
yes.
Not closed tags showed as text

+

Both line should be strikethrough but
the wrong closing tag should be showed

+
+ +