From patchwork Sat Mar 25 14:01:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Massimo Eynard X-Patchwork-Id: 40817 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a30:b0:df:834d:2c1a with SMTP id t48csp62911pzh; Sat, 25 Mar 2023 07:01:52 -0700 (PDT) X-Google-Smtp-Source: AKy350b6IU692SJjvVd3RSKB0kcShmnM51WdjEVJ+T2T3FKGHvreu0B2l361+dHtpwuvX1vrsdEv X-Received: by 2002:a17:906:7cc8:b0:932:8dc:5afe with SMTP id h8-20020a1709067cc800b0093208dc5afemr5612434ejp.67.1679752912427; Sat, 25 Mar 2023 07:01:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679752912; cv=none; d=google.com; s=arc-20160816; b=ixzQWpiHYWNyMcRgMEygiES1J+x28ftrFNz1tvblkjIIQU6iT9njzfsBsDofaEhcNW BTHnQTdNRMhcRaKvd4O6hO/p13m3rDpjtEgk91uNk/+o0k3J6BctBRMtMKeoozMLipJS WsnaVpjz0eOe0HQjqBhyx9pYozWECrLiFyjvVUT4hC9/aulc1Ltq61ukZ2ReScKXABVG zBRZ7vX+0naHDGg37klaGTET2A14zThEQHGiVe6kJUYU2I+ftS9ImgKaptXy+8uK1e3+ PLlKbMNwjoVarjrgUcuWym7/Lnh2lqNk0FcQLh/IK9PpHNnVFOZyL4VPp2J15z0lVzi+ icwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:organization:content-language:to:from:user-agent :mime-version:date:message-id:dkim-signature:dkim-filter :delivered-to; bh=xA3eo+AmzhXO8MIHRmNW+O04jKicWx1c2p4tGDZMi5o=; b=Ez7Gd1TTZrY8RyUdTYUV42/pQ6w3sN40kgC3hpp4D2cPfsjOh5peXS8U4IqEGiZCiP 71+7NGgWLKiYuVh1I7FDPlY9lTTlym52yh0fC7f1scQGMMAcn7M8WXjVzVjQ3XeDoqgk pXrpK0op/0Le9EGiTXQhb8TSydhUlLLtWyOK2jXWJvssT0McU+R0VzJ8fJ06mELI16if 4lfR5e19yjMB8apdPfHVmARl6EDk4Vuz9/L5EctcKjFCiSX2pMGX0Oph4jJ+lJzP76rn p++NHegRD6e1R/GRMHA3W2MUE/1wdZri01WVTczRoyw5UHbHUCVB+cKdPocagRpOPGuq kBRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@edu.univ-eiffel.fr header.s=AE9DBB4D-AFE8-4595-8910-BD3CC7E77531 header.b=ZlZv3MFL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=edu.univ-eiffel.fr Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l19-20020a170906795300b0093defbd6284si6543852ejo.1035.2023.03.25.07.01.51; Sat, 25 Mar 2023 07:01:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@edu.univ-eiffel.fr header.s=AE9DBB4D-AFE8-4595-8910-BD3CC7E77531 header.b=ZlZv3MFL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=edu.univ-eiffel.fr Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6B9DB68C908; Sat, 25 Mar 2023 16:01:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from smtpout02-ext4.partage.renater.fr (smtpout02-ext4.partage.renater.fr [194.254.241.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D64C968C412 for ; Sat, 25 Mar 2023 16:01:40 +0200 (EET) Received: from zmtaauth04.partage.renater.fr (zmtaauth04.partage.renater.fr [194.254.241.26]) by smtpout20.partage.renater.fr (Postfix) with ESMTP id 19BE8C952E for ; Sat, 25 Mar 2023 15:01:39 +0100 (CET) Received: from zmtaauth04.partage.renater.fr (localhost [127.0.0.1]) by zmtaauth04.partage.renater.fr (Postfix) with ESMTPS id 087301C00D6 for ; Sat, 25 Mar 2023 15:01:39 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zmtaauth04.partage.renater.fr (Postfix) with ESMTP id ED86C1C00D8 for ; Sat, 25 Mar 2023 15:01:38 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.10.3 zmtaauth04.partage.renater.fr ED86C1C00D8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=edu.univ-eiffel.fr; s=AE9DBB4D-AFE8-4595-8910-BD3CC7E77531; t=1679752898; bh=64dbXEOD7iuABeq1r0T+SlpJGYe2vOFHZcZ84r2QqWk=; h=Message-ID:Date:MIME-Version:From:To; b=ZlZv3MFLCS4sfsfo5O/hJhxVwCL/yQxgX6UHCN5ia09M964Dx55SfbudC/MGVkxGA vGf1gyiyvu/Z147NzkTMJNTUaVoMPV8KoHC7KDHXfQSGHz3lpjjSof3xQ2kGAMrQv2 KBw2HPFL1tmqCxJvpjU3z3TaNtxJ4sFZEAJ90LxiBgxbuKLJvxKd3AlcSTyqAbbfoT 5SrAvdMf8bYp5A+BaFJlcK/EZ22X0gUH8mjZt97N7RSgaJ6ucOmj+5R7krJtMMqIC8 jBd4CDpwnArTFgrkjJn67JM6mcCWtigzaoHTwO8dgwwZRrmRUH+nHMInbxDisa7YFc gRs3GYtZIx2RA== Received: from zmtaauth04.partage.renater.fr ([127.0.0.1]) by localhost (zmtaauth04.partage.renater.fr [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id az6fyhjPMVUo for ; Sat, 25 Mar 2023 15:01:38 +0100 (CET) Received: from [192.168.1.90] (unknown [194.254.241.249]) by zmtaauth04.partage.renater.fr (Postfix) with ESMTPA id AA8571C00D6 for ; Sat, 25 Mar 2023 15:01:38 +0100 (CET) Message-ID: Date: Sat, 25 Mar 2023 15:01:38 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 From: Massimo Eynard To: ffmpeg-devel@ffmpeg.org Content-Language: fr Organization: =?utf-8?q?Universit=C3=A9_Gustave_Eiffel?= X-Virus-Scanned: clamav-milter 0.103.8 at clamav01 X-Virus-Status: Clean X-Renater-Ptge-SpamState: clean X-Renater-Ptge-SpamScore: 0 X-Renater-Ptge-SpamCause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdegkedgheelucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecutffgpfetvffgtfenuceurghilhhouhhtmecufedttdenucenucfjughrpefkffggfgfhuffvohgtgfesthejredttdefjeenucfhrhhomhepofgrshhsihhmohcugfihnhgrrhguuceomhgrshhsihhmohdrvgihnhgrrhgusegvughurdhunhhivhdqvghifhhfvghlrdhfrheqnecuggftrfgrthhtvghrnhepjeelgfduudelvdevlefhleekuefgtddvkeffueffudfhkeduvdffudfhteffjedvnecuffhomhgrihhnpehvihguvgholhgrnhdrohhrghenucfkphepudelgedrvdehgedrvdeguddrvdegleenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpeduleegrddvheegrddvgedurddvgeelpdhhvghloheplgduledvrdduieekrddurdeltdgnpdhmrghilhhfrhhomhepofgrshhsihhmohcugfihnhgrrhguuceomhgrshhsihhmohdrvgihnhgrrhgusegvughurdhunhhivhdqvghifhhfvghlrdhfrheqpdhnsggprhgtphhtthhopedupdhrtghpthhtohepfhhfmhhpvghgqdguvghvvghlsehffhhmphgvghdrohhrgh Subject: [FFmpeg-devel] [PATCH] libavcodec/pgssubdec: Implement cropping X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: jDcV3080t/K0 Implement missing cropping option of subtitle bitmap, enabling complex cropping effects. A test sample as been submitted to https://streams.videolan.org/upload/. Signed-off-by: Massimo Eynard --- libavcodec/pgssubdec.c | 193 ++++++++++++++++++++++++++++------------- 1 file changed, 135 insertions(+), 58 deletions(-) int id; int w; int h; - uint8_t *rle; - unsigned int rle_buffer_size, rle_data_len; + uint8_t *rle; /**< Run Length Encoded bitmap. */ + unsigned int rle_buffer_size; + unsigned int rle_data_len; unsigned int rle_remaining_len; + uint8_t *bitmap; /**< Decoded bitmap. */ + unsigned int bitmap_buffer_size; + unsigned int bitmap_size; } PGSSubObject; typedef struct PGSSubObjects { @@ -105,8 +111,11 @@ static void flush_cache(AVCodecContext *avctx) for (i = 0; i < ctx->objects.count; i++) { av_freep(&ctx->objects.object[i].rle); - ctx->objects.object[i].rle_buffer_size = 0; + ctx->objects.object[i].rle_buffer_size = 0; ctx->objects.object[i].rle_remaining_len = 0; + av_freep(&ctx->objects.object[i].bitmap); + ctx->objects.object[i].bitmap_buffer_size = 0; + ctx->objects.object[i].bitmap_size = 0; } ctx->objects.count = 0; ctx->palettes.count = 0; @@ -149,57 +158,57 @@ static av_cold int close_decoder(AVCodecContext *avctx) } /** - * Decode the RLE data. + * Decode the RLE data of a subtitle object. * - * The subtitle is stored as a Run Length Encoded image. + * The subtitle is stored as a Run Length Encoded bitmap image. * - * @param avctx contains the current codec context - * @param sub pointer to the processed subtitle data - * @param buf pointer to the RLE data to process - * @param buf_size size of the RLE data to process + * @param avctx Contains the current codec context. + * @param object Pointer to the processed subtitle object. */ -static int decode_rle(AVCodecContext *avctx, AVSubtitleRect *rect, - const uint8_t *buf, unsigned int buf_size) +static int decode_object_rle(AVCodecContext *avctx, PGSSubObject *object) { - const uint8_t *rle_bitmap_end; + const uint8_t *rle_buf; + const uint8_t *rle_end; int pixel_count, line_count; - rle_bitmap_end = buf + buf_size; + rle_buf = object->rle; + rle_end = object->rle + object->rle_data_len; - rect->data[0] = av_malloc_array(rect->w, rect->h); - - if (!rect->data[0]) + object->bitmap_size = object->w * object->h; + av_fast_padded_malloc(&object->bitmap, &object->bitmap_buffer_size, object->bitmap_size); + if (!object->bitmap) return AVERROR(ENOMEM); pixel_count = 0; line_count = 0; - while (buf < rle_bitmap_end && line_count < rect->h) { + while (rle_buf < rle_end && line_count < object->h) { uint8_t flags, color; int run; - color = bytestream_get_byte(&buf); + color = bytestream_get_byte(&rle_buf); run = 1; if (color == 0x00) { - flags = bytestream_get_byte(&buf); + flags = bytestream_get_byte(&rle_buf); run = flags & 0x3f; if (flags & 0x40) - run = (run << 8) + bytestream_get_byte(&buf); - color = flags & 0x80 ? bytestream_get_byte(&buf) : 0; + run = (run << 8) + bytestream_get_byte(&rle_buf); + color = flags & 0x80 ? bytestream_get_byte(&rle_buf) : 0; } - if (run > 0 && pixel_count + run <= rect->w * rect->h) { - memset(rect->data[0] + pixel_count, color, run); + if (run > 0 && pixel_count + run <= object->w * object->h) { + memset(object->bitmap + pixel_count, color, run); pixel_count += run; } else if (!run) { /* * New Line. Check if correct pixels decoded, if not display warning * and adjust bitmap pointer to correct new line position. */ - if (pixel_count % rect->w > 0) { - av_log(avctx, AV_LOG_ERROR, "Decoded %d pixels, when line should be %d pixels\n", - pixel_count % rect->w, rect->w); + if (pixel_count % object->w > 0) { + av_log(avctx, AV_LOG_ERROR, + "Decoded %d pixels, when object line should be %d pixels\n", + pixel_count % object->w, object->w); if (avctx->err_recognition & AV_EF_EXPLODE) { return AVERROR_INVALIDDATA; } @@ -208,12 +217,12 @@ static int decode_rle(AVCodecContext *avctx, AVSubtitleRect *rect, } } - if (pixel_count < rect->w * rect->h) { - av_log(avctx, AV_LOG_ERROR, "Insufficient RLE data for subtitle\n"); + if (pixel_count < object->w * object->h) { + av_log(avctx, AV_LOG_ERROR, "Insufficient RLE data for object\n"); return AVERROR_INVALIDDATA; } - ff_dlog(avctx, "Pixel Count = %d, Area = %d\n", pixel_count, rect->w * rect->h); + ff_dlog(avctx, "Pixel Count = %d, Area = %d\n", pixel_count, object->w * object->h); return 0; } @@ -290,13 +299,15 @@ static int parse_object_segment(AVCodecContext *avctx, height = bytestream_get_be16(&buf); /* Make sure the bitmap is not too large */ - if (avctx->width < width || avctx->height < height || !width || !height) { - av_log(avctx, AV_LOG_ERROR, "Bitmap dimensions (%dx%d) invalid.\n", width, height); + if (MAX_OBJECT_WH < width || MAX_OBJECT_WH < height || !width || !height) { + av_log(avctx, AV_LOG_ERROR, + "Bitmap dimensions (%dx%d) invalid.\n", width, height); return AVERROR_INVALIDDATA; } object->w = width; object->h = height; + /* Dimensions against video are checked at decode after cropping. */ av_fast_padded_malloc(&object->rle, &object->rle_buffer_size, rle_bitmap_len); @@ -468,16 +479,7 @@ static int parse_presentation_segment(AVCodecContext *avctx, ff_dlog(avctx, "Subtitle Placement x=%d, y=%d\n", object->x, object->y); - - if (object->x > avctx->width || object->y > avctx->height) { - av_log(avctx, AV_LOG_ERROR, "Subtitle out of video bounds. x = %d, y = %d, video width = %d, video height = %d.\n", - object->x, object->y, - avctx->width, avctx->height); - object->y = object->x = 0; - if (avctx->err_recognition & AV_EF_EXPLODE) { - return AVERROR_INVALIDDATA; - } - } + /* Placement is checked at decode after cropping. */ } return 0; @@ -528,6 +530,7 @@ static int display_end_segment(AVCodecContext *avctx, AVSubtitle *sub, return AVERROR_INVALIDDATA; } for (i = 0; i < ctx->presentation.object_count; i++) { + const PGSSubObjectRef sub_object = ctx->presentation.objects[i]; AVSubtitleRect *const rect = av_mallocz(sizeof(*rect)); PGSSubObject *object; @@ -537,45 +540,119 @@ static int display_end_segment(AVCodecContext *avctx, AVSubtitle *sub, rect->type = SUBTITLE_BITMAP; /* Process bitmap */ - object = find_object(ctx->presentation.objects[i].id, &ctx->objects); + object = find_object(sub_object.id, &ctx->objects); if (!object) { // Missing object. Should only happen with damaged streams. av_log(avctx, AV_LOG_ERROR, "Invalid object id %d\n", - ctx->presentation.objects[i].id); + sub_object.id); if (avctx->err_recognition & AV_EF_EXPLODE) return AVERROR_INVALIDDATA; // Leaves rect empty with 0 width and height. continue; } - if (ctx->presentation.objects[i].composition_flag & 0x40) + if (sub_object.composition_flag & 0x40) rect->flags |= AV_SUBTITLE_FLAG_FORCED; - rect->x = ctx->presentation.objects[i].x; - rect->y = ctx->presentation.objects[i].y; + rect->x = sub_object.x; + rect->y = sub_object.y; if (object->rle) { - rect->w = object->w; - rect->h = object->h; + int out_of_picture = 0; + rect->w = object->w; + rect->h = object->h; rect->linesize[0] = object->w; - if (object->rle_remaining_len) { - av_log(avctx, AV_LOG_ERROR, "RLE data length %u is %u bytes shorter than expected\n", - object->rle_data_len, object->rle_remaining_len); + // Check for cropping. + if (sub_object.composition_flag & 0x80) { + int out_of_object = 0; + + if (object->w < sub_object.crop_x + sub_object.crop_w) + out_of_object = 1; + if (object->h < sub_object.crop_y + sub_object.crop_h) + out_of_object = 1; + + if (out_of_object) { + av_log(avctx, AV_LOG_ERROR, + "Subtitle cropping values are out of object. " + "obj_w = %d, obj_h = %d, crop_x = %d, crop_y = %d, " + "crop_w = %d, crop_h = %d.\n", + object->w, + object->h, + sub_object.crop_x, + sub_object.crop_y, + sub_object.crop_w, + sub_object.crop_h); + if (avctx->err_recognition & AV_EF_EXPLODE) + return AVERROR_INVALIDDATA; + } + else { + // Replace subtitle dimensions with cropping ones. + rect->w = sub_object.crop_w; + rect->h = sub_object.crop_h; + rect->linesize[0] = sub_object.crop_w; + } + } + + /* Make sure the subtitle is not out of picture. */ + if (avctx->width < rect->x + rect->w || !rect->w) + out_of_picture = 1; + if (avctx->height < rect->y + rect->h || !rect->h) + out_of_picture = 1; + if (out_of_picture) { + av_log(avctx, AV_LOG_ERROR, + "Subtitle out of video bounds. " + "x = %d, y = %d, width = %d, height = %d.\n", + rect->x, rect->y, rect->w, rect->h); if (avctx->err_recognition & AV_EF_EXPLODE) return AVERROR_INVALIDDATA; - } - ret = decode_rle(avctx, rect, object->rle, object->rle_data_len); - if (ret < 0) { - if ((avctx->err_recognition & AV_EF_EXPLODE) || - ret == AVERROR(ENOMEM)) { - return ret; - } rect->w = 0; rect->h = 0; continue; } + + if (!object->bitmap_size) { + /* Decode bitmap from RLE. */ + if (object->rle_remaining_len) { + av_log(avctx, AV_LOG_ERROR, + "RLE data length %u is %u bytes shorter than expected\n", + object->rle_data_len, object->rle_remaining_len); + if (avctx->err_recognition & AV_EF_EXPLODE) + return AVERROR_INVALIDDATA; + } + + ret = decode_object_rle(avctx, object); + if (ret < 0) { + if ((avctx->err_recognition & AV_EF_EXPLODE) || + ret == AVERROR(ENOMEM)) { + return ret; + } + rect->w = 0; + rect->h = 0; + continue; + } + } + + rect->data[0] = av_malloc_array(rect->w, rect->h); + if (!rect->data[0]) + return AVERROR(ENOMEM); + + if (sub_object.composition_flag & 0x80) { + /* Copy cropped bitmap. */ + int y; + + for (y = 0; y < sub_object.crop_h; y++) { + memcpy(&rect->data[0][y * sub_object.crop_w], + &object->bitmap[(sub_object.crop_y + y) * + object->w + sub_object.crop_x], + sub_object.crop_w); + } + } + else { + memcpy(rect->data[0], object->bitmap, object->bitmap_size); + } } + /* Allocate memory for colors */ rect->nb_colors = 256; rect->data[1] = av_mallocz(AVPALETTE_SIZE); diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c index 5f76f12615..655f39b7b0 100644 --- a/libavcodec/pgssubdec.c +++ b/libavcodec/pgssubdec.c @@ -38,6 +38,8 @@ #define MAX_EPOCH_OBJECTS 64 // Max 64 allowed per PGS epoch #define MAX_OBJECT_REFS 2 // Max objects per display set +#define MAX_OBJECT_WH 4096 // Max object width/height + enum SegmentType { PALETTE_SEGMENT = 0x14, OBJECT_SEGMENT = 0x15, @@ -70,9 +72,13 @@ typedef struct PGSSubObject {