From patchwork Tue Apr 4 09:15:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Massimo Eynard X-Patchwork-Id: 40981 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp3404530pzb; Tue, 4 Apr 2023 02:15:48 -0700 (PDT) X-Google-Smtp-Source: AKy350YochpUDtJlGEY1WVf2hrONiSY0OYTb9y04gTijXifpbbWNhC0eIl2fbvysJQ9b8tMATUn4 X-Received: by 2002:a17:907:2c66:b0:920:7a99:dcd4 with SMTP id ib6-20020a1709072c6600b009207a99dcd4mr1818024ejc.62.1680599748710; Tue, 04 Apr 2023 02:15:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680599748; cv=none; d=google.com; s=arc-20160816; b=esUWVmxEun2zn0+b2LsKkRPzCGH63lH+M4Wypu7DKTnt2aEEfwQBdNRBvG1q6dRBYU LeJtaTuMK0Bhn1g8wrSOxq9267zya/w2b/BeIdh9gwHMxxu1cD1zjsrzXolZRhrebgcQ jd7PjxFv7Kg7EkBI/XSud9GOC0NcjOYT2ntyOQ4xL2DTv6oobXQTuAD796UvI5vo5ttY ulmW03g32JtCRX7mwUkH+PVc6ndJIaDIa9u42eIefq3QpQAHrj0zB/BlUtONl9Fjg/Mb GAi6HgW5ez3nbYvTgOFy6geUlGw8xTk5odn4SOI1IKDSCivRqB9JlF4Yhq8TR/U4tQeA scEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:in-reply-to:organization:content-language :references:to:from:user-agent:mime-version:date:message-id :dkim-signature:dkim-filter:delivered-to; bh=2PrgTgYF9V+P6AsAFxNduXL/4hb5kTsXDDBzGfox/Qw=; b=U+AkeHBRcTItKOkUiw+or9FTRjm7XsmOhlWw/IvijXrZaCuF16jzbaWIILzvqSwPpF mFLxnveMPSpzkj8ANnHsefr0gIhAWzY3fnerdyO9w2Lyzj/fQJkRdvlCrttkDXW/9Zaz ThZIUG7FyaWMBbnT8DWi8JMl4+Xi6hpMxAAS2pUhoyLnQTYe7LEuJm+suPmtiFeEFkS9 ps2NcqRmoC9rtPBl5Puw0C4GnlYBabftein/VQ5+Mwv/37ZGA2k3VMVc9EWwQb3vtbNf KYXcRIgO/mgYK9xLEc5MReNILbi7wsJTIAcS8e+ZHBnGDTgT+VuWZ8y450SISzJMNxg4 /Hfw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@edu.univ-eiffel.fr header.s=AE9DBB4D-AFE8-4595-8910-BD3CC7E77531 header.b=hVrkpoRP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=edu.univ-eiffel.fr Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g12-20020a056402180c00b005021f0d5755si6836023edy.668.2023.04.04.02.15.47; Tue, 04 Apr 2023 02:15:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@edu.univ-eiffel.fr header.s=AE9DBB4D-AFE8-4595-8910-BD3CC7E77531 header.b=hVrkpoRP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=edu.univ-eiffel.fr Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DE68A689936; Tue, 4 Apr 2023 12:15:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from smtpout01-ext2.partage.renater.fr (smtpout01-ext2.partage.renater.fr [194.254.240.33]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D505568A14E for ; Tue, 4 Apr 2023 12:15:37 +0300 (EEST) Received: from zmtaauth03.partage.renater.fr (zmtaauth03.partage.renater.fr [194.254.240.26]) by smtpout10.partage.renater.fr (Postfix) with ESMTP id 1ACBF69B7E for ; Tue, 4 Apr 2023 11:15:36 +0200 (CEST) Received: from zmtaauth03.partage.renater.fr (localhost [127.0.0.1]) by zmtaauth03.partage.renater.fr (Postfix) with ESMTPS id CA8BB80171 for ; Tue, 4 Apr 2023 11:15:06 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by zmtaauth03.partage.renater.fr (Postfix) with ESMTP id B978580164 for ; Tue, 4 Apr 2023 11:15:06 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 zmtaauth03.partage.renater.fr B978580164 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=edu.univ-eiffel.fr; s=AE9DBB4D-AFE8-4595-8910-BD3CC7E77531; t=1680599706; bh=rw/IwcbXLlfOWfV49hlQEjTNFkC59MmHDcT/UvHQiv8=; h=Message-ID:Date:MIME-Version:From:To; b=hVrkpoRPkkrwu6iYNdtqBbwlGO7uDqJv5nj2i2UYMLhcK9aTQK2UiX0nDTO8JcuTo sO0nFPD6H2F9H7y/xaWRKgWAN6xsFwkgSUc0phyoS45tNIIokPjic/uyvKyP1wX9vV XcNXfxyM4H0x3vdDPfZzQwOcjkzCzfc9oUw9v0Zapx9cpuJDVtQ+jpUAV4fFnJBGCw LAXk7rHZ8Bvqt14A83Fug1b36GfZYJsjfC4DjtQYECln0v71loOsSLpbNj1oTUBANG BupiytRSD5LKJczMFJ2REXb3cTt3L5JshZZ2+oij+ggVzIA025KMuaO3gFIRgJmZVZ qPWLdDGJyxiig== Received: from zmtaauth03.partage.renater.fr ([127.0.0.1]) by localhost (zmtaauth03.partage.renater.fr [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 4Xtbt_o_ixDZ for ; Tue, 4 Apr 2023 11:15:06 +0200 (CEST) Received: from [192.168.1.90] (unknown [194.254.241.249]) by zmtaauth03.partage.renater.fr (Postfix) with ESMTPA id 704198022E for ; Tue, 4 Apr 2023 11:15:06 +0200 (CEST) Message-ID: Date: Tue, 4 Apr 2023 11:15:07 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.9.1 From: Massimo Eynard To: ffmpeg-devel@ffmpeg.org References: <307774> Content-Language: fr Organization: =?utf-8?q?Universit=C3=A9_Gustave_Eiffel?= In-Reply-To: <307774> X-Virus-Scanned: clamav-milter 0.103.8 at clamav04 X-Virus-Status: Clean X-Renater-Ptge-SpamState: clean X-Renater-Ptge-SpamScore: 0 X-Renater-Ptge-SpamCause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdeiledgudefucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecutffgpfetvffgtfenuceurghilhhouhhtmecufedttdenucenucfjughrpefkffggfgfhuffvfhhojggtgfesthejredttdefjeenucfhrhhomhepofgrshhsihhmohcugfihnhgrrhguuceomhgrshhsihhmohdrvgihnhgrrhgusegvughurdhunhhivhdqvghifhhfvghlrdhfrheqnecuggftrfgrthhtvghrnhepteffudehieelgfdvuddvudfhtdehffevvedtueffkeduheffhffhgfekgeekvdfgnecuffhomhgrihhnpehvihguvgholhgrnhdrohhrghenucfkphepudelgedrvdehgedrvdeguddrvdegleenuceurggutfgvphhuthfkphepudelgedrvdehgedrvdeguddrvdegleenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpeduleegrddvheegrddvgedurddvgeelpdhhvghloheplgduledvrdduieekrddurdeltdgnpdhmrghilhhfrhhomhepofgrshhsihhmohcugfihnhgrrhguuceomhgrshhsihhmohdrvgihnhgrrhgusegvughurdhunhhivhdqvghifhhfvghlrdhfrheqpdhnsggprhgtphhtthhopedupdhrtghpthhtohepfhhfmhhpvghgqdguvghvvghlsehffhhmphgvghdrohhrgh Subject: [FFmpeg-devel] [PATCH] libavcodec/pgssubdec: Implement cropping X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Lg+zeONsebUV Implement missing cropping option of subtitle bitmap, enabling complex cropping effects. A test sample as been submitted to https://streams.videolan.org/upload/. Attempt to correct the previous patch. Signed-off-by: Massimo Eynard --- libavcodec/pgssubdec.c | 194 ++++++++++++++++++++++++++++------------- 1 file changed, 134 insertions(+), 60 deletions(-) for (i = 0; i < ctx->objects.count; i++) { av_freep(&ctx->objects.object[i].rle); - ctx->objects.object[i].rle_buffer_size = 0; + ctx->objects.object[i].rle_buffer_size = 0; ctx->objects.object[i].rle_remaining_len = 0; + av_freep(&ctx->objects.object[i].bitmap); + ctx->objects.object[i].bitmap_buffer_size = 0; + ctx->objects.object[i].bitmap_size = 0; } ctx->objects.count = 0; ctx->palettes.count = 0; @@ -149,57 +156,57 @@ static av_cold int close_decoder(AVCodecContext *avctx) } /** - * Decode the RLE data. + * Decode the RLE data of a subtitle object. * - * The subtitle is stored as a Run Length Encoded image. + * The subtitle is stored as a Run Length Encoded bitmap image. * - * @param avctx contains the current codec context - * @param sub pointer to the processed subtitle data - * @param buf pointer to the RLE data to process - * @param buf_size size of the RLE data to process + * @param avctx Contains the current codec context. + * @param object Pointer to the processed subtitle object. */ -static int decode_rle(AVCodecContext *avctx, AVSubtitleRect *rect, - const uint8_t *buf, unsigned int buf_size) +static int decode_object_rle(AVCodecContext *avctx, PGSSubObject *object) { - const uint8_t *rle_bitmap_end; + const uint8_t *rle_buf; + const uint8_t *rle_end; int pixel_count, line_count; - rle_bitmap_end = buf + buf_size; - - rect->data[0] = av_malloc_array(rect->w, rect->h); + rle_buf = object->rle; + rle_end = object->rle + object->rle_data_len; - if (!rect->data[0]) + object->bitmap_size = object->w * object->h; + av_fast_padded_malloc(&object->bitmap, &object->bitmap_buffer_size, object->bitmap_size); + if (!object->bitmap) return AVERROR(ENOMEM); pixel_count = 0; line_count = 0; - while (buf < rle_bitmap_end && line_count < rect->h) { + while (rle_buf < rle_end && line_count < object->h) { uint8_t flags, color; int run; - color = bytestream_get_byte(&buf); + color = bytestream_get_byte(&rle_buf); run = 1; if (color == 0x00) { - flags = bytestream_get_byte(&buf); + flags = bytestream_get_byte(&rle_buf); run = flags & 0x3f; if (flags & 0x40) - run = (run << 8) + bytestream_get_byte(&buf); - color = flags & 0x80 ? bytestream_get_byte(&buf) : 0; + run = (run << 8) + bytestream_get_byte(&rle_buf); + color = flags & 0x80 ? bytestream_get_byte(&rle_buf) : 0; } - if (run > 0 && pixel_count + run <= rect->w * rect->h) { - memset(rect->data[0] + pixel_count, color, run); + if (run > 0 && pixel_count + run <= object->w * object->h) { + memset(object->bitmap + pixel_count, color, run); pixel_count += run; } else if (!run) { /* * New Line. Check if correct pixels decoded, if not display warning * and adjust bitmap pointer to correct new line position. */ - if (pixel_count % rect->w > 0) { - av_log(avctx, AV_LOG_ERROR, "Decoded %d pixels, when line should be %d pixels\n", - pixel_count % rect->w, rect->w); + if (pixel_count % object->w > 0) { + av_log(avctx, AV_LOG_ERROR, + "Decoded %d pixels, when object line should be %d pixels\n", + pixel_count % object->w, object->w); if (avctx->err_recognition & AV_EF_EXPLODE) { return AVERROR_INVALIDDATA; } @@ -208,12 +215,12 @@ static int decode_rle(AVCodecContext *avctx, AVSubtitleRect *rect, } } - if (pixel_count < rect->w * rect->h) { - av_log(avctx, AV_LOG_ERROR, "Insufficient RLE data for subtitle\n"); + if (pixel_count < object->w * object->h) { + av_log(avctx, AV_LOG_ERROR, "Insufficient RLE data for object\n"); return AVERROR_INVALIDDATA; } - ff_dlog(avctx, "Pixel Count = %d, Area = %d\n", pixel_count, rect->w * rect->h); + ff_dlog(avctx, "Pixel Count = %d, Area = %d\n", pixel_count, object->w * object->h); return 0; } @@ -289,14 +296,16 @@ static int parse_object_segment(AVCodecContext *avctx, width = bytestream_get_be16(&buf); height = bytestream_get_be16(&buf); - /* Make sure the bitmap is not too large */ - if (avctx->width < width || avctx->height < height || !width || !height) { - av_log(avctx, AV_LOG_ERROR, "Bitmap dimensions (%dx%d) invalid.\n", width, height); + /* Make sure the bitmap size is not zero */ + if (!width || !height) { + av_log(avctx, AV_LOG_ERROR, + "Bitmap dimensions (%dx%d) invalid.\n", width, height); return AVERROR_INVALIDDATA; } object->w = width; object->h = height; + /* Dimensions against video are checked at decode after cropping. */ av_fast_padded_malloc(&object->rle, &object->rle_buffer_size, rle_bitmap_len); @@ -383,7 +392,6 @@ static int parse_palette_segment(AVCodecContext *avctx, * @param avctx contains the current codec context * @param buf pointer to the packet to process * @param buf_size size of packet to process - * @todo TODO: Implement cropping */ static int parse_presentation_segment(AVCodecContext *avctx, const uint8_t *buf, int buf_size, @@ -468,16 +476,7 @@ static int parse_presentation_segment(AVCodecContext *avctx, ff_dlog(avctx, "Subtitle Placement x=%d, y=%d\n", object->x, object->y); - - if (object->x > avctx->width || object->y > avctx->height) { - av_log(avctx, AV_LOG_ERROR, "Subtitle out of video bounds. x = %d, y = %d, video width = %d, video height = %d.\n", - object->x, object->y, - avctx->width, avctx->height); - object->y = object->x = 0; - if (avctx->err_recognition & AV_EF_EXPLODE) { - return AVERROR_INVALIDDATA; - } - } + /* Placement is checked at decode after cropping. */ } return 0; @@ -528,6 +527,7 @@ static int display_end_segment(AVCodecContext *avctx, AVSubtitle *sub, return AVERROR_INVALIDDATA; } for (i = 0; i < ctx->presentation.object_count; i++) { + const PGSSubObjectRef sub_object = ctx->presentation.objects[i]; AVSubtitleRect *const rect = av_mallocz(sizeof(*rect)); PGSSubObject *object; @@ -537,45 +537,119 @@ static int display_end_segment(AVCodecContext *avctx, AVSubtitle *sub, rect->type = SUBTITLE_BITMAP; /* Process bitmap */ - object = find_object(ctx->presentation.objects[i].id, &ctx->objects); + object = find_object(sub_object.id, &ctx->objects); if (!object) { // Missing object. Should only happen with damaged streams. av_log(avctx, AV_LOG_ERROR, "Invalid object id %d\n", - ctx->presentation.objects[i].id); + sub_object.id); if (avctx->err_recognition & AV_EF_EXPLODE) return AVERROR_INVALIDDATA; // Leaves rect empty with 0 width and height. continue; } - if (ctx->presentation.objects[i].composition_flag & 0x40) + if (sub_object.composition_flag & 0x40) rect->flags |= AV_SUBTITLE_FLAG_FORCED; - rect->x = ctx->presentation.objects[i].x; - rect->y = ctx->presentation.objects[i].y; + rect->x = sub_object.x; + rect->y = sub_object.y; if (object->rle) { - rect->w = object->w; - rect->h = object->h; + int out_of_picture = 0; + rect->w = object->w; + rect->h = object->h; rect->linesize[0] = object->w; - if (object->rle_remaining_len) { - av_log(avctx, AV_LOG_ERROR, "RLE data length %u is %u bytes shorter than expected\n", - object->rle_data_len, object->rle_remaining_len); + // Check for cropping. + if (sub_object.composition_flag & 0x80) { + int out_of_object = 0; + + if (object->w < sub_object.crop_x + sub_object.crop_w) + out_of_object = 1; + if (object->h < sub_object.crop_y + sub_object.crop_h) + out_of_object = 1; + + if (out_of_object) { + av_log(avctx, AV_LOG_ERROR, + "Subtitle cropping values are out of object. " + "obj_w = %d, obj_h = %d, crop_x = %d, crop_y = %d, " + "crop_w = %d, crop_h = %d.\n", + object->w, + object->h, + sub_object.crop_x, + sub_object.crop_y, + sub_object.crop_w, + sub_object.crop_h); + if (avctx->err_recognition & AV_EF_EXPLODE) + return AVERROR_INVALIDDATA; + } + else { + // Replace subtitle dimensions with cropping ones. + rect->w = sub_object.crop_w; + rect->h = sub_object.crop_h; + rect->linesize[0] = sub_object.crop_w; + } + } + + /* Make sure the subtitle is not out of picture. */ + if (avctx->width < rect->x + rect->w || !rect->w) + out_of_picture = 1; + if (avctx->height < rect->y + rect->h || !rect->h) + out_of_picture = 1; + if (out_of_picture) { + av_log(avctx, AV_LOG_ERROR, + "Subtitle out of video bounds. " + "x = %d, y = %d, width = %d, height = %d.\n", + rect->x, rect->y, rect->w, rect->h); if (avctx->err_recognition & AV_EF_EXPLODE) return AVERROR_INVALIDDATA; - } - ret = decode_rle(avctx, rect, object->rle, object->rle_data_len); - if (ret < 0) { - if ((avctx->err_recognition & AV_EF_EXPLODE) || - ret == AVERROR(ENOMEM)) { - return ret; - } rect->w = 0; rect->h = 0; continue; } + + if (!object->bitmap_size) { + /* Decode bitmap from RLE. */ + if (object->rle_remaining_len) { + av_log(avctx, AV_LOG_ERROR, + "RLE data length %u is %u bytes shorter than expected\n", + object->rle_data_len, object->rle_remaining_len); + if (avctx->err_recognition & AV_EF_EXPLODE) + return AVERROR_INVALIDDATA; + } + + ret = decode_object_rle(avctx, object); + if (ret < 0) { + if ((avctx->err_recognition & AV_EF_EXPLODE) || + ret == AVERROR(ENOMEM)) { + return ret; + } + rect->w = 0; + rect->h = 0; + continue; + } + } + + rect->data[0] = av_malloc_array(rect->w, rect->h); + if (!rect->data[0]) + return AVERROR(ENOMEM); + + if (sub_object.composition_flag & 0x80) { + /* Copy cropped bitmap. */ + int y; + + for (y = 0; y < sub_object.crop_h; y++) { + memcpy(&rect->data[0][y * sub_object.crop_w], + &object->bitmap[(sub_object.crop_y + y) * + object->w + sub_object.crop_x], + sub_object.crop_w); + } + } + else { + memcpy(rect->data[0], object->bitmap, object->bitmap_size); + } } + /* Allocate memory for colors */ rect->nb_colors = 256; rect->data[1] = av_mallocz(AVPALETTE_SIZE); diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c index 5f76f12615..050667dbf3 100644 --- a/libavcodec/pgssubdec.c +++ b/libavcodec/pgssubdec.c @@ -70,9 +70,13 @@ typedef struct PGSSubObject { int id; int w; int h; - uint8_t *rle; - unsigned int rle_buffer_size, rle_data_len; + uint8_t *rle; /**< Run Length Encoded bitmap. */ + unsigned int rle_buffer_size; + unsigned int rle_data_len; unsigned int rle_remaining_len; + uint8_t *bitmap; /**< Decoded bitmap. */ + unsigned int bitmap_buffer_size; + unsigned int bitmap_size; } PGSSubObject; typedef struct PGSSubObjects { @@ -105,8 +109,11 @@ static void flush_cache(AVCodecContext *avctx)