From patchwork Thu May 25 15:45:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: cubi cibo X-Patchwork-Id: 41826 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c51c:b0:10c:5e6f:955f with SMTP id gm28csp645775pzb; Thu, 25 May 2023 08:47:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6T1RzcC1oRD3H2sNIA6kSx3NZXDCgbpGBreKbSdSDhH5v+xBjT69CpS1auKECkbVL9qeFF X-Received: by 2002:a50:ee82:0:b0:510:5d6d:552e with SMTP id f2-20020a50ee82000000b005105d6d552emr3975018edr.40.1685029624367; Thu, 25 May 2023 08:47:04 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h26-20020aa7c61a000000b0050e5dbb5d4asi1048763edq.126.2023.05.25.08.47.02; Thu, 25 May 2023 08:47:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b="h/N7gIUB"; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 42D0768C186; Thu, 25 May 2023 18:46:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from GBR01-CWL-obe.outbound.protection.outlook.com (mail-cwlgbr01olkn0181.outbound.protection.outlook.com [104.47.20.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4F9E168BB5A for ; Thu, 25 May 2023 18:46:51 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FoZzQ5M0Or3uu6hYwISu4v98woKSfm/zBC7Nf4wZp0w/BnPBxRK/yszeTGJLxhNv74rx0KXae9H3Js8m14xCEQgmnSG7ITnsvIAswdRuZFiwsAF2kgYDRmYZYyvr+ztTD0cxfaRPQiW+i5i8xa5N23kPnYROUK7QUVq3GkUMRJBAVF3uDypZ4K4W9GFfcS75cc4lAVQOhNpCPTfALFcjWCXWT6SDfoLuQezm6lTRBzdH+yGSQRYpPAkNDtV2mBZGsBYfq3Jez0W8xWvnpBTPqmpfom8mtr9pD2mh0F/TQU3ELSfnLW73iKN3W//+1KaEK51Zw0uSoy/RYgOJHtRDFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KDh2ln8uQ8I1bgFDylFaPTRj0RPQbGk9Qbsx5TOfPUs=; b=CiEHrFkY5PK7WauTKDFdsmBtWoMPPp4+YQcevIRMdoQPczbeqV/lYo9MAGXk4AknTb9Q5Dhw7ZilkkxK7Tab30+6fgPTwv5KNdjJw7WcwBjNOqOfDI4uEXJQbOBl63OCASP8bTRC/JmnhPvPc+JUWgxG4y1gwYjwPxuq0EucHLvlatX2bFU3nuvTdRsUnuzPCrntsEK4jn1Gn6NynUoI7RyYd2xKzQIWdQeHbmBAYBeKWOyYjhaAkwSl9bMRNuMD1ZwUNeQd38SLRGeS1U61IPA1Kb91XyND8NBWvstNrLsaeuEHwj41U3Qip0jSv/+pjviC2d5Xs3DfxzViCf1H9g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KDh2ln8uQ8I1bgFDylFaPTRj0RPQbGk9Qbsx5TOfPUs=; b=h/N7gIUBs19rJh+vkEfkuZD2TW68BG81jCeQtyPhLSJWNHu2I4qjcQAdmjH6iOUGgqyXLOgkgB0dmx5EaecA8iynkobbFtRkiojAbfum20tkzHhk1YZPcqQvy1YHobvDUqRPqGUQkh23MDkftFgX/LaBdD29js9D8RbfeELXr/yuLae4YrOR/y3kjbk9gc8i3arND5XpOEEM7q6ukvSmwGdxkyOL8tztLw3NmChbFfmdc2bIuhgJKpf0ZE5rqCbMIRWkIBE1ug5yKLBwIo/0SjdKmErMa25XRYPTvS2Qw57blK/SqIJOu4ZaA8/H0uXJ1eaPXULFtNGzOpYeLlvrqg== Received: from CWLP123MB6857.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:1ed::6) by LO6P123MB6582.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b5::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.16; Thu, 25 May 2023 15:46:49 +0000 Received: from CWLP123MB6857.GBRP123.PROD.OUTLOOK.COM ([fe80::36b6:5978:5392:39b1]) by CWLP123MB6857.GBRP123.PROD.OUTLOOK.COM ([fe80::36b6:5978:5392:39b1%7]) with mapi id 15.20.6433.017; Thu, 25 May 2023 15:46:49 +0000 From: cubicibo To: ffmpeg-devel@ffmpeg.org Date: Thu, 25 May 2023 17:45:47 +0200 Message-ID: X-Mailer: git-send-email 2.25.1 X-TMN: [shdawh8q1y+3LnKHUZlKn98a4mjACOJs] X-ClientProxiedBy: ZR0P278CA0002.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:16::12) To CWLP123MB6857.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:1ed::6) X-Microsoft-Original-Message-ID: <20230525154547.23411-1-cubicibo@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CWLP123MB6857:EE_|LO6P123MB6582:EE_ X-MS-Office365-Filtering-Correlation-Id: 5e7c67ca-abf1-4e8c-099a-08db5d374070 X-MS-Exchange-SLBlob-MailProps: znQPCv1HvwU9n5wwzlPFUZyGUM6gvszHcUQvYGuU+Uc8E+EEmZbu9stoml1iX0ZaRhyUhsHsqPl01CKglVy1CWmERUH5fcnVOFsoJdIdfHn3gZ0Pa4jZ8PBdxr1VauxeOjhQTr0ZAreTJH6TqFgjqvnXEO3GNQET6dypL0oUCl6Gqn77Z+Trm/KhoaQCbpZHf2189wmXMZrRvrwI9UDosakNLN9bA7gSoNAzQFmXvOx/r/jaXlLPG4bv0upiYX9i213QCtUBR1kEVpOW/k2dyeck4DBTc043iYMI9ymZ7gG6X9tf5qH2tJYFU226Gv0A8q1gnRgdwAR78cVoAu9eUIuo87fx6sJsGo6mcjfNXKp6JSdyN/G0DpEModvfdseakbo7Mil4QFoi9gbp86jZd3rGRUztsjXka+BeGcjjJwzqTl2Ky+om02UcdPGm7eDY46ktm20OzLlw3p1gVAATo24L1DI8xojAMiimnzP2Lap0j7J/iAPrCRGHujaBy1J4UXez2dNwAvRzrZw47e/zFOv5uVgFe6vkDOYr7Af64tpCNeqMM7viMV6VIqzxu7qPM9Ut1fyfBL0RIdTGMuRhMRkA9WcQ0H7VGzOFXxYZ+8uqNlVCfC27tV01cclAn7uQU7zAyIDZxH0iRfC32zeQC6XzAV9Q7biNOfy1l/ik7mzPoP9+X1QxpCBGlcA6dSaoVVua2TOlBFYdK2U9N+ly8Yt3RTeztA24raq8HEWmBE+ycHwz1MrPDgAk/Sey+ju7XinAzHmebRw= X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: b8x0/K+k/znwctf+eaorwVBjy2gF1rc4IwY7eQ4pCyRUd092iyjS+4JXCat9HUNApgRmEYfXwjhWmJ1KsBYdc8T+ERuUNwvn+AIka5GhrJYBQsaYFB79CoeV9duTpVFjyN1fbGXKcppzKv2Q0IgjCZyFAyx9zxpgIh6Ro/Txz0N64Rr6ak+Mzf5ztJLOzYzA+sPOBuIC8QQnkD4iPTbIdiXfVf2gIeNjPFlcYE1U7gN19ZaHY4Sp3eVj+mRO27eQrPzmdqx37q/5hPwdAlghKcpPMrcSmMtwImxMe1xm5iHTzg3JftizZ47Xg80LfSVu48NF3jSsR+OqbgR/RfwTi6XYnM07eG/XsEUEqinWvfQR31kxUjznLCAxsfJYA4hZVNKGSW+hTzXioYetY7t6M7wUXamDjMmTR0SX1L9AZbh5zhjl8IS+KE6v5XmrUTaygdQJKsNRbcie90l6v5i00l6XitE2Xta633xgdy12vB6XkOF/C1rLVHK5LyaiL1EAgF/7E6nSLtmWF65EynElDTyVGPsRVljPllpMIcTeTvR41HWulSbg8sAtoOI62EJbVzW3Pzou2VPpnxRAqlVPccGNe3mCAF4+b979OrTExTroQsV8v/lH2RPfuAuV2059 X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: nCFMJ4XSPe0ih6tEZjaL6Hf7X92nQh1s4kM/+OJpiFp67LzxU10eH/I7oLcGNloGlXnleXaMCIiHUFgWNWVZx00k5fdzPTDL8V+Gw+WHmxqk9X2QsE6AmAx8nUYZ1Yft5tpBWSUmZwztXthrvp+jqpdhYzBdAoUV3iV8Jymj5CV7eJyp2C9Hz+F2UNyGWKYWkm1jDpLBDwoIQghK9spWso9uSL/7K3VrIYaPESuer1lTDkSO5hvJ4b8fehofZkgIinVpFzgZFXaTiYc5G9D9+71C1UjBT5mNNnQjlFlx4+jSCEdOhcOaM9vza2CXYaiqu01RUrRYou5TEhWAdMLJlXHws62e3JbKND6/xyRRn8RJHVgcO67PfHl7EAPpVmvkMKjGgrtW6Q2cezzJea7gFGw0Dsv30KMaK9u38hJxJS0dGH4uK4rwVWWOxa1Xdy87orShM90yDPOP5z8EayCvHEITfteR6P8ANPE0D5Ya6GqMxqMHsxqdlYbsk6YO/7eljkva+DLY3ir3ys/inAo6Pclt4sx6ZWt9lRqYoLUbpzUa6vsSXq2rmJN1lGs5ikd4kCmVlc7vPWAQY8jwNuIn77ayjukti+SOCbF3g5U0jddxkYYdoFSEv/7PRue6r6t/IzL163hcaogO+/zAqm9MDQSchidvGLB6gEUeWEJDFvi8Ct2/mV8//02n9X1YqzzrXpveLxdE5XMvdyHkBBzHvYvYMi7ciKq5SUgnkjYzkwEtjzrdauTm+Dz3xp/8sFQQJERliCZW+JM8R53RqG+/uBd0l9rpy8NCQp/X2V4MMgNvq2vVe4vjqiugI+rE8ikaVYXeZZvixUeZr78wtFX518KR27QHQLe9nLeRWSwxgxHjSBILKhWPBlANS6MJAj3kCUvbyRLYhxDdpKzzlcjYxRUkHCyfkfeyVgum/KH+CV+6eKnbTV9zFomjeyHsZc2JvsQh2DluiHzkUQkWGvWyjY/nyuFenKWWLuQhLZpPxkaoqOgeNa+UtaT1UJho50WgV9zHGeTEZzU7DWAnjvdivspSejYD+BKlncThMinKUp2nk6TRIqRkw2jj5kDbeiWc3+80/0xLAxHP2O6n/7Sf4ZqtZB3SNoG2mMJFMkpzjuF7UCUFdVCcxCQgCoO0bpkgQSB/1ITfrPM/nz+C80lLypH/283fqRZ547mswI3RnvjW9222cTKM2oMk2m14fFtQIMG+/TsbzjrFSSf8Bvi3tA== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5e7c67ca-abf1-4e8c-099a-08db5d374070 X-MS-Exchange-CrossTenant-AuthSource: CWLP123MB6857.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 May 2023 15:46:49.1508 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO6P123MB6582 Subject: [FFmpeg-devel] [PATCH] lavc/pgssubdec: Correct rendering of palette updates. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: MXKd+LE+RuYQ Reference decoders (BD players) have a graphic plane to store the rendered output. That plane is only refreshed by a window segment. It is evaluated with the last palette and reused on palette updates. This patch adds a conditionally writable graphic plane. The decoder now differentiates appropriately palette updates, new compositions, and display sets that perform no visible actions. --- libavcodec/pgssubdec.c | 339 +++++++++++++++++++++++++---------------- 1 file changed, 210 insertions(+), 129 deletions(-) diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c index 5f76f12615..e7588b727b 100644 --- a/libavcodec/pgssubdec.c +++ b/libavcodec/pgssubdec.c @@ -47,57 +47,76 @@ enum SegmentType { }; typedef struct PGSSubObjectRef { - int id; - int window_id; - uint8_t composition_flag; - int x; - int y; - int crop_x; - int crop_y; - int crop_w; - int crop_h; + uint16_t id; + uint8_t window_id; + uint8_t composition_flag; + uint16_t x; + uint16_t y; + uint16_t crop_x; + uint16_t crop_y; + uint16_t crop_w; + uint16_t crop_h; } PGSSubObjectRef; typedef struct PGSSubPresentation { - int id_number; - int palette_id; - int object_count; + uint8_t palette_flag; + uint8_t palette_id; + uint8_t object_count; PGSSubObjectRef objects[MAX_OBJECT_REFS]; - int64_t pts; + int64_t pts; } PGSSubPresentation; typedef struct PGSSubObject { - int id; - int w; - int h; + uint16_t id; + uint16_t w; + uint16_t h; uint8_t *rle; unsigned int rle_buffer_size, rle_data_len; unsigned int rle_remaining_len; } PGSSubObject; typedef struct PGSSubObjects { - int count; + uint8_t count; PGSSubObject object[MAX_EPOCH_OBJECTS]; } PGSSubObjects; typedef struct PGSSubPalette { - int id; - uint32_t clut[256]; + uint8_t id; + uint32_t clut[AVPALETTE_COUNT]; } PGSSubPalette; typedef struct PGSSubPalettes { - int count; + uint8_t count; PGSSubPalette palette[MAX_EPOCH_PALETTES]; } PGSSubPalettes; +typedef struct PGSGraphicPlane { + uint8_t count; + uint8_t writable; + AVSubtitleRect visible_rect[MAX_OBJECT_REFS]; +} PGSGraphicPlane; + typedef struct PGSSubContext { AVClass *class; PGSSubPresentation presentation; PGSSubPalettes palettes; PGSSubObjects objects; + PGSGraphicPlane plane; int forced_subs_only; } PGSSubContext; +static void clear_graphic_plane(PGSSubContext *ctx) +{ + int i; + + for (i = 0; i < ctx->plane.count; i++) { + av_freep(&ctx->plane.visible_rect[i].data[0]); + memset(&ctx->plane.visible_rect[i], 0, sizeof(ctx->plane.visible_rect[i])); + } + ctx->plane.writable = 0; + ctx->plane.count = 0; +} + static void flush_cache(AVCodecContext *avctx) { PGSSubContext *ctx = avctx->priv_data; @@ -143,6 +162,7 @@ static av_cold int init_decoder(AVCodecContext *avctx) static av_cold int close_decoder(AVCodecContext *avctx) { + clear_graphic_plane((PGSSubContext *)avctx->priv_data); flush_cache(avctx); return 0; @@ -166,6 +186,8 @@ static int decode_rle(AVCodecContext *avctx, AVSubtitleRect *rect, rle_bitmap_end = buf + buf_size; + // Only decode if the plane has been cleared. + av_assert1(!rect->data[0]); rect->data[0] = av_malloc_array(rect->w, rect->h); if (!rect->data[0]) @@ -317,7 +339,7 @@ static int parse_object_segment(AVCodecContext *avctx, * Parse the palette segment packet. * * The palette segment contains details of the palette, - * a maximum of 256 colors can be defined. + * a maximum of 256 colors (AVPALETTE_COUNT) can be defined. * * @param avctx contains the current codec context * @param buf pointer to the packet to process @@ -390,13 +412,17 @@ static int parse_presentation_segment(AVCodecContext *avctx, int64_t pts) { PGSSubContext *ctx = avctx->priv_data; - int i, state, ret; + int ret; + uint8_t i, state; const uint8_t *buf_end = buf + buf_size; // Video descriptor int w = bytestream_get_be16(&buf); int h = bytestream_get_be16(&buf); + // On a new display set, reset writability of the graphic plane + ctx->plane.writable = 0; + ctx->presentation.pts = pts; ff_dlog(avctx, "Video Dimensions %dx%d\n", @@ -405,77 +431,80 @@ static int parse_presentation_segment(AVCodecContext *avctx, if (ret < 0) return ret; - /* Skip 1 bytes of unknown, frame rate */ - buf++; + /* Skip 3 bytes: framerate (1), presentation id number (2) */ + buf+=3; - // Composition descriptor - ctx->presentation.id_number = bytestream_get_be16(&buf); /* - * state is a 2 bit field that defines pgs epoch boundaries + * State is a 2 bit field that defines pgs epoch boundaries * 00 - Normal, previously defined objects and palettes are still valid * 01 - Acquisition point, previous objects and palettes can be released * 10 - Epoch start, previous objects and palettes can be released * 11 - Epoch continue, previous objects and palettes can be released * - * reserved 6 bits discarded + * Reserved 6 bits discarded */ state = bytestream_get_byte(&buf) >> 6; if (state != 0) { flush_cache(avctx); } + /* Reserved 7 bits discarded. */ + ctx->presentation.palette_flag = bytestream_get_byte(&buf) & 0x80; + ctx->presentation.palette_id = bytestream_get_byte(&buf); + /* - * skip palette_update_flag (0x80), + * On palette update, don't parse the compositions references, + * just evaluate the existing graphic plane with the new palette. + * (On palette update, reference decoders expects a segment with one + * composition information whose content is not decoded.) */ - buf += 1; - ctx->presentation.palette_id = bytestream_get_byte(&buf); - ctx->presentation.object_count = bytestream_get_byte(&buf); - if (ctx->presentation.object_count > MAX_OBJECT_REFS) { - av_log(avctx, AV_LOG_ERROR, - "Invalid number of presentation objects %d\n", - ctx->presentation.object_count); - ctx->presentation.object_count = 2; - if (avctx->err_recognition & AV_EF_EXPLODE) { - return AVERROR_INVALIDDATA; + if (!ctx->presentation.palette_flag) { + ctx->presentation.object_count = bytestream_get_byte(&buf); + if (ctx->presentation.object_count > MAX_OBJECT_REFS) { + av_log(avctx, AV_LOG_ERROR, + "Invalid number of presentation objects %d\n", + ctx->presentation.object_count); + ctx->presentation.object_count = 2; + if (avctx->err_recognition & AV_EF_EXPLODE) { + return AVERROR_INVALIDDATA; + } } - } + for (i = 0; i < ctx->presentation.object_count; i++) { + PGSSubObjectRef *const object = &ctx->presentation.objects[i]; - for (i = 0; i < ctx->presentation.object_count; i++) - { - PGSSubObjectRef *const object = &ctx->presentation.objects[i]; - - if (buf_end - buf < 8) { - av_log(avctx, AV_LOG_ERROR, "Insufficent space for object\n"); - ctx->presentation.object_count = i; - return AVERROR_INVALIDDATA; - } + if (buf_end - buf < 8) { + av_log(avctx, AV_LOG_ERROR, "Insufficent space for object\n"); + ctx->presentation.object_count = i; + return AVERROR_INVALIDDATA; + } - object->id = bytestream_get_be16(&buf); - object->window_id = bytestream_get_byte(&buf); - object->composition_flag = bytestream_get_byte(&buf); + object->id = bytestream_get_be16(&buf); + object->window_id = bytestream_get_byte(&buf); + object->composition_flag = bytestream_get_byte(&buf); - object->x = bytestream_get_be16(&buf); - object->y = bytestream_get_be16(&buf); + object->x = bytestream_get_be16(&buf); + object->y = bytestream_get_be16(&buf); - // If cropping - if (object->composition_flag & 0x80) { - object->crop_x = bytestream_get_be16(&buf); - object->crop_y = bytestream_get_be16(&buf); - object->crop_w = bytestream_get_be16(&buf); - object->crop_h = bytestream_get_be16(&buf); - } + // If cropping + if (object->composition_flag & 0x80) { + object->crop_x = bytestream_get_be16(&buf); + object->crop_y = bytestream_get_be16(&buf); + object->crop_w = bytestream_get_be16(&buf); + object->crop_h = bytestream_get_be16(&buf); + } - ff_dlog(avctx, "Subtitle Placement x=%d, y=%d\n", - object->x, object->y); + ff_dlog(avctx, "Subtitle Placement x=%d, y=%d\n", + object->x, object->y); - if (object->x > avctx->width || object->y > avctx->height) { - av_log(avctx, AV_LOG_ERROR, "Subtitle out of video bounds. x = %d, y = %d, video width = %d, video height = %d.\n", - object->x, object->y, - avctx->width, avctx->height); - object->y = object->x = 0; - if (avctx->err_recognition & AV_EF_EXPLODE) { - return AVERROR_INVALIDDATA; + if (object->x > avctx->width || object->y > avctx->height) { + av_log(avctx, AV_LOG_ERROR, "Subtitle out of video bounds. x = %d, y = %d, video width = %d, video height = %d.\n", + object->x, object->y, + avctx->width, avctx->height); + object->y = object->x = 0; + if (avctx->err_recognition & AV_EF_EXPLODE) { + return AVERROR_INVALIDDATA; + } } } } @@ -483,10 +512,47 @@ static int parse_presentation_segment(AVCodecContext *avctx, return 0; } +/** + * Parse the window segment packet. + * + * The window segment instructs the decoder to redraw the graphic plane + * with the composition references provided in the presentation segment + * + * @param avctx contains the current codec context + */ +static int parse_window_segment(AVCodecContext *avctx, const uint8_t *buf, + int buf_size) +{ + PGSSubContext *ctx = (PGSSubContext *)avctx->priv_data; + + // 1 byte: number of windows defined + if (bytestream_get_byte(&buf) > MAX_OBJECT_REFS) { + av_log(avctx, AV_LOG_ERROR, "Too many windows defined.\n"); + return AVERROR_INVALIDDATA; + } + + /* TODO: mask objects with windows when transfering to the graphic plane + * Window Segment Structure + * { + * 1 byte : window id, + * 2 bytes: X position of window, + * 2 bytes: Y position of window, + * 2 bytes: Width of window, + * 2 bytes: Height of window. + * } + */ + // Flush the graphic plane, it will be redrawn. + clear_graphic_plane(ctx); + ctx->plane.writable = 1; + ctx->plane.count = ctx->presentation.object_count; + return 0; +} + /** * Parse the display segment packet. * - * The display segment controls the updating of the display. + * The display segment closes the display set. The inferred data is used + * to decide if the display should be updated. * * @param avctx contains the current codec context * @param data pointer to the data pertaining the subtitle to display @@ -505,20 +571,28 @@ static int display_end_segment(AVCodecContext *avctx, AVSubtitle *sub, memset(sub, 0, sizeof(*sub)); sub->pts = pts; ctx->presentation.pts = AV_NOPTS_VALUE; - sub->start_display_time = 0; // There is no explicit end time for PGS subtitles. The end time // is defined by the start of the next sub which may contain no // objects (i.e. clears the previous sub) sub->end_display_time = UINT32_MAX; - sub->format = 0; - // Blank if last object_count was 0. - if (!ctx->presentation.object_count) + if (!ctx->presentation.palette_flag && !ctx->plane.writable) { + // This display set does not perform a display update + // E.g. it only defines new objects or palettes for future usage. + return 0; + } + + // Blank if object_count is 0 (undisplay). Stream is damaged if the + // palette update flag is set but plane count is zero. + if (!ctx->plane.count) { + avsubtitle_free(sub); return 1; - sub->rects = av_calloc(ctx->presentation.object_count, sizeof(*sub->rects)); - if (!sub->rects) { - return AVERROR(ENOMEM); } + + sub->rects = av_calloc(ctx->plane.count, sizeof(*sub->rects)); + if (!sub->rects) + return AVERROR(ENOMEM); + palette = find_palette(ctx->presentation.palette_id, &ctx->palettes); if (!palette) { // Missing palette. Should only happen with damaged streams. @@ -527,57 +601,71 @@ static int display_end_segment(AVCodecContext *avctx, AVSubtitle *sub, avsubtitle_free(sub); return AVERROR_INVALIDDATA; } - for (i = 0; i < ctx->presentation.object_count; i++) { - AVSubtitleRect *const rect = av_mallocz(sizeof(*rect)); - PGSSubObject *object; - if (!rect) - return AVERROR(ENOMEM); - sub->rects[sub->num_rects++] = rect; - rect->type = SUBTITLE_BITMAP; - - /* Process bitmap */ - object = find_object(ctx->presentation.objects[i].id, &ctx->objects); - if (!object) { - // Missing object. Should only happen with damaged streams. - av_log(avctx, AV_LOG_ERROR, "Invalid object id %d\n", - ctx->presentation.objects[i].id); - if (avctx->err_recognition & AV_EF_EXPLODE) - return AVERROR_INVALIDDATA; - // Leaves rect empty with 0 width and height. - continue; - } - if (ctx->presentation.objects[i].composition_flag & 0x40) - rect->flags |= AV_SUBTITLE_FLAG_FORCED; + for (i = 0; i < ctx->plane.count; i++) { + AVSubtitleRect *const gp_rect = &ctx->plane.visible_rect[i]; + AVSubtitleRect *rect; + gp_rect->type = SUBTITLE_BITMAP; + + // Compose the graphic plane if a window segment has been provided + if (ctx->plane.writable) { + PGSSubObject *object; + + // Process bitmap + object = find_object(ctx->presentation.objects[i].id, &ctx->objects); + if (!object) { + // Missing object. Should only happen with damaged streams. + av_log(avctx, AV_LOG_ERROR, "Invalid object id %d\n", + ctx->presentation.objects[i].id); + if (avctx->err_recognition & AV_EF_EXPLODE) + return AVERROR_INVALIDDATA; + // Leaves rect empty with 0 width and height. + continue; + } + if (ctx->presentation.objects[i].composition_flag & 0x40) + gp_rect->flags |= AV_SUBTITLE_FLAG_FORCED; - rect->x = ctx->presentation.objects[i].x; - rect->y = ctx->presentation.objects[i].y; + gp_rect->x = ctx->presentation.objects[i].x; + gp_rect->y = ctx->presentation.objects[i].y; - if (object->rle) { - rect->w = object->w; - rect->h = object->h; + if (object->rle) { + gp_rect->w = object->w; + gp_rect->h = object->h; - rect->linesize[0] = object->w; + gp_rect->linesize[0] = object->w; - if (object->rle_remaining_len) { - av_log(avctx, AV_LOG_ERROR, "RLE data length %u is %u bytes shorter than expected\n", - object->rle_data_len, object->rle_remaining_len); - if (avctx->err_recognition & AV_EF_EXPLODE) - return AVERROR_INVALIDDATA; - } - ret = decode_rle(avctx, rect, object->rle, object->rle_data_len); - if (ret < 0) { - if ((avctx->err_recognition & AV_EF_EXPLODE) || - ret == AVERROR(ENOMEM)) { - return ret; + if (object->rle_remaining_len) { + av_log(avctx, AV_LOG_ERROR, "RLE data missing %u bytes out of %u.\n", + object->rle_remaining_len, object->rle_data_len); + if (avctx->err_recognition & AV_EF_EXPLODE) + return AVERROR_INVALIDDATA; + } + ret = decode_rle(avctx, gp_rect, object->rle, object->rle_data_len); + if (ret < 0) { + if ((avctx->err_recognition & AV_EF_EXPLODE) || + ret == AVERROR(ENOMEM)) { + return ret; + } + gp_rect->w = 0; + gp_rect->h = 0; + continue; } - rect->w = 0; - rect->h = 0; - continue; } } - /* Allocate memory for colors */ - rect->nb_colors = 256; + // Export graphic plane content with latest palette + rect = av_memdup(gp_rect, sizeof(*gp_rect)); + if (!rect) + return AVERROR(ENOMEM); + + sub->rects[sub->num_rects++] = rect; + if (gp_rect->data[0]) { + rect->data[0] = av_memdup(gp_rect->data[0], rect->w*rect->h); + if (!rect->data[0]) + return AVERROR(ENOMEM); + } + + // Allocate memory for colors + rect->nb_colors = AVPALETTE_COUNT; rect->data[1] = av_mallocz(AVPALETTE_SIZE); if (!rect->data[1]) return AVERROR(ENOMEM); @@ -640,14 +728,7 @@ static int decode(AVCodecContext *avctx, AVSubtitle *sub, ret = parse_presentation_segment(avctx, buf, segment_length, sub->pts); break; case WINDOW_SEGMENT: - /* - * Window Segment Structure (No new information provided): - * 2 bytes: Unknown, - * 2 bytes: X position of subtitle, - * 2 bytes: Y position of subtitle, - * 2 bytes: Width of subtitle, - * 2 bytes: Height of subtitle. - */ + ret = parse_window_segment(avctx, buf, segment_length); break; case DISPLAY_SEGMENT: if (*got_sub_ptr) {