From patchwork Thu Oct 20 08:59:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Ross X-Patchwork-Id: 38856 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp49768pzb; Thu, 20 Oct 2022 01:59:48 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6oJEuQ0wNQDu6RQoTujFOlWVSIeMO3GlZ32axQcgMU9x+ntpDdCjsvw/LQOpnOrHM/DYAI X-Received: by 2002:a05:6402:520d:b0:45d:b6a7:cdba with SMTP id s13-20020a056402520d00b0045db6a7cdbamr11282762edd.282.1666256387710; Thu, 20 Oct 2022 01:59:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666256387; cv=none; d=google.com; s=arc-20160816; b=qLsdP5xyJzVN8ZIIxY2OHnGrcB6TR2fKmsvPnCWHlrWBp+ImaJ5XtlUv7wgY/7VAre WjBX7sFiItUcypWFICrF6MEdhDA02IPsGFuaCHMVCHv88jNkc+Bm/XrVcQ8mwucFJadh pYxtRDIXjQkSgvJjsWh9/istqUhgM3+PcmH32sueRxZMY0RQZRW+YoaYhePWN3fqdG9y b0+V1qcXAbWBKeckybnZ1yd8PINE3Hz2eOSxajQLzJqi62DWorUbq4ggd8tIgnARVpH1 t4vEZXaC4Nl5g7oGs3ANOnWChl26CubnAVz0ixTj1Q5bHu9MgfMedprMf2FeIgSV5Qby Q0wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:message-id:to:from:date:delivered-to; bh=Z0htbw2IipbKqdIGXPng3m+alb/1yF6SEkmillEbD7k=; b=M88mVS/Js60CAegCdg2cICXhh/cBYvPYAZJWD5zJ2DJ/QhavH1JqXr68FHLAip9IHm k2AjAy2NnkB4gm07fWlYG+1yCKXj1hjgKZXbz8O1k/DiQIYrbg6DALGE9aW7/3TG8/JT 19mHEVDgS+7a/68ZpV+adbXHj8aeie7WLdbRrkIgl7h1DOErIeu9tvKmstmAksbaXuZZ TSYN84IxPftmsX9UsjKjIj6FqyK2FS2XesEf2dfYzurYrHKn2PakOznmldgHmcH8CUJ6 08FA03mRaaRwmM7DtfVwiRB4ahpX1dwVNyqQg0KlsR7fvGZZOgopDFAu0lk5cJYB19yt hEuw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i9-20020a0564020f0900b0046074707fc5si1199905eda.163.2022.10.20.01.59.46; Thu, 20 Oct 2022 01:59:47 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 52D8A68BE92; Thu, 20 Oct 2022 11:59:44 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mx.sdf.org (mx.sdf.org [205.166.94.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7B04F68B96F for ; Thu, 20 Oct 2022 11:59:37 +0300 (EEST) Received: from 45cccc65f298575ed5de7e32d4efc6de ([1.145.213.234]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id 29K8xUEn025582 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits) verified NO) for ; Thu, 20 Oct 2022 08:59:34 GMT Date: Thu, 20 Oct 2022 19:59:25 +1100 From: Peter Ross To: ffmpeg-devel@ffmpeg.org Message-ID: <812f50e814dbd8cd8d854da6ccd5a5d6477551ba.1666256242.git.pross@xvid.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/mss2: calculate draw region and revise split position X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 6OF3/BhF6uXq for videos with wmv9 rectangles, the region drawn by ff_mss12_decode_rect may be less than the entire video area. the wmv9 rectangles are used to calculate the ff_mss12_decode_rect draw region. Fixes tickets #3255 and #4043 --- (will also fix identation as seperate commit on push) libavcodec/mss2.c | 70 ++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 66 insertions(+), 4 deletions(-) diff --git a/libavcodec/mss2.c b/libavcodec/mss2.c index d8a30019f7..69494d8c44 100644 --- a/libavcodec/mss2.c +++ b/libavcodec/mss2.c @@ -468,6 +468,39 @@ struct Rectangle { int coded, x, y, w, h; }; +struct Rectangle2 { + int left, right, top, bottom; +}; + +static void calc_draw_region(struct Rectangle2 * draw, const struct Rectangle2 * rect) +{ +#define COMPARE(top, bottom, left, right) \ + if (rect->top <= draw->top && rect->bottom >= draw->bottom) { \ + if (rect->left <= draw->left && rect->right >= draw->left) \ + draw->left = FFMIN(rect->right, draw->right); \ + \ + if (rect->right >= draw->right) { \ + if (rect->left >= draw->left) { \ + if (rect->left < draw->right) \ + draw->right = rect->left; \ + } else { \ + draw->right = draw->left; \ + } \ + } \ + } + + COMPARE(top, bottom, left, right) + COMPARE(left, right, top, bottom) +} + +static int calc_split_position(int split_position, const struct Rectangle2 * rect, int height) +{ + if (rect->top || rect->bottom != height) + split_position = rect->top + split_position * (rect->bottom - rect->top) / height; + + return av_clip(split_position, rect->top + 1, rect->bottom - 1); +} + #define MAX_WMV9_RECTANGLES 20 #define ARITH2_PADDING 2 @@ -485,6 +518,7 @@ static int mss2_decode_frame(AVCodecContext *avctx, AVFrame *frame, int keyframe, has_wmv9, has_mv, is_rle, is_555, ret; struct Rectangle wmv9rects[MAX_WMV9_RECTANGLES], *r; + struct Rectangle2 draw; int used_rects = 0, i, implicit_rect = 0, av_uninit(wmv9_mask); if ((ret = init_get_bits8(&gb, buf, buf_size)) < 0) @@ -671,11 +705,32 @@ static int mss2_decode_frame(AVCodecContext *avctx, AVFrame *frame, bytestream2_init(&gB, buf, buf_size + ARITH2_PADDING); arith2_init(&acoder, &gB); c->keyframe = keyframe; - if (c->corrupted = ff_mss12_decode_rect(&ctx->sc[0], &acoder, 0, 0, - avctx->width, - ctx->split_position)) + + draw.left = 0; + draw.top = 0; + draw.right = avctx->width; + draw.bottom = avctx->height; + if (wmv9_mask == -1) { + for (i = 0; i < used_rects; i++) { + struct Rectangle2 r; + r.left = wmv9rects[i].x; + r.top = wmv9rects[i].y; + r.right = r.left + wmv9rects[i].w; + r.bottom = r.top + wmv9rects[i].h; + calc_draw_region(&draw, &r); + } + } + + if (draw.left >= avctx->width || draw.right > avctx->width || + draw.top >= avctx->height || draw.bottom > avctx->height) return AVERROR_INVALIDDATA; + if (c->slice_split && draw.bottom - draw.top >= 10) { + ctx->split_position = calc_split_position(ctx->split_position, &draw, avctx->height); + if (c->corrupted = ff_mss12_decode_rect(&ctx->sc[0], &acoder, 0, draw.top, + avctx->width, + ctx->split_position - draw.top)) + return AVERROR_INVALIDDATA; buf += arith2_get_consumed_bytes(&acoder); buf_size -= arith2_get_consumed_bytes(&acoder); if (c->slice_split) { @@ -686,7 +741,14 @@ static int mss2_decode_frame(AVCodecContext *avctx, AVFrame *frame, if (c->corrupted = ff_mss12_decode_rect(&ctx->sc[1], &acoder, 0, ctx->split_position, avctx->width, - avctx->height - ctx->split_position)) + draw.bottom - ctx->split_position)) + return AVERROR_INVALIDDATA; + buf += arith2_get_consumed_bytes(&acoder); + buf_size -= arith2_get_consumed_bytes(&acoder); + } + } else { + if (c->corrupted = ff_mss12_decode_rect(&ctx->sc[0], &acoder, draw.left, draw.top, + draw.right - draw.left, draw.bottom - draw.top)) return AVERROR_INVALIDDATA; buf += arith2_get_consumed_bytes(&acoder);