From patchwork Sun Mar 11 18:30:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 7912 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp826537jad; Sun, 11 Mar 2018 11:36:18 -0700 (PDT) X-Google-Smtp-Source: AG47ELua9WVp936aDtsHk4Uq0v6e6OsRiz2JO1Pa2oeRMBPN6YkUeG+qVlYGuAZxr5QAdk9d8ldx X-Received: by 10.28.217.213 with SMTP id q204mr3365733wmg.141.1520793378260; Sun, 11 Mar 2018 11:36:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1520793378; cv=none; d=google.com; s=arc-20160816; b=NYU8yvCJ8N6o9QA+aDuHqajkEHvGFIVK+ANZ6Cbeo2PyXVWO7wWk+l3OBK759Vx9Br O88S5HNWM6MEHAFnfIVEmuU+hFCxJGeH/i2Btre40tB1LP2b5HggQ5c3O4+fIpm4Lc0x tJ0mOvqp8xt8PWuW7SnQHzkSieeaDwVHH3Be/knFmQG7FPTQFMs02bt7dt5hteMkqJlL LDMF9m1U0rlBFFUgZNFFGHTxO1pN4XdtzXKMAhj1+TH+1HRyN935HmVVJsqh4EoTDTrf 5X5t4Bv2b1a9/MINHPuOqxPvwqJQYfkdMoflRAkHZuOKvZ6r4cGXWfp2vl1DTKwce2/Z 9M/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=RmLgvUvdw0tCECRQNbgIkEIwW9zQUXEEpxgL8AOAvcw=; b=VkDuFw23IZmnwu6BahoPuIZHwR05HDEC7DQLvgo+XfxT3Oxg5gFmYw6U6OissKVtnl GCGEXD92Izdbtt9GjUg8sYWQbDHGkP/nt/WwWdXTZrcaMi3noJZxVN3ThmSCVccD9DtG hIQN11/AD1xMW2YTS/BcY3EFBLwDjDhqqlZXVFDETfbO/U6KRhyzGdEIQCjr0CWu/RYH tOZAo3yQU5O9sSyW8DlI9dsiF5XiL9VubmKVkBSuchfxkpAGWGTnCr+7TRasNPrTHYiE qhN7Ew4ped6JDHX1R2S9R7GD7VJjvt2c6ThOTDB4ki6lJqkpuKoyKAgGBsNAYZ6aL3xk zJtg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=unBpch6e; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d4si4053053wra.388.2018.03.11.11.36.17; Sun, 11 Mar 2018 11:36:18 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=unBpch6e; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BD46B68A270; Sun, 11 Mar 2018 20:36:06 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F0EC468A181 for ; Sun, 11 Mar 2018 20:35:59 +0200 (EET) Received: by mail-wm0-f67.google.com with SMTP id 188so12115847wme.1 for ; Sun, 11 Mar 2018 11:36:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id; bh=tHv1sYKzyzyJP7kZRee87cZ6FztvynA3EJtaSf7012k=; b=unBpch6eOgaNGPh5Ub+cxgsGlp/XkfV3PMLzHjjcnFIq+7Jl3sMyLK1DSeyBGP9y4U 2YuqI4MOzjradRBFyxYSOYSVDYfmbblQq8CKQq6AlTegnMZ01/LVSQuD30qlIF42yz5L oZysM6+vyo1BSiYoaY/TnirwGSE53WAD3CIu75m88jlHbb0rHvH5Kd5R7K1ed8mHvBiv A/IrqBF5dU3ANWjmO1E7HSGJ+auqTPhPpERsyfM/wPcvGsVvyjMoNUCzAHdLHKXZMfz4 Pb36+jbc81s1qiOhH3QqQGAxwjpYAczV1zkrxTTD3VcZ+osvX4sO38IP5p6wAVxJTgrb oPkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=tHv1sYKzyzyJP7kZRee87cZ6FztvynA3EJtaSf7012k=; b=AVVlsFKlTZL1bj4LiWr7NYUByh1JPOeP3IlcKjO9DNfDhn9Wx6gHx4MUQqkF5+aWTh bZWUUf4ZnpUOxU5CCmiV3uxmKS9kpE5bFizzV112xZNr4iob9LLXfXPHEY1uTdD/AT+P 8p04ehf1mET3FkULzkGm2WTlrFo/TqmxmwSQwTxuN/6ylWKLwOGgAOuZyxWjLlvspDCe 8NS0rOnjS6AjpdDEZTby49xixUYsOPCsUDhwmtsf4w2reseyjxngqL0ZmS6tP/jnyJe9 urU/GGH1r0zCMQtnbdZSVGQuioPrLhpu7vtVNC0BMdmGeafd0Caj2Nx4wXgzJ4WbYF1T 2B3Q== X-Gm-Message-State: AElRT7EwrRw2jG6cUiUViEHErCI4uqqaJmEh+sDslQV4QB8XXA+vMPoJ JFVp5dPu0ZZzhbHqf6ABfOvWDXYb X-Received: by 10.28.173.198 with SMTP id w189mr3914805wme.139.1520793025352; Sun, 11 Mar 2018 11:30:25 -0700 (PDT) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id 31sm5064514wrr.59.2018.03.11.11.30.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Mar 2018 11:30:24 -0700 (PDT) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 11 Mar 2018 18:30:14 +0000 Message-Id: <20180311183021.25556-1-sw@jkqxz.net> X-Mailer: git-send-email 2.16.1 Subject: [FFmpeg-devel] [PATCH 1/8] cbs_h264: Fix overflow in shifts X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The type of the result of a shift operation is unaffected by the type of the right operand, so some existing code overflows with undefined behaviour when the element length is 32. Add a helper macro to calculate the maximum value correctly and then use it everywhere this pattern appears. Found-by: Andreas Rheinhardt --- libavcodec/cbs_h264_syntax_template.c | 22 +++++++++++----------- libavcodec/cbs_internal.h | 4 ++++ 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/libavcodec/cbs_h264_syntax_template.c b/libavcodec/cbs_h264_syntax_template.c index f58dee8a25..b5cd0b2310 100644 --- a/libavcodec/cbs_h264_syntax_template.c +++ b/libavcodec/cbs_h264_syntax_template.c @@ -342,8 +342,8 @@ static int FUNC(sps_extension)(CodedBitstreamContext *ctx, RWContext *rw, flag(alpha_incr_flag); bits = current->bit_depth_aux_minus8 + 9; - u(bits, alpha_opaque_value, 0, (1 << bits) - 1); - u(bits, alpha_transparent_value, 0, (1 << bits) - 1); + u(bits, alpha_opaque_value, 0, MAX_UINT_BITS(bits)); + u(bits, alpha_transparent_value, 0, MAX_UINT_BITS(bits)); } flag(additional_extension_flag); @@ -483,10 +483,10 @@ static int FUNC(sei_buffering_period)(CodedBitstreamContext *ctx, RWContext *rw, length = sps->vui.nal_hrd_parameters.initial_cpb_removal_delay_length_minus1 + 1; xu(length, initial_cpb_removal_delay[SchedSelIdx], current->nal.initial_cpb_removal_delay[i], - 0, (1 << (uint64_t)length) - 1); + 1, MAX_UINT_BITS(length)); xu(length, initial_cpb_removal_delay_offset[SchedSelIdx], current->nal.initial_cpb_removal_delay_offset[i], - 0, (1 << (uint64_t)length) - 1); + 0, MAX_UINT_BITS(length)); } } @@ -495,10 +495,10 @@ static int FUNC(sei_buffering_period)(CodedBitstreamContext *ctx, RWContext *rw, length = sps->vui.vcl_hrd_parameters.initial_cpb_removal_delay_length_minus1 + 1; xu(length, initial_cpb_removal_delay[SchedSelIdx], current->vcl.initial_cpb_removal_delay[i], - 0, (1 << (uint64_t)length) - 1); + 1, MAX_UINT_BITS(length)); xu(length, initial_cpb_removal_delay_offset[SchedSelIdx], current->vcl.initial_cpb_removal_delay_offset[i], - 0, (1 << (uint64_t)length) - 1); + 0, MAX_UINT_BITS(length)); } } @@ -548,7 +548,7 @@ static int FUNC(sei_pic_timestamp)(CodedBitstreamContext *ctx, RWContext *rw, if (time_offset_length > 0) u(time_offset_length, time_offset, - 0, (1 << (uint64_t)time_offset_length) - 1); + 0, MAX_UINT_BITS(time_offset_length)); else infer(time_offset, 0); @@ -600,9 +600,9 @@ static int FUNC(sei_pic_timing)(CodedBitstreamContext *ctx, RWContext *rw, } u(hrd->cpb_removal_delay_length_minus1 + 1, cpb_removal_delay, - 0, (1 << (uint64_t)hrd->cpb_removal_delay_length_minus1) + 1); + 0, MAX_UINT_BITS(hrd->cpb_removal_delay_length_minus1 + 1)); u(hrd->dpb_output_delay_length_minus1 + 1, dpb_output_delay, - 0, (1 << (uint64_t)hrd->dpb_output_delay_length_minus1) + 1); + 0, MAX_UINT_BITS(hrd->dpb_output_delay_length_minus1 + 1)); } if (sps->vui.pic_struct_present_flag) { @@ -1123,7 +1123,7 @@ static int FUNC(slice_header)(CodedBitstreamContext *ctx, RWContext *rw, u(2, colour_plane_id, 0, 2); u(sps->log2_max_frame_num_minus4 + 4, frame_num, - 0, (1 << (sps->log2_max_frame_num_minus4 + 4)) - 1); + 0, MAX_UINT_BITS(sps->log2_max_frame_num_minus4 + 4)); if (!sps->frame_mbs_only_flag) { flag(field_pic_flag); @@ -1141,7 +1141,7 @@ static int FUNC(slice_header)(CodedBitstreamContext *ctx, RWContext *rw, if (sps->pic_order_cnt_type == 0) { u(sps->log2_max_pic_order_cnt_lsb_minus4 + 4, pic_order_cnt_lsb, - 0, (1 << (sps->log2_max_pic_order_cnt_lsb_minus4 + 4)) - 1); + 0, MAX_UINT_BITS(sps->log2_max_pic_order_cnt_lsb_minus4 + 4)); if (pps->bottom_field_pic_order_in_frame_present_flag && !current->field_pic_flag) se(delta_pic_order_cnt_bottom, INT32_MIN + 1, INT32_MAX); diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index 5674803472..be540e2a44 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -79,6 +79,10 @@ int ff_cbs_write_unsigned(CodedBitstreamContext *ctx, PutBitContext *pbc, int width, const char *name, uint32_t value, uint32_t range_min, uint32_t range_max); +// The largest value representable in N bits, suitable for use as +// range_max in the above functions. +#define MAX_UINT_BITS(length) ((UINT64_C(1) << (length)) - 1) + extern const CodedBitstreamType ff_cbs_type_h264; extern const CodedBitstreamType ff_cbs_type_h265;