diff mbox series

[FFmpeg-devel,v4] Unbreak av_malloc_max(0) API/ABI

Message ID 20201016085722.21866-1-joakim.tjernlund@infinera.com
State New
Headers show
Series [FFmpeg-devel,v4] Unbreak av_malloc_max(0) API/ABI
Related show

Checks

Context Check Description
andriy/x86_make success Make finished
andriy/x86_make_fate success Make fate finished
andriy/PPC64_make warning Make failed

Commit Message

Joakim Tjernlund Oct. 16, 2020, 8:57 a.m. UTC
From https://bugs.chromium.org/p/chromium/issues/detail?id=1095962
----------------------------
This seems to be caused by the custom handling of "av_max_alloc(0)" in
Chromium's ffmpeg fork to mean unlimited (added in [1]).

Upstream ffmpeg doesn't treat 0 as a special value; versions before 4.3 seemingly worked
because 32 was subtracted from max_alloc_size (set to 0 by Chromium) resulting in an
integer underflow, making the effective limit be SIZE_MAX - 31.

Now that the above underflow doesn't happen, the tab just crashes. The upstream change
for no longer subtracting 32 from max_alloc_size was included in ffmpeg 4.3. [2]

[1] https://chromium-review.googlesource.com/c/chromium/third_party/ffmpeg/+/73563
[2] https://github.com/FFmpeg/FFmpeg/commit/731c77589841
---------------------------

Restore av_malloc_max(0) to MAX_INT fixing MS Teams, Discord older chromium etc.

Signed-off-by: Joakim Tjernlund <joakim.tjernlund@infinera.com>
---

 v2: Cover the full API range 0-31

 v3: Closer compat with < 4.3 ffmpeg

 v4: Adjust size accoriding to Andreas Rheinhardt comments

 libavutil/mem.c | 2 ++
 1 file changed, 2 insertions(+)

Comments

Joakim Tjernlund Oct. 22, 2020, 12:17 p.m. UTC | #1
Ping ?

 Jocke

On Fri, 2020-10-16 at 10:57 +0200, Joakim Tjernlund wrote:
> From https://bugs.chromium.org/p/chromium/issues/detail?id=1095962
> ----------------------------
> This seems to be caused by the custom handling of "av_max_alloc(0)" in
> Chromium's ffmpeg fork to mean unlimited (added in [1]).
> 
> Upstream ffmpeg doesn't treat 0 as a special value; versions before 4.3 seemingly worked
> because 32 was subtracted from max_alloc_size (set to 0 by Chromium) resulting in an
> integer underflow, making the effective limit be SIZE_MAX - 31.
> 
> Now that the above underflow doesn't happen, the tab just crashes. The upstream change
> for no longer subtracting 32 from max_alloc_size was included in ffmpeg 4.3. [2]
> 
> [1] https://chromium-review.googlesource.com/c/chromium/third_party/ffmpeg/+/73563
> [2] https://github.com/FFmpeg/FFmpeg/commit/731c77589841
> ---------------------------
> 
> Restore av_malloc_max(0) to MAX_INT fixing MS Teams, Discord older chromium etc.
> 
> Signed-off-by: Joakim Tjernlund <joakim.tjernlund@infinera.com>
> ---
> 
>  v2: Cover the full API range 0-31
> 
>  v3: Closer compat with < 4.3 ffmpeg
> 
>  v4: Adjust size accoriding to Andreas Rheinhardt comments
> 
>  libavutil/mem.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/libavutil/mem.c b/libavutil/mem.c
> index cfb6d8a..44870a9 100644
> --- a/libavutil/mem.c
> +++ b/libavutil/mem.c
> @@ -71,6 +71,8 @@ void  free(void *ptr);
>  static size_t max_alloc_size= INT_MAX;
>  
> 
>  void av_max_alloc(size_t max){
> +    if (max < 32)
> +        max = SIZE_MAX - 32 + max; /* be compatible to older(< 4.3) versions */
>      max_alloc_size = max;
>  }
>  
>
diff mbox series

Patch

diff --git a/libavutil/mem.c b/libavutil/mem.c
index cfb6d8a..44870a9 100644
--- a/libavutil/mem.c
+++ b/libavutil/mem.c
@@ -71,6 +71,8 @@  void  free(void *ptr);
 static size_t max_alloc_size= INT_MAX;
 
 void av_max_alloc(size_t max){
+    if (max < 32)
+        max = SIZE_MAX - 32 + max; /* be compatible to older(< 4.3) versions */
     max_alloc_size = max;
 }