From patchwork Thu Mar 29 13:30:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wm4 X-Patchwork-Id: 8224 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp1693492jad; Thu, 29 Mar 2018 06:30:58 -0700 (PDT) X-Google-Smtp-Source: AIpwx4/c56fM57wfJkxR6DT9CVF8x+f++ZvT3MUryYefl4FVuDAxM7TAqpUdPgRJa8tnFvUHzdBB X-Received: by 10.28.69.216 with SMTP id l85mr5931865wmi.150.1522330258006; Thu, 29 Mar 2018 06:30:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522330257; cv=none; d=google.com; s=arc-20160816; b=vpe9bRvqWQF7f2dZwtiz9ydiBNVOuyJNXcDKT9RKqMvmiQ2yriQ245jnlsyayHZBH2 BWB8WJRAMFFVQx6kLKPq97wNBW3Npr1DjyH8T/BZcINKJyc7US+ga4wgHmvMppMpe5uy 3c8Au22Y1M1ycIr9k3ijMih6t7E6r5fW62E3CdC6JeW+H5/UclAXSgz65Jkek+xmG7ZO /rEl+ZjSxMIlve19LE+0gNGe1X+2VW0MF8VVpsoxgkvUBFsL0wRkT5XB5P5gsmSA5fgr toE8+7Pd6l3dAXbmp80QA1ue4ISknoqmW/T15uDOwQLID+TvdgYhe7tk6XXeikAu+o3b zvhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=bTa9UGcA4W9Z2R+RONloxGfNhHswChlaNYYmdkzKsDI=; b=ScUcoBh1sT83OCyov7DkWpFqB8Tot/UBY5IPkJG2gkctXl1s4JN2E4T0ftYHetsD7s Uj1uLxQZiQt2OzmLjsxLLyWU7PjbyIJz+8hCPKiOm/0hsOXqwC3vFQshpKss087fMo4W oiUKtvpqP9v6Yss/gQgUgXPY4/2A8DQREm5U2Eg4R8Md/8EXbTXuSUDbTVAJ5ekQzsI4 n8WuPTETo9MBxbbzvo50PS8q4WpQ2TWJ7emfs871X2a8aNF/9UTD1OhwnGI3xNkw1s8T KGObfdmtnOA+ENeYD8PWf4eJS7x7yPlKPpeNqGNsUwxoaANUMYLGI7M/LAhCd9gjXjBn oOfA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com header.s=20161025 header.b=ABoANn1S; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a138si1373760wmd.28.2018.03.29.06.30.57; Thu, 29 Mar 2018 06:30:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com header.s=20161025 header.b=ABoANn1S; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E17E7689CA0; Thu, 29 Mar 2018 16:30:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f41.google.com (mail-wm0-f41.google.com [74.125.82.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 062C5689C8A for ; Thu, 29 Mar 2018 16:30:23 +0300 (EEST) Received: by mail-wm0-f41.google.com with SMTP id o23so18594026wmf.0 for ; Thu, 29 Mar 2018 06:30:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BIfnaM9m1+Wmz4JpaiHOsSXYSaiBERSM/1oIdeDivKE=; b=ABoANn1SocdcSJe0a69St6583YRSnM4B1UXj2CrZOf6t3Q6g/bdXzxd+89iT2Vdh9R dwfBvXSUgbedXHVTXzxlrNATIp/mCgIbZiVvzrsfmO75tgFQFX7YSdX5QZPshzTDHBJF Os2685WAnbtDCqh3nYXdmGe7OBGUJ1XFogu5JhYV/iR1jc4OdYQJ8WRknU5l9bsdnQnG rj1MZKWeYjkdbVLNvNJ2ZlmERMHdJGXvwzxNOhoNVEfJTXyJ8Tr1VSJXxQnsW7NQDuis B6fnKn25x9V40AeUbc5JdxCA23KLrZZ98gT9g809J+MsPkLKRNYxRLYTnYaKA5HQh5FM R4eQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BIfnaM9m1+Wmz4JpaiHOsSXYSaiBERSM/1oIdeDivKE=; b=A3eKZCjjGG6tPeiyIGk+S/2ucKNkCD0kV4Lgg5y9aL7sZ/awJunogsgKGvfzw9gXgE PffzKlCLltys6MbYyTBDAM6eWVnmybZCervq9WwONO8LYGBYTdYBAEi/51Uz5OjfPPRw zaO+ykHTk7xjxx5sCEZZXm3mIgD4FYHrSI9yBaSjT+WcE2ZBd/rr7Tj+O4m+B6zipJPT puVHSGHSMfZhlGWuj1HwP9vNUWSPLvrP7vcx/aq0jlBcql6bs3eBoxNIGRtKvbuy9r0U NjPvV7hZtOAW2KtKiMP+pBlblsro2KhFH+lzURNPb9SHsWip3hBW2o7EDjXmeKZ1jaPI /zgQ== X-Gm-Message-State: AElRT7H8LGWb6WIwyb1suDooVIXhwK7fs4P0j2TkvHOag95UrClx1QRB PlAfCdLXkJcFEpz0xShGs+p1Nw== X-Received: by 10.80.184.198 with SMTP id l64mr7357678ede.5.1522330240592; Thu, 29 Mar 2018 06:30:40 -0700 (PDT) Received: from debian.speedport.ip (p2003006CCD4EDC480CC341E3A28E97C1.dip0.t-ipconnect.de. [2003:6c:cd4e:dc48:cc3:41e3:a28e:97c1]) by smtp.googlemail.com with ESMTPSA id k24sm4371873ede.62.2018.03.29.06.30.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 29 Mar 2018 06:30:39 -0700 (PDT) From: wm4 To: ffmpeg-devel@ffmpeg.org Date: Thu, 29 Mar 2018 15:30:43 +0200 Message-Id: <20180329133043.21890-2-nfxjfg@googlemail.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180329133043.21890-1-nfxjfg@googlemail.com> References: <20180329133043.21890-1-nfxjfg@googlemail.com> Subject: [FFmpeg-devel] [PATCH 2/2] pixdesc: deprecate AV_PIX_FMT_FLAG_PSEUDOPAL X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: wm4 MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" PSEUDOPAL pixel formats are not paletted, but carried a palette with the intention of allowing code to treat unpaletted formats as paletted. The palette simply mapped the byte values to the resulting RGB values, making it some sort of LUT for RGB conversion. It was used for 1 byte formats only: RGB4_BYTE, BGR4_BYTE, RGB8, BGR8, GRAY8. The first 4 are awfully obscure, used only by some ancient bitmap formats. The last one, GRAY8, is more common, but its treatment is grossly incorrect. It considers full range GRAY8 only, so GRAY8 coming from typical Y video planes was not mapped to the correct RGB values. Also, nothing actually used the PSEUDOPAL palette data, except xwdenc. All other code had to treat it as a special case, just to ignore or propagate palette data. In conclusion, this was just a very strange old hack that has no real justification to exist. It's negatively useful, because API users who allocate their own pixel data have to be aware that they need to allocate the palette, or FFmpeg will crash on them in _some_ situations. On top of this, there was no API to allocate the pseuo palette outside of av_frame_get_buffer(). (avpriv_set_systematic_pal2() was not public, which is good, because GRAY8 treatment is broken.) This patch not only deprecates AV_PIX_FMT_FLAG_PSEUDOPAL, but also makes the pseudo palette optional. Nothing accesses it anymore, though if it's set, it's propagated. It's still allocated and initialized for compatibility with API users that rely on this feature. But new API users do not need to allocate it. This was an explicit goal of this patch. Most changes replace AV_PIX_FMT_FLAG_PSEUDOPAL with FF_PSEUDOPAL. I first tried #ifdefing all code, but it was a mess. The FF_PSEUDOPAL macro reduces the mess, and still allows defining FF_API_PSEUDOPAL to 0. Passes FATE with FF_API_PSEUDOPAL enabled and disabled. In addition, FATE passes with FF_API_PSEUDOPAL set to 1, but with allocation functions manually changed to not allocating a palette. --- fftools/ffprobe.c | 2 ++ libavcodec/decode.c | 5 +---- libavcodec/ffv1dec.c | 2 +- libavcodec/rawdec.c | 6 ++++-- libavcodec/smvjpegdec.c | 2 +- libavfilter/drawutils.c | 2 +- libavfilter/framepool.c | 4 ++-- libavfilter/vf_crop.c | 2 +- libavfilter/vf_pixdesctest.c | 2 +- libavfilter/vf_scale.c | 5 ++--- libavfilter/vf_shuffleplanes.c | 2 +- libavutil/frame.c | 4 ++-- libavutil/imgutils.c | 13 +++++++------ libavutil/internal.h | 9 +++++++++ libavutil/pixdesc.c | 10 +++++----- libavutil/pixdesc.h | 8 ++++++++ libavutil/version.h | 3 +++ libswscale/swscale_internal.h | 14 +++++++++++--- 18 files changed, 62 insertions(+), 33 deletions(-) diff --git a/fftools/ffprobe.c b/fftools/ffprobe.c index a4ac6972a2..6965b9f679 100644 --- a/fftools/ffprobe.c +++ b/fftools/ffprobe.c @@ -3118,7 +3118,9 @@ static void ffprobe_show_pixel_formats(WriterContext *w) PRINT_PIX_FMT_FLAG(HWACCEL, "hwaccel"); PRINT_PIX_FMT_FLAG(PLANAR, "planar"); PRINT_PIX_FMT_FLAG(RGB, "rgb"); +#if FF_API_PSEUDOPAL PRINT_PIX_FMT_FLAG(PSEUDOPAL, "pseudopal"); +#endif PRINT_PIX_FMT_FLAG(ALPHA, "alpha"); writer_print_section_footer(w); } diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 40c8a8855c..d883a5f9fc 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -1614,7 +1614,7 @@ static int video_get_buffer(AVCodecContext *s, AVFrame *pic) pic->linesize[i] = 0; } if (desc->flags & AV_PIX_FMT_FLAG_PAL || - desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) + ((desc->flags & FF_PSEUDOPAL) && pic->data[1])) avpriv_set_systematic_pal2((uint32_t *)pic->data[1], pic->format); if (s->debug & FF_DEBUG_BUFFERS) @@ -1782,9 +1782,6 @@ static void validate_avframe_allocation(AVCodecContext *avctx, AVFrame *frame) for (i = 0; i < num_planes; i++) { av_assert0(frame->data[i]); } - // For now do not enforce anything for palette of pseudopal formats - if (num_planes == 1 && (flags & AV_PIX_FMT_FLAG_PSEUDOPAL)) - num_planes = 2; // For formats without data like hwaccel allow unused pointers to be non-NULL. for (i = num_planes; num_planes > 0 && i < FF_ARRAY_ELEMS(frame->data); i++) { if (frame->data[i]) diff --git a/libavcodec/ffv1dec.c b/libavcodec/ffv1dec.c index b4a183c5b7..7658a51685 100644 --- a/libavcodec/ffv1dec.c +++ b/libavcodec/ffv1dec.c @@ -950,7 +950,7 @@ static int decode_frame(AVCodecContext *avctx, void *data, int *got_frame, AVPac } if (desc->flags & AV_PIX_FMT_FLAG_PAL || - desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) { + desc->flags & FF_PSEUDOPAL) { dst[1] = p->data[1]; src[1] = f->last_picture.f->data[1]; } diff --git a/libavcodec/rawdec.c b/libavcodec/rawdec.c index 1893b26444..53f5b76e93 100644 --- a/libavcodec/rawdec.c +++ b/libavcodec/rawdec.c @@ -92,12 +92,14 @@ static av_cold int raw_init_decoder(AVCodecContext *avctx) return AVERROR(EINVAL); } - if (desc->flags & (AV_PIX_FMT_FLAG_PAL | AV_PIX_FMT_FLAG_PSEUDOPAL)) { + if (desc->flags & (AV_PIX_FMT_FLAG_PAL | FF_PSEUDOPAL)) { context->palette = av_buffer_alloc(AVPALETTE_SIZE); if (!context->palette) return AVERROR(ENOMEM); +#if FF_API_PSEUDOPAL if (desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) avpriv_set_systematic_pal2((uint32_t*)context->palette->data, avctx->pix_fmt); +#endif else { memset(context->palette->data, 0, AVPALETTE_SIZE); if (avctx->bits_per_coded_sample == 1) @@ -423,7 +425,7 @@ static int raw_decode(AVCodecContext *avctx, void *data, int *got_frame, } if ((avctx->pix_fmt == AV_PIX_FMT_PAL8 && buf_size < context->frame_size) || - (desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL)) { + (desc->flags & FF_PSEUDOPAL)) { frame->buf[1] = av_buffer_ref(context->palette); if (!frame->buf[1]) { av_buffer_unref(&frame->buf[0]); diff --git a/libavcodec/smvjpegdec.c b/libavcodec/smvjpegdec.c index 0b05d19f7b..7ea82ebfee 100644 --- a/libavcodec/smvjpegdec.c +++ b/libavcodec/smvjpegdec.c @@ -71,7 +71,7 @@ static inline void smv_img_pnt(uint8_t *dst_data[4], uint8_t *src_data[4], src_linesizes[i], h, nlines); } if (desc->flags & AV_PIX_FMT_FLAG_PAL || - desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) + desc->flags & FF_PSEUDOPAL) dst_data[1] = src_data[1]; } diff --git a/libavfilter/drawutils.c b/libavfilter/drawutils.c index 17e26c764a..db8c7a6226 100644 --- a/libavfilter/drawutils.c +++ b/libavfilter/drawutils.c @@ -184,7 +184,7 @@ int ff_draw_init(FFDrawContext *draw, enum AVPixelFormat format, unsigned flags) if (!desc || !desc->name) return AVERROR(EINVAL); - if (desc->flags & ~(AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_PSEUDOPAL | AV_PIX_FMT_FLAG_ALPHA)) + if (desc->flags & ~(AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_RGB | FF_PSEUDOPAL | AV_PIX_FMT_FLAG_ALPHA)) return AVERROR(ENOSYS); if (format == AV_PIX_FMT_P010LE || format == AV_PIX_FMT_P010BE || format == AV_PIX_FMT_P016LE || format == AV_PIX_FMT_P016BE) return AVERROR(ENOSYS); diff --git a/libavfilter/framepool.c b/libavfilter/framepool.c index da2ac5cf69..3b178cebb8 100644 --- a/libavfilter/framepool.c +++ b/libavfilter/framepool.c @@ -103,7 +103,7 @@ FFFramePool *ff_frame_pool_video_init(AVBufferRef* (*alloc)(int size), } if (desc->flags & AV_PIX_FMT_FLAG_PAL || - desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) { + desc->flags & FF_PSEUDOPAL) { pool->pools[1] = av_buffer_pool_init(AVPALETTE_SIZE, alloc); if (!pool->pools[1]) goto fail; @@ -227,7 +227,7 @@ AVFrame *ff_frame_pool_get(FFFramePool *pool) } if (desc->flags & AV_PIX_FMT_FLAG_PAL || - desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) { + desc->flags & FF_PSEUDOPAL) { enum AVPixelFormat format = pool->format == AV_PIX_FMT_PAL8 ? AV_PIX_FMT_BGR8 : pool->format; diff --git a/libavfilter/vf_crop.c b/libavfilter/vf_crop.c index 0fdc4949e3..84be4c7d0d 100644 --- a/libavfilter/vf_crop.c +++ b/libavfilter/vf_crop.c @@ -288,7 +288,7 @@ static int filter_frame(AVFilterLink *link, AVFrame *frame) frame->data[0] += s->y * frame->linesize[0]; frame->data[0] += s->x * s->max_step[0]; - if (!(desc->flags & AV_PIX_FMT_FLAG_PAL || desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL)) { + if (!(desc->flags & AV_PIX_FMT_FLAG_PAL || desc->flags & FF_PSEUDOPAL)) { for (i = 1; i < 3; i ++) { if (frame->data[i]) { frame->data[i] += (s->y >> s->vsub) * frame->linesize[i]; diff --git a/libavfilter/vf_pixdesctest.c b/libavfilter/vf_pixdesctest.c index d6423acb91..2d0749e20b 100644 --- a/libavfilter/vf_pixdesctest.c +++ b/libavfilter/vf_pixdesctest.c @@ -81,7 +81,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) /* copy palette */ if (priv->pix_desc->flags & AV_PIX_FMT_FLAG_PAL || - priv->pix_desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) + ((priv->pix_desc->flags & FF_PSEUDOPAL) && out->data[1] && in->data[1])) memcpy(out->data[1], in->data[1], AVPALETTE_SIZE); for (c = 0; c < priv->pix_desc->nb_components; c++) { diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c index 9f45032e85..f741419e7e 100644 --- a/libavfilter/vf_scale.c +++ b/libavfilter/vf_scale.c @@ -261,11 +261,10 @@ static int config_props(AVFilterLink *outlink) /* TODO: make algorithm configurable */ - scale->input_is_pal = desc->flags & AV_PIX_FMT_FLAG_PAL || - desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL; + scale->input_is_pal = desc->flags & AV_PIX_FMT_FLAG_PAL; if (outfmt == AV_PIX_FMT_PAL8) outfmt = AV_PIX_FMT_BGR8; scale->output_is_pal = av_pix_fmt_desc_get(outfmt)->flags & AV_PIX_FMT_FLAG_PAL || - av_pix_fmt_desc_get(outfmt)->flags & AV_PIX_FMT_FLAG_PSEUDOPAL; + av_pix_fmt_desc_get(outfmt)->flags & FF_PSEUDOPAL; if (scale->sws) sws_freeContext(scale->sws); diff --git a/libavfilter/vf_shuffleplanes.c b/libavfilter/vf_shuffleplanes.c index 4bc7b79f87..32d2d585f3 100644 --- a/libavfilter/vf_shuffleplanes.c +++ b/libavfilter/vf_shuffleplanes.c @@ -69,7 +69,7 @@ static av_cold int shuffleplanes_config_input(AVFilterLink *inlink) } if ((desc->flags & AV_PIX_FMT_FLAG_PAL || - desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) && + desc->flags & FF_PSEUDOPAL) && (i == 1) != (s->map[i] == 1)) { av_log(ctx, AV_LOG_ERROR, "Cannot map between a palette plane and a data plane.\n"); diff --git a/libavutil/frame.c b/libavutil/frame.c index ea13cd3ed6..00215ac29a 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -247,7 +247,7 @@ static int get_video_buffer(AVFrame *frame, int align) frame->data[i] = frame->buf[i]->data; } - if (desc->flags & AV_PIX_FMT_FLAG_PAL || desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) { + if (desc->flags & AV_PIX_FMT_FLAG_PAL || desc->flags & FF_PSEUDOPAL) { av_buffer_unref(&frame->buf[1]); frame->buf[1] = av_buffer_alloc(AVPALETTE_SIZE); if (!frame->buf[1]) @@ -848,7 +848,7 @@ static int calc_cropping_offsets(size_t offsets[4], const AVFrame *frame, int shift_x = (i == 1 || i == 2) ? desc->log2_chroma_w : 0; int shift_y = (i == 1 || i == 2) ? desc->log2_chroma_h : 0; - if (desc->flags & (AV_PIX_FMT_FLAG_PAL | AV_PIX_FMT_FLAG_PSEUDOPAL) && i == 1) { + if (desc->flags & (AV_PIX_FMT_FLAG_PAL | FF_PSEUDOPAL) && i == 1) { offsets[i] = 0; break; } diff --git a/libavutil/imgutils.c b/libavutil/imgutils.c index 5af4fc20a0..4938a7ef67 100644 --- a/libavutil/imgutils.c +++ b/libavutil/imgutils.c @@ -125,7 +125,7 @@ int av_image_fill_pointers(uint8_t *data[4], enum AVPixelFormat pix_fmt, int hei size[0] = linesizes[0] * height; if (desc->flags & AV_PIX_FMT_FLAG_PAL || - desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) { + desc->flags & FF_PSEUDOPAL) { data[1] = ptr + size[0]; /* palette is stored here as 256 32 bits words */ return size[0] + 256 * 4; } @@ -216,7 +216,7 @@ int av_image_alloc(uint8_t *pointers[4], int linesizes[4], av_free(buf); return ret; } - if (desc->flags & AV_PIX_FMT_FLAG_PAL || desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) { + if (desc->flags & AV_PIX_FMT_FLAG_PAL || (desc->flags & FF_PSEUDOPAL && pointers[1])) { avpriv_set_systematic_pal2((uint32_t*)pointers[1], pix_fmt); if (align < 4) { av_log(NULL, AV_LOG_ERROR, "Formats with a palette require a minimum alignment of 4\n"); @@ -225,7 +225,7 @@ int av_image_alloc(uint8_t *pointers[4], int linesizes[4], } if ((desc->flags & AV_PIX_FMT_FLAG_PAL || - desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) && + desc->flags & FF_PSEUDOPAL) && pointers[1] && pointers[1] - pointers[0] > linesizes[0] * h) { /* zero-initialize the padding before the palette */ memset(pointers[0] + linesizes[0] * h, 0, @@ -354,12 +354,13 @@ static void image_copy(uint8_t *dst_data[4], const ptrdiff_t dst_linesizes[4], return; if (desc->flags & AV_PIX_FMT_FLAG_PAL || - desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) { + desc->flags & FF_PSEUDOPAL) { copy_plane(dst_data[0], dst_linesizes[0], src_data[0], src_linesizes[0], width, height); /* copy the palette */ - memcpy(dst_data[1], src_data[1], 4*256); + if ((desc->flags & AV_PIX_FMT_FLAG_PAL) || (dst_data[1] && src_data[1])) + memcpy(dst_data[1], src_data[1], 4*256); } else { int i, planes_nb = 0; @@ -442,7 +443,7 @@ int av_image_get_buffer_size(enum AVPixelFormat pix_fmt, return ret; // do not include palette for these pseudo-paletted formats - if (desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL) + if (desc->flags & FF_PSEUDOPAL) return FFALIGN(width, align) * height; return av_image_fill_arrays(data, linesize, NULL, pix_fmt, diff --git a/libavutil/internal.h b/libavutil/internal.h index c77dfa7d3c..06bd561e82 100644 --- a/libavutil/internal.h +++ b/libavutil/internal.h @@ -360,4 +360,13 @@ void ff_check_pixfmt_descriptors(void); */ int avpriv_dict_set_timestamp(AVDictionary **dict, const char *key, int64_t timestamp); +// Helper macro for AV_PIX_FMT_FLAG_PSEUDOPAL deprecation. Code inside FFmpeg +// should always use FF_PSEUDOPAL. Once the public API flag gets removed, all +// code using it is dead code. +#if FF_API_PSEUDOPAL +#define FF_PSEUDOPAL AV_PIX_FMT_FLAG_PSEUDOPAL +#else +#define FF_PSEUDOPAL 0 +#endif + #endif /* AVUTIL_INTERNAL_H */ diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index e77d5f44b9..8ed52751c1 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -257,7 +257,7 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { .comp = { { 0, 1, 0, 0, 8, 0, 7, 1 }, /* Y */ }, - .flags = AV_PIX_FMT_FLAG_PSEUDOPAL, + .flags = FF_PSEUDOPAL, .alias = "gray8,y8", }, [AV_PIX_FMT_MONOWHITE] = { @@ -362,7 +362,7 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { { 0, 1, 0, 3, 3, 0, 2, 1 }, /* G */ { 0, 1, 0, 6, 2, 0, 1, 1 }, /* B */ }, - .flags = AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_PSEUDOPAL, + .flags = AV_PIX_FMT_FLAG_RGB | FF_PSEUDOPAL, }, [AV_PIX_FMT_BGR4] = { .name = "bgr4", @@ -386,7 +386,7 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { { 0, 1, 0, 1, 2, 0, 1, 1 }, /* G */ { 0, 1, 0, 3, 1, 0, 0, 1 }, /* B */ }, - .flags = AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_PSEUDOPAL, + .flags = AV_PIX_FMT_FLAG_RGB | FF_PSEUDOPAL, }, [AV_PIX_FMT_RGB8] = { .name = "rgb8", @@ -398,7 +398,7 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { { 0, 1, 0, 3, 3, 0, 2, 1 }, /* G */ { 0, 1, 0, 0, 3, 0, 2, 1 }, /* B */ }, - .flags = AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_PSEUDOPAL, + .flags = AV_PIX_FMT_FLAG_RGB | FF_PSEUDOPAL, }, [AV_PIX_FMT_RGB4] = { .name = "rgb4", @@ -422,7 +422,7 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { { 0, 1, 0, 1, 2, 0, 1, 1 }, /* G */ { 0, 1, 0, 0, 1, 0, 0, 1 }, /* B */ }, - .flags = AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_PSEUDOPAL, + .flags = AV_PIX_FMT_FLAG_RGB | FF_PSEUDOPAL, }, [AV_PIX_FMT_NV12] = { .name = "nv12", diff --git a/libavutil/pixdesc.h b/libavutil/pixdesc.h index ea046033a3..feb3eba5e0 100644 --- a/libavutil/pixdesc.h +++ b/libavutil/pixdesc.h @@ -154,6 +154,14 @@ typedef struct AVPixFmtDescriptor { * in some cases be simpler. Or the data can be interpreted purely based on * the pixel format without using the palette. * An example of a pseudo-paletted format is AV_PIX_FMT_GRAY8 + * + * @deprecated This flag is deprecated, and will be removed. When it is removed, + * the extra palette allocation in AVFrame.data[1] is removed as well. Only + * actual paletted formats (as indicated by AV_PIX_FMT_FLAG_PAL) will have a + * palette. Starting with FFmpeg versions which have this flag deprecated, the + * extra "pseudo" palette is already ignored, and API users are not required to + * allocate a palette for AV_PIX_FMT_FLAG_PSEUDOPAL formats (it was require + * before the deprecation, though). */ #define AV_PIX_FMT_FLAG_PSEUDOPAL (1 << 6) diff --git a/libavutil/version.h b/libavutil/version.h index d3dd2df544..491b841403 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -126,6 +126,9 @@ #ifndef FF_API_FRAME_GET_SET #define FF_API_FRAME_GET_SET (LIBAVUTIL_VERSION_MAJOR < 57) #endif +#ifndef FF_API_PSEUDOPAL +#define FF_API_PSEUDOPAL (LIBAVUTIL_VERSION_MAJOR < 57) +#endif /** diff --git a/libswscale/swscale_internal.h b/libswscale/swscale_internal.h index c9120d8f5f..1703856ab2 100644 --- a/libswscale/swscale_internal.h +++ b/libswscale/swscale_internal.h @@ -806,9 +806,17 @@ static av_always_inline int isPlanarRGB(enum AVPixelFormat pix_fmt) static av_always_inline int usePal(enum AVPixelFormat pix_fmt) { - const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(pix_fmt); - av_assert0(desc); - return (desc->flags & AV_PIX_FMT_FLAG_PAL) || (desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL); + switch (pix_fmt) { + case AV_PIX_FMT_PAL8: + case AV_PIX_FMT_BGR4_BYTE: + case AV_PIX_FMT_BGR8: + case AV_PIX_FMT_GRAY8: + case AV_PIX_FMT_RGB4_BYTE: + case AV_PIX_FMT_RGB8: + return 1; + default: + return 0; + } } extern const uint64_t ff_dither4[2];