From patchwork Tue Mar 15 11:59:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 34756 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:3486:0:0:0:0 with SMTP id ek6csp2944623nkb; Tue, 15 Mar 2022 05:00:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzmKLVtXNcXYx4GJ0kDJQqsu03Um6GRAtjYdn7do1rjyzTGtfshOF+dEKl5ahfdY2LcVrZr X-Received: by 2002:a17:906:d10c:b0:6cd:4aa2:cd62 with SMTP id b12-20020a170906d10c00b006cd4aa2cd62mr22857635ejz.229.1647345612251; Tue, 15 Mar 2022 05:00:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1647345612; cv=none; d=google.com; s=arc-20160816; b=LflG/EdY6xKyOgnCKFgrjFL3fzSwrEQKMhgURxCFc5IHtOA/jcSLRsU0G42yRJtkb+ lSsMPdfZ0CPKzwdnk2WfkrWMvQB43lBcWsHTFpaiCBD9nv6ULVHM5gMX6CDm8tSx5WZ2 fuHNCk08KK5/tJMFd/qurVsHGVf3aoIanLVIpHEzL7SmajB8vG0XFjlxF7Rzd+Aw+EYN qTKhVqdT9cchxbIuSQ65TnGKoEe69xIN8Pv6qh6pjx4qIcmQnFvvwzrjJmi4WShfJuI0 e/x5j25zSFDR3XweqcWpuxcqKpvOJ3yC2vrhp6blco524YFo8B2FyaTrX9FYl9If5KhU wrNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=UHj/xnF8rGYpbOkFM9GO5TxxEFIZ1pMlI8Nv+qRIdy4=; b=B35hRB9hnGk63h+tgeNBEDimpT5PtUGdgWRhJ4ltY93zlDjhI1WQtkPCBFhb3rpABW NH/XxDla9aG4h6ERp/UHcuWO7V3fINKzRYgG4lBiZbH0+s6ttcQibdNDCQ31xPMcaf1T GgOiZi9/OZqyBEMoGDtDwAo7pfnFqlN559XmrVXm/hDx5/+i8Hncm6+UW+BomLyxtQ6u uM0wkuxqcMfueBGmfdmr/Dz9hb+xbTshf2IO5TIo+z0fMys1RdOUUVH7us43FZWDJhCi 8C8fgoutX2J+RWzm9QwME+K20YVbYF8pUWx0+e2n6wEIXZuoMh/KgkIowbce3d8U7i8E bB+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=TEwDtlaT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gn17-20020a1709070d1100b006dab3af6877si11407075ejc.563.2022.03.15.05.00.11; Tue, 15 Mar 2022 05:00:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=TEwDtlaT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8B6AB68A6A3; Tue, 15 Mar 2022 14:00:07 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3525968A6A3 for ; Tue, 15 Mar 2022 14:00:00 +0200 (EET) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id DE30F49909; Tue, 15 Mar 2022 12:59:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1647345599; bh=7BvvkX1dotP1c0sI1FwWC8nJn8UsIgwb0XfIK5rurZo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TEwDtlaTgqUnpsnlXbYLqoWVTp3kehlyYGolqHVADsIquK2zCSA53yiO+7iGy+i2A 7WZdsmFhkXfHt5fSFRWGKHxFlht3OD34FCxGizfYoWEs3j7e0mXzZ2PZrFCyR6kapR nzhwUhYy+EwZAK0443diUJ+BKWE5AxDEgM0ExH88= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 15 Mar 2022 12:59:14 +0100 Message-Id: <20220315115913.45724-1-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220310131542.58255-1-ffmpeg@haasn.xyz> References: <20220310131542.58255-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2] avcodec/mjpegenc: support writing ICC profiles X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: WQiexRp2eWoO From: Niklas Haas This is mostly straightforward. The major complication is that, as a result of the 16-bit chunk size limitation, ICC profiles may need to be split up into multiple chunks. We also need to make sure to allocate enough extra space in the packet to fit the ICC profile, so modify both mpegvideo_enc.c and ljpegenc.c to take into account this extra overhead. Also add a FATE transcode test to ensure that the ICC profile gets written (and read) correctly. Note that this ICC profile is smaller than 64 kB, so this doesn't test the APP2 chunk re-arranging code at all. Signed-off-by: Niklas Haas --- Changes in v2: - Merge FATE test into this commit - Fix possible segfault (when no AVFrame is available) --- libavcodec/ljpegenc.c | 6 ++++- libavcodec/mjpegenc.c | 3 ++- libavcodec/mjpegenc_common.c | 43 +++++++++++++++++++++++++++++++++--- libavcodec/mjpegenc_common.h | 2 +- libavcodec/mpegvideo_enc.c | 3 +++ tests/fate/image.mak | 3 +++ tests/ref/fate/jpg-icc | 42 +++++++++++++++++++++++++++++++++++ 7 files changed, 96 insertions(+), 6 deletions(-) create mode 100644 tests/ref/fate/jpg-icc diff --git a/libavcodec/ljpegenc.c b/libavcodec/ljpegenc.c index e15f448f90..c54450c338 100644 --- a/libavcodec/ljpegenc.c +++ b/libavcodec/ljpegenc.c @@ -216,6 +216,7 @@ static int ljpeg_encode_frame(AVCodecContext *avctx, AVPacket *pkt, { LJpegEncContext *s = avctx->priv_data; PutBitContext pb; + const AVFrameSideData *sd; const int width = avctx->width; const int height = avctx->height; const int mb_width = (width + s->hsample[0] - 1) / s->hsample[0]; @@ -233,12 +234,15 @@ static int ljpeg_encode_frame(AVCodecContext *avctx, AVPacket *pkt, * s->hsample[0] * s->vsample[0]; } + if ((sd = av_frame_get_side_data(pict, AV_FRAME_DATA_ICC_PROFILE))) + max_pkt_size += sd->size; + if ((ret = ff_alloc_packet(avctx, pkt, max_pkt_size)) < 0) return ret; init_put_bits(&pb, pkt->data, pkt->size); - ff_mjpeg_encode_picture_header(avctx, &pb, NULL, &s->scantable, + ff_mjpeg_encode_picture_header(avctx, &pb, pict, NULL, &s->scantable, s->pred, s->matrix, s->matrix); header_bits = put_bits_count(&pb); diff --git a/libavcodec/mjpegenc.c b/libavcodec/mjpegenc.c index 08671b0df7..d7c0c763a1 100644 --- a/libavcodec/mjpegenc.c +++ b/libavcodec/mjpegenc.c @@ -77,7 +77,8 @@ static av_cold void init_uni_ac_vlc(const uint8_t huff_size_ac[256], static void mjpeg_encode_picture_header(MpegEncContext *s) { - ff_mjpeg_encode_picture_header(s->avctx, &s->pb, s->mjpeg_ctx, + const AVFrame *frame = s->picture->f; + ff_mjpeg_encode_picture_header(s->avctx, &s->pb, frame, s->mjpeg_ctx, &s->intra_scantable, 0, s->intra_matrix, s->chroma_intra_matrix); diff --git a/libavcodec/mjpegenc_common.c b/libavcodec/mjpegenc_common.c index 995e2b7670..5594a8d239 100644 --- a/libavcodec/mjpegenc_common.c +++ b/libavcodec/mjpegenc_common.c @@ -130,8 +130,10 @@ static void jpeg_table_header(AVCodecContext *avctx, PutBitContext *p, AV_WB16(ptr, size); } -static void jpeg_put_comments(AVCodecContext *avctx, PutBitContext *p) +static void jpeg_put_comments(AVCodecContext *avctx, PutBitContext *p, + const AVFrame *frame) { + const AVFrameSideData *sd = NULL; int size; uint8_t *ptr; @@ -161,6 +163,41 @@ static void jpeg_put_comments(AVCodecContext *avctx, PutBitContext *p) put_bits(p, 8, 0); /* thumbnail height */ } + /* ICC profile */ + if (frame) + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_ICC_PROFILE); + if (sd && sd->size) { + static const uint8_t hdr_size = strlen("ICC_PROFILE")+5; + static const uint16_t max_chunk_size = UINT16_MAX - hdr_size; + const uint8_t *data = sd->data; + size_t remaining = sd->size; + size_t nb_chunks = (remaining + max_chunk_size - 1) / max_chunk_size; + if (nb_chunks > 255) { + av_log(avctx, AV_LOG_WARNING, + "Cannot store %zu byte ICC profile: too large for JPEG\n", + sd->size); + nb_chunks = remaining = 0; + } + + for (int i = 0; i < nb_chunks; i++) { + size = FFMIN(remaining, max_chunk_size); + av_assert1(size > 0); + put_marker(p, APP2); + put_bits(p, 16, size + hdr_size); + ff_put_string(p, "ICC_PROFILE", 1); + put_bits(p, 8, i+1); + put_bits(p, 8, nb_chunks); + flush_put_bits(p); + ptr = put_bits_ptr(p); + skip_put_bytes(p, size); + memcpy(ptr, data, size); + remaining -= size; + data += size; + } + + av_assert1(!remaining); + } + /* comment */ if (!(avctx->flags & AV_CODEC_FLAG_BITEXACT)) { put_marker(p, COM); @@ -213,7 +250,7 @@ void ff_mjpeg_init_hvsample(AVCodecContext *avctx, int hsample[4], int vsample[4 } void ff_mjpeg_encode_picture_header(AVCodecContext *avctx, PutBitContext *pb, - MJpegContext *m, + const AVFrame *frame, struct MJpegContext *m, ScanTable *intra_scantable, int pred, uint16_t luma_intra_matrix[64], uint16_t chroma_intra_matrix[64]) @@ -233,7 +270,7 @@ void ff_mjpeg_encode_picture_header(AVCodecContext *avctx, PutBitContext *pb, if (avctx->codec_id == AV_CODEC_ID_AMV) return; - jpeg_put_comments(avctx, pb); + jpeg_put_comments(avctx, pb, frame); jpeg_table_header(avctx, pb, m, intra_scantable, luma_intra_matrix, chroma_intra_matrix, hsample); diff --git a/libavcodec/mjpegenc_common.h b/libavcodec/mjpegenc_common.h index ac753bf153..3c0a7addac 100644 --- a/libavcodec/mjpegenc_common.h +++ b/libavcodec/mjpegenc_common.h @@ -30,7 +30,7 @@ struct MJpegContext; void ff_mjpeg_encode_picture_header(AVCodecContext *avctx, PutBitContext *pb, - struct MJpegContext *m, + const AVFrame *frame, struct MJpegContext *m, ScanTable *intra_scantable, int pred, uint16_t luma_intra_matrix[64], uint16_t chroma_intra_matrix[64]); diff --git a/libavcodec/mpegvideo_enc.c b/libavcodec/mpegvideo_enc.c index c69114ea15..932dfa34f7 100644 --- a/libavcodec/mpegvideo_enc.c +++ b/libavcodec/mpegvideo_enc.c @@ -1686,10 +1686,13 @@ int ff_mpv_encode_picture(AVCodecContext *avctx, AVPacket *pkt, /* output? */ if (s->new_picture.f->data[0]) { + const AVFrameSideData *sd; int growing_buffer = context_count == 1 && !pkt->data && !s->data_partitioning; int pkt_size = growing_buffer ? FFMAX(s->mb_width*s->mb_height*64+10000, avctx->internal->byte_buffer_size) - AV_INPUT_BUFFER_PADDING_SIZE : s->mb_width*s->mb_height*(MAX_MB_BYTES+100)+10000; + if ((sd = av_frame_get_side_data(s->new_picture.f, AV_FRAME_DATA_ICC_PROFILE))) + pkt_size += sd->size; if ((ret = ff_alloc_packet(avctx, pkt, pkt_size)) < 0) return ret; if (s->mb_info) { diff --git a/tests/fate/image.mak b/tests/fate/image.mak index 573d398915..8b72890fbe 100644 --- a/tests/fate/image.mak +++ b/tests/fate/image.mak @@ -337,6 +337,9 @@ fate-jpg-12bpp: CMD = framecrc -idct simple -i $(TARGET_SAMPLES)/jpg/12bpp.jpg - FATE_JPG += fate-jpg-jfif fate-jpg-jfif: CMD = framecrc -idct simple -i $(TARGET_SAMPLES)/jpg/20242.jpg +FATE_JPG += fate-jpg-icc +fate-jpg-icc: CMD = transcode png_pipe $(TARGET_SAMPLES)/png1/lena-int_rgb24.png mjpeg "-vf scale" "" "" "-show_frames" + FATE_JPG-$(call DEMDEC, IMAGE2, MJPEG) += $(FATE_JPG) FATE_IMAGE += $(FATE_JPG-yes) fate-jpg: $(FATE_JPG-yes) diff --git a/tests/ref/fate/jpg-icc b/tests/ref/fate/jpg-icc new file mode 100644 index 0000000000..220146555e --- /dev/null +++ b/tests/ref/fate/jpg-icc @@ -0,0 +1,42 @@ +0a323df5cdfb9574e329b9831be054a6 *tests/data/fate/jpg-icc.mjpeg +11010 tests/data/fate/jpg-icc.mjpeg +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 128x128 +#sar 0: 1/1 +0, 0, 0, 1, 49152, 0xaac06b42 +[FRAME] +media_type=video +stream_index=0 +key_frame=1 +pts=0 +pts_time=0.000000 +pkt_dts=0 +pkt_dts_time=0.000000 +best_effort_timestamp=0 +best_effort_timestamp_time=0.000000 +pkt_duration=1 +pkt_duration_time=0.040000 +pkt_pos=0 +pkt_size=11010 +width=128 +height=128 +pix_fmt=yuvj444p +sample_aspect_ratio=1:1 +pict_type=I +coded_picture_number=0 +display_picture_number=0 +interlaced_frame=0 +top_field_first=0 +repeat_pict=0 +color_range=pc +color_space=bt470bg +color_primaries=unknown +color_transfer=unknown +chroma_location=center +[SIDE_DATA] +side_data_type=ICC profile +size=3144 +[/SIDE_DATA] +[/FRAME]