From patchwork Sat Apr 14 18:32:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Timo_Ter=C3=A4s?= X-Patchwork-Id: 8446 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp2054143jad; Sat, 14 Apr 2018 11:41:10 -0700 (PDT) X-Google-Smtp-Source: AIpwx49rHZ9wSGkz0gb+mykbB6ov8eGs8HAnCm+BJJoag7crhCHhOn9r7xtNkakMlKyL4ZgkGDP7 X-Received: by 10.28.234.26 with SMTP id i26mr6846320wmh.104.1523731270496; Sat, 14 Apr 2018 11:41:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1523731270; cv=none; d=google.com; s=arc-20160816; b=T0a5As7b+qeoDHKvO4cWoZm7WE8jJc5kQkv7iLS0sZxx6PuQv2Jk2knJUUeGyqOazF oUbXltPC/9i5JFEgOiN93hpdwfpwnAayzMI0DzngpORApHz3n9RiWKzb7nE6qD/P0p92 04s/3HaXy78fXQoINa5sLc2ERvusqgSTk28UDQEwja51U4m+mC4yKyZl5UBxh0T3vbqB vcHzDJ125avOtZ2kNKI8dADreKHIxzGxa72//tWV9YY94Z2Uw+pZJMHM9xi5NlYLLgIw eG5JphbBZPbutEwenzeeeJr5cDJ1ln6XyUz3Rj1sXES75QCSLRvbSFGbH/u2z5XQJshY qRdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to:arc-authentication-results; bh=xzMip/946CqxsanxUYaGXOtMjx1lDofyyW+CtIc6qMQ=; b=Tx6mDxNANUsND4Nq8EYh4r9T7ek25vwfUXCsNtCvJmikJ6x6WLHhW4lSwIszOoeVbP N7Npa4DH7czR5xFDwqvl5q+SCE2PxOL8eGBQuCYyngWu6ZZ+eqBXd3O8NkaYOHVv+Y1/ LWJjOjG78yntJNtW72f4WpylN7O5jIQK4WBeGGAgLhKhIq4CLxWRxjYwEDoR/vxTjV67 n89g221cUfBnHi0AWWAhJLguR6+W0DFUxT+/UOlHS6Ul27w5Cfop4LVLIMRjBNThMYmX b4HSTcHAzvHVWApmXRGHswMkeSP+xFhG072CbqUFrjpEP8AEsPLnDQmiQRxeCqoOfHl8 tDXg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m71si2304162wma.134.2018.04.14.11.41.09; Sat, 14 Apr 2018 11:41:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 18BC5689DB6; Sat, 14 Apr 2018 21:40:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf0-f65.google.com (mail-lf0-f65.google.com [209.85.215.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4D66F680D38 for ; Sat, 14 Apr 2018 21:40:35 +0300 (EEST) Received: by mail-lf0-f65.google.com with SMTP id q9-v6so16732250lfk.9 for ; Sat, 14 Apr 2018 11:41:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WJtP8HrP8/vvsuGtFuGhrCYN8Ue8Br/nr4RqSiKNr5w=; b=jIQE212uuGbFJb0Qp3k33eBgBDjU/EnEgBjS3KRTGtVRMS8EQVDm+1M/hoIUaHf1Ua tHZZ9WA4/set/Qear01DpGcZThVDRMXCXigDEHa/Y5hDe8a2c4VlMYFsDOSAtpLfZLZJ 0Bb+AHiPWO3hAlT4leCrSYUfjHnFYAeHobKajE89ijhOIyxqhsjT9LmpaqSJPlN0MS/B I24EKVSmmvzrvj+ygrqXEfg0sNsEz38n7Z4taZvhfT/q3TIR+75J5O3tAyCKxVwLqTFN VnADYt1vXySlZQKNQA74BajWfIV++7G42yUxzGoZuXFPzHpCDHWIxYv31gbLGd3kztkC isiA== X-Gm-Message-State: ALQs6tD3pyk7+nibvkgI51C0JCHV2h1AhptXbWMnNIND/iN+4ertIeUR PbHYgr/IWVNyC2QXH37i+1I+76Qh X-Received: by 2002:a19:4f0a:: with SMTP id d10-v6mr11350998lfb.134.1523730772990; Sat, 14 Apr 2018 11:32:52 -0700 (PDT) Received: from localhost.localdomain ([83.145.235.201]) by smtp.gmail.com with ESMTPSA id a14sm7708ljk.66.2018.04.14.11.32.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 14 Apr 2018 11:32:52 -0700 (PDT) From: =?UTF-8?q?Timo=20Ter=C3=A4s?= To: FFmpeg development discussions and patches , Rostislav Pehlivanov Date: Sat, 14 Apr 2018 21:32:11 +0300 Message-Id: <20180414183211.12906-1-timo.teras@iki.fi> X-Mailer: git-send-email 2.17.0 In-Reply-To: , <20180411205059.48c25b80@vostro> References: , <20180411205059.48c25b80@vostro> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2] avformat/movenc: support writing iTunes cover image X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: =?UTF-8?q?Timo=20Ter=C3=A4s?= Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes https://trac.ffmpeg.org/ticket/2798 This makes movenc handle AV_DISPOSITION_ATTACHED_PIC and write the associated pictures in iTunes cover atom. This corresponds to how 'mov' demuxer parses and exposes the cover images when reading. Most of the existing track handling loops properly ignore these 'virtual streams' as MOVTrack->entry is never incremented for them. However, additional tests are added as needed to ignore them. Tested to produce valid output with: ffmpeg -i movie.mp4 -i thumb.jpg -disposition:v:1 attached_pic \ -map 0 -map 1 -c copy movie-with-cover.mp4 The cover image is also copied correctly with: ffmpeg -i movie-with-cover.mp4 -map 0 -c copy out.mp4 AtomicParseley says that the attached_pic stream is properly not visible in the main tracks of the file. Signed-off-by: Timo Teräs --- v2: - Store the image in MOVTrack->cover_image instead of AVStream->attached_pic per review request libavformat/movenc.c | 88 +++++++++++++++++++++++++++++++++++++++++--- libavformat/movenc.h | 1 + 2 files changed, 84 insertions(+), 5 deletions(-) diff --git a/libavformat/movenc.c b/libavformat/movenc.c index d03d7906a1..be5daa50c2 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -142,7 +142,9 @@ static int co64_required(const MOVTrack *track) static int rtp_hinting_needed(const AVStream *st) { - /* Add hint tracks for each audio and video stream */ + /* Add hint tracks for each real audio and video stream */ + if (st->disposition & AV_DISPOSITION_ATTACHED_PIC) + return 0; return st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO || st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO; } @@ -3420,6 +3422,51 @@ static int mov_write_int8_metadata(AVFormatContext *s, AVIOContext *pb, return size; } +static int mov_write_covr(AVIOContext *pb, AVFormatContext *s) +{ + MOVMuxContext *mov = s->priv_data; + int64_t pos = 0; + int i, type; + + for (i = 0; i < s->nb_streams; i++) { + MOVTrack *trk = &mov->tracks[i]; + AVStream *st = s->streams[i]; + + if (!(st->disposition & AV_DISPOSITION_ATTACHED_PIC) || + trk->cover_image.size <= 0) + continue; + + switch (st->codecpar->codec_id) { + case AV_CODEC_ID_MJPEG: + type = 0xD; + break; + case AV_CODEC_ID_PNG: + type = 0xE; + break; + case AV_CODEC_ID_BMP: + type = 0x1B; + break; + default: + av_log(s, AV_LOG_ERROR, "unsupported codec_id (0x%x) for cover", + st->codecpar->codec_id); + continue; + } + + if (!pos) { + pos = avio_tell(pb); + avio_wb32(pb, 0); + ffio_wfourcc(pb, "covr"); + } + avio_wb32(pb, 16 + trk->cover_image.size); + ffio_wfourcc(pb, "data"); + avio_wb32(pb, type); + avio_wb32(pb , 0); + avio_write(pb, trk->cover_image.data, trk->cover_image.size); + } + + return pos ? update_size(pb, pos) : 0; +} + /* iTunes meta data list */ static int mov_write_ilst_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) @@ -3454,6 +3501,7 @@ static int mov_write_ilst_tag(AVIOContext *pb, MOVMuxContext *mov, mov_write_int8_metadata (s, pb, "hdvd", "hd_video", 1); mov_write_int8_metadata (s, pb, "pgap", "gapless_playback",1); mov_write_int8_metadata (s, pb, "cpil", "compilation", 1); + mov_write_covr(pb, s); mov_write_trkn_tag(pb, mov, s, 0); // track number mov_write_trkn_tag(pb, mov, s, 1); // disc number mov_write_tmpo_tag(pb, s); @@ -3951,6 +3999,8 @@ static int mov_write_isml_manifest(AVIOContext *pb, MOVMuxContext *mov, AVFormat } else { continue; } + if (st->disposition & AV_DISPOSITION_ATTACHED_PIC) + continue; props = (AVCPBProperties*)av_stream_get_side_data(track->st, AV_PKT_DATA_CPB_PROPERTIES, NULL); @@ -4564,6 +4614,8 @@ static int mov_write_ftyp_tag(AVIOContext *pb, AVFormatContext *s) for (i = 0; i < s->nb_streams; i++) { AVStream *st = s->streams[i]; + if (st->disposition & AV_DISPOSITION_ATTACHED_PIC) + continue; if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) has_video = 1; if (st->codecpar->codec_id == AV_CODEC_ID_H264) @@ -4712,6 +4764,8 @@ static int mov_write_identification(AVIOContext *pb, AVFormatContext *s) int video_streams_nb = 0, audio_streams_nb = 0, other_streams_nb = 0; for (i = 0; i < s->nb_streams; i++) { AVStream *st = s->streams[i]; + if (st->disposition & AV_DISPOSITION_ATTACHED_PIC) + continue; if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) video_streams_nb++; else if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) @@ -4901,7 +4955,8 @@ static int mov_flush_fragment(AVFormatContext *s, int force) int buf_size, moov_size; for (i = 0; i < mov->nb_streams; i++) - if (!mov->tracks[i].entry) + if (!mov->tracks[i].entry && + (i >= s->nb_streams || !(s->streams[i]->disposition & AV_DISPOSITION_ATTACHED_PIC))) break; /* Don't write the initial moov unless all tracks have data */ if (i < mov->nb_streams && !force) @@ -5480,13 +5535,34 @@ static int mov_write_subtitle_end_packet(AVFormatContext *s, static int mov_write_packet(AVFormatContext *s, AVPacket *pkt) { + MOVMuxContext *mov = s->priv_data; + MOVTrack *trk; + AVStream *st; + if (!pkt) { mov_flush_fragment(s, 1); return 1; + } + + st = s->streams[pkt->stream_index]; + trk = &mov->tracks[pkt->stream_index]; + + if (st->disposition & AV_DISPOSITION_ATTACHED_PIC) { + int ret; + + if (st->nb_frames >= 1) { + if (st->nb_frames == 1) + av_log(s, AV_LOG_WARNING, "Got more than one picture in stream %d," + " ignoring.\n", pkt->stream_index); + return 0; + } + + if ((ret = av_packet_ref(&trk->cover_image, pkt)) < 0) + return ret; + + return 0; } else { int i; - MOVMuxContext *mov = s->priv_data; - MOVTrack *trk = &mov->tracks[pkt->stream_index]; if (!pkt->size) return mov_write_single_packet(s, pkt); /* Passthrough. */ @@ -5733,7 +5809,8 @@ static void enable_tracks(AVFormatContext *s) AVStream *st = s->streams[i]; if (st->codecpar->codec_type <= AVMEDIA_TYPE_UNKNOWN || - st->codecpar->codec_type >= AVMEDIA_TYPE_NB) + st->codecpar->codec_type >= AVMEDIA_TYPE_NB || + st->disposition & AV_DISPOSITION_ATTACHED_PIC) continue; if (first[st->codecpar->codec_type] < 0) @@ -5776,6 +5853,7 @@ static void mov_free(AVFormatContext *s) av_freep(&mov->tracks[i].par); av_freep(&mov->tracks[i].cluster); av_freep(&mov->tracks[i].frag_info); + av_packet_unref(&mov->tracks[i].cover_image); if (mov->tracks[i].vos_len) av_freep(&mov->tracks[i].vos_data); diff --git a/libavformat/movenc.h b/libavformat/movenc.h index ca2a9c9722..c9b4072fb9 100644 --- a/libavformat/movenc.h +++ b/libavformat/movenc.h @@ -132,6 +132,7 @@ typedef struct MOVTrack { uint32_t default_size; HintSampleQueue sample_queue; + AVPacket cover_image; AVIOContext *mdat_buf; int64_t data_offset;