From patchwork Thu Nov 23 22:08:42 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 6300 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.94 with SMTP id m30csp1328344jah; Thu, 23 Nov 2017 14:09:14 -0800 (PST) X-Google-Smtp-Source: AGs4zMbt/4Zsycs1jtay85luI40tO5MmaEVZHvixQmANSkklrxKCpvwn5Nifjis1+3h1dI3pvvGl X-Received: by 10.223.157.195 with SMTP id q3mr16379744wre.241.1511474954802; Thu, 23 Nov 2017 14:09:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511474954; cv=none; d=google.com; s=arc-20160816; b=PuODQWvtnNql4czqhE9rXe2apLKGqJaGtF8SXg/ub5ZHuGicAaBM3dW8c7TCiLOyb0 5FUKxS+ZT43oMhvt6J6FmbUQ8ZCYND/fando4E1oB6E+zABw2lSadQZ6BnyKvyLNb/4S pqASRpfw55agp56D7kQAlHoA9+173hB98CCkfGSIHiByyTQVpM6DvGAnwCWcmHYcucbD qDnAO7eCJIY1BzWZGVEDFGNYWyfJQIoVZilX8sVTn3ENp9Nj7M575NA6JvOngm9tXbyZ jHcrUBoN0PNYoUS9/peiW3z2Vqhq9q6yBJNHFIDUtjJRZIIjWIHWYjjTSe2k6oP5Ve+h d3eA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=Zbb5R20NH0Ri9z5HTmejys1CSciOXUH9OtY89RBRJhg=; b=SFbRfWsjPBs7v3Qu0E+n6Rpaa5u/U+foWv8TZK1vGitfQKW9T/KwfRSMk5mhLXvUK0 X94boNxQJEeJEQzzDIEdt0J3IGPoLy1C8vMDZV29eX57AaXogSGRom1yy7EKLewfG1RN Yv7uv1iFYdQxigzMLyXLiEBwNmMYiCDWDK0A91CcDoAoqw2fMRkQK6kf5YSoC0uwxFJf sPWF575KsJqwVVi7qYiNgJDrUo79g9TZuk0yYQGKpE2PICxbb+s3HpiYc4oWh1QEgzyb +m4uakNNdKvn1kwJn8hNf5507Mg6G2nZE7duSQ9CIe6ZfE1KmXnsO79B5wazqSMs1VPp Qj9Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=fY4T8dig; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k25si13314561wrd.111.2017.11.23.14.09.14; Thu, 23 Nov 2017 14:09:14 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=fY4T8dig; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E1F7A68A1D8; Fri, 24 Nov 2017 00:09:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk0-f194.google.com (mail-qk0-f194.google.com [209.85.220.194]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3515668A184 for ; Fri, 24 Nov 2017 00:09:06 +0200 (EET) Received: by mail-qk0-f194.google.com with SMTP id p19so22488228qke.2 for ; Thu, 23 Nov 2017 14:09:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=1LJOPws/XqrHRqr/A1fxsjQZN1vBNoXQj/yYG1V32rw=; b=fY4T8dig8gyLydQUEzRd7qRK7LLNiUPEBslDzSmI/uS9R8/A/VAO4ZLf6FelWqpxvF Wdo4/OT2Os+udMPih+/wv2QDc8ZNo5ubsTXxBrjH3VYurbllO07vCYsgSDTk6VnALot2 OYDHml9Ec6FBpYEpGmriaDXfL8Th6t17YjYeqHNX3LCmyX2nTtHlJErXQwIukT/lUuif rz7RBofkTC9l35qJ96jNDyM7SmPzgyT0mQuVpLE+s6rtRbSY3Yol0zg37H9aygAABaAR Bb44JdDa82oCCPqG0M6k68kXbuuUW2BSYIRYGe0p63dkDBOhROtdmR2+wL98WeGLgNj6 GS1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=1LJOPws/XqrHRqr/A1fxsjQZN1vBNoXQj/yYG1V32rw=; b=oSK8qicM28D1i0ftEV5U+YJeSM70ETxko0MvA3XZl1GROQ/I2c8VPwB0qrRyUw4kst f1IqxpRm8CXAf873OhqshpxUgCmhlB5MXXAXQAPP56FkglphXrdlNxq6rRcxrRpsRnSt BHsuUFBuXAL/xfxRLoM7z/Km7IHCYYSB5Yh58WbjtzYSScZZ9/O2BgG09JfAqaZUDLsr mVgpS3IhuvMZqsOh+lk/uJ8YT2/T2/JePVQ/k28g/yBXqLgiWpTvIdFb4ZVz2pewA6iO q6fQbBuJ+RiRKHoIgRXp+3Zd9rRDWhjQ7mbj8Z1jotbihQVlZ2eMcIQIcm5NBo+vJM8P KpBg== X-Gm-Message-State: AJaThX79NhtRe8jvTP9Dzzej4kcvUCLQyptwVMKgC3j0/ca2ac4x5MVg QHDLvHhFdWktwAqaMbgRgXsXxA== X-Received: by 10.55.188.6 with SMTP id m6mr2756809qkf.75.1511474945508; Thu, 23 Nov 2017 14:09:05 -0800 (PST) Received: from localhost.localdomain ([181.231.59.220]) by smtp.gmail.com with ESMTPSA id m5sm13893569qkh.90.2017.11.23.14.09.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 23 Nov 2017 14:09:04 -0800 (PST) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Thu, 23 Nov 2017 19:08:42 -0300 Message-Id: <20171123220843.804-1-jamrial@gmail.com> X-Mailer: git-send-email 2.15.0 Subject: [FFmpeg-devel] [PATCH 1/2] lavf/flacenc: support writing attached pictures X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Rodger Combs Signed-off-by: James Almer --- Should be good to commit now. libavformat/flacenc.c | 286 +++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 250 insertions(+), 36 deletions(-) diff --git a/libavformat/flacenc.c b/libavformat/flacenc.c index b894f9ef61..84da54a1df 100644 --- a/libavformat/flacenc.c +++ b/libavformat/flacenc.c @@ -21,10 +21,13 @@ #include "libavutil/channel_layout.h" #include "libavutil/opt.h" +#include "libavutil/pixdesc.h" #include "libavcodec/flac.h" #include "avformat.h" #include "avio_internal.h" #include "flacenc.h" +#include "id3v2.h" +#include "internal.h" #include "vorbiscomment.h" #include "libavcodec/bytestream.h" @@ -33,8 +36,16 @@ typedef struct FlacMuxerContext { const AVClass *class; int write_header; + int audio_stream_idx; + AVPacket *pics; + int nb_pics, waiting_pics; + /* audio packets are queued here until we get all the attached pictures */ + AVPacketList *queue, *queue_end; + /* updated streaminfo sent by the encoder at the end */ uint8_t *streaminfo; + + unsigned attached_types; } FlacMuxerContext; static int flac_write_block_padding(AVIOContext *pb, unsigned int n_padding_bytes, @@ -74,31 +85,157 @@ static int flac_write_block_comment(AVIOContext *pb, AVDictionary **m, return 0; } -static int flac_write_header(struct AVFormatContext *s) +static int flac_write_picture(struct AVFormatContext *s, AVPacket *pkt) { - int ret; - int padding = s->metadata_header_padding; - AVCodecParameters *par = s->streams[0]->codecpar; - FlacMuxerContext *c = s->priv_data; - - if (!c->write_header) + FlacMuxerContext *c = s->priv_data; + AVIOContext *pb = s->pb; + const AVPixFmtDescriptor *pixdesc; + const CodecMime *mime = ff_id3v2_mime_tags; + AVDictionaryEntry *e; + const char *mimetype = NULL, *desc = ""; + const AVStream *st = s->streams[pkt->stream_index]; + int i, mimelen, desclen, type = 0; + + if (!pkt->data) return 0; - if (s->nb_streams > 1) { - av_log(s, AV_LOG_ERROR, "only one stream is supported\n"); + while (mime->id != AV_CODEC_ID_NONE) { + if (mime->id == st->codecpar->codec_id) { + mimetype = mime->str; + break; + } + mime++; + } + if (!mimetype) { + av_log(s, AV_LOG_ERROR, "No mimetype is known for stream %d, cannot " + "write an attached picture.\n", st->index); + return AVERROR(EINVAL); + } + mimelen = strlen(mimetype); + + /* get the picture type */ + e = av_dict_get(st->metadata, "comment", NULL, 0); + for (i = 0; e && i < FF_ARRAY_ELEMS(ff_id3v2_picture_types); i++) { + if (!av_strcasecmp(e->value, ff_id3v2_picture_types[i])) { + type = i; + break; + } + } + + if ((c->attached_types & (1 << type)) & 0x6) { + av_log(s, AV_LOG_ERROR, "Duplicate attachment for type '%s'\n", ff_id3v2_picture_types[type]); return AVERROR(EINVAL); } - if (par->codec_id != AV_CODEC_ID_FLAC) { - av_log(s, AV_LOG_ERROR, "unsupported codec\n"); + + if (type == 1 && (st->codecpar->codec_id != AV_CODEC_ID_PNG || + st->codecpar->width != 32 || + st->codecpar->height != 32)) { + av_log(s, AV_LOG_ERROR, "File icon attachment must be a 32x32 PNG"); return AVERROR(EINVAL); } + c->attached_types |= (1 << type); + + /* get the description */ + if ((e = av_dict_get(st->metadata, "title", NULL, 0))) + desc = e->value; + desclen = strlen(desc); + + avio_w8(pb, 0x06); + avio_wb24(pb, 4 + 4 + mimelen + 4 + desclen + 4 + 4 + 4 + 4 + 4 + pkt->size); + + avio_wb32(pb, type); + + avio_wb32(pb, mimelen); + avio_write(pb, mimetype, mimelen); + + avio_wb32(pb, desclen); + avio_write(pb, desc, desclen); + + avio_wb32(pb, st->codecpar->width); + avio_wb32(pb, st->codecpar->height); + if ((pixdesc = av_pix_fmt_desc_get(st->codecpar->format))) + avio_wb32(pb, av_get_bits_per_pixel(pixdesc)); + else + avio_wb32(pb, 0); + avio_wb32(pb, 0); + + avio_wb32(pb, pkt->size); + avio_write(pb, pkt->data, pkt->size); + return 0; +} + +static int flac_finish_header(struct AVFormatContext *s) +{ + FlacMuxerContext *c = s->priv_data; + int i, ret, padding = s->metadata_header_padding; if (padding < 0) padding = 8192; /* The FLAC specification states that 24 bits are used to represent the * size of a metadata block so we must clip this value to 2^24-1. */ padding = av_clip_uintp2(padding, 24); + for (i = 0; i < c->nb_pics; i++) { + ret = flac_write_picture(s, &c->pics[i]); + av_packet_unref(&c->pics[i]); + if (ret) + return ret; + } + + ret = flac_write_block_comment(s->pb, &s->metadata, !padding, + s->flags & AVFMT_FLAG_BITEXACT); + if (ret) + return ret; + + /* The command line flac encoder defaults to placing a seekpoint + * every 10s. So one might add padding to allow that later + * but there seems to be no simple way to get the duration here. + * So just add the amount requested by the user. */ + if (padding) + flac_write_block_padding(s->pb, padding, 1); + + return 0; +} + +static int flac_write_header(struct AVFormatContext *s) +{ + AVCodecParameters *par; + FlacMuxerContext *c = s->priv_data; + int ret, i; + + c->audio_stream_idx = -1; + for (i = 0; i < s->nb_streams; i++) { + AVStream *st = s->streams[i]; + if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { + if (c->audio_stream_idx >= 0 || st->codecpar->codec_id != AV_CODEC_ID_FLAC) { + av_log(s, AV_LOG_ERROR, "Invalid audio stream. Exactly one FLAC " + "audio stream is required.\n"); + return AVERROR(EINVAL); + } + par = s->streams[i]->codecpar; + c->audio_stream_idx = i; + } else if (st->codecpar->codec_type != AVMEDIA_TYPE_VIDEO) { + av_log(s, AV_LOG_ERROR, "Only audio streams and pictures are allowed in FLAC.\n"); + return AVERROR(EINVAL); + } else if (st->codecpar->codec_id == AV_CODEC_ID_GIF) { + av_log(s, AV_LOG_ERROR, "GIF image support is not implemented.\n"); + return AVERROR_PATCHWELCOME; + } else if (!c->write_header) { + av_log(s, AV_LOG_ERROR, "Can't write attached pictures without a header.\n"); + return AVERROR(EINVAL); + } + } + if (c->audio_stream_idx < 0) { + av_log(s, AV_LOG_ERROR, "No audio stream present.\n"); + return AVERROR(EINVAL); + } + c->waiting_pics = c->nb_pics = s->nb_streams - 1; + if (c->nb_pics && !(c->pics = av_calloc(c->nb_pics, sizeof(AVPacket)))) + return AVERROR(ENOMEM); + + if (!c->write_header) + return 0; + ret = ff_flac_write_header(s->pb, par->extradata, par->extradata_size, 0); if (ret) @@ -121,18 +258,51 @@ static int flac_write_header(struct AVFormatContext *s) } } - ret = flac_write_block_comment(s->pb, &s->metadata, !padding, - s->flags & AVFMT_FLAG_BITEXACT); - if (ret) - return ret; + if (!c->waiting_pics) + ret = flac_finish_header(s); - /* The command line flac encoder defaults to placing a seekpoint - * every 10s. So one might add padding to allow that later - * but there seems to be no simple way to get the duration here. - * So just add the amount requested by the user. */ - if (padding) - flac_write_block_padding(s->pb, padding, 1); + return ret; +} + +static int flac_write_audio_packet(struct AVFormatContext *s, AVPacket *pkt) +{ + FlacMuxerContext *c = s->priv_data; + uint8_t *streaminfo; + int streaminfo_size; + /* check for updated streaminfo */ + streaminfo = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, + &streaminfo_size); + if (streaminfo && streaminfo_size == FLAC_STREAMINFO_SIZE) { + av_freep(&c->streaminfo); + + c->streaminfo = av_malloc(FLAC_STREAMINFO_SIZE); + if (!c->streaminfo) + return AVERROR(ENOMEM); + memcpy(c->streaminfo, streaminfo, FLAC_STREAMINFO_SIZE); + } + + if (pkt->size) + avio_write(s->pb, pkt->data, pkt->size); + return 0; +} + +static int flac_queue_flush(AVFormatContext *s) +{ + FlacMuxerContext *c = s->priv_data; + AVPacketList *pktl; + int ret = 0, write = 1; + + flac_finish_header(s); + + while ((pktl = c->queue)) { + if (write && (ret = flac_write_audio_packet(s, &pktl->pkt)) < 0) + write = 0; + av_packet_unref(&pktl->pkt); + c->queue = pktl->next; + av_freep(&pktl); + } + c->queue_end = NULL; return ret; } @@ -142,7 +312,18 @@ static int flac_write_trailer(struct AVFormatContext *s) int64_t file_size; FlacMuxerContext *c = s->priv_data; uint8_t *streaminfo = c->streaminfo ? c->streaminfo : - s->streams[0]->codecpar->extradata; + s->streams[c->audio_stream_idx]->codecpar->extradata; + int i; + + if (c->waiting_pics) { + av_log(s, AV_LOG_WARNING, "No packets were sent for some of the " + "attached pictures.\n"); + flac_queue_flush(s); + } + + for (i = 0; i < c->nb_pics; i++) + av_packet_unref(&c->pics[i]); + av_freep(&c->pics); if (!c->write_header || !streaminfo) return 0; @@ -166,23 +347,56 @@ static int flac_write_trailer(struct AVFormatContext *s) static int flac_write_packet(struct AVFormatContext *s, AVPacket *pkt) { FlacMuxerContext *c = s->priv_data; - uint8_t *streaminfo; - int streaminfo_size; + int ret; - /* check for updated streaminfo */ - streaminfo = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, - &streaminfo_size); - if (streaminfo && streaminfo_size == FLAC_STREAMINFO_SIZE) { - av_freep(&c->streaminfo); + if (pkt->stream_index == c->audio_stream_idx) { + if (c->waiting_pics) { + /* buffer audio packets until we get all the pictures */ + AVPacketList *pktl = av_mallocz(sizeof(*pktl)); + + if (!pktl || av_packet_ref(&pktl->pkt, pkt) < 0) { + av_free(pktl); + if (s->error_recognition & AV_EF_EXPLODE) + return AVERROR(ENOMEM); + av_log(s, AV_LOG_ERROR, "Out of memory in packet queue; skipping attached pictures\n"); + c->waiting_pics = 0; + ret = flac_queue_flush(s); + if (ret < 0) + return ret; + return flac_write_audio_packet(s, pkt); + } + + if (c->queue_end) + c->queue_end->next = pktl; + else + c->queue = pktl; + c->queue_end = pktl; + } else + return flac_write_audio_packet(s, pkt); + } else { + int index = pkt->stream_index; - c->streaminfo = av_malloc(FLAC_STREAMINFO_SIZE); - if (!c->streaminfo) - return AVERROR(ENOMEM); - memcpy(c->streaminfo, streaminfo, FLAC_STREAMINFO_SIZE); + /* warn only once for each stream */ + if (s->streams[pkt->stream_index]->nb_frames == 1) { + av_log(s, AV_LOG_WARNING, "Got more than one picture in stream %d," + " ignoring.\n", pkt->stream_index); + } + if (!c->waiting_pics || s->streams[pkt->stream_index]->nb_frames >= 1) + return 0; + + if (index > c->audio_stream_idx) + index--; + + if ((ret = av_packet_ref(&c->pics[index], pkt)) < 0) + return ret; + c->waiting_pics--; + + /* flush the buffered audio packets */ + if (!c->waiting_pics && + (ret = flac_queue_flush(s)) < 0) + return ret; } - if (pkt->size) - avio_write(s->pb, pkt->data, pkt->size); return 0; } @@ -205,7 +419,7 @@ AVOutputFormat ff_flac_muxer = { .mime_type = "audio/x-flac", .extensions = "flac", .audio_codec = AV_CODEC_ID_FLAC, - .video_codec = AV_CODEC_ID_NONE, + .video_codec = AV_CODEC_ID_PNG, .write_header = flac_write_header, .write_packet = flac_write_packet, .write_trailer = flac_write_trailer,