From patchwork Thu Mar 1 15:58:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wm4 X-Patchwork-Id: 7776 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.181.170 with SMTP id m39csp5198331jaj; Thu, 1 Mar 2018 07:58:31 -0800 (PST) X-Google-Smtp-Source: AG47ELtkJKjdS9svBZa9Cje2XSOGDikCA2lrFoflXYECP0DRzYqxWHc2XKFKPnrXKr+3MVEvi0VE X-Received: by 10.223.150.42 with SMTP id b39mr2115137wra.55.1519919911637; Thu, 01 Mar 2018 07:58:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519919911; cv=none; d=google.com; s=arc-20160816; b=g0l9MIHDEcTmLQHr6iAnuFCfJnpPhXcrwL+6u1xSPg4oaN6cl6piLh29GZZOsFFZBg KIdVhiqTIp26RMQLdqO7TnYmUMXADXUPcVwGlazRcaiwt9i+VMQ/QHiAObfloDNvd4mI G7xajMYcDFzMfA3fWYVabP4GSokZ3srxxSSiqtKV5rLJYXrmxzHn4WtkP3Kzedhh57+a Gu+TdDkADxhyx4KMAxY5yRJtNTp7bnFgF5XHcOVUmA/51NEg4nH7zvVeByxJrWj8JcZJ P+92Iz5IxzNhchR+KqXLZAOQwxg8GauNdNOxNVx1hYkX1Yl+4EA7dy0Ium1+3oPWURXV vjuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=QV7iT6QHEEpucAQmbnszGpc0zBV33PuG1bjdAibXZJY=; b=KBlg/RUaAp8toTR/+Vmr+qZ/L8GYsF0zcvsW17MkE43FHCIAwBTyrw06AdVlBloR16 62wGcCkp6LC3imB+Art2FroIdGU40R2fQGWhtLVdVpPCTWAd4aQB4marjRUYUkrb8Ls3 BBGtiYmTsW262rABL3oPfXqsqOYgLQeOd+HzZAWdSYOzs/HMbexIkNGi8wKNZpCxMsm9 ONfiZwhvLWC+++SBWziTRnofSe551hSBBKv9TVkLrjWa0UAK3Y5a4Snpdt1U2lsVUq/X yfpGnv23szWTFTbmTwSsh0/CoeXtT5Tb57nbv7HNVCgON2W6FPs1bbnSFO6aLii1HGN2 kUwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com header.s=20161025 header.b=cOCHOCMw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o128si500859wmo.125.2018.03.01.07.58.31; Thu, 01 Mar 2018 07:58:31 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com header.s=20161025 header.b=cOCHOCMw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7F95668A439; Thu, 1 Mar 2018 17:58:16 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f182.google.com (mail-wr0-f182.google.com [209.85.128.182]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A20C668A188 for ; Thu, 1 Mar 2018 17:58:09 +0200 (EET) Received: by mail-wr0-f182.google.com with SMTP id l43so7100270wrc.2 for ; Thu, 01 Mar 2018 07:58:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ijmjEthPt24NXVgI+Ei0lG4h0xQCYnCt5VT6+4y+gbQ=; b=cOCHOCMwhbAtvzfGDj1fFvr/WTgPkPbL/cWHAxdZuX5Gnyh1kWRR/8BjHf/9/SK4vx /5Qw3fcJMgGmVpflOtJKJS24/1dlc7+NjYFqLV2Vkf/wDpeqlRhAIvb20+MzePGndr/X WOopYlN+aHAmt6RfP48vH3mZoh6qBcQnCf93doJpB1PbY8kSCeaj/KGggdxP0gvWOZuh VpXukZ2T8Lm1ab++CSC9JMc/Y7BoYtrOZsBMojmJP90YULRSrvhdCKvuJ7nZXhWD1E9r 4ITTth0baFRndwEcB4yetzeUuUMWJLQ08gfS76cRgFyco3pwZB8ieQqw4xM9UzgyJiKS kLtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ijmjEthPt24NXVgI+Ei0lG4h0xQCYnCt5VT6+4y+gbQ=; b=ojNXd7OcYP2zs4/qTYkRzqtmYCCnIAw0pAuPpMVaR4hvWk615YW9RkPKvIgxXL5hJ3 s/4VOXF1SM5XJCn3kBGgRQmAuCF9K0i6tHRO4mRJu+1wgIiXHXoCXR8SjdnqfNklWlLy Bud0IUYiNXVp/f6V+hvD9cWnx76Eo6UAntCd9aNFwyzfMOD1kdtWdgXFre5oKcDPEpua X9VbKWhev+6khoXeVZ1elrRi4KjAGgU3Y1neZhj3SWeDUgziWS+1cObC/TtbNSx9j7Ul GS1GU4z2tqI1xOSE0jp5iKCSqKkVj9+B+AyRLubzYYiCwWjcLAUc+IQj4vgf4rcePo6U iUHw== X-Gm-Message-State: APf1xPDVFVr7IAQzvreARb2sLuVKpzxvMSI7wLr0xIXlly7EKvF0i5gE YPk0OLsfUn+0u6NIqQ4YJ1byOg== X-Received: by 10.223.151.106 with SMTP id r97mr2363845wrb.203.1519919894505; Thu, 01 Mar 2018 07:58:14 -0800 (PST) Received: from debian.speedport.ip (p2003006CCD4EDC897558E017D078E109.dip0.t-ipconnect.de. [2003:6c:cd4e:dc89:7558:e017:d078:e109]) by smtp.googlemail.com with ESMTPSA id 62sm4451461wrf.24.2018.03.01.07.58.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Mar 2018 07:58:13 -0800 (PST) From: wm4 To: ffmpeg-devel@ffmpeg.org Date: Thu, 1 Mar 2018 16:58:22 +0100 Message-Id: <20180301155822.30477-2-nfxjfg@googlemail.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180301155822.30477-1-nfxjfg@googlemail.com> References: <20180301155822.30477-1-nfxjfg@googlemail.com> Subject: [FFmpeg-devel] [PATCH 2/2] lavu/frame: add QP side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: wm4 MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This adds a way for an API user to transfer QP data and metadata without having to keep the reference to AVFrame, and without having to explicitly care about QP APIs. It might also provide a way to finally remove the deprecated QP related fields. In the end, the QP table should be handled in a very similar way to e.g. AV_FRAME_DATA_MOTION_VECTORS. There are two side data types, because I didn't care about having to repack the QP data so the table and the metadata are in a single AVBufferRef. Otherwise it would have either required a copy on decoding (extra slowdown for something as obscure as the QP data), or would have required making intrusive changes to the codecs which support export of this data. The new side data types are added under deprecation guards, because I don't intend to change the status of the QP export as being deprecated (as it was before this patch too). --- libavutil/frame.c | 55 +++++++++++++++++++++++++++++++++----- libavutil/frame.h | 17 ++++++++++++ tests/ref/fate/exif-image-embedded | 6 +++++ tests/ref/fate/exif-image-jpg | 34 +++++++++++++---------- 4 files changed, 92 insertions(+), 20 deletions(-) diff --git a/libavutil/frame.c b/libavutil/frame.c index 3ecd739317..a9e76504eb 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -51,8 +51,16 @@ MAKE_ACCESSORS(AVFrame, frame, enum AVColorRange, color_range) av_get_channel_layout_nb_channels((frame)->channel_layout)) #if FF_API_FRAME_QP +struct qp_properties { + int stride; + int type; +}; + int av_frame_set_qp_table(AVFrame *f, AVBufferRef *buf, int stride, int qp_type) { + struct qp_properties *p; + AVFrameSideData *sd; + FF_DISABLE_DEPRECATION_WARNINGS av_buffer_unref(&f->qp_table_buf); @@ -62,20 +70,53 @@ FF_DISABLE_DEPRECATION_WARNINGS f->qscale_type = qp_type; FF_ENABLE_DEPRECATION_WARNINGS + av_frame_remove_side_data(f, AV_FRAME_DATA_QP_TABLE_PROPERTIES); + av_frame_remove_side_data(f, AV_FRAME_DATA_QP_TABLE_DATA); + + if (!frame_new_side_data(f, AV_FRAME_DATA_QP_TABLE_DATA, av_buffer_ref(buf))) + return AVERROR(ENOMEM); + + sd = av_frame_new_side_data(f, AV_FRAME_DATA_QP_TABLE_PROPERTIES, + sizeof(struct qp_properties)); + if (!sd) + return AVERROR(ENOMEM); + + p = (struct qp_properties *)sd->data; + p->stride = stride; + p->type = qp_type; + return 0; } int8_t *av_frame_get_qp_table(AVFrame *f, int *stride, int *type) { -FF_DISABLE_DEPRECATION_WARNINGS - *stride = f->qstride; - *type = f->qscale_type; + AVBufferRef *buf = NULL; - if (!f->qp_table_buf) - return NULL; + *stride = 0; + *type = 0; - return f->qp_table_buf->data; +FF_DISABLE_DEPRECATION_WARNINGS + if (f->qp_table_buf) { + *stride = f->qstride; + *type = f->qscale_type; + buf = f->qp_table_buf; FF_ENABLE_DEPRECATION_WARNINGS + } else { + AVFrameSideData *sd; + struct qp_properties *p; + sd = av_frame_get_side_data(f, AV_FRAME_DATA_QP_TABLE_PROPERTIES); + if (!sd) + return NULL; + p = (struct qp_properties *)sd->data; + sd = av_frame_get_side_data(f, AV_FRAME_DATA_QP_TABLE_DATA); + if (!sd) + return NULL; + *stride = p->stride; + *type = p->type; + buf = sd->buf; + } + + return buf ? buf->data : NULL; } #endif @@ -789,6 +830,8 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_CONTENT_LIGHT_LEVEL: return "Content light level metadata"; case AV_FRAME_DATA_GOP_TIMECODE: return "GOP timecode"; case AV_FRAME_DATA_ICC_PROFILE: return "ICC profile"; + case AV_FRAME_DATA_QP_TABLE_PROPERTIES: return "QP table properties"; + case AV_FRAME_DATA_QP_TABLE_DATA: return "QP table data"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index 4c3225b8bd..7c428cdfae 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -141,6 +141,23 @@ enum AVFrameSideDataType { * metadata key entry "name". */ AV_FRAME_DATA_ICC_PROFILE, + +#if FF_API_FRAME_QP + /** + * Implementation-specific description of the format of AV_FRAME_QP_TABLE_DATA. + * The contents of this side data are undocumented and internal; use + * av_frame_set_qp_table() and av_frame_get_qp_table() to access this in a + * meaningful way instead. + */ + AV_FRAME_DATA_QP_TABLE_PROPERTIES, + + /** + * Raw QP table data. Its format is described by + * AV_FRAME_DATA_QP_TABLE_PROPERTIES. Use av_frame_set_qp_table() and + * av_frame_get_qp_table() to access this instead. + */ + AV_FRAME_DATA_QP_TABLE_DATA, +#endif }; enum AVActiveFormatDescription { diff --git a/tests/ref/fate/exif-image-embedded b/tests/ref/fate/exif-image-embedded index 306ae0854b..0b640767a8 100644 --- a/tests/ref/fate/exif-image-embedded +++ b/tests/ref/fate/exif-image-embedded @@ -29,6 +29,12 @@ color_transfer=unknown chroma_location=center TAG:UserComment=AppleMark +[SIDE_DATA] +side_data_type=QP table data +[/SIDE_DATA] +[SIDE_DATA] +side_data_type=QP table properties +[/SIDE_DATA] [/FRAME] [FRAME] media_type=audio diff --git a/tests/ref/fate/exif-image-jpg b/tests/ref/fate/exif-image-jpg index b266501191..dec2b78737 100644 --- a/tests/ref/fate/exif-image-jpg +++ b/tests/ref/fate/exif-image-jpg @@ -27,30 +27,30 @@ color_space=bt470bg color_primaries=unknown color_transfer=unknown chroma_location=center -TAG:ImageDescription= +TAG:ImageDescription= TAG:Make=Canon TAG:Model=Canon PowerShot SX200 IS TAG:Orientation= 1 -TAG:XResolution= 180:1 -TAG:YResolution= 180:1 +TAG:XResolution= 180:1 +TAG:YResolution= 180:1 TAG:ResolutionUnit= 2 TAG:DateTime=2013:07:18 13:12:03 TAG:YCbCrPositioning= 2 -TAG:ExposureTime= 1:1250 -TAG:FNumber= 40:10 +TAG:ExposureTime= 1:1250 +TAG:FNumber= 40:10 TAG:ISOSpeedRatings= 160 TAG:ExifVersion= 48, 50, 50, 49 TAG:DateTimeOriginal=2013:07:18 13:12:03 TAG:DateTimeDigitized=2013:07:18 13:12:03 TAG:ComponentsConfiguration= 1, 2, 3, 0 -TAG:CompressedBitsPerPixel= 3:1 -TAG:ShutterSpeedValue= 329:32 -TAG:ApertureValue= 128:32 -TAG:ExposureBiasValue= 0:3 -TAG:MaxApertureValue= 113:32 +TAG:CompressedBitsPerPixel= 3:1 +TAG:ShutterSpeedValue= 329:32 +TAG:ApertureValue= 128:32 +TAG:ExposureBiasValue= 0:3 +TAG:MaxApertureValue= 113:32 TAG:MeteringMode= 5 TAG:Flash= 16 -TAG:FocalLength= 5000:1000 +TAG:FocalLength= 5000:1000 TAG:MakerNote= 25, 0, 1, 0, 3, 0, 48, 0, 0, 0, 28, 4, 0, 0, 2, 0 3, 0, 4, 0, 0, 0, 124, 4, 0, 0, 3, 0, 3, 0, 4, 0 @@ -219,14 +219,20 @@ TAG:GPSLatitudeRef=R98 TAG:GPSLatitude= 48, 49, 48, 48 TAG:0x1001= 4000 TAG:0x1002= 2248 -TAG:FocalPlaneXResolution=4000000:244 -TAG:FocalPlaneYResolution=2248000:183 +TAG:FocalPlaneXResolution=4000000:244 +TAG:FocalPlaneYResolution=2248000:183 TAG:FocalPlaneResolutionUnit= 2 TAG:SensingMethod= 2 TAG:FileSource= 3 TAG:CustomRendered= 0 TAG:ExposureMode= 0 TAG:WhiteBalance= 0 -TAG:DigitalZoomRatio= 4000:4000 +TAG:DigitalZoomRatio= 4000:4000 TAG:SceneCaptureType= 0 +[SIDE_DATA] +side_data_type=QP table data +[/SIDE_DATA] +[SIDE_DATA] +side_data_type=QP table properties +[/SIDE_DATA] [/FRAME]