From patchwork Fri Mar 2 11:16:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wm4 X-Patchwork-Id: 7793 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.181.170 with SMTP id m39csp6115220jaj; Fri, 2 Mar 2018 03:16:35 -0800 (PST) X-Google-Smtp-Source: AG47ELuwmnJ6HtlleuaDNvtw1KGs1W5ZRTBpNYujTQOU40t01WzWYzRui7cDt+Snp4288cPUqS8t X-Received: by 10.28.132.203 with SMTP id g194mr1289960wmd.77.1519989395780; Fri, 02 Mar 2018 03:16:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519989395; cv=none; d=google.com; s=arc-20160816; b=RnE+TTQNje/kQtmmqDvcJjpMo151oi4XuKd61FTBag3etQOGuHbC2iUMVfzz0N8PT+ IAQx0iUnnjHieUwhl6w9QxbWWlVx29NCSVD9mR9oHUtrjsvIIeuG0dy1yf2+CPmGE2WY iJLw4+CsG/5Y2ilT0s/L6WPMy5EfuBblGDfxhnPQtt1CQYoCm/wJtltj3EgJwSCVRpdA IhM3rbb8LJ9zNtCl6udzzQ0nwWVy7DpMTWasXBkFsMUwrYKQsJq7SyDIAF39o7AHznF7 8OGXYABI7lrSPcK7Y/sHjnxYafG6R3kTHt85vDnXIxl4kvWbt4Q8HoAgFPkWQxANtz0j vOYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=28aPqWgOH71QUrkwFZjCOwPjvlaZhmvMCGZzxo5JOO4=; b=gi3cSYLFUH3D2/Bts9EzcQ+K8SjiiXoOaXP/lIwKSukxvdHbyJ+kJfAL//TzMx+k3z HVPfYfi9G//zHv/HRmAdHpKlJgk6DKZRW0S4CxQiIZKulxNYBz/sLk0GRetQtZBRqjJH Nmkz75OnmiCkRILWXyf96S1t7+78kXgl1jnDjfqoIV3jAE547rTUvKy+fd2cr66574f1 RnpUa6x1oKi4tK7fFlMy8kgRDub1odzssR+ShT7QShBmUpxgErZFND8QbUJ0HNj2N1Hx LmPmzXcU8bAqelPvPBFpXvkbgGE2AugSMWaclRYgwAbgBqVQ2WEg265EUEOHhvDwp8lY CPPA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com header.s=20161025 header.b=iataIL64; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 35si4399942wrc.20.2018.03.02.03.16.35; Fri, 02 Mar 2018 03:16:35 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com header.s=20161025 header.b=iataIL64; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 46C6768A5C7; Fri, 2 Mar 2018 13:16:28 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f43.google.com (mail-wm0-f43.google.com [74.125.82.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 272BC68A555 for ; Fri, 2 Mar 2018 13:16:22 +0200 (EET) Received: by mail-wm0-f43.google.com with SMTP id z9so2369593wmb.3 for ; Fri, 02 Mar 2018 03:16:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QS0v/E0jq8vjWy52uTGlA4oN7tKy3VjvBNpaGEg9lR4=; b=iataIL64pFlq7iKs6KMV7nOaK/B0ut/dGh9fUEWOhNMn7RgilStt5D+i49xMiHqfqX /Ln4udcnrwapZjS/KsO7TFN2r2WpSv5sVZ/hZ5NhOldp/pAgmWUoF+e4RZ+BxNJ+krro FtZN1fnaud4I1ZXEB5LLxfkOg0r43C4KkfgDy8DRiY6mwvRhk2GqvWG2miw7CWXyDLcd zLMka+U7QRQmDomqfsgFUYph2SsOC8lIDbq8FwJOXDyOuKoS4KOz90dpUbG5kC0iZ/N+ BV1giFI8d5VNdO7fHk+7Z+/YiSh4x79GyHdaydFOvJo2pIVTpC48GOabhy/fkXNr1M7y emZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QS0v/E0jq8vjWy52uTGlA4oN7tKy3VjvBNpaGEg9lR4=; b=hLeyVAqV/qQ2OnL0asRTfOwnapd25+xDKlbIzZWqne2MppFbET9NaSZknWyVYCP/Kt a9OWJhL5PP+0Pd/O5CNoprfFDeOnBkPwxDf3jU6mgx03T1Z7vF9KeowcV2h0Z0hlLUml wsL/Yrf5qKu6f18G4rPmJo7j7WYPS5iSqE6s8G/SDIFsC03j90IX6UGwAaxp0aBuiwiW xkH3xIzNLb1jZ+zdjIQSPmQHYVQ/TMhlQYpGIUmeEEha6tT6nHfo7V/FTB5hrItWGWLb nO9s0uZjtR9eybZbelwfdC/1T0hqf5vC1gjIGlv3aX8G//5BEljzcl6tIAbnv4lA8wJ9 70Yw== X-Gm-Message-State: AElRT7H6nFxuTUfzmVbBiGjXLU+sqaoocbWEp3hJvare50lFqpSkmnxf uKq61hobufw+uW7spsF9SHC50w== X-Received: by 10.28.40.214 with SMTP id o205mr1153792wmo.22.1519989387050; Fri, 02 Mar 2018 03:16:27 -0800 (PST) Received: from debian.speedport.ip (p2003006CCD4EDC507558E017D078E109.dip0.t-ipconnect.de. [2003:6c:cd4e:dc50:7558:e017:d078:e109]) by smtp.googlemail.com with ESMTPSA id f127sm159777wmg.46.2018.03.02.03.16.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Mar 2018 03:16:26 -0800 (PST) From: wm4 To: ffmpeg-devel@ffmpeg.org Date: Fri, 2 Mar 2018 12:16:37 +0100 Message-Id: <20180302111637.27597-1-nfxjfg@googlemail.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180302011018.GD23018@michaelspb> References: <20180302011018.GD23018@michaelspb> Subject: [FFmpeg-devel] [PATCH v2] lavu/frame: add QP side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: wm4 MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This adds a way for an API user to transfer QP data and metadata without having to keep the reference to AVFrame, and without having to explicitly care about QP APIs. It might also provide a way to finally remove the deprecated QP related fields. In the end, the QP table should be handled in a very similar way to e.g. AV_FRAME_DATA_MOTION_VECTORS. There are two side data types, because I didn't care about having to repack the QP data so the table and the metadata are in a single AVBufferRef. Otherwise it would have either required a copy on decoding (extra slowdown for something as obscure as the QP data), or would have required making intrusive changes to the codecs which support export of this data. The new side data types are added under deprecation guards, because I don't intend to change the status of the QP export as being deprecated (as it was before this patch too). --- libavutil/frame.c | 59 ++++++++++++++++++++++++++++++++++---- libavutil/frame.h | 17 +++++++++++ tests/ref/fate/exif-image-embedded | 6 ++++ tests/ref/fate/exif-image-jpg | 34 +++++++++++++--------- 4 files changed, 96 insertions(+), 20 deletions(-) diff --git a/libavutil/frame.c b/libavutil/frame.c index 0db2a2d57b..ea13cd3ed6 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -46,8 +46,17 @@ MAKE_ACCESSORS(AVFrame, frame, enum AVColorRange, color_range) av_get_channel_layout_nb_channels((frame)->channel_layout)) #if FF_API_FRAME_QP +struct qp_properties { + int stride; + int type; +}; + int av_frame_set_qp_table(AVFrame *f, AVBufferRef *buf, int stride, int qp_type) { + struct qp_properties *p; + AVFrameSideData *sd; + AVBufferRef *ref; + FF_DISABLE_DEPRECATION_WARNINGS av_buffer_unref(&f->qp_table_buf); @@ -57,20 +66,56 @@ FF_DISABLE_DEPRECATION_WARNINGS f->qscale_type = qp_type; FF_ENABLE_DEPRECATION_WARNINGS + av_frame_remove_side_data(f, AV_FRAME_DATA_QP_TABLE_PROPERTIES); + av_frame_remove_side_data(f, AV_FRAME_DATA_QP_TABLE_DATA); + + ref = av_buffer_ref(buf); + if (!av_frame_new_side_data_from_buf(f, AV_FRAME_DATA_QP_TABLE_DATA, ref)) { + av_buffer_unref(&ref); + return AVERROR(ENOMEM); + } + + sd = av_frame_new_side_data(f, AV_FRAME_DATA_QP_TABLE_PROPERTIES, + sizeof(struct qp_properties)); + if (!sd) + return AVERROR(ENOMEM); + + p = (struct qp_properties *)sd->data; + p->stride = stride; + p->type = qp_type; + return 0; } int8_t *av_frame_get_qp_table(AVFrame *f, int *stride, int *type) { -FF_DISABLE_DEPRECATION_WARNINGS - *stride = f->qstride; - *type = f->qscale_type; + AVBufferRef *buf = NULL; - if (!f->qp_table_buf) - return NULL; + *stride = 0; + *type = 0; - return f->qp_table_buf->data; +FF_DISABLE_DEPRECATION_WARNINGS + if (f->qp_table_buf) { + *stride = f->qstride; + *type = f->qscale_type; + buf = f->qp_table_buf; FF_ENABLE_DEPRECATION_WARNINGS + } else { + AVFrameSideData *sd; + struct qp_properties *p; + sd = av_frame_get_side_data(f, AV_FRAME_DATA_QP_TABLE_PROPERTIES); + if (!sd) + return NULL; + p = (struct qp_properties *)sd->data; + sd = av_frame_get_side_data(f, AV_FRAME_DATA_QP_TABLE_DATA); + if (!sd) + return NULL; + *stride = p->stride; + *type = p->type; + buf = sd->buf; + } + + return buf ? buf->data : NULL; } #endif @@ -787,6 +832,8 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_CONTENT_LIGHT_LEVEL: return "Content light level metadata"; case AV_FRAME_DATA_GOP_TIMECODE: return "GOP timecode"; case AV_FRAME_DATA_ICC_PROFILE: return "ICC profile"; + case AV_FRAME_DATA_QP_TABLE_PROPERTIES: return "QP table properties"; + case AV_FRAME_DATA_QP_TABLE_DATA: return "QP table data"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index ddbac3156d..9d57d6ce66 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -141,6 +141,23 @@ enum AVFrameSideDataType { * metadata key entry "name". */ AV_FRAME_DATA_ICC_PROFILE, + +#if FF_API_FRAME_QP + /** + * Implementation-specific description of the format of AV_FRAME_QP_TABLE_DATA. + * The contents of this side data are undocumented and internal; use + * av_frame_set_qp_table() and av_frame_get_qp_table() to access this in a + * meaningful way instead. + */ + AV_FRAME_DATA_QP_TABLE_PROPERTIES, + + /** + * Raw QP table data. Its format is described by + * AV_FRAME_DATA_QP_TABLE_PROPERTIES. Use av_frame_set_qp_table() and + * av_frame_get_qp_table() to access this instead. + */ + AV_FRAME_DATA_QP_TABLE_DATA, +#endif }; enum AVActiveFormatDescription { diff --git a/tests/ref/fate/exif-image-embedded b/tests/ref/fate/exif-image-embedded index 306ae0854b..0b640767a8 100644 --- a/tests/ref/fate/exif-image-embedded +++ b/tests/ref/fate/exif-image-embedded @@ -29,6 +29,12 @@ color_transfer=unknown chroma_location=center TAG:UserComment=AppleMark +[SIDE_DATA] +side_data_type=QP table data +[/SIDE_DATA] +[SIDE_DATA] +side_data_type=QP table properties +[/SIDE_DATA] [/FRAME] [FRAME] media_type=audio diff --git a/tests/ref/fate/exif-image-jpg b/tests/ref/fate/exif-image-jpg index b266501191..dec2b78737 100644 --- a/tests/ref/fate/exif-image-jpg +++ b/tests/ref/fate/exif-image-jpg @@ -27,30 +27,30 @@ color_space=bt470bg color_primaries=unknown color_transfer=unknown chroma_location=center -TAG:ImageDescription= +TAG:ImageDescription= TAG:Make=Canon TAG:Model=Canon PowerShot SX200 IS TAG:Orientation= 1 -TAG:XResolution= 180:1 -TAG:YResolution= 180:1 +TAG:XResolution= 180:1 +TAG:YResolution= 180:1 TAG:ResolutionUnit= 2 TAG:DateTime=2013:07:18 13:12:03 TAG:YCbCrPositioning= 2 -TAG:ExposureTime= 1:1250 -TAG:FNumber= 40:10 +TAG:ExposureTime= 1:1250 +TAG:FNumber= 40:10 TAG:ISOSpeedRatings= 160 TAG:ExifVersion= 48, 50, 50, 49 TAG:DateTimeOriginal=2013:07:18 13:12:03 TAG:DateTimeDigitized=2013:07:18 13:12:03 TAG:ComponentsConfiguration= 1, 2, 3, 0 -TAG:CompressedBitsPerPixel= 3:1 -TAG:ShutterSpeedValue= 329:32 -TAG:ApertureValue= 128:32 -TAG:ExposureBiasValue= 0:3 -TAG:MaxApertureValue= 113:32 +TAG:CompressedBitsPerPixel= 3:1 +TAG:ShutterSpeedValue= 329:32 +TAG:ApertureValue= 128:32 +TAG:ExposureBiasValue= 0:3 +TAG:MaxApertureValue= 113:32 TAG:MeteringMode= 5 TAG:Flash= 16 -TAG:FocalLength= 5000:1000 +TAG:FocalLength= 5000:1000 TAG:MakerNote= 25, 0, 1, 0, 3, 0, 48, 0, 0, 0, 28, 4, 0, 0, 2, 0 3, 0, 4, 0, 0, 0, 124, 4, 0, 0, 3, 0, 3, 0, 4, 0 @@ -219,14 +219,20 @@ TAG:GPSLatitudeRef=R98 TAG:GPSLatitude= 48, 49, 48, 48 TAG:0x1001= 4000 TAG:0x1002= 2248 -TAG:FocalPlaneXResolution=4000000:244 -TAG:FocalPlaneYResolution=2248000:183 +TAG:FocalPlaneXResolution=4000000:244 +TAG:FocalPlaneYResolution=2248000:183 TAG:FocalPlaneResolutionUnit= 2 TAG:SensingMethod= 2 TAG:FileSource= 3 TAG:CustomRendered= 0 TAG:ExposureMode= 0 TAG:WhiteBalance= 0 -TAG:DigitalZoomRatio= 4000:4000 +TAG:DigitalZoomRatio= 4000:4000 TAG:SceneCaptureType= 0 +[SIDE_DATA] +side_data_type=QP table data +[/SIDE_DATA] +[SIDE_DATA] +side_data_type=QP table properties +[/SIDE_DATA] [/FRAME]