From patchwork Mon Apr 27 04:27:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neil Birkbeck X-Patchwork-Id: 19290 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1687944B060 for ; Mon, 27 Apr 2020 07:28:08 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C2E7A68C389; Mon, 27 Apr 2020 07:28:07 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B5A7568C341 for ; Mon, 27 Apr 2020 07:28:01 +0300 (EEST) Received: by mail-pf1-f173.google.com with SMTP id x2so4651894pfx.7 for ; Sun, 26 Apr 2020 21:28:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=aRL2v3m7hcCtSbUs5kE/S8NsdXnrKVwC8NC9dOZOLyk=; b=NaL3KVpFABv/688wyzv0F7MghSXnGY86o9dxZPKUwKAt2pJmW2W2960I96FFQAaReh bWc+mVjDr7pHfj8STttI31CCYRblNcmYrGCjgJkzzklyKwc4edZB2m+VsYpAOvtXOV1J 3D7SmAVfCU3w5jT9dmEJIHQGJZtPI5yHIh1k4WbWtc59TSKfmJCqTpY0u7yC5YBzJ0wU L5rmThHwf74wL6MLpEjhnGcmJ4ALdG/dUSW0yaMTaR6lD7RkQz4qq3nV2ZbK17PvHXYH m4UHyjcsuY9mbGDUu+DfKYRv+2I2D9LATzVLZHpIp4Mnm92EZon+ytG7w9mexgTyX5/u RJwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=aRL2v3m7hcCtSbUs5kE/S8NsdXnrKVwC8NC9dOZOLyk=; b=aVBW1NeTAN6X23RXAQWtLeXSAYSpkboOhYf256yqevigY/yy9jmBahQXXtGHSeP9MF 3EvCDHOCUxZ+zjaA4J4INJ98wHo8aj1IDeteCE8Npq0ZubtXjuSMeOHxhxSWpNQ6+rpN xfofwem5oNohC77D7ACRZWGpHgmP5PXfLL1/BeUeJlQVsZo14kTi6swJ9zQrvpcXtza/ 7MZ6+LORRmDg7MVQmrVyx5W38QzSN/8hlbXUM6ZuNciCLkYlYb8kh+PwG74HpNpxnHGl nC+RjyyV/Vb+k3+oU/Gq9YFfRHe8Zv9W4UP7jbLbgsXdBE0KW0ev2McBrjxaOCv0idHi exGg== X-Gm-Message-State: AGi0PuYJI0/hTzLd1TQTf/UmGLR3HvRVeTU8J0UozQETDUw10UIRNG/c O0FMZphvwx4bBjwP5GNzTypWRWzK X-Google-Smtp-Source: APiQypIYIW/JGD3dQ4kP3+0DmLBfYSsCL9IWyYWfLZBl5/V87wYPOqY0VmYnYQFbvx4d9GtEZxSZZQ== X-Received: by 2002:a63:f70f:: with SMTP id x15mr19043521pgh.199.1587961679196; Sun, 26 Apr 2020 21:27:59 -0700 (PDT) Received: from birkbeck-glaptop2.roam.corp.google.com (c-73-189-189-30.hsd1.ca.comcast.net. [73.189.189.30]) by smtp.gmail.com with ESMTPSA id i190sm11216795pfe.114.2020.04.26.21.27.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2020 21:27:58 -0700 (PDT) From: Neil Birkbeck To: ffmpeg-devel@ffmpeg.org Date: Sun, 26 Apr 2020 21:27:53 -0700 Message-Id: <20200427042753.166716-1-neil.birkbeck@gmail.com> X-Mailer: git-send-email 2.26.2.303.gf8c07b1a785-goog MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] libavutil: add clean aperture (CLAP) side data. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The clean aperature represents a cropping of the stored image data used to relate the image data to a canonical video system and exists as container metadata (see 'clap' section in https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap3/qtff3.html) Addition of the side data is a first step towards demuxing CLAP atom metadata, helping to resolve https://trac.ffmpeg.org/ticket/7437 This CleanAperture representation can also carry PixelCrop fields from MKV. Side data was suggested as a way to carry such PixelCrop fields in: https://ffmpeg.org/pipermail/ffmpeg-devel/2016-March/192302.html Transmuxing the side data can then be added (MOV to/from MKV), and auto-application could optionally be enabled like autorotate in ffmpeg_filter.c. Signed-off-by: Neil Birkbeck --- libavcodec/avpacket.c | 1 + libavcodec/packet.h | 12 ++++++++ libavformat/dump.c | 15 +++++++++ libavutil/Makefile | 2 ++ libavutil/clean_aperture.c | 30 ++++++++++++++++++ libavutil/clean_aperture.h | 63 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 123 insertions(+) create mode 100644 libavutil/clean_aperture.c create mode 100644 libavutil/clean_aperture.h diff --git a/libavcodec/avpacket.c b/libavcodec/avpacket.c index 55b509108e..2ff779720b 100644 --- a/libavcodec/avpacket.c +++ b/libavcodec/avpacket.c @@ -397,6 +397,7 @@ const char *av_packet_side_data_name(enum AVPacketSideDataType type) case AV_PKT_DATA_AFD: return "Active Format Description data"; case AV_PKT_DATA_ICC_PROFILE: return "ICC Profile"; case AV_PKT_DATA_DOVI_CONF: return "DOVI configuration record"; + case AV_PKT_DATA_CLEAN_APERTURE: return "Clean Aperture"; } return NULL; } diff --git a/libavcodec/packet.h b/libavcodec/packet.h index 41485f4527..a0f8c29a33 100644 --- a/libavcodec/packet.h +++ b/libavcodec/packet.h @@ -282,6 +282,18 @@ enum AVPacketSideDataType { */ AV_PKT_DATA_DOVI_CONF, + /** + * This side data contains a crop region (clean aperture) that defines the + * region of the stored image that should be cropped. + * + * It is intended to be used to store crop-related metadata from the + * container. For example, the CLAP atom in MOV files, and PixelCrop fields + * in MKV. + * + * The data is of type AVCleanAperture (see libavutil/clean_aperture.h) + */ + AV_PKT_DATA_CLEAN_APERTURE, + /** * The number of side data types. * This is not part of the public API/ABI in the sense that it may diff --git a/libavformat/dump.c b/libavformat/dump.c index 5e9a03185f..060e7b2d10 100644 --- a/libavformat/dump.c +++ b/libavformat/dump.c @@ -26,6 +26,7 @@ #include "libavutil/display.h" #include "libavutil/intreadwrite.h" #include "libavutil/log.h" +#include "libavutil/clean_aperture.h" #include "libavutil/mastering_display_metadata.h" #include "libavutil/dovi_meta.h" #include "libavutil/mathematics.h" @@ -402,6 +403,16 @@ static void dump_dovi_conf(void *ctx, AVPacketSideData* sd) dovi->dv_bl_signal_compatibility_id); } +static void dump_clean_aperture(void *ctx, AVPacketSideData *sd) +{ + AVCleanAperture *clap = (AVCleanAperture *)sd->data; + av_log(ctx, AV_LOG_INFO, "[width %d/%d height:%d/%d h_offset:%d/%d v_offset:%d/%d]", + clap->width.num, clap->width.den, + clap->height.num, clap->height.den, + clap->horizontal_offset.num, clap->horizontal_offset.den, + clap->vertical_offset.num, clap->vertical_offset.den); +} + static void dump_sidedata(void *ctx, AVStream *st, const char *indent) { int i; @@ -468,6 +479,10 @@ static void dump_sidedata(void *ctx, AVStream *st, const char *indent) av_log(ctx, AV_LOG_INFO, "DOVI configuration record: "); dump_dovi_conf(ctx, &sd); break; + case AV_PKT_DATA_CLEAN_APERTURE: + av_log(ctx, AV_LOG_INFO, "Clean aperture:"); + dump_clean_aperture(ctx, &sd); + break; default: av_log(ctx, AV_LOG_INFO, "unknown side data type %d (%d bytes)", sd.type, sd.size); diff --git a/libavutil/Makefile b/libavutil/Makefile index 966eec41aa..e9c9b2bff1 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -17,6 +17,7 @@ HEADERS = adler32.h \ cast5.h \ camellia.h \ channel_layout.h \ + clean_aperture.h \ common.h \ cpu.h \ crc.h \ @@ -107,6 +108,7 @@ OBJS = adler32.o \ cast5.o \ camellia.o \ channel_layout.o \ + clean_aperture.o \ color_utils.o \ cpu.o \ crc.o \ diff --git a/libavutil/clean_aperture.c b/libavutil/clean_aperture.c new file mode 100644 index 0000000000..0be51725f3 --- /dev/null +++ b/libavutil/clean_aperture.c @@ -0,0 +1,30 @@ +/** + * Copyright (c) 2020 Neil Birkbeck + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include + +#include "clean_aperture.h" +#include "mem.h" + +AVCleanAperture *av_clean_aperture_alloc(void) +{ + return av_mallocz(sizeof(AVCleanAperture)); +} diff --git a/libavutil/clean_aperture.h b/libavutil/clean_aperture.h new file mode 100644 index 0000000000..116cf36976 --- /dev/null +++ b/libavutil/clean_aperture.h @@ -0,0 +1,63 @@ +/* + * Copyright (c) 2020 Neil Birkbeck + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_CLEAN_APERTURE_H +#define AVUTIL_CLEAN_APERTURE_H + +#include "frame.h" +#include "rational.h" + + +/** + * Clean aperture represents the subset of stored pixels (as width, height + * and a center offset) that relates the pixels of the stored image to a + * canonical display area. + * + * The clean aperture structure is meant to be allocated as stream side data. + * It represents the Clean Aperture Atom (CLAP) and container-level cropping + * information (e.g., PixelCrop fields in MKV). + * + * @note The struct should be allocated with av_clean_aperture_alloc() + * and its size is not a part of the public ABI. + */ +typedef struct AVCleanAperture { + // The width of the aperture window (<= stored image width) + AVRational width; + + // The height of the aperture window (<= stored image height) + AVRational height; + + // The horizontal offset of the center (relative to stored image center) + AVRational horizontal_offset; + + // The vertical offset of the center (relative to stored image center) + AVRational vertical_offset; +} AVCleanAperture; + +/** + * Allocate an AVCleanAperture structure and set its fields to + * default values. The resulting struct can be freed using av_freep(). + * + * @return An AVCleanAperture filled with default values or NULL + * on failure. + */ +AVCleanAperture *av_clean_aperture_alloc(void); + +#endif /* AVUTIL_CLEAN_APERTURE_H */