From patchwork Fri Apr 29 07:59:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 35499 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp153142pzb; Fri, 29 Apr 2022 01:06:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzp0XwAMHaHk/SCkG9MAeM8a681YmJ/7Ap9AKFIWlJdbnQXQu5NsDQWbfhWEk69C5883fbe X-Received: by 2002:a17:906:9c83:b0:6df:839a:a6d0 with SMTP id fj3-20020a1709069c8300b006df839aa6d0mr34418580ejc.419.1651219574683; Fri, 29 Apr 2022 01:06:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1651219574; cv=none; d=google.com; s=arc-20160816; b=0On1FWzAClrUs1C9VP5pP+S+w6t0p+rXFlZDmwJFk5gEKz6mhzirN/t8L1IbdRCVhf S/PrBZ3BsBg5UQS3lg0guMcEoKJ0ObjXO7sdeb2EcaGg24P/nrZSzlA8hezhHG7dcSt2 p1EMv6s6zbtpfJpaFu7gu52Hpe5ncAej6lHRsJqxkcI4Eiogxa5EYyB4tlKOT0sXm5H9 mNaOt1WhlMV9jxdWa7RmF+wEuunMhPcZdy9aPdiu5X48oYeMRAkVeF1nx2m91oatqWYB quXwl9vN5Qtoc7G6cD5EqZk4QfT/q0bJ7952G51KmaoN7rPeAJwwkkN5/rKNTMmJGZf7 IABA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=BJrZSP84AO4T7EAObDMSTOq5SRcQONMtz8bPxJ5FbFY=; b=ruqKTdtWt3dvv7N0S+i8hr7S+iwY6vnaNH5DE5T7UebQndjOmX8IeUOYxb6eqQKaWB GQGmZPm7Jq8O4ikLYx2OertfFYvO76p4kS/wRCgwXhuflYvb3xlNmpQS6w8pYCLW1Hyz OMETF0vZe8w42Hr1ejGp5enVPyCjHJOPhKHa1C0cgCVCaO12ytHS/F2tn2PwfU7HjvLU 8npkAwBfuDdi8df38MOgw4MQOJrZP9O+FuTHELbk8g4lxSxf1G3sOIzveez5nqn6BT65 +q/oXGHxONW3UBm4FhZPiKW8WrehPz7F0cOaIIuF6huEOAl8UK66IkabiUf+AJOkJjP1 mHXg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=YqZmTfxW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k25-20020a05640212d900b00425c55eb110si5202162edx.245.2022.04.29.01.06.13; Fri, 29 Apr 2022 01:06:14 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=YqZmTfxW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1554268AF1C; Fri, 29 Apr 2022 11:06:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7321368AF1C for ; Fri, 29 Apr 2022 11:06:02 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1651219567; x=1682755567; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=xCy8h2kkkS+sEQDGBXbJVzo5F7eT6GX1TsnN/0t2ULg=; b=YqZmTfxWOMckJpL07VFD7xYXPNJotLTf8wY5H5YTecEhWiC+fH7UdGGa VSeut/h7LrwLg5ZYN1zFG6fzcpD06UxZ6JPI32fieyT7dkax6J5PkMmj5 2G8aP9dXVYAIz5OxaDctnUzfHDeqtDMhXwlicfKfCGb6Hf7tvxXtiEIk4 NxzziY7+AdjVOE7jRXdI5BO8hUOmc7gh4e+AltNTRDlBfdqQEXFNBve5B 6nCA+cE15RNUKOmQL6Qb2hb0NZDH/k3IPUvYpcKNw94fH+T7Vgkzz7PMQ uwBgervURw1AoFsw5BjEtxyw+SXSTkOntdnqpOVAgxj6Ga0BajE1f+g0X w==; X-IronPort-AV: E=McAfee;i="6400,9594,10331"; a="327065229" X-IronPort-AV: E=Sophos;i="5.91,297,1647327600"; d="scan'208";a="327065229" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Apr 2022 01:06:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,297,1647327600"; d="scan'208";a="581977644" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga008.jf.intel.com with ESMTP; 29 Apr 2022 01:05:59 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Fri, 29 Apr 2022 15:59:38 +0800 Message-Id: <20220429075941.1844370-1-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 1/4] lavu: add sub frame side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: BbMN6+4yHSbG Sub frame side data allows attach another AVFrame as side data into the target AVFrame. Signed-off-by: Fei Wang --- libavutil/Makefile | 2 ++ libavutil/frame.c | 5 ++- libavutil/frame.h | 5 +++ libavutil/sub_frame_metadata.c | 61 ++++++++++++++++++++++++++++++++++ libavutil/sub_frame_metadata.h | 35 +++++++++++++++++++ libavutil/version.h | 4 +-- 6 files changed, 109 insertions(+), 3 deletions(-) create mode 100644 libavutil/sub_frame_metadata.c create mode 100644 libavutil/sub_frame_metadata.h diff --git a/libavutil/Makefile b/libavutil/Makefile index 81df3b0640..09333a0b48 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -88,6 +88,7 @@ HEADERS = adler32.h \ tea.h \ tx.h \ film_grain_params.h \ + sub_frame_metadata.h \ ARCH_HEADERS = bswap.h \ intmath.h \ @@ -176,6 +177,7 @@ OBJS = adler32.o \ tx_int32.o \ video_enc_params.o \ film_grain_params.o \ + sub_frame_metadata.o \ OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o diff --git a/libavutil/frame.c b/libavutil/frame.c index fbb869fffa..b21a15e652 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -315,7 +315,9 @@ static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy) if ( sd_src->type == AV_FRAME_DATA_PANSCAN && (src->width != dst->width || src->height != dst->height)) continue; - if (force_copy) { + /* Don't copy sub frame side data, otherwise sub frame's pointers in + * dst may be invalid. */ + if (force_copy && sd_src->type != AV_FRAME_DATA_SUB_FRAME) { sd_dst = av_frame_new_side_data(dst, sd_src->type, sd_src->size); if (!sd_dst) { @@ -815,6 +817,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_DETECTION_BBOXES: return "Bounding boxes for object detection and classification"; case AV_FRAME_DATA_DOVI_RPU_BUFFER: return "Dolby Vision RPU Data"; case AV_FRAME_DATA_DOVI_METADATA: return "Dolby Vision Metadata"; + case AV_FRAME_DATA_SUB_FRAME: return "Sub frame Metadata"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index 33fac2054c..9ae1286100 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -209,6 +209,11 @@ enum AVFrameSideDataType { * volume transform - CUVA 005.1-2021. */ AV_FRAME_DATA_DYNAMIC_HDR_VIVID, + + /** + * Sub frame of a target frame, as described by AVFrame. + */ + AV_FRAME_DATA_SUB_FRAME, }; enum AVActiveFormatDescription { diff --git a/libavutil/sub_frame_metadata.c b/libavutil/sub_frame_metadata.c new file mode 100644 index 0000000000..82ea32383f --- /dev/null +++ b/libavutil/sub_frame_metadata.c @@ -0,0 +1,61 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "sub_frame_metadata.h" + +static void sub_frame_free(void *opaque, uint8_t *data) +{ + AVFrame *frame = (AVFrame*)data; + + av_frame_free(&frame); +} + +static AVFrame *sub_frame_alloc(size_t *out_size) +{ + AVFrame *sub_frame = av_frame_alloc(); + if (!sub_frame) + return NULL; + + *out_size = sizeof(*sub_frame); + + return sub_frame; +} + +AVFrame *av_sub_frame_create_side_data(AVFrame *frame) +{ + AVBufferRef *buf; + AVFrame *sub_frame; + size_t size; + + sub_frame = sub_frame_alloc(&size); + if (!sub_frame) + return NULL; + + buf = av_buffer_create((uint8_t *)sub_frame, size, &sub_frame_free, NULL, 0); + if (!buf) { + av_frame_free(&sub_frame); + return NULL; + } + + if (!av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_SUB_FRAME, buf)) { + av_buffer_unref(&buf); + return NULL; + } + + return sub_frame; +} diff --git a/libavutil/sub_frame_metadata.h b/libavutil/sub_frame_metadata.h new file mode 100644 index 0000000000..621fb31e42 --- /dev/null +++ b/libavutil/sub_frame_metadata.h @@ -0,0 +1,35 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_SUB_FRAME_METADATA_H +#define AVUTIL_SUB_FRAME_METADATA_H + +#include "frame.h" + +/** + * Allocate a AVFrame structure and add it to the input frame as + * the side data. The allocated AVFrame will be freed automatically + * once the buf of created side data reference count decrease to zero. + * + * @param frame The frame which side data is added to. + * + * @return The AVFrame structure to be filled by caller. + */ +AVFrame *av_sub_frame_create_side_data(AVFrame *frame); + +#endif /* AVUTIL_SUB_FRAME_METADATA_H */ diff --git a/libavutil/version.h b/libavutil/version.h index 6735c20090..dd7d20a9fa 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,8 +79,8 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 24 -#define LIBAVUTIL_VERSION_MICRO 101 +#define LIBAVUTIL_VERSION_MINOR 25 +#define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ LIBAVUTIL_VERSION_MINOR, \