From patchwork Sat Mar 27 00:58:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suji Velupillai X-Patchwork-Id: 26624 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 9062F44AFC0 for ; Sat, 27 Mar 2021 02:59:41 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5D58F68A569; Sat, 27 Mar 2021 02:59:41 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ot1-f97.google.com (mail-ot1-f97.google.com [209.85.210.97]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8FCDD689CA7 for ; Sat, 27 Mar 2021 02:59:34 +0200 (EET) Received: by mail-ot1-f97.google.com with SMTP id h6-20020a0568300346b02901b71a850ab4so6904640ote.6 for ; Fri, 26 Mar 2021 17:59:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id; bh=scAYOLrQNuY4Bz5xE9r2TfYa3CAATNwQI/Ul5ItojtU=; b=QCRvzipJJc0CFy/chtu4gBbrbJsDWJW/cv7O+71u5uUR4V9erirWoMwpqKHnz/KJ0B l26kOGVfUb50+v+vm9wZahjVys4VCKQMV4nn0AQeVB8vW+MopSEBaiekRF96J2iLpgHE ax7Ex4WtYQAssBcJIajT/DAiSrouiirGIAnAs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=scAYOLrQNuY4Bz5xE9r2TfYa3CAATNwQI/Ul5ItojtU=; b=VtnXhsPNzpat6ITHxyFGaOpofMfjBROCFl2KeLzFAZVKDriTBMHA/lblALRRzoh/hS HiRUgsjoCWRvce3cvIZC3nshwrHRvROP6LDNmpZBco07MEeM8PUdt7GBBkv4ZUtwv40h MuKfxNYKeSXr4IrcsDh3gkrhHstuSyYgQQbNtFyNEByPGuQTE2Oy8bAxznzyCXj9/dZC 3bl5v2uEoALJtN3Vxs4LBTXN+684a1XxLfbEK+e/gWD+qTg/6Br2ByQJn6wyiafOCMY7 w6jXiBgUE9iFbdBid9EWMDlrF/Rx4uKi6Yrf12BDZwbUkzYpWMHEBdOrzEHpEV5fN59x 8OMA== X-Gm-Message-State: AOAM532qjdVOAPPmDA6auS+HnSAQqTxOVwWVRkN2hAuejTRVd4KTgaZf ZnMc0f9M/J2ZDw0Kqf/h50zdyopRnv83uuu7WWNklxrk3YmCj6+kHxAWrEZzmBOUTxF8XoaALKi vOku8u079p/e6d6z+gsTbKUdFX8pbmqPRuj8TKd6fg2J+zhh1NoEPmKnFOm5MZ7Pm9BTc34vuuR mXLyvBGKIc8qQCPkM1s0A= X-Google-Smtp-Source: ABdhPJxYJ4q5Pz/sv7Cw2kJMcDR7YTYXpQl6/vAWfonc1yIi+kF3eBG4e7YtX1fTmQ1ChNfR/drS+4g6OPyq X-Received: by 2002:a05:6830:1542:: with SMTP id l2mr12859616otp.95.1616806772777; Fri, 26 Mar 2021 17:59:32 -0700 (PDT) Received: from suji-Vostro-460.ric.broadcom.com ([192.19.228.250]) by smtp-relay.gmail.com with ESMTPS id y15sm5200667ooh.5.2021.03.26.17.59.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Mar 2021 17:59:32 -0700 (PDT) X-Relaying-Domain: broadcom.com From: suji.velupillai@broadcom.com To: ffmpeg-devel@ffmpeg.org Date: Fri, 26 Mar 2021 17:58:33 -0700 Message-Id: <20210327005833.23853-1-suji.velupillai@broadcom.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH v2] lavu: add BCMVK hwcontext implementation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: shamez.kurji@broadcom.com, Suji Velupillai , patrick.rault@broadcom.com MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Suji Velupillai Initial commit to add BCMVK hardware accelerator implementation. The depedency component vkil source code can be obtained from github https://github.com/Broadcom/vkil Signed-off-by: Suji Velupillai --- configure | 8 +- doc/APIchanges | 6 + libavutil/Makefile | 3 + libavutil/hwcontext.c | 4 + libavutil/hwcontext.h | 1 + libavutil/hwcontext_bcmvk.c | 427 +++++++++++++++++++++++++++++++++ libavutil/hwcontext_bcmvk.h | 65 +++++ libavutil/hwcontext_internal.h | 1 + libavutil/pixdesc.c | 4 + libavutil/pixfmt.h | 7 + 10 files changed, 525 insertions(+), 1 deletion(-) create mode 100644 libavutil/hwcontext_bcmvk.c create mode 100644 libavutil/hwcontext_bcmvk.h diff --git a/configure b/configure index d7a3f507e8..9d89d230ac 100755 --- a/configure +++ b/configure @@ -349,6 +349,7 @@ External library support: --disable-vaapi disable Video Acceleration API (mainly Unix/Intel) code [autodetect] --disable-vdpau disable Nvidia Video Decode and Presentation API for Unix code [autodetect] --disable-videotoolbox disable VideoToolbox code [autodetect] + --disable-bcmvk disable BCMVK code [autodetect] Toolchain options: --arch=ARCH select architecture [$arch] @@ -1848,6 +1849,7 @@ HWACCEL_AUTODETECT_LIBRARY_LIST=" videotoolbox v4l2_m2m xvmc + bcmvk " # catchall list of things that require external libs to link @@ -2923,6 +2925,7 @@ vaapi_x11_deps="xlib" videotoolbox_hwaccel_deps="videotoolbox pthreads" videotoolbox_hwaccel_extralibs="-framework QuartzCore" xvmc_deps="X11_extensions_XvMClib_h" +bcmvk_deps="libvkil" av1_d3d11va_hwaccel_deps="d3d11va DXVA_PicParams_AV1" av1_d3d11va_hwaccel_select="av1_decoder" @@ -3715,7 +3718,7 @@ avformat_deps="avcodec avutil" avformat_suggest="libm network zlib" avresample_deps="avutil" avresample_suggest="libm" -avutil_suggest="clock_gettime ffnvcodec libm libdrm libmfx opencl user32 vaapi vulkan videotoolbox corefoundation corevideo coremedia bcrypt" +avutil_suggest="clock_gettime ffnvcodec libm libdrm libmfx opencl user32 vaapi vulkan bcmvk videotoolbox corefoundation corevideo coremedia bcrypt" postproc_deps="avutil gpl" postproc_suggest="libm" swresample_deps="avutil" @@ -6743,6 +6746,9 @@ enabled vdpau && enabled vdpau && check_lib vdpau_x11 "vdpau/vdpau.h vdpau/vdpau_x11.h" vdp_device_create_x11 -lvdpau -lX11 +enabled bcmvk && + check_lib libvkil vkil_api.h vkil_create_api -lvkil + enabled crystalhd && check_lib crystalhd "stdint.h libcrystalhd/libcrystalhd_if.h" DtsCrystalHDVersion -lcrystalhd enabled vulkan && diff --git a/doc/APIchanges b/doc/APIchanges index b41dadee8d..e445644f33 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -15,6 +15,12 @@ libavutil: 2017-10-21 API changes, most recent first: +2021-03-26 - xxxxxxxxxx - lavu yy.yy.yyy - hwcontext.h + Add AV_PIX_FMT_BCMVK + Add AV_HWDEVICE_TYPE_BCMVK and implementation. + Add struct AVBCMVKDeviceContext + Add struct AVBCMVKFramesContext + 2021-03-21 - xxxxxxxxxx - lavu 56.72.100 - frame.h Deprecated av_get_colorspace_name(). Use av_color_space_name() instead. diff --git a/libavutil/Makefile b/libavutil/Makefile index 27bafe9e12..5acd187fe3 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -35,6 +35,7 @@ HEADERS = adler32.h \ hdr_dynamic_metadata.h \ hmac.h \ hwcontext.h \ + hwcontext_bcmvk.h \ hwcontext_cuda.h \ hwcontext_d3d11va.h \ hwcontext_drm.h \ @@ -174,6 +175,7 @@ OBJS = adler32.o \ film_grain_params.o \ +OBJS-$(CONFIG_BCMVK) += hwcontext_bcmvk.o OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o OBJS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.o OBJS-$(CONFIG_DXVA2) += hwcontext_dxva2.o @@ -192,6 +194,7 @@ OBJS += $(COMPAT_OBJS:%=../compat/%) # Windows resource file SLIBOBJS-$(HAVE_GNU_WINDRES) += avutilres.o +SKIPHEADERS-$(CONFIG_BCMVK) += hwcontext_bcmvk.h SKIPHEADERS-$(HAVE_CUDA_H) += hwcontext_cuda.h SKIPHEADERS-$(CONFIG_CUDA) += hwcontext_cuda_internal.h \ cuda_check.h diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c index d13d0f7c9b..4ba6f3e59d 100644 --- a/libavutil/hwcontext.c +++ b/libavutil/hwcontext.c @@ -61,6 +61,9 @@ static const HWContextType * const hw_table[] = { #endif #if CONFIG_VULKAN &ff_hwcontext_type_vulkan, +#endif +#if CONFIG_BCMVK + &ff_hwcontext_type_bcmvk, #endif NULL, }; @@ -77,6 +80,7 @@ static const char *const hw_type_names[] = { [AV_HWDEVICE_TYPE_VIDEOTOOLBOX] = "videotoolbox", [AV_HWDEVICE_TYPE_MEDIACODEC] = "mediacodec", [AV_HWDEVICE_TYPE_VULKAN] = "vulkan", + [AV_HWDEVICE_TYPE_BCMVK] = "bcmvk", }; enum AVHWDeviceType av_hwdevice_find_type_by_name(const char *name) diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h index 04d19d89c2..b195a10b02 100644 --- a/libavutil/hwcontext.h +++ b/libavutil/hwcontext.h @@ -37,6 +37,7 @@ enum AVHWDeviceType { AV_HWDEVICE_TYPE_OPENCL, AV_HWDEVICE_TYPE_MEDIACODEC, AV_HWDEVICE_TYPE_VULKAN, + AV_HWDEVICE_TYPE_BCMVK, }; typedef struct AVHWDeviceInternal AVHWDeviceInternal; diff --git a/libavutil/hwcontext_bcmvk.c b/libavutil/hwcontext_bcmvk.c new file mode 100644 index 0000000000..49917b7a7c --- /dev/null +++ b/libavutil/hwcontext_bcmvk.c @@ -0,0 +1,427 @@ +/* + * Copyright (c) 2018 Broadcom + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "avassert.h" +#include "buffer.h" +#include "buffer_internal.h" +#include "common.h" +#include "hwcontext.h" +#include "hwcontext_bcmvk.h" +#include "hwcontext_internal.h" +#include "imgutils.h" +#include "mem.h" +#include "pixdesc.h" +#include "pixfmt.h" + +#define BCMVK_FRAME_ALIGNMENT 256 + +static const enum AVPixelFormat supported_formats[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_NV21, + AV_PIX_FMT_P010, + AV_PIX_FMT_P016, + AV_PIX_FMT_BCMVK, + AV_PIX_FMT_NONE, +}; + +static int bcmvk_fmt_is_in(int fmt, const int *fmts) +{ + const int *p; + + for (p = fmts; *p != -1; p++) { + if (fmt == *p) + return 1; + } + + return 0; +} + +const bcmvk_format_type av_bcmvk_fmt_from_pixfmt(enum AVPixelFormat pixfmt) +{ + switch (pixfmt) { + case AV_PIX_FMT_NV12: + return BCMVK_FORMAT_NV12; + case AV_PIX_FMT_NV21: + return BCMVK_FORMAT_NV21; + case AV_PIX_FMT_P010: + return BCMVK_FORMAT_P010; + case AV_PIX_FMT_BCMVK: + return BCMVK_FORMAT_YOL2; + default: + return BCMVK_FORMAT_UNDEF; + } +} + +static int bcmvk_get_ptr2surface(const AVFrame *frame, void *psurface) +{ + void **surface = psurface; + + if (frame->format != AV_PIX_FMT_BCMVK) + return AVERROR_BUG; + + *surface = frame->data[AVBCMVK_VKIL_BUF_DESC_IDX]; + + if (!(*surface)) + return AVERROR_BUG; + + return 0; +} + +static void bcmvk_pool_release(void *opaque, uint8_t *data) +{ + av_free(data); +} + +static AVBufferRef *bcmvk_pool_alloc(void *opaque, int size) +{ + AVHWFramesContext *ctx = opaque; + AVBufferRef *ret = NULL; + vkil_buffer_surface *surface; + + surface = av_mallocz(sizeof(vkil_buffer_surface)); + if (!surface) { + av_log(ctx, AV_LOG_ERROR, "av_mallocz failed\n"); + goto out; + } + + ret = av_buffer_create((uint8_t *)surface, + sizeof(*surface), bcmvk_pool_release, + NULL, AV_BUFFER_FLAG_READONLY); + if (!ret) { + av_log(ctx, AV_LOG_ERROR, "av_buffer_create failed\n"); + av_free(surface); + } + +out: + return ret; +} + +static int bcmvk_frames_init(AVHWFramesContext *ctx) +{ + int aligned_width = FFALIGN(ctx->width, BCMVK_FRAME_ALIGNMENT); + int ret = AVERROR(EINVAL); + int valid_format = bcmvk_fmt_is_in(ctx->sw_format, supported_formats); + + if (!valid_format) + goto fail; + + if (!ctx->pool) { + int size; + + switch (ctx->sw_format) { + + case AV_PIX_FMT_BCMVK: + size = sizeof(void *); + break; + case AV_PIX_FMT_NV12: + case AV_PIX_FMT_NV21: + size = (aligned_width * ctx->height * 3) >> 1; + break; + case AV_PIX_FMT_P010: + case AV_PIX_FMT_P016: + size = aligned_width * ctx->height * 3; + break; + default: + goto fail; + } + + ctx->internal->pool_internal = av_buffer_pool_init2(size, ctx, bcmvk_pool_alloc, NULL); + if (!ctx->internal->pool_internal) { + ret = AVERROR(ENOMEM); + goto fail; + } + } + + return 0; + +fail: + av_log(ctx, AV_LOG_ERROR, "bcmvk_frames_init failed on error %d\n", ret); + return ret; + +} + +static int bcmvk_get_buffer(AVHWFramesContext *ctx, AVFrame *frame) +{ + int ret = 0; + + frame->buf[0] = av_buffer_pool_get(ctx->pool); + if (!frame->buf[0]) { + av_log(ctx, AV_LOG_ERROR, "av_buffer_pool_get failed\n"); + ret = AVERROR(ENOMEM); + } else { + // vkil data are stored in frame->data[AVBCMVK_VKIL_BUF_DESC_IDX] + frame->data[AVBCMVK_VKIL_BUF_DESC_IDX] = frame->buf[0]->data; + frame->format = AV_PIX_FMT_BCMVK; + frame->width = ctx->width; + frame->height = ctx->height; + } + + return ret; +} + +static int bcmvk_transfer_get_formats(AVHWFramesContext *ctx, + enum AVHWFrameTransferDirection dir, + enum AVPixelFormat **formats) +{ + enum AVPixelFormat *fmts; + int ret = 0; + + // this allocation freed in hwcontext::transfer_data_alloc + fmts = av_malloc_array(2, sizeof(*fmts)); + if (!fmts) { + av_log(ctx, AV_LOG_ERROR, "bcmvk_transfer_get_formats failed\n"); + ret = AVERROR(ENOMEM); + } else { + fmts[0] = ctx->sw_format; + fmts[1] = AV_PIX_FMT_NONE; + *formats = fmts; + } + + return ret; +} + +static int bcmvk_convert_av2surface(const AVFrame *src, + vkil_buffer_surface *dst) +{ + int i; + + av_assert0(src->interlaced_frame == 0); + + dst->max_size.width = src->width; + dst->max_size.height = src->height; + dst->format = av_bcmvk_fmt_from_pixfmt(src->format); + dst->quality = src->quality; + dst->prefix.type = VKIL_BUF_SURFACE; + dst->prefix.port_id = 0; // set to default (first port) + dst->prefix.flags = 0; // other fields set to zero (default value) + + for (i = 0; i < VKIL_BUF_NPLANES; i++) { + dst->plane_top[i] = src->data[i]; + dst->plane_bot[i] = src->data[i + VKIL_BUF_NPLANES]; + dst->stride[i] = src->linesize[i]; + // sanity check: vk structure requires 32 bits alignment + if (((uintptr_t)dst->plane_top[i] & (VKIL_BUF_ALIGN -1)) || + ((uintptr_t)dst->plane_bot[i] & (VKIL_BUF_ALIGN -1)) || + (dst->stride[i] &(VKIL_BUF_ALIGN -1))) + return AVERROR(EINVAL); + } + + return 0; +} + +static int bcmvk_transfer_data_from(AVHWFramesContext *ctx, AVFrame *dst, + const AVFrame *src) +{ + int ret; + int32_t size; + AVBCMVKFramesContext *bcmvk_fc; + AVHWFramesContext *avhw_fc; + AVBCMVKDeviceContext *bcmvk_dc = ctx->device_ctx->hwctx; + vkil_buffer_surface *surface; + vkil_buffer metadata; + + av_assert0(src->hw_frames_ctx->data); + avhw_fc = (AVHWFramesContext *)src->hw_frames_ctx->data; + + av_assert0(avhw_fc->hwctx); + bcmvk_fc = avhw_fc->hwctx; + ret = bcmvk_get_ptr2surface(src, &surface); + if (ret < 0) + goto fail; + + ret = bcmvk_convert_av2surface(dst, surface); + if (ret < 0) + goto fail; + + // blocking call as ffmpeg assume the transfer is complete on return + ret = bcmvk_dc->ilapi->transfer_buffer2(bcmvk_fc->ilctx, surface, + VK_CMD_DOWNLOAD | VK_CMD_OPT_BLOCKING, + &size); + if (ret < 0) { + ret = AVERROR_EXTERNAL; + goto fail; + } + + if (src->data[AVBCMVK_METADATA_HNDL_IDX]) { + // Dereference if any transferred frame that has an associated metadata buffer + // no need to transfer it back to host + metadata.handle = (uint32_t)src->data[AVBCMVK_METADATA_HNDL_IDX]; + metadata.type = VKIL_BUF_META_DATA; + metadata.ref = 1; + ret = bcmvk_dc->ilapi->xref_buffer(bcmvk_fc->ilctx, &metadata, -1, + VK_CMD_RUN | VK_CMD_OPT_BLOCKING); + if (ret) { + ret = AVERROR_EXTERNAL; + goto fail; + } + } + + return 0; + +fail: + av_log(ctx, AV_LOG_ERROR, "failure %d on bcmvk_transfer_data_to\n", ret); + return ret; +} + +static int bcmvk_transfer_data_to(AVHWFramesContext *ctx, AVFrame *dst, + const AVFrame *src) +{ + int ret; + int32_t i, size = 0; + AVBCMVKFramesContext *bcmvk_fc; + AVHWFramesContext *avhw_fc; + AVBCMVKDeviceContext *bcmvk_dc = ctx->device_ctx->hwctx; + vkil_buffer_surface *surface; + vkil_context *ilctx; + uint8_t *tmp_data[4] = {NULL, NULL, NULL, NULL}; + int linesize[4]; + + av_assert0(VKIL_BUF_NPLANES * 2 <= 4); + + av_assert0(dst->hw_frames_ctx->data); + avhw_fc = (AVHWFramesContext *)dst->hw_frames_ctx->data; + + av_assert0(avhw_fc->hwctx); + bcmvk_fc = avhw_fc->hwctx; + + ret = bcmvk_get_ptr2surface(dst, &surface); + if (ret < 0) + goto fail; + + ret = bcmvk_convert_av2surface(src, surface); + if (ret) { + ret = av_image_alloc(tmp_data, linesize, src->width, src->height, + src->format, VKIL_BUF_ALIGN); + if (ret < 0) + goto fail; + + av_image_copy(tmp_data, linesize, (const uint8_t **)src->data, + src->linesize, src->format, src->width, src->height); + + for (i = 0; i < VKIL_BUF_NPLANES; i++) { + surface->plane_top[i]= tmp_data[i]; + surface->plane_bot[i]= tmp_data[VKIL_BUF_NPLANES + i]; + surface->stride[i] = linesize[i]; + } + } + + surface->quality = dst->quality; + + ilctx = bcmvk_fc->ilctx; + if (!ilctx) { + ret = AVERROR(EINVAL); + goto fail; + } + + // blocking call as ffmpeg assumes the transfer is complete on return + ret = bcmvk_dc->ilapi->transfer_buffer2(ilctx, surface, + VK_CMD_UPLOAD | VK_CMD_OPT_BLOCKING, + &size); + if (ret < 0) { + ret = AVERROR_EXTERNAL; + goto fail; + } + + bcmvk_fc->handle = surface->prefix.handle; + + if (tmp_data[0]) + av_free(tmp_data[0]); + + return 0; + +fail: + if (tmp_data[0]) + av_free(tmp_data[0]); + + av_log(ctx, AV_LOG_ERROR, "failure %d on bcmvk_transfer_data_to\n", ret); + return ret; +} + +static void bcmvk_device_uninit(AVHWDeviceContext *ctx) +{ + AVBCMVKDeviceContext *hwctx; + + av_assert0(ctx); + av_assert0(ctx->hwctx); + + hwctx = ctx->hwctx; + + if (hwctx->ilapi) + vkil_destroy_api((void **)&hwctx->ilapi); +} + +static int bcmvk_device_init(AVHWDeviceContext *ctx) +{ + AVBCMVKDeviceContext *hwctx; + int ret = AVERROR_EXTERNAL; + + av_assert0(ctx); + av_assert0(ctx->hwctx); + + hwctx = ctx->hwctx; + if (!(hwctx->ilapi = vkil_create_api())) { + av_log(ctx, AV_LOG_ERROR, "ctx->ilapi failed to be created\n"); + goto out; + } + + if (!hwctx->ilapi->init) { + av_log(ctx, AV_LOG_ERROR, "ctx->ilapi not properly initialized\n"); + goto out; + } + + ret = 0; + +out: + if (ret) + bcmvk_device_uninit(ctx); + + return ret; +} + +static int bcmvk_device_create(AVHWDeviceContext *ctx, const char *device, + AVDictionary *opts, int flags) +{ + /* nothing to do at this time */ + return 0; +} + +const HWContextType ff_hwcontext_type_bcmvk = { + .type = AV_HWDEVICE_TYPE_BCMVK, + .name = "BCMVK", + + .device_hwctx_size = sizeof(AVBCMVKDeviceContext), + .frames_hwctx_size = sizeof(AVBCMVKFramesContext), + + .device_create = bcmvk_device_create, + .device_init = bcmvk_device_init, + .device_uninit = bcmvk_device_uninit, + + .frames_init = bcmvk_frames_init, + .frames_get_buffer = bcmvk_get_buffer, + + .transfer_get_formats = bcmvk_transfer_get_formats, + .transfer_data_to = bcmvk_transfer_data_to, + .transfer_data_from = bcmvk_transfer_data_from, + + .pix_fmts = (const enum AVPixelFormat[]) { + AV_PIX_FMT_BCMVK, + AV_PIX_FMT_NONE + }, +}; diff --git a/libavutil/hwcontext_bcmvk.h b/libavutil/hwcontext_bcmvk.h new file mode 100644 index 0000000000..44b0363c59 --- /dev/null +++ b/libavutil/hwcontext_bcmvk.h @@ -0,0 +1,65 @@ +/* + * Copyright (c) 2018 Broadcom + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_HWCONTEXT_BCMVK_H +#define AVUTIL_HWCONTEXT_BCMVK_H + +#include + +/** + * @file + * API-specific header for AV_HWDEVICE_TYPE_BCMVK. + */ + +#define AVBCMVK_VKIL_BUF_DESC_IDX 0 +#define AVBCMVK_METADATA_HNDL_IDX 1 + +/** + * Allocated as AVHWDeviceContext.hwctx + */ +typedef struct AVBCMVKDeviceContext { + /** + * Holds pointers to hardware specific functions + */ + vkil_api *ilapi; + +} AVBCMVKDeviceContext; + +/** + * Allocated as AVHWFramesContext.hwctx + */ +typedef struct AVBCMVKFramesContext { + /** + * Handle to a hardware frame context + */ + uint32_t handle; + /** + * ilcontext associated to the frame context + */ + vkil_context *ilctx; +} AVBCMVKFramesContext; + +/** + * Convert AVPixelFormat to BCMVK equivalent pixel format + * Return BCMVK_FORMAT_UNDEF on unsupported formats + */ +const bcmvk_format_type av_bcmvk_fmt_from_pixfmt(enum AVPixelFormat pixfmt); + +#endif /* AVUTIL_HWCONTEXT_BCMVK_H */ diff --git a/libavutil/hwcontext_internal.h b/libavutil/hwcontext_internal.h index e6266494ac..657562f90a 100644 --- a/libavutil/hwcontext_internal.h +++ b/libavutil/hwcontext_internal.h @@ -174,5 +174,6 @@ extern const HWContextType ff_hwcontext_type_vdpau; extern const HWContextType ff_hwcontext_type_videotoolbox; extern const HWContextType ff_hwcontext_type_mediacodec; extern const HWContextType ff_hwcontext_type_vulkan; +extern const HWContextType ff_hwcontext_type_bcmvk; #endif /* AVUTIL_HWCONTEXT_INTERNAL_H */ diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index 2a919461a5..536552b9d6 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2395,6 +2395,10 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { .name = "vulkan", .flags = AV_PIX_FMT_FLAG_HWACCEL, }, + [AV_PIX_FMT_BCMVK] = { + .name = "bcmvk", + .flags = AV_PIX_FMT_FLAG_HWACCEL, + }, }; #if FF_API_PLUS1_MINUS1 FF_ENABLE_DEPRECATION_WARNINGS diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index 46ef211add..4e82f52890 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -360,6 +360,13 @@ enum AVPixelFormat { AV_PIX_FMT_X2RGB10LE, ///< packed RGB 10:10:10, 30bpp, (msb)2X 10R 10G 10B(lsb), little-endian, X=unused/undefined AV_PIX_FMT_X2RGB10BE, ///< packed RGB 10:10:10, 30bpp, (msb)2X 10R 10G 10B(lsb), big-endian, X=unused/undefined + + /** + * BCMPI hardware acceleration. + * data[0] contains a pointer to the vkil_buffer_surface structure + * data[1] contains handle to meta data + */ + AV_PIX_FMT_BCMVK, AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions };