Message ID | 5f1ebc54-9fe5-a129-7f16-272d13133887@jkqxz.net |
---|---|
State | Superseded |
Headers | show |
On Thu, 6 Jul 2017 00:02:08 +0100 Mark Thompson <sw@jkqxz.net> wrote: > --- > Updated following discussion: > * Back to using nested arrays. > * Documentation improved. > * Configure option now called libdrm. > * Other minor fixups. > > > configure | 3 + > libavutil/Makefile | 2 + > libavutil/hwcontext.c | 4 + > libavutil/hwcontext.h | 1 + > libavutil/hwcontext_drm.c | 294 +++++++++++++++++++++++++++++++++++++++++ > libavutil/hwcontext_drm.h | 166 +++++++++++++++++++++++ > libavutil/hwcontext_internal.h | 1 + > libavutil/pixdesc.c | 4 + > libavutil/pixfmt.h | 4 + > 9 files changed, 479 insertions(+) > create mode 100644 libavutil/hwcontext_drm.c > create mode 100644 libavutil/hwcontext_drm.h Generally LGTM, though I don't know when LongChair can try it. > +#include <fcntl.h> > +#include <sys/mman.h> > +#if HAVE_UNISTD_H > +# include <unistd.h> > +#endif Is there really going to be a system that has fcntl.h and sys/mman.h, but not unistd.h? This code is going to work on POSIX only anyway, which requires this header. > +typedef struct AVDRMObjectDescriptor { > + /** > + * DRM PRIME fd for the object. > + */ > + int fd; > + /** > + * Total size of the object. > + * > + * (This includes any parts not which do not contain image data.) > + */ > + size_t size; > + /** > + * Format modifier applied to the object (DRM_FORMAT_MOD_*). > + */ > + uint64_t format_modifier; > +} AVDRMObjectDescriptor; So I guess this thing really is worth the trouble, but I still don't entirely understand it. Why is the size needed, and what do the format_modifiers do?
On 06/07/17 09:00, wm4 wrote: > On Thu, 6 Jul 2017 00:02:08 +0100 > Mark Thompson <sw@jkqxz.net> wrote: > >> --- >> Updated following discussion: >> * Back to using nested arrays. >> * Documentation improved. >> * Configure option now called libdrm. >> * Other minor fixups. >> >> >> configure | 3 + >> libavutil/Makefile | 2 + >> libavutil/hwcontext.c | 4 + >> libavutil/hwcontext.h | 1 + >> libavutil/hwcontext_drm.c | 294 +++++++++++++++++++++++++++++++++++++++++ >> libavutil/hwcontext_drm.h | 166 +++++++++++++++++++++++ >> libavutil/hwcontext_internal.h | 1 + >> libavutil/pixdesc.c | 4 + >> libavutil/pixfmt.h | 4 + >> 9 files changed, 479 insertions(+) >> create mode 100644 libavutil/hwcontext_drm.c >> create mode 100644 libavutil/hwcontext_drm.h > > Generally LGTM, though I don't know when LongChair can try it. I won't apply it without that implementation showing it being used. >> +#include <fcntl.h> >> +#include <sys/mman.h> >> +#if HAVE_UNISTD_H >> +# include <unistd.h> >> +#endif > > Is there really going to be a system that has fcntl.h and sys/mman.h, > but not unistd.h? This code is going to work on POSIX only anyway, > which requires this header. This was just copied from other uses. Agree; removed. >> +typedef struct AVDRMObjectDescriptor { >> + /** >> + * DRM PRIME fd for the object. >> + */ >> + int fd; >> + /** >> + * Total size of the object. >> + * >> + * (This includes any parts not which do not contain image data.) >> + */ >> + size_t size; >> + /** >> + * Format modifier applied to the object (DRM_FORMAT_MOD_*). >> + */ >> + uint64_t format_modifier; >> +} AVDRMObjectDescriptor; > > So I guess this thing really is worth the trouble, but I still don't > entirely understand it. Why is the size needed, and what do the > format_modifiers do? Size is needed for some import APIs: <https://cgit.freedesktop.org/beignet/tree/include/CL/cl_intel.h#n141> <https://github.com/01org/libva/blob/master/va/va.h#L920> Having it also makes the generic mapping sensible - without it, we would need to map each plane individually, which would require knowledge of what all of the formats mean to know how big they are. Format modifiers are a relatively new mechanism to communicate tiling modes and other related metadata in a way common to all drivers (with current single-driver setups this is already done in some cases by magic internal properties). EGL import supports them: <https://www.khronos.org/registry/EGL/extensions/EXT/EGL_EXT_image_dma_buf_import_modifiers.txt> (Ignore the fact that modifiers there are per-plane - the extension was codified before the decision that mixing modifiers within the same object was madness and that feature completely removed: <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bae781b259269590109e8a4a8227331362b88212>.)
On 2017/7/6 7:02, Mark Thompson wrote: > --- > Updated following discussion: > * Back to using nested arrays. > * Documentation improved. > * Configure option now called libdrm. > * Other minor fixups. > > > configure | 3 + > libavutil/Makefile | 2 + > libavutil/hwcontext.c | 4 + > libavutil/hwcontext.h | 1 + > libavutil/hwcontext_drm.c | 294 +++++++++++++++++++++++++++++++++++++++++ > libavutil/hwcontext_drm.h | 166 +++++++++++++++++++++++ > libavutil/hwcontext_internal.h | 1 + > libavutil/pixdesc.c | 4 + > libavutil/pixfmt.h | 4 + > 9 files changed, 479 insertions(+) > create mode 100644 libavutil/hwcontext_drm.c > create mode 100644 libavutil/hwcontext_drm.h > > diff --git a/configure b/configure > index 9509bdf09c..3225c3cca7 100755 > --- a/configure > +++ b/configure > @@ -298,6 +298,7 @@ External library support: > --disable-cuvid disable Nvidia CUVID support [autodetect] > --disable-d3d11va disable Microsoft Direct3D 11 video acceleration code [autodetect] > --disable-dxva2 disable Microsoft DirectX 9 video acceleration code [autodetect] > + --enable-libdrm enable DRM code (Linux) [no] > --enable-libmfx enable Intel MediaSDK (AKA Quick Sync Video) code via libmfx [no] > --enable-libnpp enable Nvidia Performance Primitives-based code [no] > --enable-mmal enable Broadcom Multi-Media Abstraction Layer (Raspberry Pi) via MMAL [no] > @@ -1539,6 +1540,7 @@ EXTERNAL_LIBRARY_LIST=" > libcaca > libcelt > libdc1394 > + libdrm > libflite > libfontconfig > libfreetype > @@ -5797,6 +5799,7 @@ enabled libcelt && require libcelt celt/celt.h celt_decode -lcelt0 && > die "ERROR: libcelt must be installed and version must be >= 0.11.0."; } > enabled libcaca && require_pkg_config caca caca.h caca_create_canvas > enabled libdc1394 && require_pkg_config libdc1394-2 dc1394/dc1394.h dc1394_new > +enabled libdrm && require_pkg_config libdrm xf86drm.h drmGetVersion > enabled libfdk_aac && { use_pkg_config fdk-aac "fdk-aac/aacenc_lib.h" aacEncOpen || > { require libfdk_aac fdk-aac/aacenc_lib.h aacEncOpen -lfdk-aac && > warn "using libfdk without pkg-config"; } } > diff --git a/libavutil/Makefile b/libavutil/Makefile > index 4fe81fdd07..991b2f3fbd 100644 > --- a/libavutil/Makefile > +++ b/libavutil/Makefile > @@ -33,6 +33,7 @@ HEADERS = adler32.h \ > hmac.h \ > hwcontext.h \ > hwcontext_cuda.h \ > + hwcontext_drm.h \ > hwcontext_dxva2.h \ > hwcontext_qsv.h \ > hwcontext_vaapi.h \ > @@ -158,6 +159,7 @@ OBJS-$(!HAVE_ATOMICS_NATIVE) += atomic.o \ > OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o > OBJS-$(CONFIG_DXVA2) += hwcontext_dxva2.o > OBJS-$(CONFIG_QSV) += hwcontext_qsv.o > +OBJS-$(CONFIG_LIBDRM) += hwcontext_drm.o > OBJS-$(CONFIG_LZO) += lzo.o > OBJS-$(CONFIG_OPENCL) += opencl.o opencl_internal.o > OBJS-$(CONFIG_VAAPI) += hwcontext_vaapi.o > diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c > index 5a22194716..e40ceb12a1 100644 > --- a/libavutil/hwcontext.c > +++ b/libavutil/hwcontext.c > @@ -32,6 +32,9 @@ static const HWContextType *const hw_table[] = { > #if CONFIG_CUDA > &ff_hwcontext_type_cuda, > #endif > +#if CONFIG_LIBDRM > + &ff_hwcontext_type_drm, > +#endif > #if CONFIG_DXVA2 > &ff_hwcontext_type_dxva2, > #endif > @@ -52,6 +55,7 @@ static const HWContextType *const hw_table[] = { > > static const char *const hw_type_names[] = { > [AV_HWDEVICE_TYPE_CUDA] = "cuda", > + [AV_HWDEVICE_TYPE_DRM] = "drm", > [AV_HWDEVICE_TYPE_DXVA2] = "dxva2", > [AV_HWDEVICE_TYPE_QSV] = "qsv", > [AV_HWDEVICE_TYPE_VAAPI] = "vaapi", > diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h > index edf12cc631..fe7613b379 100644 > --- a/libavutil/hwcontext.h > +++ b/libavutil/hwcontext.h > @@ -32,6 +32,7 @@ enum AVHWDeviceType { > AV_HWDEVICE_TYPE_QSV, > AV_HWDEVICE_TYPE_VIDEOTOOLBOX, > AV_HWDEVICE_TYPE_NONE, > + AV_HWDEVICE_TYPE_DRM, > }; AV_HWDEVICE_TYPE_DRM = AV_HWDEVICE_TYPE_NONE + 1 ? > > typedef struct AVHWDeviceInternal AVHWDeviceInternal; > diff --git a/libavutil/hwcontext_drm.c b/libavutil/hwcontext_drm.c > new file mode 100644 > index 0000000000..c6be3295f9 > --- /dev/null > +++ b/libavutil/hwcontext_drm.c > @@ -0,0 +1,294 @@ > +/* > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include "config.h" > + > +#include <fcntl.h> > +#include <sys/mman.h> > +#if HAVE_UNISTD_H > +# include <unistd.h> > +#endif > + > +#include <drm.h> > +#include <xf86drm.h> > + > +#include "avassert.h" > +#include "hwcontext.h" > +#include "hwcontext_drm.h" > +#include "hwcontext_internal.h" > +#include "imgutils.h" > + > + > +static void drm_device_free(AVHWDeviceContext *hwdev) > +{ > + AVDRMDeviceContext *hwctx = hwdev->hwctx; > + > + close(hwctx->fd); > +} > + > +static int drm_device_create(AVHWDeviceContext *hwdev, const char *device, > + AVDictionary *opts, int flags) > +{ > + AVDRMDeviceContext *hwctx = hwdev->hwctx; > + drmVersionPtr version; > + > + hwctx->fd = open(device, O_RDWR); > + if (hwctx->fd < 0) > + return AVERROR_UNKNOWN; > + > + version = drmGetVersion(hwctx->fd); > + if (!version) { > + av_log(hwdev, AV_LOG_ERROR, "Failed to DRM version information " > + "from %s: probably not a DRM device?\n", device); > + close(hwctx->fd); > + return AVERROR(EINVAL); > + } > + > + av_log(hwdev, AV_LOG_VERBOSE, "Opened DRM device %s: driver %s " > + "version %d.%d.%d.\n", device, version->name, > + version->version_major, version->version_minor, > + version->version_patchlevel); > + > + drmFreeVersion(version); > + > + hwdev->free = &drm_device_free; > + > + return 0; > +} > + > +static int drm_get_buffer(AVHWFramesContext *hwfc, AVFrame *frame) > +{ > + frame->buf[0] = av_buffer_pool_get(hwfc->pool); > + if (!frame->buf[0]) > + return AVERROR(ENOMEM); > + > + frame->data[0] = (uint8_t*)frame->buf[0]->data; > + > + frame->format = AV_PIX_FMT_DRM_PRIME; > + frame->width = hwfc->width; > + frame->height = hwfc->height; > + > + return 0; > +} > + > +typedef struct DRMMapping { > + // Address and length of each mmap()ed region. > + int nb_regions; > + void *address[AV_DRM_MAX_PLANES]; > + size_t length[AV_DRM_MAX_PLANES]; > +} DRMMapping; > + > +static void drm_unmap_frame(AVHWFramesContext *hwfc, > + HWMapDescriptor *hwmap) > +{ > + DRMMapping *map = hwmap->priv; > + int i; > + > + for (i = 0; i < map->nb_regions; i++) { > + if (map->address[i]) > + munmap(map->address[i], map->length[i]); > + } > + > + av_free(map); > +} > + > +static int drm_map_frame(AVHWFramesContext *hwfc, > + AVFrame *dst, const AVFrame *src, int flags) > +{ > + const AVDRMFrameDescriptor *desc = (AVDRMFrameDescriptor*)src->data[0]; > + DRMMapping *map; > + int err, i, p, plane; > + int mmap_prot; > + void *addr; > + > + map = av_mallocz(sizeof(*map)); > + if (!map) > + return AVERROR(ENOMEM); > + > + mmap_prot = 0; > + if (flags & AV_HWFRAME_MAP_READ) > + mmap_prot |= PROT_READ; > + if (flags & AV_HWFRAME_MAP_WRITE) > + mmap_prot |= PROT_WRITE; > + > + av_assert0(desc->nb_objects <= AV_DRM_MAX_PLANES); > + for (i = 0; i < desc->nb_objects; i++) { > + addr = mmap(NULL, desc->objects[i].size, mmap_prot, MAP_SHARED, > + desc->objects[i].fd, 0); > + if (addr == MAP_FAILED) { > + err = AVERROR(errno); > + av_log(hwfc, AV_LOG_ERROR, "Failed to map DRM object %d to " > + "memory: %d.\n", desc->objects[i].fd, errno); > + goto fail; > + } > + > + map->address[i] = addr; > + map->length[i] = desc->objects[i].size; > + } > + map->nb_regions = i; > + > + plane = 0; > + for (i = 0; i < desc->nb_layers; i++) { > + const AVDRMLayerDescriptor *layer = &desc->layers[i]; > + for (p = 0; p < layer->nb_planes; p++) { > + dst->data[plane] = > + (uint8_t*)map->address[layer->planes[p].object_index] + > + layer->planes[p].offset; > + dst->linesize[plane] = layer->planes[p].pitch; > + ++plane; > + } > + } > + av_assert0(plane <= AV_DRM_MAX_PLANES); > + > + dst->width = src->width; > + dst->height = src->height; > + > + err = ff_hwframe_map_create(src->hw_frames_ctx, dst, src, > + &drm_unmap_frame, map); > + if (err < 0) > + return err; > + > + return 0; > + > +fail: > + for (i = 0; i < desc->nb_objects; i++) { > + if (map->address[i]) > + munmap(map->address[i], map->length[i]); > + } > + av_free(map); > + return err; > +} > + > +static int drm_transfer_get_formats(AVHWFramesContext *ctx, > + enum AVHWFrameTransferDirection dir, > + enum AVPixelFormat **formats) > +{ > + enum AVPixelFormat *pix_fmts; > + > + pix_fmts = av_malloc_array(2, sizeof(*pix_fmts)); > + if (!pix_fmts) > + return AVERROR(ENOMEM); > + > + pix_fmts[0] = ctx->sw_format; > + pix_fmts[1] = AV_PIX_FMT_NONE; > + > + *formats = pix_fmts; > + return 0; > +} > + > +static int drm_transfer_data_from(AVHWFramesContext *hwfc, > + AVFrame *dst, const AVFrame *src) > +{ > + AVFrame *map; > + int err; > + > + if (dst->width > hwfc->width || dst->height > hwfc->height) > + return AVERROR(EINVAL); > + > + map = av_frame_alloc(); > + if (!map) > + return AVERROR(ENOMEM); > + map->format = dst->format; > + > + err = drm_map_frame(hwfc, map, src, AV_HWFRAME_MAP_READ); > + if (err) > + goto fail; > + > + map->width = dst->width; > + map->height = dst->height; > + > + err = av_frame_copy(dst, map); > + if (err) > + goto fail; > + > + err = 0; > +fail: > + av_frame_free(&map); > + return err; > +} > + > +static int drm_transfer_data_to(AVHWFramesContext *hwfc, > + AVFrame *dst, const AVFrame *src) > +{ > + AVFrame *map; > + int err; > + > + if (src->width > hwfc->width || src->height > hwfc->height) > + return AVERROR(EINVAL); > + > + map = av_frame_alloc(); > + if (!map) > + return AVERROR(ENOMEM); > + map->format = src->format; > + > + err = drm_map_frame(hwfc, map, dst, AV_HWFRAME_MAP_WRITE | AV_HWFRAME_MAP_OVERWRITE); > + if (err) > + goto fail; > + > + map->width = src->width; > + map->height = src->height; > + > + err = av_frame_copy(map, src); > + if (err) > + goto fail; > + > + err = 0; > +fail: > + av_frame_free(&map); > + return err; > +} > + > +static int drm_map_from(AVHWFramesContext *hwfc, AVFrame *dst, > + const AVFrame *src, int flags) > +{ > + int err; > + > + if (hwfc->sw_format != dst->format) > + return AVERROR(ENOSYS); > + > + err = drm_map_frame(hwfc, dst, src, flags); > + if (err) > + return err; > + > + err = av_frame_copy_props(dst, src); > + if (err) > + return err; > + > + return 0; > +} > + > +const HWContextType ff_hwcontext_type_drm = { > + .type = AV_HWDEVICE_TYPE_DRM, > + .name = "DRM", > + > + .device_hwctx_size = sizeof(AVDRMDeviceContext), > + > + .device_create = &drm_device_create, > + > + .frames_get_buffer = &drm_get_buffer, > + > + .transfer_get_formats = &drm_transfer_get_formats, > + .transfer_data_to = &drm_transfer_data_to, > + .transfer_data_from = &drm_transfer_data_from, > + .map_from = &drm_map_from, > + > + .pix_fmts = (const enum AVPixelFormat[]) { > + AV_PIX_FMT_DRM_PRIME, > + AV_PIX_FMT_NONE > + }, > +}; > diff --git a/libavutil/hwcontext_drm.h b/libavutil/hwcontext_drm.h > new file mode 100644 > index 0000000000..2e225451e1 > --- /dev/null > +++ b/libavutil/hwcontext_drm.h > @@ -0,0 +1,166 @@ > +/* > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#ifndef AVUTIL_HWCONTEXT_DRM_H > +#define AVUTIL_HWCONTEXT_DRM_H > + > +#include <stddef.h> > +#include <stdint.h> > + > +/** > + * @file > + * API-specific header for AV_HWDEVICE_TYPE_DRM. > + * > + * Internal frame allocation is not currently supported - all frames > + * must be allocated by the user. Thus AVHWFramesContext is always > + * NULL, though this may change if support for frame allocation is > + * added in future. > + */ > + > +enum { > + /** > + * The maximum number of layers/planes in a DRM frame. > + */ > + AV_DRM_MAX_PLANES = 4 > +}; > + > +/** > + * DRM object descriptor. > + * > + * Describes a single DRM object, addressing it as a PRIME file > + * descriptor. > + */ > +typedef struct AVDRMObjectDescriptor { > + /** > + * DRM PRIME fd for the object. > + */ > + int fd; > + /** > + * Total size of the object. > + * > + * (This includes any parts not which do not contain image data.) > + */ > + size_t size; > + /** > + * Format modifier applied to the object (DRM_FORMAT_MOD_*). > + */ > + uint64_t format_modifier; > +} AVDRMObjectDescriptor; > + > +/** > + * DRM plane descriptor. > + * > + * Describes a single plane of a layer, which is contained within > + * a single object. > + */ > +typedef struct AVDRMPlaneDescriptor { > + /** > + * Index of the object containing this plane in the objects > + * array of the enclosing frame descriptor. > + */ > + int object_index; > + /** > + * Offset within that object of this plane. > + */ > + ptrdiff_t offset; > + /** > + * Pitch (linesize) of this plane. > + */ > + ptrdiff_t pitch; > +} AVDRMPlaneDescriptor; > + > +/** > + * DRM layer descriptor. > + * > + * Describes a single layer within a frame. This has the structure > + * defined by its format, and will contain one or more planes. > + */ > +typedef struct AVDRMLayerDescriptor { > + /** > + * Format of the layer (DRM_FORMAT_*). > + */ > + uint32_t format; > + /** > + * Number of planes in the layer. > + * > + * This must match the number of planes required by format. > + */ > + int nb_planes; > + /** > + * Array of planes in this layer. > + */ > + AVDRMPlaneDescriptor planes[AV_DRM_MAX_PLANES]; > +} AVDRMLayerDescriptor; > + > +/** > + * DRM frame descriptor. > + * > + * This is used as the data pointer for AV_PIX_FMT_DRM_PRIME frames. > + * It is also used by user-allocated frame pools - allocating in > + * AVHWFramesContext.pool must return AVBufferRefs which contain > + * an object of this type. > + * > + * The fields of this structure should be set such it can be > + * imported directly by EGL using the EGL_EXT_image_dma_buf_import > + * and EGL_EXT_image_dma_buf_import_modifiers extensions. > + * (Note that the exact layout of a particular format may vary between > + * platforms - we only specify that the same platform should be able > + * to import it.) > + * > + * The total number of planes must not exceed AV_DRM_MAX_PLANES, and > + * the order of the planes by increasing layer index followed by > + * increasing plane index must be the same as the order which would > + * be used for the data pointers in the equivalent software format. > + */ > +typedef struct AVDRMFrameDescriptor { > + /** > + * Number of DRM objects making up this frame. > + */ > + int nb_objects; > + /** > + * Array of objects making up the frame. > + */ > + AVDRMObjectDescriptor objects[AV_DRM_MAX_PLANES]; > + /** > + * Number of layers in the frame. > + */ > + int nb_layers; > + /** > + * Array of layers in the frame. > + */ > + AVDRMLayerDescriptor layers[AV_DRM_MAX_PLANES]; > +} AVDRMFrameDescriptor; > + > +/** > + * DRM device. > + * > + * Allocated as AVHWDeviceContext.hwctx. > + */ > +typedef struct AVDRMDeviceContext { > + /** > + * File descriptor of DRM device. > + * > + * This is used as the device to create frames on, and may also be > + * used in some derivation and mapping operations. > + * > + * If no device is required, set to -1. > + */ > + int fd; > +} AVDRMDeviceContext; > + > +#endif /* AVUTIL_HWCONTEXT_DRM_H */ > diff --git a/libavutil/hwcontext_internal.h b/libavutil/hwcontext_internal.h > index 68f78c0a1f..314c3656a7 100644 > --- a/libavutil/hwcontext_internal.h > +++ b/libavutil/hwcontext_internal.h > @@ -158,6 +158,7 @@ int ff_hwframe_map_create(AVBufferRef *hwframe_ref, > > > extern const HWContextType ff_hwcontext_type_cuda; > +extern const HWContextType ff_hwcontext_type_drm; > extern const HWContextType ff_hwcontext_type_dxva2; > extern const HWContextType ff_hwcontext_type_qsv; > extern const HWContextType ff_hwcontext_type_vaapi; > diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c > index d4a7a8ba3b..bc1142d571 100644 > --- a/libavutil/pixdesc.c > +++ b/libavutil/pixdesc.c > @@ -2158,6 +2158,10 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { > .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_PLANAR | > AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_ALPHA, > }, > + [AV_PIX_FMT_DRM_PRIME] = { > + .name = "drm_prime", > + .flags = AV_PIX_FMT_FLAG_HWACCEL, > + }, > }; > #if FF_API_PLUS1_MINUS1 > FF_ENABLE_DEPRECATION_WARNINGS > diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h > index 6b7eea8c4e..5a6a732061 100644 > --- a/libavutil/pixfmt.h > +++ b/libavutil/pixfmt.h > @@ -314,6 +314,10 @@ enum AVPixelFormat { > AV_PIX_FMT_P016LE, ///< like NV12, with 16bpp per component, little-endian > AV_PIX_FMT_P016BE, ///< like NV12, with 16bpp per component, big-endian > > + // DRM-managed buffers exposed through PRIME buffer sharing. > + // data[0] points to an AVDRMFrameDescriptor. > + AV_PIX_FMT_DRM_PRIME, > + > AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions > }; > >
On Fri, 7 Jul 2017 08:51:51 +0800 Jun Zhao <mypopydev@gmail.com> wrote: > On 2017/7/6 7:02, Mark Thompson wrote: > > diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h > > index edf12cc631..fe7613b379 100644 > > --- a/libavutil/hwcontext.h > > +++ b/libavutil/hwcontext.h > > @@ -32,6 +32,7 @@ enum AVHWDeviceType { > > AV_HWDEVICE_TYPE_QSV, > > AV_HWDEVICE_TYPE_VIDEOTOOLBOX, > > AV_HWDEVICE_TYPE_NONE, > > + AV_HWDEVICE_TYPE_DRM, > > }; > > AV_HWDEVICE_TYPE_DRM = AV_HWDEVICE_TYPE_NONE + 1 ? Yes, for ABI reasons. AV_HWDEVICE_TYPE_NONE should really be 0 and on the beginning of the enum, but we can fix that only on the next ABI bump.
diff --git a/configure b/configure index 9509bdf09c..3225c3cca7 100755 --- a/configure +++ b/configure @@ -298,6 +298,7 @@ External library support: --disable-cuvid disable Nvidia CUVID support [autodetect] --disable-d3d11va disable Microsoft Direct3D 11 video acceleration code [autodetect] --disable-dxva2 disable Microsoft DirectX 9 video acceleration code [autodetect] + --enable-libdrm enable DRM code (Linux) [no] --enable-libmfx enable Intel MediaSDK (AKA Quick Sync Video) code via libmfx [no] --enable-libnpp enable Nvidia Performance Primitives-based code [no] --enable-mmal enable Broadcom Multi-Media Abstraction Layer (Raspberry Pi) via MMAL [no] @@ -1539,6 +1540,7 @@ EXTERNAL_LIBRARY_LIST=" libcaca libcelt libdc1394 + libdrm libflite libfontconfig libfreetype @@ -5797,6 +5799,7 @@ enabled libcelt && require libcelt celt/celt.h celt_decode -lcelt0 && die "ERROR: libcelt must be installed and version must be >= 0.11.0."; } enabled libcaca && require_pkg_config caca caca.h caca_create_canvas enabled libdc1394 && require_pkg_config libdc1394-2 dc1394/dc1394.h dc1394_new +enabled libdrm && require_pkg_config libdrm xf86drm.h drmGetVersion enabled libfdk_aac && { use_pkg_config fdk-aac "fdk-aac/aacenc_lib.h" aacEncOpen || { require libfdk_aac fdk-aac/aacenc_lib.h aacEncOpen -lfdk-aac && warn "using libfdk without pkg-config"; } } diff --git a/libavutil/Makefile b/libavutil/Makefile index 4fe81fdd07..991b2f3fbd 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -33,6 +33,7 @@ HEADERS = adler32.h \ hmac.h \ hwcontext.h \ hwcontext_cuda.h \ + hwcontext_drm.h \ hwcontext_dxva2.h \ hwcontext_qsv.h \ hwcontext_vaapi.h \ @@ -158,6 +159,7 @@ OBJS-$(!HAVE_ATOMICS_NATIVE) += atomic.o \ OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o OBJS-$(CONFIG_DXVA2) += hwcontext_dxva2.o OBJS-$(CONFIG_QSV) += hwcontext_qsv.o +OBJS-$(CONFIG_LIBDRM) += hwcontext_drm.o OBJS-$(CONFIG_LZO) += lzo.o OBJS-$(CONFIG_OPENCL) += opencl.o opencl_internal.o OBJS-$(CONFIG_VAAPI) += hwcontext_vaapi.o diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c index 5a22194716..e40ceb12a1 100644 --- a/libavutil/hwcontext.c +++ b/libavutil/hwcontext.c @@ -32,6 +32,9 @@ static const HWContextType *const hw_table[] = { #if CONFIG_CUDA &ff_hwcontext_type_cuda, #endif +#if CONFIG_LIBDRM + &ff_hwcontext_type_drm, +#endif #if CONFIG_DXVA2 &ff_hwcontext_type_dxva2, #endif @@ -52,6 +55,7 @@ static const HWContextType *const hw_table[] = { static const char *const hw_type_names[] = { [AV_HWDEVICE_TYPE_CUDA] = "cuda", + [AV_HWDEVICE_TYPE_DRM] = "drm", [AV_HWDEVICE_TYPE_DXVA2] = "dxva2", [AV_HWDEVICE_TYPE_QSV] = "qsv", [AV_HWDEVICE_TYPE_VAAPI] = "vaapi", diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h index edf12cc631..fe7613b379 100644 --- a/libavutil/hwcontext.h +++ b/libavutil/hwcontext.h @@ -32,6 +32,7 @@ enum AVHWDeviceType { AV_HWDEVICE_TYPE_QSV, AV_HWDEVICE_TYPE_VIDEOTOOLBOX, AV_HWDEVICE_TYPE_NONE, + AV_HWDEVICE_TYPE_DRM, }; typedef struct AVHWDeviceInternal AVHWDeviceInternal; diff --git a/libavutil/hwcontext_drm.c b/libavutil/hwcontext_drm.c new file mode 100644 index 0000000000..c6be3295f9 --- /dev/null +++ b/libavutil/hwcontext_drm.c @@ -0,0 +1,294 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config.h" + +#include <fcntl.h> +#include <sys/mman.h> +#if HAVE_UNISTD_H +# include <unistd.h> +#endif + +#include <drm.h> +#include <xf86drm.h> + +#include "avassert.h" +#include "hwcontext.h" +#include "hwcontext_drm.h" +#include "hwcontext_internal.h" +#include "imgutils.h" + + +static void drm_device_free(AVHWDeviceContext *hwdev) +{ + AVDRMDeviceContext *hwctx = hwdev->hwctx; + + close(hwctx->fd); +} + +static int drm_device_create(AVHWDeviceContext *hwdev, const char *device, + AVDictionary *opts, int flags) +{ + AVDRMDeviceContext *hwctx = hwdev->hwctx; + drmVersionPtr version; + + hwctx->fd = open(device, O_RDWR); + if (hwctx->fd < 0) + return AVERROR_UNKNOWN; + + version = drmGetVersion(hwctx->fd); + if (!version) { + av_log(hwdev, AV_LOG_ERROR, "Failed to DRM version information " + "from %s: probably not a DRM device?\n", device); + close(hwctx->fd); + return AVERROR(EINVAL); + } + + av_log(hwdev, AV_LOG_VERBOSE, "Opened DRM device %s: driver %s " + "version %d.%d.%d.\n", device, version->name, + version->version_major, version->version_minor, + version->version_patchlevel); + + drmFreeVersion(version); + + hwdev->free = &drm_device_free; + + return 0; +} + +static int drm_get_buffer(AVHWFramesContext *hwfc, AVFrame *frame) +{ + frame->buf[0] = av_buffer_pool_get(hwfc->pool); + if (!frame->buf[0]) + return AVERROR(ENOMEM); + + frame->data[0] = (uint8_t*)frame->buf[0]->data; + + frame->format = AV_PIX_FMT_DRM_PRIME; + frame->width = hwfc->width; + frame->height = hwfc->height; + + return 0; +} + +typedef struct DRMMapping { + // Address and length of each mmap()ed region. + int nb_regions; + void *address[AV_DRM_MAX_PLANES]; + size_t length[AV_DRM_MAX_PLANES]; +} DRMMapping; + +static void drm_unmap_frame(AVHWFramesContext *hwfc, + HWMapDescriptor *hwmap) +{ + DRMMapping *map = hwmap->priv; + int i; + + for (i = 0; i < map->nb_regions; i++) { + if (map->address[i]) + munmap(map->address[i], map->length[i]); + } + + av_free(map); +} + +static int drm_map_frame(AVHWFramesContext *hwfc, + AVFrame *dst, const AVFrame *src, int flags) +{ + const AVDRMFrameDescriptor *desc = (AVDRMFrameDescriptor*)src->data[0]; + DRMMapping *map; + int err, i, p, plane; + int mmap_prot; + void *addr; + + map = av_mallocz(sizeof(*map)); + if (!map) + return AVERROR(ENOMEM); + + mmap_prot = 0; + if (flags & AV_HWFRAME_MAP_READ) + mmap_prot |= PROT_READ; + if (flags & AV_HWFRAME_MAP_WRITE) + mmap_prot |= PROT_WRITE; + + av_assert0(desc->nb_objects <= AV_DRM_MAX_PLANES); + for (i = 0; i < desc->nb_objects; i++) { + addr = mmap(NULL, desc->objects[i].size, mmap_prot, MAP_SHARED, + desc->objects[i].fd, 0); + if (addr == MAP_FAILED) { + err = AVERROR(errno); + av_log(hwfc, AV_LOG_ERROR, "Failed to map DRM object %d to " + "memory: %d.\n", desc->objects[i].fd, errno); + goto fail; + } + + map->address[i] = addr; + map->length[i] = desc->objects[i].size; + } + map->nb_regions = i; + + plane = 0; + for (i = 0; i < desc->nb_layers; i++) { + const AVDRMLayerDescriptor *layer = &desc->layers[i]; + for (p = 0; p < layer->nb_planes; p++) { + dst->data[plane] = + (uint8_t*)map->address[layer->planes[p].object_index] + + layer->planes[p].offset; + dst->linesize[plane] = layer->planes[p].pitch; + ++plane; + } + } + av_assert0(plane <= AV_DRM_MAX_PLANES); + + dst->width = src->width; + dst->height = src->height; + + err = ff_hwframe_map_create(src->hw_frames_ctx, dst, src, + &drm_unmap_frame, map); + if (err < 0) + return err; + + return 0; + +fail: + for (i = 0; i < desc->nb_objects; i++) { + if (map->address[i]) + munmap(map->address[i], map->length[i]); + } + av_free(map); + return err; +} + +static int drm_transfer_get_formats(AVHWFramesContext *ctx, + enum AVHWFrameTransferDirection dir, + enum AVPixelFormat **formats) +{ + enum AVPixelFormat *pix_fmts; + + pix_fmts = av_malloc_array(2, sizeof(*pix_fmts)); + if (!pix_fmts) + return AVERROR(ENOMEM); + + pix_fmts[0] = ctx->sw_format; + pix_fmts[1] = AV_PIX_FMT_NONE; + + *formats = pix_fmts; + return 0; +} + +static int drm_transfer_data_from(AVHWFramesContext *hwfc, + AVFrame *dst, const AVFrame *src) +{ + AVFrame *map; + int err; + + if (dst->width > hwfc->width || dst->height > hwfc->height) + return AVERROR(EINVAL); + + map = av_frame_alloc(); + if (!map) + return AVERROR(ENOMEM); + map->format = dst->format; + + err = drm_map_frame(hwfc, map, src, AV_HWFRAME_MAP_READ); + if (err) + goto fail; + + map->width = dst->width; + map->height = dst->height; + + err = av_frame_copy(dst, map); + if (err) + goto fail; + + err = 0; +fail: + av_frame_free(&map); + return err; +} + +static int drm_transfer_data_to(AVHWFramesContext *hwfc, + AVFrame *dst, const AVFrame *src) +{ + AVFrame *map; + int err; + + if (src->width > hwfc->width || src->height > hwfc->height) + return AVERROR(EINVAL); + + map = av_frame_alloc(); + if (!map) + return AVERROR(ENOMEM); + map->format = src->format; + + err = drm_map_frame(hwfc, map, dst, AV_HWFRAME_MAP_WRITE | AV_HWFRAME_MAP_OVERWRITE); + if (err) + goto fail; + + map->width = src->width; + map->height = src->height; + + err = av_frame_copy(map, src); + if (err) + goto fail; + + err = 0; +fail: + av_frame_free(&map); + return err; +} + +static int drm_map_from(AVHWFramesContext *hwfc, AVFrame *dst, + const AVFrame *src, int flags) +{ + int err; + + if (hwfc->sw_format != dst->format) + return AVERROR(ENOSYS); + + err = drm_map_frame(hwfc, dst, src, flags); + if (err) + return err; + + err = av_frame_copy_props(dst, src); + if (err) + return err; + + return 0; +} + +const HWContextType ff_hwcontext_type_drm = { + .type = AV_HWDEVICE_TYPE_DRM, + .name = "DRM", + + .device_hwctx_size = sizeof(AVDRMDeviceContext), + + .device_create = &drm_device_create, + + .frames_get_buffer = &drm_get_buffer, + + .transfer_get_formats = &drm_transfer_get_formats, + .transfer_data_to = &drm_transfer_data_to, + .transfer_data_from = &drm_transfer_data_from, + .map_from = &drm_map_from, + + .pix_fmts = (const enum AVPixelFormat[]) { + AV_PIX_FMT_DRM_PRIME, + AV_PIX_FMT_NONE + }, +}; diff --git a/libavutil/hwcontext_drm.h b/libavutil/hwcontext_drm.h new file mode 100644 index 0000000000..2e225451e1 --- /dev/null +++ b/libavutil/hwcontext_drm.h @@ -0,0 +1,166 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_HWCONTEXT_DRM_H +#define AVUTIL_HWCONTEXT_DRM_H + +#include <stddef.h> +#include <stdint.h> + +/** + * @file + * API-specific header for AV_HWDEVICE_TYPE_DRM. + * + * Internal frame allocation is not currently supported - all frames + * must be allocated by the user. Thus AVHWFramesContext is always + * NULL, though this may change if support for frame allocation is + * added in future. + */ + +enum { + /** + * The maximum number of layers/planes in a DRM frame. + */ + AV_DRM_MAX_PLANES = 4 +}; + +/** + * DRM object descriptor. + * + * Describes a single DRM object, addressing it as a PRIME file + * descriptor. + */ +typedef struct AVDRMObjectDescriptor { + /** + * DRM PRIME fd for the object. + */ + int fd; + /** + * Total size of the object. + * + * (This includes any parts not which do not contain image data.) + */ + size_t size; + /** + * Format modifier applied to the object (DRM_FORMAT_MOD_*). + */ + uint64_t format_modifier; +} AVDRMObjectDescriptor; + +/** + * DRM plane descriptor. + * + * Describes a single plane of a layer, which is contained within + * a single object. + */ +typedef struct AVDRMPlaneDescriptor { + /** + * Index of the object containing this plane in the objects + * array of the enclosing frame descriptor. + */ + int object_index; + /** + * Offset within that object of this plane. + */ + ptrdiff_t offset; + /** + * Pitch (linesize) of this plane. + */ + ptrdiff_t pitch; +} AVDRMPlaneDescriptor; + +/** + * DRM layer descriptor. + * + * Describes a single layer within a frame. This has the structure + * defined by its format, and will contain one or more planes. + */ +typedef struct AVDRMLayerDescriptor { + /** + * Format of the layer (DRM_FORMAT_*). + */ + uint32_t format; + /** + * Number of planes in the layer. + * + * This must match the number of planes required by format. + */ + int nb_planes; + /** + * Array of planes in this layer. + */ + AVDRMPlaneDescriptor planes[AV_DRM_MAX_PLANES]; +} AVDRMLayerDescriptor; + +/** + * DRM frame descriptor. + * + * This is used as the data pointer for AV_PIX_FMT_DRM_PRIME frames. + * It is also used by user-allocated frame pools - allocating in + * AVHWFramesContext.pool must return AVBufferRefs which contain + * an object of this type. + * + * The fields of this structure should be set such it can be + * imported directly by EGL using the EGL_EXT_image_dma_buf_import + * and EGL_EXT_image_dma_buf_import_modifiers extensions. + * (Note that the exact layout of a particular format may vary between + * platforms - we only specify that the same platform should be able + * to import it.) + * + * The total number of planes must not exceed AV_DRM_MAX_PLANES, and + * the order of the planes by increasing layer index followed by + * increasing plane index must be the same as the order which would + * be used for the data pointers in the equivalent software format. + */ +typedef struct AVDRMFrameDescriptor { + /** + * Number of DRM objects making up this frame. + */ + int nb_objects; + /** + * Array of objects making up the frame. + */ + AVDRMObjectDescriptor objects[AV_DRM_MAX_PLANES]; + /** + * Number of layers in the frame. + */ + int nb_layers; + /** + * Array of layers in the frame. + */ + AVDRMLayerDescriptor layers[AV_DRM_MAX_PLANES]; +} AVDRMFrameDescriptor; + +/** + * DRM device. + * + * Allocated as AVHWDeviceContext.hwctx. + */ +typedef struct AVDRMDeviceContext { + /** + * File descriptor of DRM device. + * + * This is used as the device to create frames on, and may also be + * used in some derivation and mapping operations. + * + * If no device is required, set to -1. + */ + int fd; +} AVDRMDeviceContext; + +#endif /* AVUTIL_HWCONTEXT_DRM_H */ diff --git a/libavutil/hwcontext_internal.h b/libavutil/hwcontext_internal.h index 68f78c0a1f..314c3656a7 100644 --- a/libavutil/hwcontext_internal.h +++ b/libavutil/hwcontext_internal.h @@ -158,6 +158,7 @@ int ff_hwframe_map_create(AVBufferRef *hwframe_ref, extern const HWContextType ff_hwcontext_type_cuda; +extern const HWContextType ff_hwcontext_type_drm; extern const HWContextType ff_hwcontext_type_dxva2; extern const HWContextType ff_hwcontext_type_qsv; extern const HWContextType ff_hwcontext_type_vaapi; diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index d4a7a8ba3b..bc1142d571 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2158,6 +2158,10 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_ALPHA, }, + [AV_PIX_FMT_DRM_PRIME] = { + .name = "drm_prime", + .flags = AV_PIX_FMT_FLAG_HWACCEL, + }, }; #if FF_API_PLUS1_MINUS1 FF_ENABLE_DEPRECATION_WARNINGS diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index 6b7eea8c4e..5a6a732061 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -314,6 +314,10 @@ enum AVPixelFormat { AV_PIX_FMT_P016LE, ///< like NV12, with 16bpp per component, little-endian AV_PIX_FMT_P016BE, ///< like NV12, with 16bpp per component, big-endian + // DRM-managed buffers exposed through PRIME buffer sharing. + // data[0] points to an AVDRMFrameDescriptor. + AV_PIX_FMT_DRM_PRIME, + AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions };