From patchwork Sat Jan 13 06:06:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wm4 X-Patchwork-Id: 7273 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.78.2 with SMTP id r2csp357015jaa; Fri, 12 Jan 2018 22:06:07 -0800 (PST) X-Google-Smtp-Source: ACJfBouBPQBkis2yqeHbG7YGHuNcVoqCQRKAKrGVqrfRyuV2BNdtgiTOHNze0eEw4IJ/NDD71BaX X-Received: by 10.28.214.20 with SMTP id n20mr6172506wmg.59.1515823567353; Fri, 12 Jan 2018 22:06:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515823567; cv=none; d=google.com; s=arc-20160816; b=u1VGNUcA6aW65J1RVAZuAIa3RukUWTvaWy4r1kuhe80a+b+6NevdYU1mO8jnlYOP4s vm0344ZqowtCyIWm8UgK5FDFEAWWune+OW3EpO97+AcTYuZ/dWaa/tKvPWoXRqEi1q0q MdU0gaCXfC8ZBjvAf11F3JIJaeFBG4jf55uNst6FWEmz9MFJHR+Cf2jJu3lBvDCzLqIt ngchfLFn+gNHcPGkOJrs1KpsWVuyxEzz5r5u4IYf4ddb6aUv00V7RUUXqdwO06p8UpDX gu5kan00XKGZ8fp/62oLPux3W29W4APS0Pu2I7pzX+3m+1tX1u3N74ZoXxVISTvOG8+y Cm1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=tGeCVhLgpm6aWYRKVtApCjmUR8TEMXJn+/p4lIq6Ls4=; b=aS2O5ci7UQTynT1/2gz+Op5YbNlf4bnMVdJ35jR59uGCDCWGDJbuVO1nFbESLNHLyN omcaOE1GcIJfqY/mcCwwwP7hHZEjLes/N0Nm8Reh91IC4K3MQkTU7ab1NHnUp7g36oOf 3Ti4obBlAHhN98aFGy77akmLI5KmAvJAChOWHuC4qyp8DJb3Wz3z+w/IscMLfo9O9JcJ BRMntAYOFXSmBMGDSc6CuNtKx7N75SvQB+nj0Ep3iK7j+k8sqX5men7MA5+GrtvCAPef Cm+NglnC0Nr945RG06yapj3c2xSjtK/GE3hPpjo1aXLRDeWIOjKKKGd1htQQYoAHceG6 h15Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com header.s=20161025 header.b=VZtRj1br; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j22si3448334wmi.149.2018.01.12.22.06.06; Fri, 12 Jan 2018 22:06:07 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com header.s=20161025 header.b=VZtRj1br; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 76F33689D60; Sat, 13 Jan 2018 08:06:04 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9B418689C7D for ; Sat, 13 Jan 2018 08:05:57 +0200 (EET) Received: by mail-wm0-f67.google.com with SMTP id 81so5045798wmb.1 for ; Fri, 12 Jan 2018 22:05:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=kiEy2OTOlUJWFg2xVIgt1cv6nZNgXWXY47knMTT/7Yo=; b=VZtRj1brlt9We2bcy4cTnwSVJg0e3MoMff2Y0Wf3M0BQtoOb219UeEwI/vTCB3ew/Z sDoQlnnG90fUq2kjei2wWpD8pL6TqnrUkkkbEMuhCz3KVzszd5FpayCCd1rvBHXc7w0A Bf846FD+GwhH6Ox/V3xDcPjyXcE+BtBW/5RCk9TOZI+t6zhOFkBRBPLgF5QCrDp2JaXc RIdso5ZT+5UEB9Ichmbrdx+75iMC/tFR4IaosOQoJTno4cKzdYzC9QqnXXktlyTA0+w8 slBcfvM5hXZ50ayTyC1pdAconlD4H+qzIj2wz1ajgPumzouM6gGWWlWMbkgJJyO5nOAy Hy0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=kiEy2OTOlUJWFg2xVIgt1cv6nZNgXWXY47knMTT/7Yo=; b=o9pLLDnbBheKttso6rCCyMoDwro1wsNls21B5gkmOHlAdSkNNxy9w5YpP16OgFUQ3j QFObEl+ILIx1mBwKsMJoFmj9Hyjd5ZuykHr8+YnIfFmm5trs001qt6tb6FvTHcNVwphk ei0u/K6hw8WeQjzNyBDBoqeOREjiOImaAZ/UB5uC4m/YmWZqdsnt8KzPAzeJU/nLGrpf lLJE2lAnlrzLkKiTpOV8Rr7GjjNCtE36ty0FgBU2onvcRe7+KzUX43Kb6RVLXnL2d2Oe XYCybVhS0P+Go9Hw5Z8vAWQivXpcHh4oFju/JmY3bNYAjfSTK0C52p7V2AvBMr+bqx2s sLoQ== X-Gm-Message-State: AKwxyteVOX7nerrp8CHSXJs63uCyIe7kd4EbLyjczVTBRQxcaEMwwl7r hwaRg42xMVVpHphzjLK1vHPmrQ== X-Received: by 10.80.190.197 with SMTP id e5mr19816189edk.48.1515823557350; Fri, 12 Jan 2018 22:05:57 -0800 (PST) Received: from debian.speedport.ip (p2003006CCD4EDC182090E2A8DCD919CC.dip0.t-ipconnect.de. [2003:6c:cd4e:dc18:2090:e2a8:dcd9:19cc]) by smtp.googlemail.com with ESMTPSA id g54sm189096edc.91.2018.01.12.22.05.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Jan 2018 22:05:56 -0800 (PST) From: wm4 To: ffmpeg-devel@ffmpeg.org Date: Sat, 13 Jan 2018 07:06:02 +0100 Message-Id: <20180113060602.9713-1-nfxjfg@googlemail.com> X-Mailer: git-send-email 2.15.1 Subject: [FFmpeg-devel] [PATCH] hwcontext_vdpau: implement av_hwdevice_get_hwframe_constraints() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: wm4 MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" In addition, this does not allow creating frames contexts with sw_format for which no known transfer formats exist. In theory, we should check whether the chroma format (i.e. the sw_format) is supported at all by the vdpau driver, but checking for transfer formats has the same effect. Note that the pre-existing code adds 1 to priv->nb_pix_fmts[i] for unknown reason, and some checks need to account for that to check for empty lists. They are not off-by-one errors. --- libavutil/hwcontext_vdpau.c | 55 ++++++++++++++++++++++++++++++++------------- 1 file changed, 39 insertions(+), 16 deletions(-) diff --git a/libavutil/hwcontext_vdpau.c b/libavutil/hwcontext_vdpau.c index 9b8f839647..c11c3cfdab 100644 --- a/libavutil/hwcontext_vdpau.c +++ b/libavutil/hwcontext_vdpau.c @@ -79,11 +79,12 @@ static const VDPAUPixFmtMap pix_fmts_444[] = { static const struct { VdpChromaType chroma_type; + enum AVPixelFormat frames_sw_format; const VDPAUPixFmtMap *map; } vdpau_pix_fmts[] = { - { VDP_CHROMA_TYPE_420, pix_fmts_420 }, - { VDP_CHROMA_TYPE_422, pix_fmts_422 }, - { VDP_CHROMA_TYPE_444, pix_fmts_444 }, + { VDP_CHROMA_TYPE_420, AV_PIX_FMT_YUV420P, pix_fmts_420 }, + { VDP_CHROMA_TYPE_422, AV_PIX_FMT_YUV422P, pix_fmts_422 }, + { VDP_CHROMA_TYPE_444, AV_PIX_FMT_YUV444P, pix_fmts_444 }, }; static int count_pixfmts(const VDPAUPixFmtMap *map) @@ -170,6 +171,35 @@ static void vdpau_device_uninit(AVHWDeviceContext *ctx) av_freep(&priv->pix_fmts[i]); } +static int vdpau_frames_get_constraints(AVHWDeviceContext *ctx, + const void *hwconfig, + AVHWFramesConstraints *constraints) +{ + VDPAUDeviceContext *priv = ctx->internal->priv; + int nb_sw_formats = 0; + int i; + + constraints->valid_sw_formats = av_malloc_array(FF_ARRAY_ELEMS(vdpau_pix_fmts) + 1, + sizeof(*constraints->valid_sw_formats)); + if (!constraints->valid_sw_formats) + return AVERROR(ENOMEM); + + for (i = 0; i < FF_ARRAY_ELEMS(vdpau_pix_fmts); i++) { + if (priv->nb_pix_fmts[i] > 1) + constraints->valid_sw_formats[nb_sw_formats++] = vdpau_pix_fmts[i].frames_sw_format; + } + constraints->valid_sw_formats[nb_sw_formats] = AV_PIX_FMT_NONE; + + constraints->valid_hw_formats = av_malloc_array(2, sizeof(*constraints->valid_hw_formats)); + if (!constraints->valid_hw_formats) + return AVERROR(ENOMEM); + + constraints->valid_hw_formats[0] = AV_PIX_FMT_VDPAU; + constraints->valid_hw_formats[1] = AV_PIX_FMT_NONE; + + return 0; +} + static void vdpau_buffer_free(void *opaque, uint8_t *data) { AVHWFramesContext *ctx = opaque; @@ -214,26 +244,18 @@ static int vdpau_frames_init(AVHWFramesContext *ctx) int i; - switch (ctx->sw_format) { - case AV_PIX_FMT_YUV420P: priv->chroma_type = VDP_CHROMA_TYPE_420; break; - case AV_PIX_FMT_YUV422P: priv->chroma_type = VDP_CHROMA_TYPE_422; break; - case AV_PIX_FMT_YUV444P: priv->chroma_type = VDP_CHROMA_TYPE_444; break; - default: - av_log(ctx, AV_LOG_ERROR, "Unsupported data layout: %s\n", - av_get_pix_fmt_name(ctx->sw_format)); - return AVERROR(ENOSYS); - } - for (i = 0; i < FF_ARRAY_ELEMS(vdpau_pix_fmts); i++) { - if (vdpau_pix_fmts[i].chroma_type == priv->chroma_type) { + if (vdpau_pix_fmts[i].frames_sw_format == ctx->sw_format) { + priv->chroma_type = vdpau_pix_fmts[i].chroma_type; priv->chroma_idx = i; priv->pix_fmts = device_priv->pix_fmts[i]; priv->nb_pix_fmts = device_priv->nb_pix_fmts[i]; break; } } - if (!priv->pix_fmts) { - av_log(ctx, AV_LOG_ERROR, "Unsupported chroma type: %d\n", priv->chroma_type); + if (priv->nb_pix_fmts < 2) { + av_log(ctx, AV_LOG_ERROR, "Unsupported sw format: %s\n", + av_get_pix_fmt_name(ctx->sw_format)); return AVERROR(ENOSYS); } @@ -468,6 +490,7 @@ const HWContextType ff_hwcontext_type_vdpau = { #endif .device_init = vdpau_device_init, .device_uninit = vdpau_device_uninit, + .frames_get_constraints = vdpau_frames_get_constraints, .frames_init = vdpau_frames_init, .frames_get_buffer = vdpau_get_buffer, .transfer_get_formats = vdpau_transfer_get_formats,