From patchwork Fri Nov 13 23:15:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bas Nieuwenhuizen X-Patchwork-Id: 23620 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1A4A4448D84 for ; Sat, 14 Nov 2020 02:17:57 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EE08C68C176; Sat, 14 Nov 2020 02:17:56 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1A8B668BF4B for ; Sat, 14 Nov 2020 02:17:49 +0200 (EET) Received: by mail-wr1-f68.google.com with SMTP id d12so12042081wrr.13 for ; Fri, 13 Nov 2020 16:17:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=basnieuwenhuizen.nl; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=wdraLNhYZ9Stj6w9rTq8e9/gYBy8QSA95tgI7ihdqyI=; b=gj9VrDOLGPsjOFof99k2d0IEKNs3hlWTL2DQjV3Yvweg+Co3Pt4aNmx/jzV0P0dao0 CaYnTm/nyz8lUuiOPsD08kaOKgLwx//qgK4uGw+3Qcaz8GX0PQYpSQgY3vpTqjz31HqL W3kB1vh3Mni3jywKHsrzEDc3gAncEFpshoDB5Ss/w8pxvESyek/zs0CVYQX+PcRhFo+T vAHgSz6hFhLczFZWUzyBHL6x6jCClkyFpE8djDi6d/two//aK+rO9b6eQDqY8lDRtpNO oEn2oH9vwpYF8MshRJOyhEDR+znqD0zhvlffiS51dKWSFQJDHvIjObhxJgKjKftcet3F HTjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=wdraLNhYZ9Stj6w9rTq8e9/gYBy8QSA95tgI7ihdqyI=; b=p37mX1ispZP2XD1KsdpJTQPOAA/jgyirzyDs4XiSrf0nGoUOnn94a2ez0iqZwAnrHM 0jTBEPyKWLKG0tDZfnM28apd0LyuqBroQAwB1u2K+ocRwYrKaqH/wo9tQZpqJip1abKM veEWa4cEsmFJMQ4HGLjokpsVQliy3wTY2NYOUFKBzDonxzObA2V8cqLw4OsKqYhEtLUU /slehnCofHdHhJ/Joi7hcCEtQIdGbHvlzeJzzOD3RCsCWFigPz+D5EN8D5BLZVuESUpH lTqmStOYVHpqaBAaDzBh/TtgrAlEZJliECjkNBsXQ8f8kN2oLPNcOjfUrXdUVl79C47y wRZQ== X-Gm-Message-State: AOAM530O9s6bMuoK9qDAS+qnvXN+CYJly5Ttt26JiASRsv8Rb2ljtgjN CXS2/Xq1uXBwvHETxhj+htnwnQRFc/cuCk6O X-Google-Smtp-Source: ABdhPJyxGvJF4iXUGenARcMSP2itDLXL205w4CP4Sc0RH0WSTQFNMRDB3ayXNpRpUAQX8gIQ1ghBJw== X-Received: by 2002:adf:cd8d:: with SMTP id q13mr6387665wrj.61.1605309343409; Fri, 13 Nov 2020 15:15:43 -0800 (PST) Received: from localhost.localdomain ([2a02:aa12:a77f:2000:4cea:81e7:5fd4:93f7]) by smtp.gmail.com with ESMTPSA id a17sm13633512wra.61.2020.11.13.15.15.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Nov 2020 15:15:42 -0800 (PST) From: Bas Nieuwenhuizen To: ffmpeg-devel@ffmpeg.org Date: Sat, 14 Nov 2020 00:15:46 +0100 Message-Id: <20201113231548.869559-1-bas@basnieuwenhuizen.nl> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/3] kmsgrab: Use invalid modifier if modifiers weren't used. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: sw@jkqxz.net, Bas Nieuwenhuizen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The kernel defaults to initializing the field to 0 when modifiers are not used and this happens to be linear. If we end up actually passing the modifier to a driver, tiling issues happen. So if the kernel doesn't return a modifier set it explicitly to INVALID. That way later processing knows there is no explicit modifier. v2: Fix support for modifier overrides in case the getfb2 call does not return a modifier. --- libavdevice/kmsgrab.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/libavdevice/kmsgrab.c b/libavdevice/kmsgrab.c index a0aa9dc22f..b740a32171 100644 --- a/libavdevice/kmsgrab.c +++ b/libavdevice/kmsgrab.c @@ -160,6 +160,7 @@ static int kmsgrab_get_fb2(AVFormatContext *avctx, KMSGrabContext *ctx = avctx->priv_data; drmModeFB2 *fb; int err, i, nb_objects; + uint64_t modifier = ctx->drm_format_modifier; fb = drmModeGetFB2(ctx->hwctx->fd, plane->fb_id); if (!fb) { @@ -195,6 +196,9 @@ static int kmsgrab_get_fb2(AVFormatContext *avctx, goto fail; } + if (fb->flags & DRM_MODE_FB_MODIFIERS) + modifier = fb->modifier; + *desc = (AVDRMFrameDescriptor) { .nb_layers = 1, .layers[0] = { @@ -243,7 +247,7 @@ static int kmsgrab_get_fb2(AVFormatContext *avctx, desc->objects[obj] = (AVDRMObjectDescriptor) { .fd = fd, .size = size, - .format_modifier = fb->modifier, + .format_modifier = modifier, }; desc->layers[0].planes[i] = (AVDRMPlaneDescriptor) { .object_index = obj, @@ -557,15 +561,18 @@ static av_cold int kmsgrab_read_header(AVFormatContext *avctx) err = AVERROR(EINVAL); goto fail; } - if (ctx->drm_format_modifier != DRM_FORMAT_MOD_INVALID && - ctx->drm_format_modifier != fb2->modifier) { - av_log(avctx, AV_LOG_ERROR, "Framebuffer format modifier " - "%"PRIx64" does not match expected modifier.\n", - fb2->modifier); - err = AVERROR(EINVAL); - goto fail; - } else { - ctx->drm_format_modifier = fb2->modifier; + + if (fb2->flags & DRM_MODE_FB_MODIFIERS) { + if (ctx->drm_format_modifier != DRM_FORMAT_MOD_INVALID && + ctx->drm_format_modifier != fb2->modifier) { + av_log(avctx, AV_LOG_ERROR, "Framebuffer format modifier " + "%"PRIx64" does not match expected modifier.\n", + fb2->modifier); + err = AVERROR(EINVAL); + goto fail; + } else { + ctx->drm_format_modifier = fb2->modifier; + } } av_log(avctx, AV_LOG_VERBOSE, "Format is %s, from " "DRM format %"PRIx32" modifier %"PRIx64".\n",