From patchwork Sun Oct 20 05:58:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koushik Dutta X-Patchwork-Id: 52403 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:143:b0:48e:c0f8:d0de with SMTP id h3csp2011095vqi; Sun, 20 Oct 2024 05:02:56 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXH+3OC+s7u+1gMjh9RghNxQABhHoGu5FkrUK4PZ2e8HvOGn9Ra3qQEIlTgGZZBr/sgT5SKlVvsyotUYRrZPXsj@gmail.com X-Google-Smtp-Source: AGHT+IGXZDWdI/N8z7x31xaJfS79nAzw99I7RWo8LpPpeJgc3BbP/yTogureOFVq9Lbu5R663FtJ X-Received: by 2002:a05:6512:3b98:b0:52b:de5b:1b30 with SMTP id 2adb3069b0e04-53a15449538mr4465144e87.44.1729425776519; Sun, 20 Oct 2024 05:02:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1729425776; cv=none; d=google.com; s=arc-20240605; b=LchQtUuhzPcS2h9U3E99oj0nQFnUG6dKJp9NzVhr2tVJBdUbbTY/5L6MoXS3NKmW4U DkXtid7lbyFCsmFl2HFY65tYWS7c1+o5fF8ta00HXv1JcNt6xRM0uSy4B7PgkVnLIsY6 lIciYS1aI7mWoNHZND3sh37guOaqMALLQv1/KKdFQ7Hu6rRnRdwnzGeSmklJ65juhx84 O6mugCqsiLAohFGsSwMMMk2yg9W8ySNAYuAW+ZgWhmv178UC3sREX+9Q9/Jsjr/ydFt/ 5BvOygxXamjkBLD93XaZOVy1s6so9rIlnGEboct3gGrAXx0epD74z9ZC23CfrTrFP9oU tjHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=+FiIhn7uU25M4I5uu6czChDgFXGDWcV8qDngJesbQrQ=; fh=vf1rgm8KpEMxcRKiIfYPC5TR7MMWdHSIxg51vjgwMpE=; b=AvbiuX1Q35leTgntsw5GSpgH/G38L58d1kZJT9XjFq5pwDQOfpsS87SeKHj4SUwnuc ifoorB4/BpcL7ArOLtzhlWrU/XZrGPikseVlNzm+kSNLab1krPt28V5zMGF6vSY0KE5R JKDhMLYjdbEzI2kCy3RNzN1E3HagHy6ze++f7bc7pikwrXV15bSoh8pKZT5irBwmmUcU tLuvg+S2v/8AVof/eiDxIUhB71muFiWI5+tdYXnY9QuAQ3aLMQv2QuvLD4a4A8Yt9i9E QMvGmp61itfVPxgwFiNWfnB9NL0cC+0QaSxXjit9Qb7clwUgjKsxpMpenDKk42CITvhI 25Pg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=jRBiKeN1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-53a223ec25fsi444147e87.16.2024.10.20.05.02.56; Sun, 20 Oct 2024 05:02:56 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=jRBiKeN1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 55A9868DB03; Sun, 20 Oct 2024 15:02:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 86CB368D9B9 for ; Sun, 20 Oct 2024 15:02:46 +0300 (EEST) Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-2e2e6a1042dso2909338a91.2 for ; Sun, 20 Oct 2024 05:02:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729425765; x=1730030565; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Lr6ltoIF6qg+pIfjZ0787S4YLIYVVmhrRz35xRvCN/A=; b=jRBiKeN1w577lZMz0L8Pab1qNa8QsUIGCNOmgAOaeiBE8+fdJW3+Q3kxlmH810G2Um PQIMUm4xxXQD7sk0BVfLOChrNXapmrcM0bWj4ZwOwq9rwaArweB/i8w0rNnngQg7MzUt LgVl2cCUbr5dZRUcmpzY2AAhm8AiYptP5fJmWpyXKnWhbgyVx45KdL7ZZ2gatl9rR4IO xSSCy5OVoDoezvc8oIFMM6eOQJNKcWqw+X5MJrNOZYSswgWSBzhv7kimnaGrBEU5+JNd RV+8G130uAlr6c2PTukBADv+T+/yo1u8dTYFjSECYYMAYoIcd0JNjdlqe7vx2s691IDy BXrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729425765; x=1730030565; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Lr6ltoIF6qg+pIfjZ0787S4YLIYVVmhrRz35xRvCN/A=; b=FKMGsqd76sCEi4vZUcZ5dUJQ9Qsl3XUgEQaqsHmPoE0mU8r2SXZ8bavd4PbjZtm+Y/ aBa5rJP666USxni9+azqZx90Mzh3JeMbU+WDz2OiRqc7yVr3XV4PrKm707f5l2eoZpzC SoL7LgkF+jEktOKY7y8Co8M6QrVGTIaxgzEF3aqGhr8r7J2+Q5XVMfVryhmAcTJ/2glh pBrUtGjeAkFyytEMVJ1SKt4ZWK6adYZG93woDpISDc5GyGiPM8rz9GiwnXg0+9iCxthr 5xdEZgq+4Q+3XtVO56WNjXRebDqu3ysZZv7zLxxqPufJgY0/DKjB14pRSRDcnn23b544 37lA== X-Gm-Message-State: AOJu0YwypkIVWezXTa2DYkoCdwYMvp2gvW7Kr6Rd4A1jc8Fqh4ogkiXD IgjTV32NyPV/NjB9EtTz1H3aQE69wBLZPQ3Q2g6Kyoj3B41vBHMc+/R3Mg== X-Received: by 2002:a05:6a00:1741:b0:71d:f7ea:89f6 with SMTP id d2e1a72fcca58-71ea32eb202mr9594638b3a.18.1729403931571; Sat, 19 Oct 2024 22:58:51 -0700 (PDT) Received: from Koushik-MacStudio.tail05204.ts.net ([2001:559:76c:0:dd25:f71c:c57:e92]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ec1407f63sm608169b3a.204.2024.10.19.22.58.51 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sat, 19 Oct 2024 22:58:51 -0700 (PDT) From: Koushik Dutta To: ffmpeg-devel@ffmpeg.org Date: Sat, 19 Oct 2024 22:58:43 -0700 Message-Id: <20241020055846.85936-1-koushd@gmail.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/4] avfilter/scale_vt: implement frame crop X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Koushik Dutta Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: cieiaOB2/cU6 The crop filter has no effect on scale_vt: -vf crop=100:100,scale_vt=300x300 Hardware frames (AV_PIX_FMT_FLAG_HWACCEL) are expected to use the crop_* properties, as seen in the implementation vf_crop.c. The current workaround is to hwdownload the full frame and perform the crop on CPU. Signed-off-by: Koushik Dutta --- libavfilter/vf_scale_vt.c | 50 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/libavfilter/vf_scale_vt.c b/libavfilter/vf_scale_vt.c index 05f4e7b797..3da46a6cd5 100644 --- a/libavfilter/vf_scale_vt.c +++ b/libavfilter/vf_scale_vt.c @@ -109,6 +109,8 @@ static av_cold int scale_vt_init(AVFilterContext *avctx) VTSessionSetProperty(s->transfer, kVTPixelTransferPropertyKey_DestinationYCbCrMatrix, value); } + VTSessionSetProperty(s->transfer, kVTPixelTransferPropertyKey_ScalingMode, kVTScalingMode_CropSourceToCleanAperture); + return 0; } @@ -132,6 +134,18 @@ static int scale_vt_filter_frame(AVFilterLink *link, AVFrame *in) CVPixelBufferRef src; CVPixelBufferRef dst; + int left; + int top; + int width; + int height; + CFNumberRef crop_width_num; + CFNumberRef crop_height_num; + CFNumberRef crop_offset_left_num; + CFNumberRef crop_offset_top_num; + const void *clean_aperture_keys[4]; + const void *source_clean_aperture_values[4]; + CFDictionaryRef source_clean_aperture; + AVFrame *out = ff_get_video_buffer(outlink, outlink->w, outlink->h); if (!out) { ret = AVERROR(ENOMEM); @@ -153,8 +167,43 @@ static int scale_vt_filter_frame(AVFilterLink *link, AVFrame *in) if (s->colour_matrix != AVCOL_SPC_UNSPECIFIED) out->colorspace = s->colour_matrix; + width = (in->width - in->crop_right) - in->crop_left; + height = (in->height - in->crop_bottom) - in->crop_top; + // The crop offsets are relative to the center of the frame. + // the crop width and crop height are relative to the center of the crop rect, not top left as normal. + left = in->crop_left - in->width / 2 + width / 2; + top = in->crop_top - in->height / 2 + height / 2; + crop_width_num = CFNumberCreate(kCFAllocatorDefault, kCFNumberIntType, &width); + crop_height_num = CFNumberCreate(kCFAllocatorDefault, kCFNumberIntType, &height); + crop_offset_left_num = CFNumberCreate(kCFAllocatorDefault, kCFNumberIntType, &left); + crop_offset_top_num = CFNumberCreate(kCFAllocatorDefault, kCFNumberIntType, &top); + + clean_aperture_keys[0] = kCVImageBufferCleanApertureWidthKey; + clean_aperture_keys[1] = kCVImageBufferCleanApertureHeightKey; + clean_aperture_keys[2] = kCVImageBufferCleanApertureHorizontalOffsetKey; + clean_aperture_keys[3] = kCVImageBufferCleanApertureVerticalOffsetKey; + + source_clean_aperture_values[0] = crop_width_num; + source_clean_aperture_values[1] = crop_height_num; + source_clean_aperture_values[2] = crop_offset_left_num; + source_clean_aperture_values[3] = crop_offset_top_num; + + source_clean_aperture = CFDictionaryCreate(kCFAllocatorDefault, + clean_aperture_keys, + source_clean_aperture_values, + 4, + &kCFTypeDictionaryKeyCallBacks, + &kCFTypeDictionaryValueCallBacks); + + CFRelease(crop_width_num); + CFRelease(crop_height_num); + CFRelease(crop_offset_left_num); + CFRelease(crop_offset_top_num); + src = (CVPixelBufferRef)in->data[3]; dst = (CVPixelBufferRef)out->data[3]; + CVBufferSetAttachment(src, kCVImageBufferCleanApertureKey, + source_clean_aperture, kCVAttachmentMode_ShouldPropagate); ret = VTPixelTransferSessionTransferImage(s->transfer, src, dst); if (ret != noErr) { av_log(ctx, AV_LOG_ERROR, "transfer image failed, %d\n", ret); @@ -162,6 +211,7 @@ static int scale_vt_filter_frame(AVFilterLink *link, AVFrame *in) goto fail; } + CFRelease(source_clean_aperture); av_frame_free(&in); return ff_filter_frame(outlink, out);