From patchwork Wed Feb 27 22:00:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 12165 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id CBCE4448F93 for ; Thu, 28 Feb 2019 00:06:35 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AB6B8689F14; Thu, 28 Feb 2019 00:06:35 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5082368080A for ; Thu, 28 Feb 2019 00:06:29 +0200 (EET) Received: by mail-wr1-f65.google.com with SMTP id d17so19686057wre.10 for ; Wed, 27 Feb 2019 14:06:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=bjQXL5FlAA7ANhCuGJvJETxq8KB+1IJgr7gSXr0j+4g=; b=XNger2/svyOm+ou0G3S9JjrpGyRVyEsqTy6rNXm5QHMWGOeFRptd9KDfLPyXFnvnQQ k2sNW0edPbAOjOa6KdoGAi9lOGNlVf6JxHMfwyY/FTwJEL/upzCbQoVLR/EwRzux4pX9 zrCdJt1mOeZiqsKCmNb0X8pDiGQw0st/3IIVeD5DIuJeC+7WPOzg8wEUAblGv/q845WQ ATASy+uPLwPIj78oa3/ITIFqeQhxUSs66SSXr4dhj9uO3U+NhZ52r1uXluOMF1iJdZff UQLq0jq4CTxBVa+es94pILFvmFY29trjr+jQgOwvlJU0PmGBioNrihq++duyCpFHVyKs 6/Sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bjQXL5FlAA7ANhCuGJvJETxq8KB+1IJgr7gSXr0j+4g=; b=dIU6KHTm39RKxiBHU0T+YeuzCaf/HDDm5/HRGkIoxa/51nYTAdGTYOi+vpiDefj8sF Wn7EKRDS4OZ7PPTQq7/38rPd1FrfRO6vuW+ujc4rggi8bAV3B9XiL1IjqW8Hb5NA9Lfw 2fL39Qq728L43893xTYK4SmPbe/z5Ycr9WKBlrAMPRPftoOSOhkr+KKX8KncWM7PdeTx 0CQeFtJKSgHX/kqK+7z4hR6osnI+RL28mH4eb/I1fEgFOtiiIpgKLEBjPLPwxEwv/NGC DEl9yKq+F6djBYmAHbExH1vHQ3h6NwQnaa3JPf9+ymxyMhvueD1is68ndjND3pj9/Dzf H2+g== X-Gm-Message-State: APjAAAXVIQH96yL5QdIDJNwD8+b3Pio2twyNRXVSTM8+PmjRswiUgBFH vDQkYtHIgPQymzvSxPDs+2MYMRnoUMeNDw== X-Google-Smtp-Source: APXvYqwikwoMjFKo4QmUHtiK/s/ubEJzHvpRQ/OnIaJZIn4YovBFOy+k5UYaOrGNMT65OyZSATMH+Q== X-Received: by 2002:adf:8294:: with SMTP id 20mr4081761wrc.223.1551304835447; Wed, 27 Feb 2019 14:00:35 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id n14sm20586073wrx.24.2019.02.27.14.00.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Feb 2019 14:00:34 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Wed, 27 Feb 2019 22:00:22 +0000 Message-Id: <20190227220023.16040-4-sw@jkqxz.net> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20190227220023.16040-1-sw@jkqxz.net> References: <20190227220023.16040-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/5] vaapi_encode: Add ROI support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/vaapi_encode.c | 129 ++++++++++++++++++++++++++++++++ libavcodec/vaapi_encode.h | 9 +++ libavcodec/vaapi_encode_h264.c | 2 + libavcodec/vaapi_encode_h265.c | 2 + libavcodec/vaapi_encode_mpeg2.c | 2 + libavcodec/vaapi_encode_vp8.c | 2 + libavcodec/vaapi_encode_vp9.c | 2 + 7 files changed, 148 insertions(+) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 2dda451882..1865006c1a 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -143,6 +143,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, int err, i; char data[MAX_PARAM_BUFFER_SIZE]; size_t bit_len; + AVFrameSideData *sd; av_log(avctx, AV_LOG_DEBUG, "Issuing encode for pic %"PRId64"/%"PRId64" " "as type %s.\n", pic->display_order, pic->encode_order, @@ -412,6 +413,92 @@ static int vaapi_encode_issue(AVCodecContext *avctx, } } + sd = av_frame_get_side_data(pic->input_image, + AV_FRAME_DATA_REGIONS_OF_INTEREST); + +#if VA_CHECK_VERSION(1, 0, 0) + if (sd && ctx->roi_allowed) { + const AVRegionOfInterest *roi; + int nb_roi; + VAEncMiscParameterBuffer param_misc = { + .type = VAEncMiscParameterTypeROI, + }; + VAEncMiscParameterBufferROI param_roi; + // VAAPI requires the second structure to immediately follow the + // first in the parameter buffer, but they have different natural + // alignment on 64-bit architectures (4 vs. 8, since the ROI + // structure contains a pointer). This means we can't make a + // single working structure, nor can we make the buffer + // separately and map it because dereferencing the second pointer + // would be undefined. Therefore, construct the two parts + // separately and combine them into a single character buffer + // before uploading. + char param_buffer[sizeof(param_misc) + sizeof(param_roi)]; + int i, v; + + roi = (const AVRegionOfInterest*)sd->data; + av_assert0(roi->self_size && sd->size % roi->self_size == 0); + nb_roi = sd->size / roi->self_size; + if (nb_roi > ctx->roi_regions) { + if (!ctx->roi_warned) { + av_log(avctx, AV_LOG_WARNING, "More ROIs set than " + "supported by driver (%d > %d).\n", + nb_roi, ctx->roi_regions); + ctx->roi_warned = 1; + } + nb_roi = ctx->roi_regions; + } + + pic->roi = av_mallocz_array(nb_roi, sizeof(*pic->roi)); + if (!pic->roi) { + err = AVERROR(ENOMEM); + goto fail; + } + for (i = 0; i < nb_roi; i++) { + roi = (const AVRegionOfInterest*)(sd->data + roi->self_size * i); + + v = roi->qoffset.num * ctx->roi_quant_range / roi->qoffset.den; + av_log(avctx, AV_LOG_DEBUG, "ROI region: (%d-%d,%d-%d) -> %+d.\n", + roi->left, avctx->width - roi->right, + roi->top, avctx->height - roi->bottom, v); + + pic->roi[i] = (VAEncROI) { + .roi_rectangle = { + .x = roi->left, + .y = roi->top, + .width = avctx->width - roi->right - roi->left, + .height = avctx->height - roi->bottom - roi->top, + }, + .roi_value = av_clip_c(v, INT8_MIN, INT8_MAX), + }; + } + + param_roi = (VAEncMiscParameterBufferROI) { + .num_roi = nb_roi, + .max_delta_qp = INT8_MAX, + .min_delta_qp = 0, + .roi = pic->roi, + .roi_flags.bits.roi_value_is_qp_delta = 1, + }; + + memcpy(param_buffer, ¶m_misc, sizeof(param_misc)); + memcpy(param_buffer + sizeof(param_misc), + ¶m_roi, sizeof(param_roi)); + + err = vaapi_encode_make_param_buffer(avctx, pic, + VAEncMiscParameterBufferType, + param_buffer, + sizeof(param_buffer)); + if (err < 0) + goto fail; + } else +#endif + if (sd && !ctx->roi_warned) { + av_log(avctx, AV_LOG_WARNING, "ROI side data on input " + "frames ignored due to lack of driver support.\n"); + ctx->roi_warned = 1; + } + vas = vaBeginPicture(ctx->hwctx->display, ctx->va_context, pic->input_surface); if (vas != VA_STATUS_SUCCESS) { @@ -477,6 +564,7 @@ fail_at_end: av_freep(&pic->codec_picture_params); av_freep(&pic->param_buffers); av_freep(&pic->slices); + av_freep(&pic->roi); av_frame_free(&pic->recon_image); av_buffer_unref(&pic->output_buffer_ref); pic->output_buffer = VA_INVALID_ID; @@ -611,6 +699,7 @@ static int vaapi_encode_free(AVCodecContext *avctx, av_freep(&pic->priv_data); av_freep(&pic->codec_picture_params); + av_freep(&pic->roi); av_free(pic); @@ -1899,6 +1988,42 @@ static av_cold int vaapi_encode_init_quality(AVCodecContext *avctx) return 0; } +static av_cold int vaapi_encode_init_roi(AVCodecContext *avctx) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + +#if VA_CHECK_VERSION(1, 0, 0) + VAStatus vas; + VAConfigAttrib attr = { VAConfigAttribEncROI }; + + vas = vaGetConfigAttributes(ctx->hwctx->display, + ctx->va_profile, + ctx->va_entrypoint, + &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to query ROI " + "config attribute: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR_EXTERNAL; + } + + if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { + ctx->roi_allowed = 0; + } else { + VAConfigAttribValEncROI roi = { + .value = attr.value, + }; + + ctx->roi_regions = roi.bits.num_roi_regions; + ctx->roi_allowed = ctx->roi_regions > 0 && + (ctx->va_rc_mode == VA_RC_CQP || + roi.bits.roi_rc_qp_delta_support); + } +#else + ctx->roi_allowed = 0; +#endif + return 0; +} + static void vaapi_encode_free_output_buffer(void *opaque, uint8_t *data) { @@ -2089,6 +2214,10 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) if (err < 0) goto fail; + err = vaapi_encode_init_roi(avctx); + if (err < 0) + goto fail; + if (avctx->compression_level >= 0) { err = vaapi_encode_init_quality(avctx); if (err < 0) diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 44a8db566e..be99b80989 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -69,6 +69,9 @@ typedef struct VAAPIEncodePicture { int64_t pts; int force_idr; + // ROI regions. + VAEncROI *roi; + int type; int b_depth; int encode_issued; @@ -314,6 +317,12 @@ typedef struct VAAPIEncodeContext { int idr_counter; int gop_counter; int end_of_stream; + + // ROI state. + int roi_allowed; + int roi_regions; + int roi_quant_range; + int roi_warned; } VAAPIEncodeContext; enum { diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index 91be33f99f..7c55b8a4a7 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -1123,6 +1123,8 @@ static av_cold int vaapi_encode_h264_configure(AVCodecContext *avctx) } } + ctx->roi_quant_range = 51 + 6 * (ctx->profile->depth - 8); + return 0; } diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 758bd40a37..538862a9d5 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -1102,6 +1102,8 @@ static av_cold int vaapi_encode_h265_configure(AVCodecContext *avctx) priv->fixed_qp_b = 30; } + ctx->roi_quant_range = 51 + 6 * (ctx->profile->depth - 8); + return 0; } diff --git a/libavcodec/vaapi_encode_mpeg2.c b/libavcodec/vaapi_encode_mpeg2.c index fb1ef71fdc..bac9ea1fa6 100644 --- a/libavcodec/vaapi_encode_mpeg2.c +++ b/libavcodec/vaapi_encode_mpeg2.c @@ -552,6 +552,8 @@ static av_cold int vaapi_encode_mpeg2_configure(AVCodecContext *avctx) ctx->nb_slices = ctx->slice_block_rows; ctx->slice_size = 1; + ctx->roi_quant_range = 31; + return 0; } diff --git a/libavcodec/vaapi_encode_vp8.c b/libavcodec/vaapi_encode_vp8.c index ddbe4c9075..6e7bf9d106 100644 --- a/libavcodec/vaapi_encode_vp8.c +++ b/libavcodec/vaapi_encode_vp8.c @@ -173,6 +173,8 @@ static av_cold int vaapi_encode_vp8_configure(AVCodecContext *avctx) else priv->q_index_i = priv->q_index_p; + ctx->roi_quant_range = VP8_MAX_QUANT; + return 0; } diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index f89fd0d07a..d7f415d704 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -202,6 +202,8 @@ static av_cold int vaapi_encode_vp9_configure(AVCodecContext *avctx) priv->q_idx_idr = priv->q_idx_p = priv->q_idx_b = 100; } + ctx->roi_quant_range = VP9_MAX_QUANT; + return 0; }