From patchwork Fri Oct 2 18:26:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 22694 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1F33344B263 for ; Fri, 2 Oct 2020 21:26:34 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E8806689CFF; Fri, 2 Oct 2020 21:26:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0F089680812 for ; Fri, 2 Oct 2020 21:26:27 +0300 (EEST) Received: by mail-qt1-f176.google.com with SMTP id d1so2156096qtr.6 for ; Fri, 02 Oct 2020 11:26:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=rDKjC8911iIWxq22wRE/EidKyoiV7/12Dd/eGMskQ/E=; b=A7zZpQ1XY+HqZhkMeDk37KdOCVtL2Nn713ig4ZMFMC83WDIe9cQuHXTCI0HViaXWaQ 0HQ/D8kys/SxPJJefXNp63+WcJ6lOWhFTfAzaqlxrNhf4gibEtX+NaDV37KXKSnJJcb0 CaPtOFECygeFX5/oyptsQ342m5ufgA8fg5oKcu1CUX/yWkCkf/CCTp4KsA3ZJQ46pC6J Zvj6KHaGhU2+M9OlWAdfFLLlKezJgZDe8h83b6KG79aHx5LkrSfBnwj52jmEirmhGxKT RrFE1hzQk8DsGFfXON9F9aJ6zude5QIsgs8ErsC243cFuHI9ZilfVshQUCHbLEc56i9K KKzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rDKjC8911iIWxq22wRE/EidKyoiV7/12Dd/eGMskQ/E=; b=G7egH1YhZD0UJnaRz/S/HgO1zDXsr+UGXd7hvnrs21fhtu0JN5Nyhy4OcCSgcF46kO FeQ+3Qr81+r1undUkhDoBtZNmuK63h9xSx0Knp2PKpOgEXGU9k1aOnNrDhIckkoZwQmu 7XfX/jtZ5cSNIlHM+XMu4W9LBMCLjKCDaxT0uJiHLFY+abZ3Z/TimybjyA0AKM4nbJdp b6xu9iu7fZza6BlQWWwrQKB4wurRQxBbn2L2l835ZVe3w5uvoV+v3JDbgnkEt61ixYYc 82DuPBpUliAmLaq1YuculPvInbjJos9zO5YnvJZsJxoMJvUEb0uWy5LbL6Semwia6td1 eSfw== X-Gm-Message-State: AOAM533TR7hBfItWnIuZC1CdqmKK8OHJJ9By1NZOfqHu722cSzaywdAj 0g29aTDDhXWmRFKnV7sOc2NJ+PCgxpY= X-Google-Smtp-Source: ABdhPJwowEbW6u6iTJcwjzbNvoe3zujEVw29LxM4quVdPldPw10SFnLLK679T0YBHnRvnMrZYsiPeA== X-Received: by 2002:ac8:5d4d:: with SMTP id g13mr3665350qtx.348.1601663184970; Fri, 02 Oct 2020 11:26:24 -0700 (PDT) Received: from localhost.localdomain ([181.23.78.153]) by smtp.gmail.com with ESMTPSA id h26sm1653222qtm.9.2020.10.02.11.26.23 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 11:26:24 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Fri, 2 Oct 2020 15:26:10 -0300 Message-Id: <20201002182610.61516-1-jamrial@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/4 v2] avcodec/cbs_av1: add an option to select an operating point X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This implements the function drop_obu() as defined in Setion 6.2.1 from the spec. In a reading only scenario, units that belong to an operating point the caller doesn't want should not be parsed. Signed-off-by: James Almer --- libavcodec/cbs_av1.c | 30 +++++++++++++++++++++++++++++- libavcodec/cbs_av1.h | 5 +++++ 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c index 65cf45fcfa..753e089e8f 100644 --- a/libavcodec/cbs_av1.c +++ b/libavcodec/cbs_av1.c @@ -18,6 +18,7 @@ #include "libavutil/avassert.h" #include "libavutil/pixfmt.h" +#include "libavutil/opt.h" #include "cbs.h" #include "cbs_internal.h" @@ -883,7 +884,7 @@ static int cbs_av1_read_unit(CodedBitstreamContext *ctx, int in_spatial_layer = (priv->operating_point_idc >> (priv->spatial_id + 8)) & 1; if (!in_temporal_layer || !in_spatial_layer) { - // Decoding will drop this OBU at this operating point. + return AVERROR(EAGAIN); // drop_obu() } } } @@ -896,6 +897,18 @@ static int cbs_av1_read_unit(CodedBitstreamContext *ctx, if (err < 0) return err; + if (priv->operating_point >= 0) { + AV1RawSequenceHeader *sequence_header = &obu->obu.sequence_header; + + if (priv->operating_point > sequence_header->operating_points_cnt_minus_1) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid Operating Point %d requested. " + "Must not be higher than %u.\n", + priv->operating_point, sequence_header->operating_points_cnt_minus_1); + return AVERROR(EINVAL); + } + priv->operating_point_idc = sequence_header->operating_point_idc[priv->operating_point]; + } + av_buffer_unref(&priv->sequence_header_ref); priv->sequence_header = NULL; @@ -1251,9 +1264,24 @@ static const CodedBitstreamUnitTypeDescriptor cbs_av1_unit_types[] = { CBS_UNIT_TYPE_END_OF_LIST }; +#define OFFSET(x) offsetof(CodedBitstreamAV1Context, x) +static const AVOption cbs_av1_options[] = { + { "operating_point", "Set operating point to select layers to parse from a scalable bitstream", + OFFSET(operating_point), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AV1_MAX_OPERATING_POINTS - 1, 0 }, + { NULL } +}; + +static const AVClass cbs_av1_class = { + .class_name = "cbs_av1", + .item_name = av_default_item_name, + .option = cbs_av1_options, + .version = LIBAVUTIL_VERSION_INT, +}; + const CodedBitstreamType ff_cbs_type_av1 = { .codec_id = AV_CODEC_ID_AV1, + .priv_class = &cbs_av1_class, .priv_data_size = sizeof(CodedBitstreamAV1Context), .unit_types = cbs_av1_unit_types, diff --git a/libavcodec/cbs_av1.h b/libavcodec/cbs_av1.h index 7a0c08c596..27b44d68ff 100644 --- a/libavcodec/cbs_av1.h +++ b/libavcodec/cbs_av1.h @@ -416,6 +416,8 @@ typedef struct AV1ReferenceFrameState { } AV1ReferenceFrameState; typedef struct CodedBitstreamAV1Context { + const AVClass *class; + AV1RawSequenceHeader *sequence_header; AVBufferRef *sequence_header_ref; @@ -443,6 +445,9 @@ typedef struct CodedBitstreamAV1Context { int tile_rows; AV1ReferenceFrameState ref[AV1_NUM_REF_FRAMES]; + + // AVOptions + int operating_point; } CodedBitstreamAV1Context; From patchwork Fri Oct 2 18:37:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 22695 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 060A9449EC2 for ; Fri, 2 Oct 2020 21:37:34 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DD052689C95; Fri, 2 Oct 2020 21:37:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 617EC6880B1 for ; Fri, 2 Oct 2020 21:37:27 +0300 (EEST) Received: by mail-qt1-f176.google.com with SMTP id d1so2221682qtr.6 for ; Fri, 02 Oct 2020 11:37:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=L+z69+bDVsqQ5PPQOA9sftEpW/7aTGCEzTYyk02GmEc=; b=gNIF/jbgujjbItSZYC+x2hlIh0cqNBZZbyz0OfgzwYpdJm+5jVNHA2Qxr8BI+yLzxn RO5TUFAxpNOpwi/I00A6vXrBJnUG21bVrlX8Ff1fP5odNw0gU8k8OEKdQkj8Q0y18jWD JMMC7o8TFLoZDJBCz29Xt6LxikVetxh0yx3dPSvA2nAtTUzuxM/HzQDYz4cfiSe7BdQi QDVqbNdTvZ4Wa3GkFN1/m54gY88jGnoWdt7RCa3XoDkTF5+PjBzM+dnJzn5hleiOPkut OxQjWlRJgsYacbOQm3qZpBXcykhMpiJT+EZ1oXE3F9miLFRL7qyWdLu2+HO2ZP3DEILs 49YQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L+z69+bDVsqQ5PPQOA9sftEpW/7aTGCEzTYyk02GmEc=; b=a9L79wXzDdzj5RQVXXvGtHebyGqyok5LSjdUxCBWgt6rSLzeXyYnov4KHLjMbnKBQa slmOGSXVW4jt4iLTsdd6aLHguKRuSV7N9sQYpEBXu2Lj/A6GZuC/w3DETo46UxKeLlfa naq2d1Fp/cpSArr7WLhjLCxUwKpDt6W6i+W2XPgLk9V5Trg9J4Ls+R4do6TZtUaPEUy/ tfDLLP2UAWNN+yg1AFdvA+q4Ngx2HBvt0kbnZM9yZso3NWlxwBbROKhI0Rn6lFk0+Vct K+xdZV8ny3X0wcDHglSTWAm8/Q+7gYO53MR26NffiofAIEeeb/oXEHEt6B0blV3rxOK6 tCsQ== X-Gm-Message-State: AOAM5320LgRFTDU3SGMroCdVjyHJuh2d7C9vqYQQsRI8A8cHy7vzvxCl obDmz3vnoWYElHVsZPT48DdbXxnSOAk= X-Google-Smtp-Source: ABdhPJw/itxDT9VV/BH3QXeaTfWxvqnJ7hLwGsGNCtuuw5V+YT5q2lNerzXEaOp5Rxv7Eer+Yy926w== X-Received: by 2002:ac8:224f:: with SMTP id p15mr3756844qtp.300.1601663845725; Fri, 02 Oct 2020 11:37:25 -0700 (PDT) Received: from localhost.localdomain ([181.23.78.153]) by smtp.gmail.com with ESMTPSA id f8sm1596331qtx.81.2020.10.02.11.37.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 11:37:25 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Fri, 2 Oct 2020 15:37:08 -0300 Message-Id: <20201002183708.61996-1-jamrial@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200920172443.4763-4-jamrial@gmail.com> References: <20200920172443.4763-4-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/4 v2] avcodec/av1dec: add an option to select an operating point X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: James Almer --- doc/decoders.texi | 13 +++++++++++++ libavcodec/av1dec.c | 34 ++++++++++++++++++++++++++++++++++ libavcodec/av1dec.h | 7 +++++++ 3 files changed, 54 insertions(+) diff --git a/doc/decoders.texi b/doc/decoders.texi index 8649f83e28..667941652c 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -25,6 +25,19 @@ enabled decoders. A description of some of the currently available video decoders follows. +@section av1 + +AOMedia Video 1 (AV1) decoder. + +@subsection Options + +@table @option + +@item operating_point +Select an operating point of a scalable AV1 bitstream (0 - 31). Default is 0. + +@end table + @section rawvideo Raw video decoder. diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index 07026b7aeb..eff2fc1232 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -19,6 +19,7 @@ */ #include "libavutil/pixdesc.h" +#include "libavutil/opt.h" #include "avcodec.h" #include "av1dec.h" #include "bytestream.h" @@ -343,6 +344,7 @@ static void av1_frame_unref(AVCodecContext *avctx, AV1Frame *f) ff_thread_release_buffer(avctx, &f->tf); av_buffer_unref(&f->hwaccel_priv_buf); f->hwaccel_picture_private = NULL; + f->spatial_id = f->temporal_id = 0; } static int av1_frame_ref(AVCodecContext *avctx, AV1Frame *dst, const AV1Frame *src) @@ -360,6 +362,8 @@ static int av1_frame_ref(AVCodecContext *avctx, AV1Frame *dst, const AV1Frame *s dst->hwaccel_picture_private = dst->hwaccel_priv_buf->data; } + dst->spatial_id = src->spatial_id; + dst->temporal_id = src->temporal_id; dst->loop_filter_delta_enabled = src->loop_filter_delta_enabled; memcpy(dst->loop_filter_ref_deltas, src->loop_filter_ref_deltas, @@ -505,6 +509,8 @@ static av_cold int av1_decode_init(AVCodecContext *avctx) if (ret < 0) return ret; + av_opt_set_int(s->cbc->priv_data, "operating_point", s->operating_point, 0); + if (avctx->extradata && avctx->extradata_size) { ret = ff_cbs_read(s->cbc, &s->current_obu, avctx->extradata, avctx->extradata_size); @@ -588,6 +594,11 @@ static int set_output_frame(AVCodecContext *avctx, AVFrame *frame, const AVFrame *srcframe = s->cur_frame.tf.f; int ret; + // TODO: all layers + if (s->operating_point_idc && + av_log2(s->operating_point_idc >> 8) > s->cur_frame.spatial_id) + return 0; + ret = av_frame_ref(frame, srcframe); if (ret < 0) return ret; @@ -670,6 +681,7 @@ static int av1_decode_frame(AVCodecContext *avctx, void *frame, for (int i = 0; i < s->current_obu.nb_units; i++) { CodedBitstreamUnit *unit = &s->current_obu.units[i]; AV1RawOBU *obu = unit->content; + const AV1RawOBUHeader *header = &obu->header; av_log(avctx, AV_LOG_DEBUG, "Obu idx:%d, obu type:%d.\n", i, unit->type); switch (unit->type) { @@ -689,6 +701,8 @@ static int av1_decode_frame(AVCodecContext *avctx, void *frame, goto end; } + s->operating_point_idc = s->raw_seq->operating_point_idc[s->operating_point]; + if (s->pix_fmt == AV_PIX_FMT_NONE) { ret = get_pixel_format(avctx); if (ret < 0) { @@ -763,6 +777,9 @@ static int av1_decode_frame(AVCodecContext *avctx, void *frame, goto end; } + s->cur_frame.spatial_id = header->spatial_id; + s->cur_frame.temporal_id = header->temporal_id; + if (avctx->hwaccel) { ret = avctx->hwaccel->start_frame(avctx, unit->data, unit->data_size); @@ -852,12 +869,28 @@ static void av1_decode_flush(AVCodecContext *avctx) av1_frame_unref(avctx, &s->ref[i]); av1_frame_unref(avctx, &s->cur_frame); + s->operating_point_idc = 0; s->raw_frame_header = NULL; s->raw_seq = NULL; ff_cbs_flush(s->cbc); } +#define OFFSET(x) offsetof(AV1DecContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM +static const AVOption av1_options[] = { + { "operating_point", "Select an operating point of the scalable bitstream", + OFFSET(operating_point), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, AV1_MAX_OPERATING_POINTS - 1, VD }, + { NULL } +}; + +static const AVClass av1_class = { + .class_name = "AV1 decoder", + .item_name = av_default_item_name, + .option = av1_options, + .version = LIBAVUTIL_VERSION_INT, +}; + AVCodec ff_av1_decoder = { .name = "av1", .long_name = NULL_IF_CONFIG_SMALL("Alliance for Open Media AV1"), @@ -873,6 +906,7 @@ AVCodec ff_av1_decoder = { FF_CODEC_CAP_SETS_PKT_DTS, .flush = av1_decode_flush, .profiles = NULL_IF_CONFIG_SMALL(ff_av1_profiles), + .priv_class = &av1_class, .hw_configs = (const AVCodecHWConfigInternal * []) { NULL }, diff --git a/libavcodec/av1dec.h b/libavcodec/av1dec.h index 3604870299..67ae4c2c9f 100644 --- a/libavcodec/av1dec.h +++ b/libavcodec/av1dec.h @@ -36,6 +36,9 @@ typedef struct AV1Frame { AVBufferRef *hwaccel_priv_buf; void *hwaccel_picture_private; + int temporal_id; + int spatial_id; + uint8_t loop_filter_delta_enabled; int8_t loop_filter_ref_deltas[AV1_NUM_REF_FRAMES]; int8_t loop_filter_mode_deltas[2]; @@ -67,9 +70,13 @@ typedef struct AV1DecContext { uint16_t tg_start; uint16_t tg_end; + int operating_point_idc; + AV1Frame ref[AV1_NUM_REF_FRAMES]; AV1Frame cur_frame; + // AVOptions + int operating_point; } AV1DecContext; #endif /* AVCODEC_AV1DEC_H */