[FFmpeg-devel] avutil: add API for mb_types

Submitted by Michael Niedermayer on Nov. 11, 2017, 11:21 p.m.

Details

Message ID 20171111232154.25593-1-michael@niedermayer.cc
State New
Headers show

Commit Message

Michael Niedermayer Nov. 11, 2017, 11:21 p.m.
This is based on motion_type.h

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavutil/block_type.h | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 89 insertions(+)
 create mode 100644 libavutil/block_type.h

Comments

Ronald S. Bultje Nov. 12, 2017, 1:44 a.m.
Hi,

On Sat, Nov 11, 2017 at 6:21 PM, Michael Niedermayer <michael@niedermayer.cc
> wrote:

> +    /**
> +     * Width and height of the block.
> +     */
> +    uint8_t w, h;
>

av1 already has up to 128x128 block sizes, so I'm assuming this won't be
enough for av2. Maybe that's Ok, but it's easy to make it (u)int16_t now.

+    /**
> +     * Quantization parameter, a value of -128 means lossless, 127 means
> not applicable
> +     */
> +    int8_t qp;
>

vp8, vp9 and av1 have 256 quantizers.


> +    /**
> +     * Direction of prediction, 0 is horizontal, 48 is vertical,
> +     * values 0 to 95 follow clockwise direction.
> +     * 254 means the block uses planar prediction
> +     * 255 means that the block has no prediction direction.
> +     */
> +    uint8_t pred_direction;
>

Assuming this is intra - I think we want some more specificity here, e.g.
is 255 DC? What about tm (vp8/9) or smooth_pred (av1). What about blocks
that used mixed inter+intra pred (av1)?

+    /**
> +     * Direction of transform, 0 is horizontal, 48 is vertical,
> +     * values 0 to 95 follow clockwise direction.
> +     * 255 means that the block has no directional transform.
> +     */
> +    uint8_t transform_direction;
>

Av1, hevc and vp9 also have the concept of transform type.

For a lot of this, I think we want to be more specific and explicit how
these magic numbers (255, 254, -127) are mapped to concepts in
hevc/h264/av1/vp9. People using this API will want to use it for codecs
that are popular now. If we don't specify what it means for these codecs,
it'll be obsolete and useless before it's even introduced.

Ronald
Rostislav Pehlivanov Nov. 12, 2017, 1:55 a.m.
On 12 November 2017 at 01:44, Ronald S. Bultje <rsbultje@gmail.com> wrote:

> Hi,
>
> On Sat, Nov 11, 2017 at 6:21 PM, Michael Niedermayer
> <michael@niedermayer.cc
> > wrote:
>
> > +    /**
> > +     * Width and height of the block.
> > +     */
> > +    uint8_t w, h;
> >
>
> av1 already has up to 128x128 block sizes, so I'm assuming this won't be
> enough for av2. Maybe that's Ok, but it's easy to make it (u)int16_t now.
>
> +    /**
> > +     * Quantization parameter, a value of -128 means lossless, 127 means
> > not applicable
> > +     */
> > +    int8_t qp;
> >
>
> vp8, vp9 and av1 have 256 quantizers.
>
>
I agree, I think this should be a uint16_t with 0 defined as lossless and
UINT16_MAX as N/A.
The range should be derived from the codec ID.




> > +    /**
> > +     * Direction of prediction, 0 is horizontal, 48 is vertical,
> > +     * values 0 to 95 follow clockwise direction.
> > +     * 254 means the block uses planar prediction
> > +     * 255 means that the block has no prediction direction.
> > +     */
> > +    uint8_t pred_direction;
> >
>
> Assuming this is intra - I think we want some more specificity here, e.g.
> is 255 DC? What about tm (vp8/9) or smooth_pred (av1). What about blocks
> that used mixed inter+intra pred (av1)?
>
> +    /**
> > +     * Direction of transform, 0 is horizontal, 48 is vertical,
> > +     * values 0 to 95 follow clockwise direction.
> > +     * 255 means that the block has no directional transform.
> > +     */
> > +    uint8_t transform_direction;
> >
>
> Av1, hevc and vp9 also have the concept of transform type.
>


Yeah, I think this should be renamed to "transform_type" and interpreted
based on codec ID.
Clément Bœsch Nov. 12, 2017, 9:18 a.m.
On Sun, Nov 12, 2017 at 01:55:14AM +0000, Rostislav Pehlivanov wrote:
[...]
> The range should be derived from the codec ID.

The frame doesn't contain the codec ID, so this information should be
added to that struct somehow (it needs to reach the filters who are
agnostics).

[...]
Moritz Barsnick Nov. 13, 2017, 9:30 a.m.
On Sun, Nov 12, 2017 at 00:21:54 +0100, Michael Niedermayer wrote:

Some nitpicking, since it's doxygen:

> +#define AV_BLOCK_TYPE_FLAG_INTRA      0x0000001        ///< Block is a intra block, else inter
                                                                          ^ an
> +#define AV_BLOCK_TYPE_FLAG_SKIP       0x0000008        ///< Block is skiped
                                                                        ^ skipped
> +#define AV_BLOCK_TYPE_FLAG_CONCEALED  0x0010000        ///< Block has been replaced by error conclealment
                                                                                                ^ concealment

Moritz

Patch hide | download patch | download mbox

diff --git a/libavutil/block_type.h b/libavutil/block_type.h
new file mode 100644
index 0000000000..1531c7b420
--- /dev/null
+++ b/libavutil/block_type.h
@@ -0,0 +1,89 @@ 
+/*
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVUTIL_BLOCK_TYPE_H
+#define AVUTIL_BLOCK_TYPE_H
+
+#include <stdint.h>
+
+typedef struct AVBlockType {
+    /**
+     * Extra flag information.
+     */
+    uint64_t flags;
+#define AV_BLOCK_TYPE_FLAG_INTRA      0x0000001        ///< Block is a intra block, else inter
+
+#define AV_BLOCK_TYPE_FLAG_BIDIR      0x0000002        ///< Block uses bidirectional motion compensation
+#define AV_BLOCK_TYPE_FLAG_GMC        0x0000004        ///< Block uses global motion compensation
+#define AV_BLOCK_TYPE_FLAG_SKIP       0x0000008        ///< Block is skiped
+#define AV_BLOCK_TYPE_FLAG_CMV        0x0000010        ///< Block has concealment motion vector(s)
+#define AV_BLOCK_TYPE_FLAG_LOOP       0x0000020        ///< Block uses a loop filter
+#define AV_BLOCK_TYPE_FLAG_WEIGHT     0x0000040        ///< Block uses weighted prediction
+#define AV_BLOCK_TYPE_FLAG_OBMC       0x0000080        ///< Block uses overlapped block MC
+
+#define AV_BLOCK_TYPE_FLAG_DCPRED     0x0000100        ///< Block uses spatial DC prediction
+#define AV_BLOCK_TYPE_FLAG_ACPRED     0x0000200        ///< Block uses spatial AC prediction
+#define AV_BLOCK_TYPE_FLAG_PCM        0x0000400        ///< Block is PCM coded
+#define AV_BLOCK_TYPE_FLAG_IPRED      0x0000800        ///< Block uses interlaced prediction
+
+#define AV_BLOCK_TYPE_FLAG_LEAF       0x0001000        ///< Block is a leaf block, it is not further split
+#define AV_BLOCK_TYPE_FLAG_ROOT       0x0002000        ///< Block is a root block, it is not contained in a larger block
+
+#define AV_BLOCK_TYPE_FLAG_CONCEALED  0x0010000        ///< Block has been replaced by error conclealment
+#define AV_BLOCK_TYPE_FLAG_DC_DAMAGED 0x0020000        ///< Block has damaged DC coeffs
+#define AV_BLOCK_TYPE_FLAG_AC_DAMAGED 0x0040000        ///< Block has damaged AC coeffs
+#define AV_BLOCK_TYPE_FLAG_MV_DAMAGED 0x0080000        ///< Block has damaged MV
+#define AV_BLOCK_TYPE_FLAG_CODED_DC   0x0100000        ///< Block has coded DC coefficients
+#define AV_BLOCK_TYPE_FLAG_CODED_AC   0x0200000        ///< Block has coded AC coefficients
+#define AV_BLOCK_TYPE_FLAG_CODED_MV   0x0400000        ///< Block has coded MV diff
+
+#define AV_BLOCK_TYPE_FLAG_ITRANSFORM 0x0800000        ///< Block uses interlaced transform
+
+
+    /**
+     * Left Top corner position.
+     */
+    uint16_t x, y;
+    /**
+     * Width and height of the block.
+     */
+    uint8_t w, h;
+    /**
+     * Quantization parameter, a value of -128 means lossless, 127 means not applicable
+     */
+    int8_t qp;
+    /**
+     * Direction of prediction, 0 is horizontal, 48 is vertical,
+     * values 0 to 95 follow clockwise direction.
+     * 254 means the block uses planar prediction
+     * 255 means that the block has no prediction direction.
+     */
+    uint8_t pred_direction;
+    /**
+     * Direction of transform, 0 is horizontal, 48 is vertical,
+     * values 0 to 95 follow clockwise direction.
+     * 255 means that the block has no directional transform.
+     */
+    uint8_t transform_direction;
+    /**
+     * Number identifying the slice the MB is in.
+     */
+    uint8_t slice_num;
+} AVBlockType;
+
+#endif /* AVUTIL_BLOCK_TYPE_H */