diff mbox series

[FFmpeg-devel,1/9] libavutil: add API for exporting video frame quantizers

Message ID 20200418101415.26409-2-anton@khirnov.net
State Accepted
Headers show
Series [FFmpeg-devel,1/9] libavutil: add API for exporting video frame quantizers
Related show

Checks

Context Check Description
andriy/default pending
andriy/make_warn warning New warnings during build
andriy/make success Make finished
andriy/make_fate success Make fate finished

Commit Message

Anton Khirnov April 18, 2020, 10:14 a.m. UTC
From: Juan De León <juandl-at-google.com@ffmpeg.org>

This is intended to replace the deprecated the AV_FRAME_DATA_QP_TABLE*
API and extend it to a wider range of codecs.

In the future, it may also be extended to support other encoding
parameters such as motion vectors.

Additional changes by Anton Khirnov <anton@khirnov.net> with suggestions
by Lynne <dev@lynne.ee>.

Signed-off-by: Juan De León <juandl@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
---
 doc/APIchanges               |   4 ++
 libavutil/Makefile           |   5 +-
 libavutil/frame.c            |   1 +
 libavutil/frame.h            |   5 ++
 libavutil/version.h          |   4 +-
 libavutil/video_enc_params.c |  79 ++++++++++++++++++++
 libavutil/video_enc_params.h | 135 +++++++++++++++++++++++++++++++++++
 7 files changed, 230 insertions(+), 3 deletions(-)
 create mode 100644 libavutil/video_enc_params.c
 create mode 100644 libavutil/video_enc_params.h

Comments

Lynne April 18, 2020, 12:01 p.m. UTC | #1
Apr 18, 2020, 11:14 by anton@khirnov.net:

> From: Juan De León <juandl-at-google.com@ffmpeg.org>
>
> This is intended to replace the deprecated the AV_FRAME_DATA_QP_TABLE*
> API and extend it to a wider range of codecs.
>
> In the future, it may also be extended to support other encoding
> parameters such as motion vectors.
>
> Additional changes by Anton Khirnov <anton@khirnov.net> with suggestions
> by Lynne <dev@lynne.ee>.
>
> Signed-off-by: Juan De León <juandl@google.com>
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> Signed-off-by: Anton Khirnov <anton@khirnov.net>
>

API looks good, patchset LGTM.
Thierry Foucu May 7, 2020, 7:14 p.m. UTC | #2
On Sat, Apr 18, 2020 at 5:02 AM Lynne <dev@lynne.ee> wrote:

> Apr 18, 2020, 11:14 by anton@khirnov.net:
>
> > From: Juan De León <juandl-at-google.com@ffmpeg.org>
> >
> > This is intended to replace the deprecated the AV_FRAME_DATA_QP_TABLE*
> > API and extend it to a wider range of codecs.
> >
> > In the future, it may also be extended to support other encoding
> > parameters such as motion vectors.
> >
> > Additional changes by Anton Khirnov <anton@khirnov.net> with suggestions
> > by Lynne <dev@lynne.ee>.
> >
> > Signed-off-by: Juan De León <juandl@google.com>
> > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > Signed-off-by: Anton Khirnov <anton@khirnov.net>
> >
>
> API looks good, patchset LGTM.
>

Any chance to have this patch set applied?

Thanks



> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Anton Khirnov May 11, 2020, 9:25 a.m. UTC | #3
Quoting Thierry Foucu (2020-05-07 21:14:15)
> On Sat, Apr 18, 2020 at 5:02 AM Lynne <dev@lynne.ee> wrote:
> 
> > Apr 18, 2020, 11:14 by anton@khirnov.net:
> >
> > > From: Juan De León <juandl-at-google.com@ffmpeg.org>
> > >
> > > This is intended to replace the deprecated the AV_FRAME_DATA_QP_TABLE*
> > > API and extend it to a wider range of codecs.
> > >
> > > In the future, it may also be extended to support other encoding
> > > parameters such as motion vectors.
> > >
> > > Additional changes by Anton Khirnov <anton@khirnov.net> with suggestions
> > > by Lynne <dev@lynne.ee>.
> > >
> > > Signed-off-by: Juan De León <juandl@google.com>
> > > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > > Signed-off-by: Anton Khirnov <anton@khirnov.net>
> > >
> >
> > API looks good, patchset LGTM.
> >
> 
> Any chance to have this patch set applied?

Sure, guess I don't need to wait for the legacy filters to be resolved
and can push everything else.
diff mbox series

Patch

diff --git a/doc/APIchanges b/doc/APIchanges
index e30148dc90..2fc9428827 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -15,6 +15,10 @@  libavutil:     2017-10-21
 
 API changes, most recent first:
 
+2020-xx-xx - xxxxxxxxxx - lavu 56.43.100 - video_enc_params.h
+  Add a new API for exporting video encoding information.
+  Replaces the deprecated API for exporting QP tables from decoders.
+
 2020-04-15 - xxxxxxxxxx - lavc 58.79.100 - avcodec.h
   Add formal support for calling avcodec_flush_buffers() on encoders.
   Encoders that set the cap AV_CODEC_CAP_ENCODER_FLUSH will be flushed.
diff --git a/libavutil/Makefile b/libavutil/Makefile
index 8feb029a3a..d44b62877f 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -79,6 +79,7 @@  HEADERS = adler32.h                                                     \
           tree.h                                                        \
           twofish.h                                                     \
           version.h                                                     \
+          video_enc_params.h                                            \
           xtea.h                                                        \
           tea.h                                                         \
           tx.h                                                          \
@@ -165,7 +166,9 @@  OBJS = adler32.o                                                        \
        tx.o                                                             \
        tx_float.o                                                       \
        tx_double.o                                                      \
-       tx_int32.o
+       tx_int32.o                                                       \
+       video_enc_params.o                                               \
+
 
 OBJS-$(CONFIG_CUDA)                     += hwcontext_cuda.o
 OBJS-$(CONFIG_D3D11VA)                  += hwcontext_d3d11va.o
diff --git a/libavutil/frame.c b/libavutil/frame.c
index 769851ceac..53581e4862 100644
--- a/libavutil/frame.c
+++ b/libavutil/frame.c
@@ -850,6 +850,7 @@  const char *av_frame_side_data_name(enum AVFrameSideDataType type)
 #endif
     case AV_FRAME_DATA_DYNAMIC_HDR_PLUS: return "HDR Dynamic Metadata SMPTE2094-40 (HDR10+)";
     case AV_FRAME_DATA_REGIONS_OF_INTEREST: return "Regions Of Interest";
+    case AV_FRAME_DATA_VIDEO_ENC_PARAMS:            return "Video encoding parameters";
     }
     return NULL;
 }
diff --git a/libavutil/frame.h b/libavutil/frame.h
index b5afb58634..fc67db0f6c 100644
--- a/libavutil/frame.h
+++ b/libavutil/frame.h
@@ -179,6 +179,11 @@  enum AVFrameSideDataType {
      * array element is implied by AVFrameSideData.size / AVRegionOfInterest.self_size.
      */
     AV_FRAME_DATA_REGIONS_OF_INTEREST,
+
+    /**
+     * Encoding parameters for a video frame, as described by AVVideoEncParams.
+     */
+    AV_FRAME_DATA_VIDEO_ENC_PARAMS,
 };
 
 enum AVActiveFormatDescription {
diff --git a/libavutil/version.h b/libavutil/version.h
index 70836a5d52..ea9363e8e9 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -79,8 +79,8 @@ 
  */
 
 #define LIBAVUTIL_VERSION_MAJOR  56
-#define LIBAVUTIL_VERSION_MINOR  42
-#define LIBAVUTIL_VERSION_MICRO 102
+#define LIBAVUTIL_VERSION_MINOR  43
+#define LIBAVUTIL_VERSION_MICRO 100
 
 #define LIBAVUTIL_VERSION_INT   AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \
                                                LIBAVUTIL_VERSION_MINOR, \
diff --git a/libavutil/video_enc_params.c b/libavutil/video_enc_params.c
new file mode 100644
index 0000000000..c46c0f1dc6
--- /dev/null
+++ b/libavutil/video_enc_params.c
@@ -0,0 +1,79 @@ 
+/*
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include <limits.h>
+#include <stddef.h>
+#include <stdint.h>
+
+#include "buffer.h"
+#include "common.h"
+#include "frame.h"
+#include "mem.h"
+#include "video_enc_params.h"
+
+AVVideoEncParams *av_video_enc_params_alloc(enum AVVideoEncParamsType type,
+                                            unsigned int nb_blocks, size_t *out_size)
+{
+    AVVideoEncParams *par;
+    size_t size;
+
+    size = sizeof(*par);
+    if (nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams) ||
+        nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size)
+        return NULL;
+    size += sizeof(AVVideoBlockParams) * nb_blocks;
+
+    par = av_mallocz(size);
+    if (!par)
+        return NULL;
+
+    par->type          = type;
+    par->nb_blocks     = nb_blocks;
+    par->block_size    = sizeof(AVVideoBlockParams);
+    par->blocks_offset = sizeof(*par);
+
+    if (out_size)
+        *out_size = size;
+
+    return par;
+}
+
+AVVideoEncParams*
+av_video_enc_params_create_side_data(AVFrame *frame, enum AVVideoEncParamsType type,
+                                     unsigned int nb_blocks)
+{
+    AVBufferRef      *buf;
+    AVVideoEncParams *par;
+    size_t size;
+
+    par = av_video_enc_params_alloc(type, nb_blocks, &size);
+    if (!par)
+        return NULL;
+    buf = av_buffer_create((uint8_t *)par, size, NULL, NULL, 0);
+    if (!buf) {
+        av_freep(&par);
+        return NULL;
+    }
+
+    if (!av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_VIDEO_ENC_PARAMS, buf)) {
+        av_buffer_unref(&buf);
+        return NULL;
+    }
+
+    return par;
+}
diff --git a/libavutil/video_enc_params.h b/libavutil/video_enc_params.h
new file mode 100644
index 0000000000..f64864caae
--- /dev/null
+++ b/libavutil/video_enc_params.h
@@ -0,0 +1,135 @@ 
+/*
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVUTIL_VIDEO_ENC_PARAMS_H
+#define AVUTIL_VIDEO_ENC_PARAMS_H
+
+#include <stddef.h>
+#include <stdint.h>
+
+#include "libavutil/avassert.h"
+#include "libavutil/frame.h"
+
+enum AVVideoEncParamsType {
+    AV_VIDEO_ENC_PARAMS_NONE = -1,
+};
+
+/**
+ * Video encoding parameters for a given frame. This struct is allocated along
+ * with an optional array of per-block AVVideoBlockParams descriptors.
+ * Must be allocated with av_video_enc_params_alloc().
+ */
+typedef struct AVVideoEncParams {
+    /**
+     * Number of blocks in the array.
+     *
+     * May be 0, in which case no per-block information is present. In this case
+     * the values of blocks_offset / block_size are unspecified and should not
+     * be accessed.
+     */
+    unsigned int nb_blocks;
+    /**
+     * Offset in bytes from the beginning of this structure at which the array
+     * of blocks starts.
+     */
+    size_t blocks_offset;
+    /*
+     * Size of each block in bytes. May not match sizeof(AVVideoBlockParams).
+     */
+    size_t block_size;
+
+    /**
+     * Type of the parameters (the codec they are used with).
+     */
+    enum AVVideoEncParamsType type;
+
+    /**
+     * Base quantisation parameter for the frame. The final quantiser for a
+     * given block in a given plane is obtained from this value, possibly
+     * combined with {@code delta_qp} and the per-block delta in a manner
+     * documented for each type.
+     */
+    int32_t qp;
+
+    /**
+     * Quantisation parameter offset from the base (per-frame) qp for a given
+     * plane (first index) and AC/DC coefficients (second index).
+     */
+    int32_t delta_qp[4][2];
+} AVVideoEncParams;
+
+/**
+ * Data structure for storing block-level encoding information.
+ * It is allocated as a part of AVVideoEncParams and should be retrieved with
+ * av_video_enc_params_block().
+ *
+ * sizeof(AVVideoBlockParams) is not a part of the ABI and new fields may be
+ * added to it.
+ */
+typedef struct AVVideoBlockParams {
+    /**
+     * Distance in luma pixels from the top-left corner of the visible frame
+     * to the top-left corner of the block.
+     * Can be negative if top/right padding is present on the coded frame.
+     */
+    int src_x, src_y;
+    /**
+     * Width and height of the block in luma pixels.
+     */
+    int w, h;
+
+    /**
+     * Difference between this block's final quantization parameter and the
+     * corresponding per-frame value.
+     */
+    int32_t delta_qp;
+} AVVideoBlockParams;
+
+/*
+ * Get the block at the specified {@code idx}. Must be between 0 and nb_blocks.
+ */
+static av_always_inline AVVideoBlockParams*
+av_video_enc_params_block(AVVideoEncParams *par, unsigned int idx)
+{
+    av_assert0(idx < par->nb_blocks);
+    return (AVVideoBlockParams *)((uint8_t *)par + par->blocks_offset +
+                                  idx * par->block_size);
+}
+
+/**
+ * Allocates memory for AVVideoEncParams of the given type, plus an array of
+ * {@code nb_blocks} AVVideoBlockParams and initializes the variables. Can be
+ * freed with a normal av_free() call.
+ *
+ * @param out_size if non-NULL, the size in bytes of the resulting data array is
+ * written here.
+ */
+AVVideoEncParams *av_video_enc_params_alloc(enum AVVideoEncParamsType type,
+                                            unsigned int nb_blocks, size_t *out_size);
+
+/**
+ * Allocates memory for AVEncodeInfoFrame plus an array of
+ * {@code nb_blocks} AVEncodeInfoBlock in the given AVFrame {@code frame}
+ * as AVFrameSideData of type AV_FRAME_DATA_ENCODE_INFO
+ * and initializes the variables.
+ */
+AVVideoEncParams*
+av_video_enc_params_create_side_data(AVFrame *frame, enum AVVideoEncParamsType type,
+                                     unsigned int nb_blocks);
+
+#endif /* AVUTIL_VIDEO_ENC_PARAMS_H */