[FFmpeg-devel,V6,1/2] avutil: add ROI (Region Of Interest) data struct and bump version

Submitted by Guo, Yejun on Jan. 7, 2019, 7:54 p.m.

Details

Message ID 1546890854-7115-1-git-send-email-yejun.guo@intel.com
State Accepted
Headers show

Commit Message

Guo, Yejun Jan. 7, 2019, 7:54 p.m.
The encoders such as libx264 support different QPs offset for different MBs,
it makes possible for ROI-based encoding. It makes sense to add support
within ffmpeg to generate/accept ROI infos and pass into encoders.

Typical usage: After AVFrame is decoded, a ffmpeg filter or user's code
generates ROI info for that frame, and the encoder finally does the
ROI-based encoding.

The ROI info is maintained as side data of AVFrame.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
---
 libavutil/frame.c   |  1 +
 libavutil/frame.h   | 35 +++++++++++++++++++++++++++++++++++
 libavutil/version.h |  2 +-
 3 files changed, 37 insertions(+), 1 deletion(-)

Comments

Michael Niedermayer Jan. 7, 2019, 7:20 p.m.
On Tue, Jan 08, 2019 at 03:54:14AM +0800, Guo, Yejun wrote:
> The encoders such as libx264 support different QPs offset for different MBs,
> it makes possible for ROI-based encoding. It makes sense to add support
> within ffmpeg to generate/accept ROI infos and pass into encoders.
> 
> Typical usage: After AVFrame is decoded, a ffmpeg filter or user's code
> generates ROI info for that frame, and the encoder finally does the
> ROI-based encoding.
> 
> The ROI info is maintained as side data of AVFrame.
> 
> Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
> ---
>  libavutil/frame.c   |  1 +
>  libavutil/frame.h   | 35 +++++++++++++++++++++++++++++++++++
>  libavutil/version.h |  2 +-
>  3 files changed, 37 insertions(+), 1 deletion(-)

needs a update to doc/APIchanges

thx

[...]
Guo, Yejun Jan. 8, 2019, 1:13 a.m.
> -----Original Message-----
> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf
> Of Michael Niedermayer
> Sent: Tuesday, January 08, 2019 3:21 AM
> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH V6 1/2] avutil: add ROI (Region Of
> Interest) data struct and bump version
> 
> On Tue, Jan 08, 2019 at 03:54:14AM +0800, Guo, Yejun wrote:
> > The encoders such as libx264 support different QPs offset for
> > different MBs, it makes possible for ROI-based encoding. It makes
> > sense to add support within ffmpeg to generate/accept ROI infos and pass
> into encoders.
> >
> > Typical usage: After AVFrame is decoded, a ffmpeg filter or user's
> > code generates ROI info for that frame, and the encoder finally does
> > the ROI-based encoding.
> >
> > The ROI info is maintained as side data of AVFrame.
> >
> > Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
> > ---
> >  libavutil/frame.c   |  1 +
> >  libavutil/frame.h   | 35 +++++++++++++++++++++++++++++++++++
> >  libavutil/version.h |  2 +-
> >  3 files changed, 37 insertions(+), 1 deletion(-)
> 
> needs a update to doc/APIchanges

thanks, will update in a separate patch. (looks that this file is usually updated in a separated patch).

> 
> thx
> 
> [...]
> --
> Michael     GnuPG fingerprint:
> 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> Those who would give up essential Liberty, to purchase a little temporary
> Safety, deserve neither Liberty nor Safety -- Benjamin Franklin

Patch hide | download patch | download mbox

diff --git a/libavutil/frame.c b/libavutil/frame.c
index 34a6210..dcf1fc3 100644
--- a/libavutil/frame.c
+++ b/libavutil/frame.c
@@ -841,6 +841,7 @@  const char *av_frame_side_data_name(enum AVFrameSideDataType type)
     case AV_FRAME_DATA_QP_TABLE_DATA:               return "QP table data";
 #endif
     case AV_FRAME_DATA_DYNAMIC_HDR_PLUS: return "HDR Dynamic Metadata SMPTE2094-40 (HDR10+)";
+    case AV_FRAME_DATA_REGIONS_OF_INTEREST: return "Regions Of Interest";
     }
     return NULL;
 }
diff --git a/libavutil/frame.h b/libavutil/frame.h
index 582ac47..8d0dfed 100644
--- a/libavutil/frame.h
+++ b/libavutil/frame.h
@@ -173,6 +173,12 @@  enum AVFrameSideDataType {
      * volume transform - application 4 of SMPTE 2094-40:2016 standard.
      */
     AV_FRAME_DATA_DYNAMIC_HDR_PLUS,
+
+    /**
+     * Regions Of Interest, the data is an array of AVRegionOfInterest type, the number of
+     * array element is implied by AVFrameSideData.size / AVRegionOfInterest.self_size.
+     */
+    AV_FRAME_DATA_REGIONS_OF_INTEREST,
 };
 
 enum AVActiveFormatDescription {
@@ -201,6 +207,35 @@  typedef struct AVFrameSideData {
 } AVFrameSideData;
 
 /**
+ * Structure to hold Region Of Interest.
+ *
+ * self_size specifies the size of this data structure. This value
+ * should be set to sizeof(AVRegionOfInterest). EINVAL is returned if self_size is zero.
+ *
+ * Number of pixels to discard from the top/bottom/left/right border of
+ * the frame to obtain the region of interest of the frame.
+ * They are encoder dependent and will be extended internally
+ * if the codec requires an alignment.
+ * If the regions overlap, the last value in the list will be used.
+ *
+ * qoffset is quant offset, and base rule here:
+ * returns EINVAL if AVRational.den is zero.
+ * the value (num/den) range is [-1.0, 1.0], clamp to +-1.0 if out of range.
+ * 0 means no picture quality change,
+ * negtive offset asks for better quality (and the best with value -1.0),
+ * positive offset asks for worse quality (and the worst with value 1.0).
+ * How to explain/implement the different quilaity requirement is encoder dependent.
+ */
+typedef struct AVRegionOfInterest {
+    uint32_t self_size;
+    int top;
+    int bottom;
+    int left;
+    int right;
+    AVRational qoffset;
+} AVRegionOfInterest;
+
+/**
  * This structure describes decoded (raw) audio or video data.
  *
  * AVFrame must be allocated using av_frame_alloc(). Note that this only
diff --git a/libavutil/version.h b/libavutil/version.h
index f997615..1fcdea9 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -79,7 +79,7 @@ 
  */
 
 #define LIBAVUTIL_VERSION_MAJOR  56
-#define LIBAVUTIL_VERSION_MINOR  25
+#define LIBAVUTIL_VERSION_MINOR  26
 #define LIBAVUTIL_VERSION_MICRO 100
 
 #define LIBAVUTIL_VERSION_INT   AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \