| Message ID | 20210219065945.15594-2-yejun.guo@intel.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [FFmpeg-devel,1/4] libavfilter/bbox.h: add BoundingBoxHeader and BoundingBox | expand |
| Context | Check | Description |
|---|---|---|
| andriy/x86_make | success | Make finished |
| andriy/x86_make_fate | success | Make fate finished |
| andriy/PPC64_make | success | Make finished |
| andriy/PPC64_make_fate | success | Make fate finished |
Feb 19, 2021, 07:59 by yejun.guo@intel.com: > Signed-off-by: Guo, Yejun <yejun.guo@intel.com> > --- > doc/APIchanges | 2 ++ > libavutil/frame.c | 1 + > libavutil/frame.h | 7 +++++++ > libavutil/version.h | 2 +- > 4 files changed, 11 insertions(+), 1 deletion(-) > > diff --git a/doc/APIchanges b/doc/APIchanges > index c353d2d281..3c6e9e351e 100644 > --- a/doc/APIchanges > +++ b/doc/APIchanges > @@ -14,6 +14,8 @@ libavutil: 2017-10-21 > This won't really work. Side data must be public but the header it's specified in is neither installed, neither in lavu. There's no such thing as ffmpeg-only private data. If we do plan to make this side data public, it'll require a lot more considerations, and versioning like with the film grain side data, since as you pointed out it may be used by codecs. For now, although it's somewhat less appropriate, you can replace the side data with frame->private_ref. It's an AVBufferRef that's for use by a single internal library, and we don't use it anywhere except in lavc for now.
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Lynne > Sent: 2021年2月20日 6:54 > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/4] libavutil: add side data > AV_FRAME_DATA_BOUNDING_BOXES > > Feb 19, 2021, 07:59 by yejun.guo@intel.com: > > > Signed-off-by: Guo, Yejun <yejun.guo@intel.com> > > --- > > doc/APIchanges | 2 ++ > > libavutil/frame.c | 1 + > > libavutil/frame.h | 7 +++++++ > > libavutil/version.h | 2 +- > > 4 files changed, 11 insertions(+), 1 deletion(-) > > > > diff --git a/doc/APIchanges b/doc/APIchanges > > index c353d2d281..3c6e9e351e 100644 > > --- a/doc/APIchanges > > +++ b/doc/APIchanges > > @@ -14,6 +14,8 @@ libavutil: 2017-10-21 > > > > This won't really work. > Side data must be public but the header it's specified in is neither installed, > neither in lavu. There's no such thing as ffmpeg-only private data. > > If we do plan to make this side data public, it'll require a lot more > considerations, > and versioning like with the film grain side data, since as you pointed out it > may > be used by codecs. Thanks for your feedback. Could you share a bit more about the 'versioning' of film grain? thanks. I checked file film_grain_params.h/c and do not find it is version relative. As for the bounding box, the BoundingBoxHeader.bbox_size acts like a version checker. (different bbox_size if the struct is changed later) > > For now, although it's somewhat less appropriate, you can replace > the side data with frame->private_ref. It's an AVBufferRef that's for > use by a single internal library, and we don't use it anywhere except > in lavc for now. Thanks for pointing this possible way, I'll investigate it. I would prefer side data which is possible the final solution. > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Feb 20, 2021, 15:13 by yejun.guo@intel.com: > > >> -----Original Message----- >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Lynne >> Sent: 2021年2月20日 6:54 >> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> >> Subject: Re: [FFmpeg-devel] [PATCH 2/4] libavutil: add side data >> AV_FRAME_DATA_BOUNDING_BOXES >> >> Feb 19, 2021, 07:59 by yejun.guo@intel.com: >> >> > Signed-off-by: Guo, Yejun <yejun.guo@intel.com> >> > --- >> > doc/APIchanges | 2 ++ >> > libavutil/frame.c | 1 + >> > libavutil/frame.h | 7 +++++++ >> > libavutil/version.h | 2 +- >> > 4 files changed, 11 insertions(+), 1 deletion(-) >> > >> > diff --git a/doc/APIchanges b/doc/APIchanges >> > index c353d2d281..3c6e9e351e 100644 >> > --- a/doc/APIchanges >> > +++ b/doc/APIchanges >> > @@ -14,6 +14,8 @@ libavutil: 2017-10-21 >> > >> >> This won't really work. >> Side data must be public but the header it's specified in is neither installed, >> neither in lavu. There's no such thing as ffmpeg-only private data. >> >> If we do plan to make this side data public, it'll require a lot more >> considerations, >> and versioning like with the film grain side data, since as you pointed out it >> may >> be used by codecs. >> > > Thanks for your feedback. > > Could you share a bit more about the 'versioning' of film grain? thanks. > > I checked file film_grain_params.h/c and do not find it is version relative. > > As for the bounding box, the BoundingBoxHeader.bbox_size acts like > a version checker. (different bbox_size if the struct is changed later) > That's not really a good solution. I meant having a dedicated and documented version field which switches structs. Changing the struct size means we can only add fields, so if we need to alter the behavior of previous fields we'll end up with a data format more convoluted than mpeg audio codecs. >> For now, although it's somewhat less appropriate, you can replace >> the side data with frame->private_ref. It's an AVBufferRef that's for >> use by a single internal library, and we don't use it anywhere except >> in lavc for now. >> > > Thanks for pointing this possible way, I'll investigate it. > I would prefer side data which is possible the final solution. > I'd rather not commit us to a side data format that may change in the future and has no lavfi-external use. Could you implement it as a private_ref, at least for the time being? It's lavfi-only, so it makes sense for now, plus we can keep changing it, and once an external user appears we can make it proper public side data?
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Lynne > Sent: Saturday, February 20, 2021 11:36 PM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/4] libavutil: add side data > AV_FRAME_DATA_BOUNDING_BOXES > > Feb 20, 2021, 15:13 by yejun.guo@intel.com: > > > > > > >> -----Original Message----- > >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > >> Lynne > >> Sent: 2021年2月20日 6:54 > >> To: FFmpeg development discussions and patches > >> <ffmpeg-devel@ffmpeg.org> > >> Subject: Re: [FFmpeg-devel] [PATCH 2/4] libavutil: add side data > >> AV_FRAME_DATA_BOUNDING_BOXES > >> > >> Feb 19, 2021, 07:59 by yejun.guo@intel.com: > >> > >> > Signed-off-by: Guo, Yejun <yejun.guo@intel.com> > >> > --- > >> > doc/APIchanges | 2 ++ > >> > libavutil/frame.c | 1 + > >> > libavutil/frame.h | 7 +++++++ > >> > libavutil/version.h | 2 +- > >> > 4 files changed, 11 insertions(+), 1 deletion(-) > >> > > >> > diff --git a/doc/APIchanges b/doc/APIchanges index > >> > c353d2d281..3c6e9e351e 100644 > >> > --- a/doc/APIchanges > >> > +++ b/doc/APIchanges > >> > @@ -14,6 +14,8 @@ libavutil: 2017-10-21 > >> > > >> > >> This won't really work. > >> Side data must be public but the header it's specified in is neither > >> installed, neither in lavu. There's no such thing as ffmpeg-only private > data. > >> > >> If we do plan to make this side data public, it'll require a lot more > >> considerations, and versioning like with the film grain side data, > >> since as you pointed out it may be used by codecs. > >> > > > > Thanks for your feedback. > > > > Could you share a bit more about the 'versioning' of film grain? thanks. > > > > I checked file film_grain_params.h/c and do not find it is version relative. > > > > As for the bounding box, the BoundingBoxHeader.bbox_size acts like a > > version checker. (different bbox_size if the struct is changed later) > > > > That's not really a good solution. I meant having a dedicated and > documented version field which switches structs. > Changing the struct size means we can only add fields, so if we need to alter > the behavior of previous fields we'll end up with a data format more > convoluted than mpeg audio codecs. Thanks, got it. > > > >> For now, although it's somewhat less appropriate, you can replace the > >> side data with frame->private_ref. It's an AVBufferRef that's for use > >> by a single internal library, and we don't use it anywhere except in > >> lavc for now. > >> > > > > Thanks for pointing this possible way, I'll investigate it. > > I would prefer side data which is possible the final solution. > > > > I'd rather not commit us to a side data format that may change in the future > and has no lavfi-external use. Could you implement it as a private_ref, at > least for the time being? It's lavfi-only, so it makes sense for now, plus we > can keep changing it, and once an external user appears we can make it > proper public side data? Sure, I'll look at this method, thanks. > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org > with subject "unsubscribe".
diff --git a/doc/APIchanges b/doc/APIchanges index c353d2d281..3c6e9e351e 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -14,6 +14,8 @@ libavutil: 2017-10-21 API changes, most recent first: +2021-02-xx - xxxxxxxxxx - lavu 56.xx.100 - frame.h + Add AV_FRAME_DATA_BOUNDING_BOXES 2021-02-14 - xxxxxxxxxx - lavd 58.12.100 - avdevice.h Deprecated avdevice_capabilities_create() and diff --git a/libavutil/frame.c b/libavutil/frame.c index eab51b6a32..53868a2410 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -852,6 +852,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_VIDEO_ENC_PARAMS: return "Video encoding parameters"; case AV_FRAME_DATA_SEI_UNREGISTERED: return "H.26[45] User Data Unregistered SEI message"; case AV_FRAME_DATA_FILM_GRAIN_PARAMS: return "Film grain parameters"; + case AV_FRAME_DATA_BOUNDING_BOXES: return "Bounding boxes"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index 1aeafef6de..e213e0a646 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -198,6 +198,13 @@ enum AVFrameSideDataType { * Must be present for every frame which should have film grain applied. */ AV_FRAME_DATA_FILM_GRAIN_PARAMS, + + /** + * Bounding boxes for object detection and classification, the data is a BoundingBoxHeader + * followed with an array of BoudingBox, the number of array element is implied by + * (AVFrameSideData.size - sizeof(BoundingBoxHeader)) / BoundingBoxHeader.bbox_size. + */ + AV_FRAME_DATA_BOUNDING_BOXES, }; enum AVActiveFormatDescription { diff --git a/libavutil/version.h b/libavutil/version.h index b2165754f9..b7c5892a37 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 56 -#define LIBAVUTIL_VERSION_MINOR 65 +#define LIBAVUTIL_VERSION_MINOR 66 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \
Signed-off-by: Guo, Yejun <yejun.guo@intel.com> --- doc/APIchanges | 2 ++ libavutil/frame.c | 1 + libavutil/frame.h | 7 +++++++ libavutil/version.h | 2 +- 4 files changed, 11 insertions(+), 1 deletion(-)