Message ID | 20220831230918.1736362-2-chemag@gmail.com |
---|---|
State | New |
Headers | show |
Series | add motion_vec (motion vector) debug mode | expand |
Context | Check | Description |
---|---|---|
yinshiyou/commit_msg_loongarch64 | warning | Please wrap lines in the body of the commit message between 60 and 72 characters. |
andriy/commit_msg_x86 | warning | Please wrap lines in the body of the commit message between 60 and 72 characters. |
yinshiyou/make_loongarch64 | success | Make finished |
yinshiyou/make_fate_loongarch64 | success | Make fate finished |
andriy/make_x86 | success | Make finished |
andriy/make_fate_x86 | success | Make fate finished |
Hi, On Wed, Aug 31, 2022 at 7:10 PM Chema Gonzalez <chemag@gmail.com> wrote: > Add a new debug mode ("motion_vec") that prints the values of the > motion vectors (MV). > Isn't this super-hacky? In a well-designed system, a codec might export its MVs using designated structs (which we already have) and then use these structs to print it in a codec-agnostic way using a filter (which IIRC we already have?). Ronald
Hi, All the other debug modes (qp, skip, mb_type) use the same approach. -Chema On Thu, Sep 1, 2022 at 6:30 AM Ronald S. Bultje <rsbultje@gmail.com> wrote: > > Hi, > > On Wed, Aug 31, 2022 at 7:10 PM Chema Gonzalez <chemag@gmail.com> wrote: >> >> Add a new debug mode ("motion_vec") that prints the values of the >> motion vectors (MV). > > > Isn't this super-hacky? In a well-designed system, a codec might export its MVs using designated structs (which we already have) and then use these structs to print it in a codec-agnostic way using a filter (which IIRC we already have?). > > Ronald
On Thu, Sep 1, 2022 at 5:06 PM Chema Gonzalez <chema@berkeley.edu> wrote: > Hi, > > All the other debug modes (qp, skip, mb_type) use the same approach. > > This adds to much code for something that is already possible. > -Chema > > > On Thu, Sep 1, 2022 at 6:30 AM Ronald S. Bultje <rsbultje@gmail.com> > wrote: > > > > Hi, > > > > On Wed, Aug 31, 2022 at 7:10 PM Chema Gonzalez <chemag@gmail.com> wrote: > >> > >> Add a new debug mode ("motion_vec") that prints the values of the > >> motion vectors (MV). > > > > > > Isn't this super-hacky? In a well-designed system, a codec might export > its MVs using designated structs (which we already have) and then use these > structs to print it in a codec-agnostic way using a filter (which IIRC we > already have?). > > > > Ronald > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Hi Paul, Can you please describe how you can get the MV values? Thanks, -Chema On Thu, Sep 1, 2022 at 10:52 AM Paul B Mahol <onemda@gmail.com> wrote: > > On Thu, Sep 1, 2022 at 5:06 PM Chema Gonzalez <chema@berkeley.edu> wrote: > > > Hi, > > > > All the other debug modes (qp, skip, mb_type) use the same approach. > > > > > This adds to much code for something that is already possible. > > > > -Chema > > > > > > On Thu, Sep 1, 2022 at 6:30 AM Ronald S. Bultje <rsbultje@gmail.com> > > wrote: > > > > > > Hi, > > > > > > On Wed, Aug 31, 2022 at 7:10 PM Chema Gonzalez <chemag@gmail.com> wrote: > > >> > > >> Add a new debug mode ("motion_vec") that prints the values of the > > >> motion vectors (MV). > > > > > > > > > Isn't this super-hacky? In a well-designed system, a codec might export > > its MVs using designated structs (which we already have) and then use these > > structs to print it in a codec-agnostic way using a filter (which IIRC we > > already have?). > > > > > > Ronald > > _______________________________________________ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On Thu, Sep 1, 2022 at 8:43 PM Chema Gonzalez <chema@berkeley.edu> wrote: > Hi Paul, > > Can you please describe how you can get the MV values? > See how filters do it. They visualize MV but using similar logic one could dump them to ton of texts. > > Thanks, > -Chema > > > On Thu, Sep 1, 2022 at 10:52 AM Paul B Mahol <onemda@gmail.com> wrote: > > > > On Thu, Sep 1, 2022 at 5:06 PM Chema Gonzalez <chema@berkeley.edu> > wrote: > > > > > Hi, > > > > > > All the other debug modes (qp, skip, mb_type) use the same approach. > > > > > > > > This adds to much code for something that is already possible. > > > > > > > -Chema > > > > > > > > > On Thu, Sep 1, 2022 at 6:30 AM Ronald S. Bultje <rsbultje@gmail.com> > > > wrote: > > > > > > > > Hi, > > > > > > > > On Wed, Aug 31, 2022 at 7:10 PM Chema Gonzalez <chemag@gmail.com> > wrote: > > > >> > > > >> Add a new debug mode ("motion_vec") that prints the values of the > > > >> motion vectors (MV). > > > > > > > > > > > > Isn't this super-hacky? In a well-designed system, a codec might > export > > > its MVs using designated structs (which we already have) and then use > these > > > structs to print it in a codec-agnostic way using a filter (which IIRC > we > > > already have?). > > > > > > > > Ronald > > > _______________________________________________ > > > ffmpeg-devel mailing list > > > ffmpeg-devel@ffmpeg.org > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > > To unsubscribe, visit link above, or email > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > > > > _______________________________________________ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
So is there a filter that already dumps this information? Thanks, -Chema On Fri, Sep 2, 2022 at 12:56 AM Paul B Mahol <onemda@gmail.com> wrote: > > On Thu, Sep 1, 2022 at 8:43 PM Chema Gonzalez <chema@berkeley.edu> wrote: > > > Hi Paul, > > > > Can you please describe how you can get the MV values? > > > > See how filters do it. They visualize MV but using similar logic one could > dump them to ton of texts. > > > > > > > Thanks, > > -Chema > > > > > > On Thu, Sep 1, 2022 at 10:52 AM Paul B Mahol <onemda@gmail.com> wrote: > > > > > > On Thu, Sep 1, 2022 at 5:06 PM Chema Gonzalez <chema@berkeley.edu> > > wrote: > > > > > > > Hi, > > > > > > > > All the other debug modes (qp, skip, mb_type) use the same approach. > > > > > > > > > > > This adds to much code for something that is already possible. > > > > > > > > > > -Chema > > > > > > > > > > > > On Thu, Sep 1, 2022 at 6:30 AM Ronald S. Bultje <rsbultje@gmail.com> > > > > wrote: > > > > > > > > > > Hi, > > > > > > > > > > On Wed, Aug 31, 2022 at 7:10 PM Chema Gonzalez <chemag@gmail.com> > > wrote: > > > > >> > > > > >> Add a new debug mode ("motion_vec") that prints the values of the > > > > >> motion vectors (MV). > > > > > > > > > > > > > > > Isn't this super-hacky? In a well-designed system, a codec might > > export > > > > its MVs using designated structs (which we already have) and then use > > these > > > > structs to print it in a codec-agnostic way using a filter (which IIRC > > we > > > > already have?). > > > > > > > > > > Ronald > > > > _______________________________________________ > > > > ffmpeg-devel mailing list > > > > ffmpeg-devel@ffmpeg.org > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > > > > To unsubscribe, visit link above, or email > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > > > > > > _______________________________________________ > > > ffmpeg-devel mailing list > > > ffmpeg-devel@ffmpeg.org > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > > To unsubscribe, visit link above, or email > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > _______________________________________________ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On Fri, Sep 2, 2022 at 5:12 PM Chema Gonzalez <chema@berkeley.edu> wrote: > So is there a filter that already dumps this information? > Not in text but in visual, note that you can write own code that uses library to dump MV as text, because MV are exported when certain flag is enabled. > > Thanks, > -Chema > > On Fri, Sep 2, 2022 at 12:56 AM Paul B Mahol <onemda@gmail.com> wrote: > > > > On Thu, Sep 1, 2022 at 8:43 PM Chema Gonzalez <chema@berkeley.edu> > wrote: > > > > > Hi Paul, > > > > > > Can you please describe how you can get the MV values? > > > > > > > See how filters do it. They visualize MV but using similar logic one > could > > dump them to ton of texts. > > > > > > > > > > > > Thanks, > > > -Chema > > > > > > > > > On Thu, Sep 1, 2022 at 10:52 AM Paul B Mahol <onemda@gmail.com> wrote: > > > > > > > > On Thu, Sep 1, 2022 at 5:06 PM Chema Gonzalez <chema@berkeley.edu> > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > All the other debug modes (qp, skip, mb_type) use the same > approach. > > > > > > > > > > > > > > This adds to much code for something that is already possible. > > > > > > > > > > > > > -Chema > > > > > > > > > > > > > > > On Thu, Sep 1, 2022 at 6:30 AM Ronald S. Bultje < > rsbultje@gmail.com> > > > > > wrote: > > > > > > > > > > > > Hi, > > > > > > > > > > > > On Wed, Aug 31, 2022 at 7:10 PM Chema Gonzalez <chemag@gmail.com > > > > > wrote: > > > > > >> > > > > > >> Add a new debug mode ("motion_vec") that prints the values of > the > > > > > >> motion vectors (MV). > > > > > > > > > > > > > > > > > > Isn't this super-hacky? In a well-designed system, a codec might > > > export > > > > > its MVs using designated structs (which we already have) and then > use > > > these > > > > > structs to print it in a codec-agnostic way using a filter (which > IIRC > > > we > > > > > already have?). > > > > > > > > > > > > Ronald > > > > > _______________________________________________ > > > > > ffmpeg-devel mailing list > > > > > ffmpeg-devel@ffmpeg.org > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > > > > > > To unsubscribe, visit link above, or email > > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > > > > > > > > _______________________________________________ > > > > ffmpeg-devel mailing list > > > > ffmpeg-devel@ffmpeg.org > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > > > > To unsubscribe, visit link above, or email > > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > > _______________________________________________ > > > ffmpeg-devel mailing list > > > ffmpeg-devel@ffmpeg.org > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > > To unsubscribe, visit link above, or email > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > > > > _______________________________________________ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Hi again, On Fri, Sep 2, 2022 at 9:10 AM Paul B Mahol <onemda@gmail.com> wrote: > > On Fri, Sep 2, 2022 at 5:12 PM Chema Gonzalez <chema@berkeley.edu> wrote: > > > So is there a filter that already dumps this information? > > > > Not in text but in visual, note that you can write own code that uses > library to dump MV as text, because MV are exported when certain flag is > enabled. Not sure if I follow this. Is the suggestion to use external code that just uses the ffmpeg libraries (libavcodec in this case)? Thanks, -Chema
On Fri, Sep 2, 2022 at 8:04 PM Chema Gonzalez <chema@berkeley.edu> wrote: > Hi again, > > On Fri, Sep 2, 2022 at 9:10 AM Paul B Mahol <onemda@gmail.com> wrote: > > > > On Fri, Sep 2, 2022 at 5:12 PM Chema Gonzalez <chema@berkeley.edu> > wrote: > > > > > So is there a filter that already dumps this information? > > > > > > > Not in text but in visual, note that you can write own code that uses > > library to dump MV as text, because MV are exported when certain flag is > > enabled. > > Not sure if I follow this. Is the suggestion to use external code that > just uses the ffmpeg libraries (libavcodec in this case)? > Yes, no point to add bunch of printfs if code already exports MV alternative way. > > Thanks, > -Chema > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Hi again, On Fri, Sep 2, 2022 at 2:05 PM Paul B Mahol <onemda@gmail.com> wrote: > > > > So is there a filter that already dumps this information? > > > Not in text but in visual, note that you can write own code that uses > > > library to dump MV as text, because MV are exported when certain flag is > > > enabled. > > Not sure if I follow this. Is the suggestion to use external code that > > just uses the ffmpeg libraries (libavcodec in this case)? > Yes, no point to add bunch of printfs if code already exports MV > alternative way. Hmm... IIUC, your suggestion is not to provide the MV vector list using an ffmpeg debug sub-mode. Instead, have the user write her own code using one of the ffmpeg libraries. What you mean by "visual" is something similar to [this](https://www.ffmpeg.org/doxygen/0.6/api-example_8c-source.html), right? Couple of questions: * is there a (sane) way to have this code included in ffmpeg? Something like a series of C files/scripts where this type of functionality would live. FWIW some companies charge thousands of dollars for h264 stream parsers, where some of the functionality allows you to access these values using a GUI. * is there any reason why we have the other ffmpeg debug sub-modes (mb_type, qp, skip)? Thanks, -Chema
On Sat, Sep 3, 2022 at 2:46 AM Chema Gonzalez <chema@berkeley.edu> wrote: > Hi again, > > On Fri, Sep 2, 2022 at 2:05 PM Paul B Mahol <onemda@gmail.com> wrote: > > > > > So is there a filter that already dumps this information? > > > > Not in text but in visual, note that you can write own code that uses > > > > library to dump MV as text, because MV are exported when certain > flag is > > > > enabled. > > > Not sure if I follow this. Is the suggestion to use external code that > > > just uses the ffmpeg libraries (libavcodec in this case)? > > Yes, no point to add bunch of printfs if code already exports MV > > alternative way. > > Hmm... IIUC, your suggestion is not to provide the MV vector list > using an ffmpeg debug sub-mode. Instead, have the user write her own > code using one of the ffmpeg libraries. What you mean by "visual" is > something similar to > [this](https://www.ffmpeg.org/doxygen/0.6/api-example_8c-source.html), > right? > > Couple of questions: > * is there a (sane) way to have this code included in ffmpeg? > Something like a series of C files/scripts where this type of > functionality would live. FWIW some companies charge thousands of > dollars for h264 stream parsers, where some of the functionality > allows you to access these values using a GUI. > And why is this related at all with this clumsy printfs patch? There is already functionality in FFmpeg libs to give you MVs. Nobody blocks you to write sane GUI tool that will import MVs from ffmpeg API and sell it for millions to customers. > * is there any reason why we have the other ffmpeg debug sub-modes > (mb_type, qp, skip)? > They should be ultimately removed and in better way exported. > > Thanks, > -Chema > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Hi Chema, On Fri, Sep 2, 2022 at 11:12 AM Chema Gonzalez <chema@berkeley.edu> wrote: > So is there a filter that already dumps this information? > Yes, it's called codecview. We can help understand how it works if you ask more specific questions, but something like "git grep EXPORT_DATA_MVS ../libavcodec/mpeg*.c" and checking the complement code in codecview should explain the basics. Ronald
Hi Paul, I tried to understand the per-frame side-data (and metadata) mechanism. Adding my notes here in case they help a future reader. Metadata and side-data seem like similar mechanisms to add auxiliary information to each frame. Main difference seems to be that metadata is simpler (a key/value dictionary), while side-data allows adding any data struct. Metadata operation is also simpler: There are filters that generate metadata (e.g. "signalstats" generates key-values such as `lavfi.signalstats.YMIN=3`, while "silencedetect" generates audio-related key-values like `lavfi.silence_start=0`). There are also 2x filters that print metadata ("vf_metadata" and "af_ametadata"). So e.g. to see what signalstats/silencedetect are generating, you can do: ``` $ ffmpeg -i in.264 -vf signalstats,metadata=mode=print -f null - ... [Parsed_metadata_1 @ ...] frame:0 pts:0 pts_time:0 [Parsed_metadata_1 @ ...] lavfi.signalstats.YMIN=3 ... ``` or: ``` $ ffmpeg -y -i in.wav -af "silencedetect=n=-10dB:d=1,ametadata=print" /tmp/out.wav ... [Parsed_ametadata_1 @ ...] frame:23 pts:47104 pts_time:0.981333 [Parsed_ametadata_1 @ ...] lavfi.silence_start=0 ... ``` Side-data operation is more complicated. There is some side-data information already generated (e.g. SEI_UNREGISTERED side-data). Some information requires explicitly asking for it. For example, the `MOTION_VECTORS` side-data, you need to enable AV_CODEC_FLAG2_EXPORT_MVS (which means calling ffmpeg/ffplay/ffprobe with "-flags2 +export_mvs"). The main filter to print side-data information is showinfo (`vf_showinfo` and `af_ashowinfo`). Now, the `vf_showinfo` filter only knows how to dump some of the side-data structs. In particular, it does not know how to dump MOTION_VECTORS side-data. So, if we add the motion vectors, and then ask showinfo to print it, we see: ``` $ ffmpeg -hide_banner -flags2 +export_mvs -export_side_data +mvs -export_side_data +prft -export_side_data +venc_params -export_side_data +film_grain -i /tmp/in.264 -vf showinfo -f null /dev/null ... -- frame 0 is a key frame: We can see SEI_UNREGISTERED and VIDEO_ENC_PARAMS info [Parsed_showinfo_0 @ 0x308fd40] config in time_base: 1/1200000, frame_rate: 25/1 [Parsed_showinfo_0 @ 0x308fd40] config out time_base: 0/0, frame_rate: 0/0 [Parsed_showinfo_0 @ 0x308fd40] n: 0 pts: 0 pts_time:0 duration: 48000 duration_time:0.04 pos: 0 fmt:yuv420p sar:0/1 s:1920x1080 i:P iskey:1 type:I checksum:F6BBEA9F plane_checksum:[AFB1432E 63F2F255 2887B50D] mean:[109 119 138] stdev:[43.0 12.7 13.3] [Parsed_showinfo_0 @ 0x308fd40] side data - User Data Unregistered: [Parsed_showinfo_0 @ 0x308fd40] UUID=47564adc-5c4c-433f-94ef-c5113cd143a8 [Parsed_showinfo_0 @ 0x308fd40] User Data=01ffccccff0200e4dd42 [Parsed_showinfo_0 @ 0x308fd40] [Parsed_showinfo_0 @ 0x308fd40] side data - video encoding parameters: type 1; qp=26; 8160 blocks; [Parsed_showinfo_0 @ 0x308fd40] color_range:tv color_space:bt709 color_primaries:bt709 color_trc:bt709 ... -- frame 1 is a P-frame: we can see VIDEO_ENC_PARAMS info, and a complain about "side-data type 8" (MOTION_VECTORS) [Parsed_showinfo_0 @ 0x308fd40] n: 1 pts: 48000 pts_time:0.04 duration: 48000 duration_time:0.04 pos: 259304 fmt:yuv420p sar:0/1 s:1920x1080 i:P iskey:0 type:B checksum:BC4E5C12 plane_checksum:[AEA8857A 34697DA4 805E58E5] mean:[109 119 138] stdev:[43.0 12.6 13.3] [Parsed_showinfo_0 @ 0x308fd40] side data - video encoding parameters: type 1; qp=26; 8160 blocks; -- showinfo does not dump MOTION_VECTORS side-data [Parsed_showinfo_0 @ 0x308fd40] side data - unknown side data type 8 (547280 bytes) [Parsed_showinfo_0 @ 0x308fd40] [Parsed_showinfo_0 @ 0x308fd40] color_range:tv color_space:bt709 color_primaries:bt709 color_trc:bt709 ... ``` So the best way right now to see the MVs is to use `doc/examples/extract_mvs`, which does exactly that: ``` $ make examples -j ... $ doc/examples/extract_mvs in.264 | head -40 | \ csvcut -C framenum,source,flags |csvlook | blockw | blockh | srcx | srcy | dstx | dsty | motion_x | motion_y | motion_scale | | ------ | ------ | ----- | ---- | ----- | ---- | -------- | -------- | ------------ | | 16 | 16 | 20 | 26 | 8 | 8 | 49 | 72 | 4 | | 16 | 16 | 152 | 15 | 136 | 8 | 65 | 28 | 4 | | 16 | 8 | 360 | 3 | 360 | 4 | 1 | -6 | 4 | | 16 | 8 | 360 | 13 | 360 | 12 | -1 | 4 | 4 | | 16 | 16 | 440 | 10 | 440 | 8 | 3 | 10 | 4 | | 8 | 16 | 829 | 7 | 836 | 8 | -31 | -6 | 4 | | 8 | 16 | 844 | 7 | 844 | 8 | -1 | -4 | 4 | ``` > Yes, it's called codecview. We can help understand how it works if you ask more specific questions, but something like "git grep EXPORT_DATA_MVS ../libavcodec/mpeg*.c" and checking the complement code in codecview should explain the basics. codecview will get the motion vectors (assuming you asked for them), and will overlay arrows showing the MV on top of the original video. ``` $ ffmpeg -flags2 +export_mvs -i input -vf codecview=mv=pf+bf+bb output.%04d.png ``` Adding one of the resulting frames in the next email. -Chema On Sat, Sep 3, 2022 at 5:08 AM Ronald S. Bultje <rsbultje@gmail.com> wrote: > > Hi Chema, > > On Fri, Sep 2, 2022 at 11:12 AM Chema Gonzalez <chema@berkeley.edu> wrote: >> >> So is there a filter that already dumps this information? > > > Yes, it's called codecview. We can help understand how it works if you ask more specific questions, but something like "git grep EXPORT_DATA_MVS ../libavcodec/mpeg*.c" and checking the complement code in codecview should explain the basics. > > Ronald
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 65c8535359..4287555a31 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1344,6 +1344,7 @@ typedef struct AVCodecContext { #define FF_DEBUG_THREADS 0x00010000 #define FF_DEBUG_GREEN_MD 0x00800000 #define FF_DEBUG_NOMC 0x01000000 +#define FF_DEBUG_MOTION_VEC 0x02000000 /** * Error recognition; may misdetect some more or less valid parts as errors. diff --git a/libavcodec/mpegutils.c b/libavcodec/mpegutils.c index ff9418232b..d0a94784fb 100644 --- a/libavcodec/mpegutils.c +++ b/libavcodec/mpegutils.c @@ -246,8 +246,11 @@ void ff_print_debug_info2(AVCodecContext *avctx, AVFrame *pict, return; - if (avctx->debug & (FF_DEBUG_SKIP | FF_DEBUG_QP | FF_DEBUG_MB_TYPE)) { + if (avctx->debug & (FF_DEBUG_SKIP | FF_DEBUG_QP | FF_DEBUG_MB_TYPE | FF_DEBUG_MOTION_VEC)) { int x,y; + const int mv_sample_log2 = avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id == AV_CODEC_ID_SVQ3 ? 2 : 1; + const int mv_stride = (mb_width << mv_sample_log2) + + (avctx->codec->id == AV_CODEC_ID_H264 ? 0 : 1); av_log(avctx, AV_LOG_DEBUG, "New frame, type: %c\n", av_get_picture_type_char(pict->pict_type)); @@ -271,6 +274,46 @@ void ff_print_debug_info2(AVCodecContext *avctx, AVFrame *pict, get_segmentation_char(mb_type), get_interlacement_char(mb_type)); } + if (avctx->debug & FF_DEBUG_MOTION_VEC) { + int i, direction, mb_type = mbtype_table[x + y * mb_stride]; + if (IS_8X8(mb_type)) { + for (i = 0; i < 4; i++) { + int xy = (x * 2 + (i & 1) + + (y * 2 + (i >> 1)) * mv_stride) << (mv_sample_log2 - 1); + int mx = motion_val[direction][xy][0]; + int my = motion_val[direction][xy][1]; + av_log(avctx, AV_LOG_DEBUG, "%4d %4d", mx, my); + } + } else if (IS_16X8(mb_type)) { + for (i = 0; i < 2; i++) { + int xy = (x * 2 + (y * 2 + i) * mv_stride) << (mv_sample_log2 - 1); + int mx = motion_val[direction][xy][0]; + int my = motion_val[direction][xy][1]; + if (IS_INTERLACED(mb_type)) + my *= 2; + av_log(avctx, AV_LOG_DEBUG, "%4d %4d", mx, my); + av_log(avctx, AV_LOG_DEBUG, "%4d %4d", mx, my); + } + } else if (IS_8X16(mb_type)) { + for (i = 0; i < 2; i++) { + int xy = (x * 2 + i + y * 2 * mv_stride) << (mv_sample_log2 - 1); + int mx = motion_val[direction][xy][0]; + int my = motion_val[direction][xy][1]; + if (IS_INTERLACED(mb_type)) + my *= 2; + av_log(avctx, AV_LOG_DEBUG, "%4d %4d", mx, my); + av_log(avctx, AV_LOG_DEBUG, "%4d %4d", mx, my); + } + } else { + int xy = (x + y * mv_stride) << mv_sample_log2; + int mx = motion_val[direction][xy][0]; + int my = motion_val[direction][xy][1]; + av_log(avctx, AV_LOG_DEBUG, "%4d %4d", mx, my); + av_log(avctx, AV_LOG_DEBUG, "%4d %4d", mx, my); + av_log(avctx, AV_LOG_DEBUG, "%4d %4d", mx, my); + av_log(avctx, AV_LOG_DEBUG, "%4d %4d", mx, my); + } + } } av_log(avctx, AV_LOG_DEBUG, "\n"); } diff --git a/libavcodec/options_table.h b/libavcodec/options_table.h index cd02f5096f..0d100069e1 100644 --- a/libavcodec/options_table.h +++ b/libavcodec/options_table.h @@ -198,6 +198,7 @@ static const AVOption avcodec_options[] = { {"startcode", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = FF_DEBUG_STARTCODE }, INT_MIN, INT_MAX, V|D, "debug"}, {"er", "error recognition", 0, AV_OPT_TYPE_CONST, {.i64 = FF_DEBUG_ER }, INT_MIN, INT_MAX, V|D, "debug"}, {"mmco", "memory management control operations (H.264)", 0, AV_OPT_TYPE_CONST, {.i64 = FF_DEBUG_MMCO }, INT_MIN, INT_MAX, V|D, "debug"}, +{"motion_vec", "macroblock (MB) motion vector values", 0, AV_OPT_TYPE_CONST, {.i64 = FF_DEBUG_MOTION_VEC }, INT_MIN, INT_MAX, V|D, "debug"}, {"bugs", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = FF_DEBUG_BUGS }, INT_MIN, INT_MAX, V|D, "debug"}, {"buffers", "picture buffer allocations", 0, AV_OPT_TYPE_CONST, {.i64 = FF_DEBUG_BUFFERS }, INT_MIN, INT_MAX, V|D, "debug"}, {"thread_ops", "threading operations", 0, AV_OPT_TYPE_CONST, {.i64 = FF_DEBUG_THREADS }, INT_MIN, INT_MAX, V|A|D, "debug"},