[FFmpeg-devel,v2,1/2] fftools/ffmpeg_filter, ffplay: Add flip support to rotation

Submitted by Jun Li on May 6, 2019, 3:41 a.m.

Details

Message ID 20190506034128.22929-1-junli1026@gmail.com
State New
Headers show

Commit Message

Jun Li May 6, 2019, 3:41 a.m.
Current implemantion for autoratation does not support flip.
That is, if the matrix contains flip info, the API get_rotation
only reflects partial information. This change is for adding
support for hflip (vflip can be achieved by rotation+hflip).
---
 fftools/cmdutils.c        |  4 ++--
 fftools/cmdutils.h        |  2 +-
 fftools/ffmpeg_filter.c   | 31 ++++++++++++++++++++++++++-----
 fftools/ffplay.c          | 22 ++++++++++++++++++----
 libavutil/display.c       | 12 ++++++++++++
 libavutil/display.h       | 14 ++++++++++++++
 libavutil/tests/display.c |  8 ++++++++
 tests/ref/fate/display    |  4 ++++
 8 files changed, 85 insertions(+), 12 deletions(-)

Comments

Michael Niedermayer May 7, 2019, 10:10 p.m.
On Sun, May 05, 2019 at 08:41:27PM -0700, Jun Li wrote:
> Current implemantion for autoratation does not support flip.
> That is, if the matrix contains flip info, the API get_rotation
> only reflects partial information. This change is for adding
> support for hflip (vflip can be achieved by rotation+hflip).
> ---
>  fftools/cmdutils.c        |  4 ++--
>  fftools/cmdutils.h        |  2 +-
>  fftools/ffmpeg_filter.c   | 31 ++++++++++++++++++++++++++-----
>  fftools/ffplay.c          | 22 ++++++++++++++++++----
>  libavutil/display.c       | 12 ++++++++++++
>  libavutil/display.h       | 14 ++++++++++++++
>  libavutil/tests/display.c |  8 ++++++++
>  tests/ref/fate/display    |  4 ++++
>  8 files changed, 85 insertions(+), 12 deletions(-)

this caused people to hang from the ceiling in:
./ffplay ~/issues/1086/47j9R7PXBep.mov

  Metadata:
    major_brand     : qt  
    minor_version   : 537199360
    compatible_brands: qt  
    creation_time   : 2009-05-05T17:20:23.000000Z
    com.apple.quicktime.player.movie.audio.gain: 1.000000
    com.apple.quicktime.player.movie.audio.treble: 0.000000
    com.apple.quicktime.player.movie.audio.bass: 0.000000
    com.apple.quicktime.player.movie.audio.balance: 0.000000
    com.apple.quicktime.player.movie.audio.pitchshift: 0.000000
    com.apple.quicktime.player.movie.audio.mute: 
    com.apple.quicktime.player.movie.visual.brightness: 0.000000
    com.apple.quicktime.player.movie.visual.color: 1.000000
    com.apple.quicktime.player.movie.visual.tint: 0.000000
    com.apple.quicktime.player.movie.visual.contrast: 1.000000
    com.apple.quicktime.player.version: 7.6 (7.6)
    com.apple.quicktime.version: 7.6.0 (1290) 0x7608000 (Mac OS X, 10.5.6, 9G55)
    compilation     : 0
    encoder         : Sorenson Squeeze 4.5
  Duration: 00:00:30.58, start: 0.000000, bitrate: 2006 kb/s
    Stream #0:0: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      rotate          : 0
      creation_time   : 2009-05-05T17:20:23.000000Z
      handler_name    : Apple Sound Media Handler
    Stream #0:1: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 320x240, 1544 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      rotate          : 0
      creation_time   : 2009-05-05T17:20:23.000000Z
      handler_name    : Apple Video Media Handler
      encoder         : AVC Coding
    Side data:
      displaymatrix: rotation of -0.00 degrees
    Stream #0:2: Data: none (mp4s / 0x7334706D) (default)
    Metadata:
      rotate          : 0
      creation_time   : 2009-05-05T17:20:23.000000Z
      handler_name    : Apple MPEG-4 Scene Media Handler
    Stream #0:3: Data: none (mp4s / 0x7334706D) (default)
    Metadata:
      rotate          : 0
      creation_time   : 2009-05-05T17:20:23.000000Z
      handler_name    : Apple MPEG-4 ODSM Media Handler
    Stream #0:4(eng): Data: none (rtp  / 0x20707472), 14 kb/s
    Metadata:
      rotate          : 0
      creation_time   : 2009-05-05T17:20:23.000000Z
      handler_name    : hint media handler
    Stream #0:5(eng): Data: none (rtp  / 0x20707472), 64 kb/s
    Metadata:
      rotate          : 0
      creation_time   : 2009-05-05T17:20:23.000000Z
      handler_name    : hint media handler
[...]
Jun Li May 7, 2019, 10:15 p.m.
On Tue, May 7, 2019 at 3:10 PM Michael Niedermayer <michael@niedermayer.cc>
wrote:

> On Sun, May 05, 2019 at 08:41:27PM -0700, Jun Li wrote:
> > Current implemantion for autoratation does not support flip.
> > That is, if the matrix contains flip info, the API get_rotation
> > only reflects partial information. This change is for adding
> > support for hflip (vflip can be achieved by rotation+hflip).
> > ---
> >  fftools/cmdutils.c        |  4 ++--
> >  fftools/cmdutils.h        |  2 +-
> >  fftools/ffmpeg_filter.c   | 31 ++++++++++++++++++++++++++-----
> >  fftools/ffplay.c          | 22 ++++++++++++++++++----
> >  libavutil/display.c       | 12 ++++++++++++
> >  libavutil/display.h       | 14 ++++++++++++++
> >  libavutil/tests/display.c |  8 ++++++++
> >  tests/ref/fate/display    |  4 ++++
> >  8 files changed, 85 insertions(+), 12 deletions(-)
>
> this caused people to hang from the ceiling in:
> ./ffplay ~/issues/1086/47j9R7PXBep.mov
>
>   Metadata:
>     major_brand     : qt
>     minor_version   : 537199360
>     compatible_brands: qt
>     creation_time   : 2009-05-05T17:20:23.000000Z
>     com.apple.quicktime.player.movie.audio.gain: 1.000000
>     com.apple.quicktime.player.movie.audio.treble: 0.000000
>     com.apple.quicktime.player.movie.audio.bass: 0.000000
>     com.apple.quicktime.player.movie.audio.balance: 0.000000
>     com.apple.quicktime.player.movie.audio.pitchshift: 0.000000
>     com.apple.quicktime.player.movie.audio.mute:
>     com.apple.quicktime.player.movie.visual.brightness: 0.000000
>     com.apple.quicktime.player.movie.visual.color: 1.000000
>     com.apple.quicktime.player.movie.visual.tint: 0.000000
>     com.apple.quicktime.player.movie.visual.contrast: 1.000000
>     com.apple.quicktime.player.version: 7.6 (7.6)
>     com.apple.quicktime.version: 7.6.0 (1290) 0x7608000 (Mac OS X, 10.5.6,
> 9G55)
>     compilation     : 0
>     encoder         : Sorenson Squeeze 4.5
>   Duration: 00:00:30.58, start: 0.000000, bitrate: 2006 kb/s
>     Stream #0:0: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo,
> fltp, 127 kb/s (default)
>     Metadata:
>       rotate          : 0
>       creation_time   : 2009-05-05T17:20:23.000000Z
>       handler_name    : Apple Sound Media Handler
>     Stream #0:1: Video: h264 (Constrained Baseline) (avc1 / 0x31637661),
> yuv420p, 320x240, 1544 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 180k tbc
> (default)
>     Metadata:
>       rotate          : 0
>       creation_time   : 2009-05-05T17:20:23.000000Z
>       handler_name    : Apple Video Media Handler
>       encoder         : AVC Coding
>     Side data:
>       displaymatrix: rotation of -0.00 degrees
>     Stream #0:2: Data: none (mp4s / 0x7334706D) (default)
>     Metadata:
>       rotate          : 0
>       creation_time   : 2009-05-05T17:20:23.000000Z
>       handler_name    : Apple MPEG-4 Scene Media Handler
>     Stream #0:3: Data: none (mp4s / 0x7334706D) (default)
>     Metadata:
>       rotate          : 0
>       creation_time   : 2009-05-05T17:20:23.000000Z
>       handler_name    : Apple MPEG-4 ODSM Media Handler
>     Stream #0:4(eng): Data: none (rtp  / 0x20707472), 14 kb/s
>     Metadata:
>       rotate          : 0
>       creation_time   : 2009-05-05T17:20:23.000000Z
>       handler_name    : hint media handler
>     Stream #0:5(eng): Data: none (rtp  / 0x20707472), 64 kb/s
>     Metadata:
>       rotate          : 0
>       creation_time   : 2009-05-05T17:20:23.000000Z
>       handler_name    : hint media handler
> [...]
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> The real ebay dictionary, page 2
> "100% positive feedback" - "All either got their money back or didnt
> complain"
> "Best seller ever, very honest" - "Seller refunded buyer after failed scam"
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".


Thanks Michael for testing.
Do you mind sharing the content "~/issues/1086/47j9R7PXBep.mov" ?

Best Regards,
Jun
Michael Niedermayer May 7, 2019, 10:33 p.m.
On Tue, May 07, 2019 at 03:15:33PM -0700, Jun Li wrote:
> On Tue, May 7, 2019 at 3:10 PM Michael Niedermayer <michael@niedermayer.cc>
> wrote:
> 
> > On Sun, May 05, 2019 at 08:41:27PM -0700, Jun Li wrote:
> > > Current implemantion for autoratation does not support flip.
> > > That is, if the matrix contains flip info, the API get_rotation
> > > only reflects partial information. This change is for adding
> > > support for hflip (vflip can be achieved by rotation+hflip).
> > > ---
> > >  fftools/cmdutils.c        |  4 ++--
> > >  fftools/cmdutils.h        |  2 +-
> > >  fftools/ffmpeg_filter.c   | 31 ++++++++++++++++++++++++++-----
> > >  fftools/ffplay.c          | 22 ++++++++++++++++++----
> > >  libavutil/display.c       | 12 ++++++++++++
> > >  libavutil/display.h       | 14 ++++++++++++++
> > >  libavutil/tests/display.c |  8 ++++++++
> > >  tests/ref/fate/display    |  4 ++++
> > >  8 files changed, 85 insertions(+), 12 deletions(-)
> >
> > this caused people to hang from the ceiling in:
> > ./ffplay ~/issues/1086/47j9R7PXBep.mov
> >
> >   Metadata:
> >     major_brand     : qt
> >     minor_version   : 537199360
> >     compatible_brands: qt
> >     creation_time   : 2009-05-05T17:20:23.000000Z
> >     com.apple.quicktime.player.movie.audio.gain: 1.000000
> >     com.apple.quicktime.player.movie.audio.treble: 0.000000
> >     com.apple.quicktime.player.movie.audio.bass: 0.000000
> >     com.apple.quicktime.player.movie.audio.balance: 0.000000
> >     com.apple.quicktime.player.movie.audio.pitchshift: 0.000000
> >     com.apple.quicktime.player.movie.audio.mute:
> >     com.apple.quicktime.player.movie.visual.brightness: 0.000000
> >     com.apple.quicktime.player.movie.visual.color: 1.000000
> >     com.apple.quicktime.player.movie.visual.tint: 0.000000
> >     com.apple.quicktime.player.movie.visual.contrast: 1.000000
> >     com.apple.quicktime.player.version: 7.6 (7.6)
> >     com.apple.quicktime.version: 7.6.0 (1290) 0x7608000 (Mac OS X, 10.5.6,
> > 9G55)
> >     compilation     : 0
> >     encoder         : Sorenson Squeeze 4.5
> >   Duration: 00:00:30.58, start: 0.000000, bitrate: 2006 kb/s
> >     Stream #0:0: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo,
> > fltp, 127 kb/s (default)
> >     Metadata:
> >       rotate          : 0
> >       creation_time   : 2009-05-05T17:20:23.000000Z
> >       handler_name    : Apple Sound Media Handler
> >     Stream #0:1: Video: h264 (Constrained Baseline) (avc1 / 0x31637661),
> > yuv420p, 320x240, 1544 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 180k tbc
> > (default)
> >     Metadata:
> >       rotate          : 0
> >       creation_time   : 2009-05-05T17:20:23.000000Z
> >       handler_name    : Apple Video Media Handler
> >       encoder         : AVC Coding
> >     Side data:
> >       displaymatrix: rotation of -0.00 degrees
> >     Stream #0:2: Data: none (mp4s / 0x7334706D) (default)
> >     Metadata:
> >       rotate          : 0
> >       creation_time   : 2009-05-05T17:20:23.000000Z
> >       handler_name    : Apple MPEG-4 Scene Media Handler
> >     Stream #0:3: Data: none (mp4s / 0x7334706D) (default)
> >     Metadata:
> >       rotate          : 0
> >       creation_time   : 2009-05-05T17:20:23.000000Z
> >       handler_name    : Apple MPEG-4 ODSM Media Handler
> >     Stream #0:4(eng): Data: none (rtp  / 0x20707472), 14 kb/s
> >     Metadata:
> >       rotate          : 0
> >       creation_time   : 2009-05-05T17:20:23.000000Z
> >       handler_name    : hint media handler
> >     Stream #0:5(eng): Data: none (rtp  / 0x20707472), 64 kb/s
> >     Metadata:
> >       rotate          : 0
> >       creation_time   : 2009-05-05T17:20:23.000000Z
> >       handler_name    : hint media handler
> > [...]
> > --
> > Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> >
> > The real ebay dictionary, page 2
> > "100% positive feedback" - "All either got their money back or didnt
> > complain"
> > "Best seller ever, very honest" - "Seller refunded buyer after failed scam"
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> 
> 
> Thanks Michael for testing.
> Do you mind sharing the content "~/issues/1086/47j9R7PXBep.mov" ?

sent privatly 

Thanks

[...]

Patch hide | download patch | download mbox

diff --git a/fftools/cmdutils.c b/fftools/cmdutils.c
index 9cfbc45c2b..1235a3dd7b 100644
--- a/fftools/cmdutils.c
+++ b/fftools/cmdutils.c
@@ -2172,13 +2172,13 @@  void *grow_array(void *array, int elem_size, int *size, int new_size)
     return array;
 }
 
-double get_rotation(AVStream *st)
+double get_rotation_hflip(AVStream *st, int* hflip)
 {
     uint8_t* displaymatrix = av_stream_get_side_data(st,
                                                      AV_PKT_DATA_DISPLAYMATRIX, NULL);
     double theta = 0;
     if (displaymatrix)
-        theta = -av_display_rotation_get((int32_t*) displaymatrix);
+        theta = -av_display_rotation_hflip_get((int32_t*) displaymatrix, hflip);
 
     theta -= 360*floor(theta/360 + 0.9/360);
 
diff --git a/fftools/cmdutils.h b/fftools/cmdutils.h
index 6e2e0a2acb..0349d1bea7 100644
--- a/fftools/cmdutils.h
+++ b/fftools/cmdutils.h
@@ -643,6 +643,6 @@  void *grow_array(void *array, int elem_size, int *size, int new_size);
     char name[128];\
     av_get_channel_layout_string(name, sizeof(name), 0, ch_layout);
 
-double get_rotation(AVStream *st);
+double get_rotation_hflip(AVStream *st, int* hflip);
 
 #endif /* FFTOOLS_CMDUTILS_H */
diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index 72838de1e2..b000958015 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -807,22 +807,43 @@  static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter,
     last_filter = ifilter->filter;
 
     if (ist->autorotate) {
-        double theta = get_rotation(ist->st);
+        int hflip = 0;
+        double theta = get_rotation_hflip(ist->st, &hflip);
 
-        if (fabs(theta - 90) < 1.0) {
+        if (fabs(theta) < 1.0) {
+            if (hflip)
+                ret = insert_filter(&last_filter, &pad_idx, "hflip", NULL);
+        } else if (fabs(theta - 90) < 1.0) {
             ret = insert_filter(&last_filter, &pad_idx, "transpose", "clock");
-        } else if (fabs(theta - 180) < 1.0) {
-            ret = insert_filter(&last_filter, &pad_idx, "hflip", NULL);
             if (ret < 0)
                 return ret;
-            ret = insert_filter(&last_filter, &pad_idx, "vflip", NULL);
+            if (hflip)
+                ret = insert_filter(&last_filter, &pad_idx, "hflip", NULL);
+        } else if (fabs(theta - 180) < 1.0) {
+            if (hflip) { // rotate 180 and hflip equals vflip
+                ret = insert_filter(&last_filter, &pad_idx, "vflip", NULL);
+            } else {
+                ret = insert_filter(&last_filter, &pad_idx, "hflip", NULL);
+                if (ret < 0)
+                    return ret;
+                ret = insert_filter(&last_filter, &pad_idx, "vflip", NULL);
+            }
         } else if (fabs(theta - 270) < 1.0) {
             ret = insert_filter(&last_filter, &pad_idx, "transpose", "cclock");
+            if (ret < 0)
+                return ret;
+            if (hflip)
+                ret = insert_filter(&last_filter, &pad_idx, "hflip", NULL);
         } else if (fabs(theta) > 1.0) {
             char rotate_buf[64];
             snprintf(rotate_buf, sizeof(rotate_buf), "%f*PI/180", theta);
             ret = insert_filter(&last_filter, &pad_idx, "rotate", rotate_buf);
+            if (ret < 0)
+                return ret;
+            if (hflip)
+                ret = insert_filter(&last_filter, &pad_idx, "hflip", NULL);
         }
+
         if (ret < 0)
             return ret;
     }
diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index 8f050e16e6..2c77612193 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -1914,19 +1914,33 @@  static int configure_video_filters(AVFilterGraph *graph, VideoState *is, const c
 } while (0)
 
     if (autorotate) {
-        double theta  = get_rotation(is->video_st);
+        int hflip;
+        double theta  = get_rotation_hflip(is->video_st, &hflip);
 
-        if (fabs(theta - 90) < 1.0) {
+        if (fabs(theta) < 1.0) {
+            if (hflip)
+                INSERT_FILT("hflip", NULL);
+        } else if (fabs(theta - 90) < 1.0) {
             INSERT_FILT("transpose", "clock");
+            if (hflip)
+                INSERT_FILT("hflip", NULL);
         } else if (fabs(theta - 180) < 1.0) {
-            INSERT_FILT("hflip", NULL);
-            INSERT_FILT("vflip", NULL);
+            if (hflip) { // rotate 180 and hflip equals vflip
+                INSERT_FILT("vflip", NULL);
+            } else {
+                INSERT_FILT("hflip", NULL);
+                INSERT_FILT("vflip", NULL);
+            }
         } else if (fabs(theta - 270) < 1.0) {
             INSERT_FILT("transpose", "cclock");
+            if (hflip)
+                INSERT_FILT("hflip", NULL);
         } else if (fabs(theta) > 1.0) {
             char rotate_buf[64];
             snprintf(rotate_buf, sizeof(rotate_buf), "%f*PI/180", theta);
             INSERT_FILT("rotate", rotate_buf);
+            if (hflip)
+                INSERT_FILT("hflip", NULL);
         }
     }
 
diff --git a/libavutil/display.c b/libavutil/display.c
index a0076e067b..37c4b9be5d 100644
--- a/libavutil/display.c
+++ b/libavutil/display.c
@@ -71,3 +71,15 @@  void av_display_matrix_flip(int32_t matrix[9], int hflip, int vflip)
         for (i = 0; i < 9; i++)
             matrix[i] *= flip[i % 3];
 }
+
+double av_display_rotation_hflip_get(const int32_t matrix[9], int *hflip)
+{
+    int32_t m[9];
+    *hflip = 0;
+    memcpy(m, matrix, sizeof(int32_t) * 9);
+    if (m[0] != m[4] || m[1] != -m[3]) {
+        *hflip = 1;
+        av_display_matrix_flip(m, 1, 0);
+    }
+    return av_display_rotation_get(m);
+}
\ No newline at end of file
diff --git a/libavutil/display.h b/libavutil/display.h
index 515adad795..23d71a0bc5 100644
--- a/libavutil/display.h
+++ b/libavutil/display.h
@@ -106,6 +106,20 @@  void av_display_rotation_set(int32_t matrix[9], double angle);
  */
 void av_display_matrix_flip(int32_t matrix[9], int hflip, int vflip);
 
+/**
+ * Extract the rotation component and hflip status of the transformation matrix.
+ *
+ * @param matrix the transformation matrix
+ * @param hflip wille be set to 1 if the matrix need horizontal flipped
+ * @return the angle (in degrees) by which the transformation rotates the frame
+ *         counterclockwise. The angle will be in range [-180.0, 180.0],
+ *         or NaN if the matrix is singular.
+ *
+ * @note floating point numbers are inherently inexact, so callers are
+ *       recommended to round the return value to nearest integer before use.
+ */
+double av_display_rotation_hflip_get(const int32_t matrix[9], int* hflip);
+
 /**
  * @}
  * @}
diff --git a/libavutil/tests/display.c b/libavutil/tests/display.c
index 893ebb5543..65a0971e7b 100644
--- a/libavutil/tests/display.c
+++ b/libavutil/tests/display.c
@@ -35,6 +35,8 @@  static void print_matrix(int32_t matrix[9])
 int main(void)
 {
     int32_t matrix[9];
+    int hflip = 0;
+    double degree;
 
     // Set the matrix to 90 degrees
     av_display_rotation_set(matrix, 90);
@@ -56,6 +58,12 @@  int main(void)
     print_matrix(matrix);
     printf("degrees: %f\n", av_display_rotation_get(matrix));
 
+    // flip vertical
+    av_display_matrix_flip(matrix, 0, 1);
+    print_matrix(matrix);
+    degree = av_display_rotation_hflip_get(matrix, &hflip);
+    printf("degrees: %f, hflip: %i\n", degree, hflip);
+
     return 0;
 
 }
diff --git a/tests/ref/fate/display b/tests/ref/fate/display
index 251e7e0cdf..841a7235b8 100644
--- a/tests/ref/fate/display
+++ b/tests/ref/fate/display
@@ -14,3 +14,7 @@  degrees: 135.000000
 -46340 -46340 0
 0 0 1073741824
 degrees: -135.000000
+-46340 -46340 0
+-46340 46340 0
+0 0 1073741824
+degrees: 45.000000, hflip: 1
\ No newline at end of file