[FFmpeg-devel,2/2] swscale: Add swscale and fate support for AYUV

Submitted by Linjie Fu on Aug. 28, 2019, 6:22 a.m.

Details

Message ID 1566973361-19277-1-git-send-email-linjie.fu@intel.com
State New
Headers show

Commit Message

Linjie Fu Aug. 28, 2019, 6:22 a.m.
Add swscale support for AYUV and make it more robust.

Also update the reference in fate.

Signed-off-by: Linjie Fu <linjie.fu@intel.com>
---
 libswscale/input.c                       | 26 +++++++++++++++++
 libswscale/output.c                      | 50 ++++++++++++++++++++++++++++++++
 libswscale/utils.c                       |  1 +
 libswscale/version.h                     |  2 +-
 tests/ref/fate/filter-pixdesc-ayuv       |  1 +
 tests/ref/fate/filter-pixfmts-copy       |  1 +
 tests/ref/fate/filter-pixfmts-crop       |  1 +
 tests/ref/fate/filter-pixfmts-field      |  1 +
 tests/ref/fate/filter-pixfmts-fieldorder |  1 +
 tests/ref/fate/filter-pixfmts-hflip      |  1 +
 tests/ref/fate/filter-pixfmts-il         |  1 +
 tests/ref/fate/filter-pixfmts-null       |  1 +
 tests/ref/fate/filter-pixfmts-pad        |  1 +
 tests/ref/fate/filter-pixfmts-scale      |  1 +
 tests/ref/fate/filter-pixfmts-transpose  |  1 +
 tests/ref/fate/filter-pixfmts-vflip      |  1 +
 tests/ref/fate/pixfmt_best               |  2 +-
 tests/ref/fate/sws-pixdesc-query         |  3 ++
 18 files changed, 94 insertions(+), 2 deletions(-)
 create mode 100644 tests/ref/fate/filter-pixdesc-ayuv

Comments

Carl Eugen Hoyos Aug. 28, 2019, 6:58 a.m.
> Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>:
> 
> Add swscale support for AYUV and make it more robust.

The path that converts from AYUV to planar YUV(A) seems to be missing the handling of alpha values.

Carl Eugen
Linjie Fu Aug. 28, 2019, 7:51 a.m.
> -----Original Message-----

> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf

> Of Carl Eugen Hoyos

> Sent: Wednesday, August 28, 2019 14:58

> To: FFmpeg development discussions and patches <ffmpeg-

> devel@ffmpeg.org>

> Subject: Re: [FFmpeg-devel] [PATCH 2/2] swscale: Add swscale and fate

> support for AYUV

> 

> 

> 

> > Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>:

> >

> > Add swscale support for AYUV and make it more robust.

> 

> The path that converts from AYUV to planar YUV(A) seems to be missing the

> handling of alpha values.


Alpha channel information in AYUV are treated in the same way for RGBA/BGRA and
is reusing rgbaToA_c to handle the alpha values.

When converting AYUV to YUA/YUVA, two methods are acceptable:
- For YUV, just cut the alpha and discard the transparency 
- For format which also contains an alpha channel(YUV444P), copy the alpha information seems good.
Carl Eugen Hoyos Aug. 28, 2019, 9:17 p.m.
Am Mi., 28. Aug. 2019 um 09:52 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>:
>
> > -----Original Message-----
> > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf
> > Of Carl Eugen Hoyos
> > Sent: Wednesday, August 28, 2019 14:58
> > To: FFmpeg development discussions and patches <ffmpeg-
> > devel@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH 2/2] swscale: Add swscale and fate
> > support for AYUV
> >
> >
> >
> > > Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>:
> > >
> > > Add swscale support for AYUV and make it more robust.
> >
> > The path that converts from AYUV to planar YUV(A) seems to be missing the
> > handling of alpha values.
>
> Alpha channel information in AYUV are treated in the same way for RGBA/BGRA and
> is reusing rgbaToA_c to handle the alpha values.
>
> When converting AYUV to YUA/YUVA, two methods are acceptable:
> - For YUV, just cut the alpha and discard the transparency
> - For format which also contains an alpha channel(YUV444P), copy the
> alpha information seems good.

But since the input does not contain (valid) alpha, this does not apply.

Carl Eugen
Linjie Fu Aug. 29, 2019, 1:49 a.m.
> -----Original Message-----

> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf

> Of Carl Eugen Hoyos

> Sent: Thursday, August 29, 2019 05:18

> To: FFmpeg development discussions and patches <ffmpeg-

> devel@ffmpeg.org>

> Subject: Re: [FFmpeg-devel] [PATCH 2/2] swscale: Add swscale and fate

> support for AYUV

> 

> Am Mi., 28. Aug. 2019 um 09:52 Uhr schrieb Fu, Linjie <linjie.fu@intel.com>:

> >

> > > -----Original Message-----

> > > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On

> Behalf

> > > Of Carl Eugen Hoyos

> > > Sent: Wednesday, August 28, 2019 14:58

> > > To: FFmpeg development discussions and patches <ffmpeg-

> > > devel@ffmpeg.org>

> > > Subject: Re: [FFmpeg-devel] [PATCH 2/2] swscale: Add swscale and fate

> > > support for AYUV

> > >

> > >

> > >

> > > > Am 28.08.2019 um 08:22 schrieb Linjie Fu <linjie.fu@intel.com>:

> > > >

> > > > Add swscale support for AYUV and make it more robust.

> > >

> > > The path that converts from AYUV to planar YUV(A) seems to be missing

> the

> > > handling of alpha values.

> >

> > Alpha channel information in AYUV are treated in the same way for

> RGBA/BGRA and

> > is reusing rgbaToA_c to handle the alpha values.

> >

> > When converting AYUV to YUA/YUVA, two methods are acceptable:

> > - For YUV, just cut the alpha and discard the transparency

> > - For format which also contains an alpha channel(YUV444P), copy the

> > alpha information seems good.

> 

> But since the input does not contain (valid) alpha, this does not apply.


If we convert from a input with valid alpha, then it makes sense. (like ARGB to AYUV)

And IMHO, swscale function should focus on CSC itself and should be able to cope with
the color space convert even if the alpha data is kind of not valid.
The validation judgement for alpha may be better if it was implemented in the user level.

- linjie

Patch hide | download patch | download mbox

diff --git a/libswscale/input.c b/libswscale/input.c
index 064f8da..73d1aa9 100644
--- a/libswscale/input.c
+++ b/libswscale/input.c
@@ -552,6 +552,25 @@  static void yvy2ToUV_c(uint8_t *dstU, uint8_t *dstV, const uint8_t *unused0, con
     av_assert1(src1 == src2);
 }
 
+static void ayuvToY_c(uint8_t *dst, const uint8_t *src, const uint8_t *unused1, const uint8_t *unused2,  int width,
+                      uint32_t *unused)
+{
+    int i;
+    for (i = 0; i < width; i++)
+        dst[i] = src[4 * i + 2];
+}
+
+static void ayuvToUV_c(uint8_t *dstU, uint8_t *dstV, const uint8_t *unused0, const uint8_t *src1,
+                       const uint8_t *src2, int width, uint32_t *unused)
+{
+    int i;
+    for (i = 0; i < width; i++) {
+        dstV[i] = src1[4 * i];
+        dstU[i] = src1[4 * i + 1];
+    }
+    av_assert1(src1 == src2);
+}
+
 static void bswap16Y_c(uint8_t *_dst, const uint8_t *_src, const uint8_t *unused1, const uint8_t *unused2, int width,
                        uint32_t *unused)
 {
@@ -1154,6 +1173,9 @@  av_cold void ff_sws_init_input_funcs(SwsContext *c)
     case AV_PIX_FMT_P016BE:
         c->chrToYV12 = p016BEToUV_c;
         break;
+    case AV_PIX_FMT_AYUV:
+        c->chrToYV12 = ayuvToUV_c;
+        break;
     }
     if (c->chrSrcHSubSample) {
         switch (srcFormat) {
@@ -1586,6 +1608,9 @@  av_cold void ff_sws_init_input_funcs(SwsContext *c)
         c->lumToYV12 = grayf32ToY16_bswap_c;
 #endif
         break;
+    case AV_PIX_FMT_AYUV:
+        c->lumToYV12 = ayuvToY_c;
+        break;
     }
     if (c->needAlpha) {
         if (is16BPS(srcFormat) || isNBPS(srcFormat)) {
@@ -1599,6 +1624,7 @@  av_cold void ff_sws_init_input_funcs(SwsContext *c)
         case AV_PIX_FMT_RGBA64BE:  c->alpToYV12 = rgba64beToA_c; break;
         case AV_PIX_FMT_BGRA:
         case AV_PIX_FMT_RGBA:
+        case AV_PIX_FMT_AYUV:
             c->alpToYV12 = rgbaToA_c;
             break;
         case AV_PIX_FMT_ABGR:
diff --git a/libswscale/output.c b/libswscale/output.c
index 26b0ff3..d7c0818 100644
--- a/libswscale/output.c
+++ b/libswscale/output.c
@@ -2403,6 +2403,53 @@  yuv2ya8_X_c(SwsContext *c, const int16_t *lumFilter,
 }
 
 static void
+yuv2ayuv_X_c(SwsContext *c, const int16_t *lumFilter,
+                 const int16_t **lumSrc, int lumFilterSize,
+                 const int16_t *chrFilter, const int16_t **chrUSrc,
+                 const int16_t **chrVSrc, int chrFilterSize,
+                 const int16_t **alpSrc, uint8_t *dest, int dstW, int y)
+{
+    int hasAlpha = !!alpSrc;
+    int i;
+
+    for (i = 0; i < dstW; i++) {
+        int j;
+        int A = 1 << 18;
+        int Y = 1 << 18;
+        int U = 1 << 18;
+        int V = 1 << 18;
+
+        for (j = 0; j < lumFilterSize; j++) {
+            Y += lumSrc[j][i]  * lumFilter[j];
+        }
+        for (j = 0; j < chrFilterSize; j++) {
+            U += chrUSrc[j][i] * chrFilter[j];
+            V += chrVSrc[j][i] * chrFilter[j];
+        }
+        if (hasAlpha)
+            for (j = 0; j < lumFilterSize; j++)
+                A += alpSrc[j][i] * lumFilter[j];
+        A >>= 19;
+        Y >>= 19;
+        U >>= 19;
+        V >>= 19;
+        A = hasAlpha ? A : 255;
+
+        if ((A | Y | U | V) & 0x100) {
+            A = av_clip_uint8(A);
+            Y = av_clip_uint8(Y);
+            U = av_clip_uint8(U);
+            V = av_clip_uint8(V);
+        }
+
+        dest[4*i]     = V;
+        dest[4*i + 1] = U;
+        dest[4*i + 2] = Y;
+        dest[4*i + 3] = A;
+    }
+}
+
+static void
 yuv2ayuv64le_X_c(SwsContext *c, const int16_t *lumFilter,
                  const int16_t **_lumSrc, int lumFilterSize,
                  const int16_t *chrFilter, const int16_t **_chrUSrc,
@@ -2932,6 +2979,9 @@  av_cold void ff_sws_init_output_funcs(SwsContext *c,
         *yuv2packed2 = yuv2ya16be_2_c;
         *yuv2packedX = yuv2ya16be_X_c;
         break;
+    case AV_PIX_FMT_AYUV:
+        *yuv2packedX = yuv2ayuv_X_c;
+        break;
     case AV_PIX_FMT_AYUV64LE:
         *yuv2packedX = yuv2ayuv64le_X_c;
         break;
diff --git a/libswscale/utils.c b/libswscale/utils.c
index 1b1f779..1e12e21 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -266,6 +266,7 @@  static const FormatEntry format_entries[AV_PIX_FMT_NB] = {
     [AV_PIX_FMT_YUVA444P12LE] = { 1, 1 },
     [AV_PIX_FMT_NV24]        = { 1, 1 },
     [AV_PIX_FMT_NV42]        = { 1, 1 },
+    [AV_PIX_FMT_AYUV]        = { 1, 1 },
 };
 
 int sws_isSupportedInput(enum AVPixelFormat pix_fmt)
diff --git a/libswscale/version.h b/libswscale/version.h
index cc434c0..1dbb561 100644
--- a/libswscale/version.h
+++ b/libswscale/version.h
@@ -28,7 +28,7 @@ 
 
 #define LIBSWSCALE_VERSION_MAJOR   5
 #define LIBSWSCALE_VERSION_MINOR   6
-#define LIBSWSCALE_VERSION_MICRO 100
+#define LIBSWSCALE_VERSION_MICRO 101
 
 #define LIBSWSCALE_VERSION_INT  AV_VERSION_INT(LIBSWSCALE_VERSION_MAJOR, \
                                                LIBSWSCALE_VERSION_MINOR, \
diff --git a/tests/ref/fate/filter-pixdesc-ayuv b/tests/ref/fate/filter-pixdesc-ayuv
new file mode 100644
index 0000000..8e26ac8
--- /dev/null
+++ b/tests/ref/fate/filter-pixdesc-ayuv
@@ -0,0 +1 @@ 
+pixdesc-ayuv        a27703ac894af1a90df131dc3c590833
diff --git a/tests/ref/fate/filter-pixfmts-copy b/tests/ref/fate/filter-pixfmts-copy
index 4675b6e..18b4746 100644
--- a/tests/ref/fate/filter-pixfmts-copy
+++ b/tests/ref/fate/filter-pixfmts-copy
@@ -2,6 +2,7 @@ 
 0rgb                527ef3d164c8fd0700493733959689c2
 abgr                023ecf6396d324edb113e4a483b79ba2
 argb                f003b555ef429222005d33844cca9325
+ayuv                0af13a42f9d0932c5a9bb6a8a5d1c5ee
 ayuv64le            07b9c969dfbe4add4c0626773b151d4f
 bgr0                6fcd67c8e6cec723dab21c70cf53dc16
 bgr24               4cff3814819f02ecf5824edfd768d2b1
diff --git a/tests/ref/fate/filter-pixfmts-crop b/tests/ref/fate/filter-pixfmts-crop
index 4b9f67c..7a4430c 100644
--- a/tests/ref/fate/filter-pixfmts-crop
+++ b/tests/ref/fate/filter-pixfmts-crop
@@ -2,6 +2,7 @@ 
 0rgb                974833c777e6abe6d84dc59af2ca5625
 abgr                1d21f5b8a20186ac9dd54459c986a2a7
 argb                8b822972049a1e207000763f2564d6e0
+ayuv                615241c5406eb556fca0ad8606c23a02
 ayuv64le            ab2f7bc8f150af47c42c778e3ea28bce
 bgr0                38a84849a9198667c348c686802e3b52
 bgr24               1dacd8e04bf0eff163e82250d01a9cc7
diff --git a/tests/ref/fate/filter-pixfmts-field b/tests/ref/fate/filter-pixfmts-field
index 059347e..b8adb8a 100644
--- a/tests/ref/fate/filter-pixfmts-field
+++ b/tests/ref/fate/filter-pixfmts-field
@@ -2,6 +2,7 @@ 
 0rgb                e2c35753a2271d1f9455b1809bc0e907
 abgr                c0eb95959edf5d40ff8af315e62d0f8a
 argb                6dca4f2987b49b7d63f702d17bace630
+ayuv                3d02eeab336d0a8106f6fdd91be61073
 ayuv64le            d9836decca6323ba88b3b3d02257c0b6
 bgr0                1da3fdbac616b3b410d081e39ed7a1f6
 bgr24               573c76d77b1cbe6534ea7c0267dc1b13
diff --git a/tests/ref/fate/filter-pixfmts-fieldorder b/tests/ref/fate/filter-pixfmts-fieldorder
index 066b944..ea35c41 100644
--- a/tests/ref/fate/filter-pixfmts-fieldorder
+++ b/tests/ref/fate/filter-pixfmts-fieldorder
@@ -2,6 +2,7 @@ 
 0rgb                2b0f066cfa0bef378a492875d541de8f
 abgr                832924b5351361db68dbdbb96c60ae55
 argb                80d08e68cb91bc8f2f817516e65f0bd0
+ayuv                9e4480c5fcb7c091ec3e517420764ef3
 ayuv64le            84ef6260fe02427da946d4a2207fb54c
 bgr0                d2c676224ea80ac3ce01afde325ea1a0
 bgr24               b7fdbcd10f20e6ea2d40aae0f329f80d
diff --git a/tests/ref/fate/filter-pixfmts-hflip b/tests/ref/fate/filter-pixfmts-hflip
index 100dd70..839dc71 100644
--- a/tests/ref/fate/filter-pixfmts-hflip
+++ b/tests/ref/fate/filter-pixfmts-hflip
@@ -2,6 +2,7 @@ 
 0rgb                ada57572ee2b35f86edac9b911ce8523
 abgr                d2da6c3ee72e4a89a7cd011dd08566b2
 argb                36cf791c52c5463bfc52a070de54337e
+ayuv                f1d087284fb1556d76e6def5f94bf273
 ayuv64le            4cedbc38b3d4dcb26cdab170ce6d667b
 bgr0                66e9fda4e658d73bfe4fc9d792542271
 bgr24               db074979bd684ca4547e28681ad3f6ab
diff --git a/tests/ref/fate/filter-pixfmts-il b/tests/ref/fate/filter-pixfmts-il
index 979eb0c..d3027b2 100644
--- a/tests/ref/fate/filter-pixfmts-il
+++ b/tests/ref/fate/filter-pixfmts-il
@@ -2,6 +2,7 @@ 
 0rgb                53efe0182723cd1dedfdbf56357c76f5
 abgr                97603869e6248a8e5d8501563a11b114
 argb                9e50e6ef02c83f28e97865a1f46ddfcd
+ayuv                4251d94ee49e6a3cc1c10c09cd331308
 ayuv64le            6f45f683e99ddf4180c7c7f47719efcc
 bgr0                590dcd1297d1dd4541eea217381db604
 bgr24               73afe7b447b083a7c2d682abe8dd451a
diff --git a/tests/ref/fate/filter-pixfmts-null b/tests/ref/fate/filter-pixfmts-null
index 4675b6e..18b4746 100644
--- a/tests/ref/fate/filter-pixfmts-null
+++ b/tests/ref/fate/filter-pixfmts-null
@@ -2,6 +2,7 @@ 
 0rgb                527ef3d164c8fd0700493733959689c2
 abgr                023ecf6396d324edb113e4a483b79ba2
 argb                f003b555ef429222005d33844cca9325
+ayuv                0af13a42f9d0932c5a9bb6a8a5d1c5ee
 ayuv64le            07b9c969dfbe4add4c0626773b151d4f
 bgr0                6fcd67c8e6cec723dab21c70cf53dc16
 bgr24               4cff3814819f02ecf5824edfd768d2b1
diff --git a/tests/ref/fate/filter-pixfmts-pad b/tests/ref/fate/filter-pixfmts-pad
index 41ccec8..2795844 100644
--- a/tests/ref/fate/filter-pixfmts-pad
+++ b/tests/ref/fate/filter-pixfmts-pad
@@ -2,6 +2,7 @@ 
 0rgb                ff12e0f1e576b47a4c962729d5c0b868
 abgr                52738042432893de555e6a3833172806
 argb                2a10108ac524b422b8a2393c064b3eab
+ayuv                93fca69f640574dc7d8cce6282f72e96
 bgr0                32207a2de1b2ac7937e940a8459b97c0
 bgr24               f8b65ad845905c7d0c93ca28dfbb826f
 bgra                929aac15e848038e367c250037575f9f
diff --git a/tests/ref/fate/filter-pixfmts-scale b/tests/ref/fate/filter-pixfmts-scale
index 2f38241..c6b0f2c 100644
--- a/tests/ref/fate/filter-pixfmts-scale
+++ b/tests/ref/fate/filter-pixfmts-scale
@@ -2,6 +2,7 @@ 
 0rgb                80a58af8c639743307207ab4b69ca863
 abgr                63f2eaa8712ea6108985f4a0b83587c9
 argb                f0e17c71a40643c33a5bcfb481f6d8f8
+ayuv                34b705030d4ac10668cd88e0d069136a
 ayuv64le            59fb016f9874062d0be77cb3920ffed2
 bgr0                243d58ca64f97b2f415b4c63cb79f0e1
 bgr24               18744aaab4b8bce065a7144dc0ccf921
diff --git a/tests/ref/fate/filter-pixfmts-transpose b/tests/ref/fate/filter-pixfmts-transpose
index b2ab3b7..2031288 100644
--- a/tests/ref/fate/filter-pixfmts-transpose
+++ b/tests/ref/fate/filter-pixfmts-transpose
@@ -2,6 +2,7 @@ 
 0rgb                cf1bedd0784a3efd3ab00c4e44005c37
 abgr                6d6f896f853a6c6f93ee70dba9af3d17
 argb                87bbd23debb94d486ac3a6b6c0b005f9
+ayuv                46b5b821d7ee6ddedb3ddafd1e5b007c
 ayuv64le            e4c07e0d5b333b3bc9eb4f3ce6af3a2c
 bgr0                df3a6eedd4939ce09a357b655ac2962a
 bgr24               f9a08135e5d58c0b2a5509c369a88414
diff --git a/tests/ref/fate/filter-pixfmts-vflip b/tests/ref/fate/filter-pixfmts-vflip
index e4d58f9..785c9cd 100644
--- a/tests/ref/fate/filter-pixfmts-vflip
+++ b/tests/ref/fate/filter-pixfmts-vflip
@@ -2,6 +2,7 @@ 
 0rgb                76b792f8ce8a72925e04294dc2f25b36
 abgr                8b94f489e68802d76f1e2844688a4911
 argb                3fd6af7ef2364d8aa845d45db289a04a
+ayuv                ed7de87da324b39090a8961dfd56ca5a
 ayuv64le            558671dd31d0754cfa6344eaf441df78
 bgr0                7117438cf000254610f23625265769b5
 bgr24               52b2c21cbc166978a38a646c354b6858
diff --git a/tests/ref/fate/pixfmt_best b/tests/ref/fate/pixfmt_best
index 5f51e2d..1da1846 100644
--- a/tests/ref/fate/pixfmt_best
+++ b/tests/ref/fate/pixfmt_best
@@ -1 +1 @@ 
-73 tests passed, 0 tests failed.
+74 tests passed, 0 tests failed.
diff --git a/tests/ref/fate/sws-pixdesc-query b/tests/ref/fate/sws-pixdesc-query
index e234922..28696fe 100644
--- a/tests/ref/fate/sws-pixdesc-query
+++ b/tests/ref/fate/sws-pixdesc-query
@@ -171,6 +171,7 @@  isBE:
   yuva444p9be
 
 isYUV:
+  ayuv
   ayuv64be
   ayuv64le
   nv12
@@ -579,6 +580,7 @@  AnyRGB:
   rgba64le
 
 ALPHA:
+  ayuv
   ayuv64be
   ayuv64le
   bgr32
@@ -631,6 +633,7 @@  ALPHA:
 Packed:
   0bgr
   0rgb
+  ayuv
   ayuv64be
   ayuv64le
   bayer_bggr16be