[FFmpeg-devel] avfilter: add zoneplate video source filter

Attached.

On date Sunday 2023-05-07 18:04:37 +0200, Paul B Mahol wrote:
> Attached.

> From 7b0a8586adc0a142f0b7afcdbdf36ce526f4cd34 Mon Sep 17 00:00:00 2001
> From: Paul B Mahol <onemda@gmail.com>
> Date: Sat, 6 May 2023 22:52:47 +0200
> Subject: [PATCH] avfilter: add zoneplate video test source
> 
> Signed-off-by: Paul B Mahol <onemda@gmail.com>
> ---
>  libavfilter/Makefile       |   1 +
>  libavfilter/allfilters.c   |   1 +
>  libavfilter/vsrc_testsrc.c | 142 +++++++++++++++++++++++++++++++++++++
>  3 files changed, 144 insertions(+)

please add docs

[...]

New version, faster and with slice threading and docs.

On date Monday 2023-05-08 01:30:34 +0200, Paul B Mahol wrote:
> New version, faster and with slice threading and docs.

> From 8b9ab6e3401d69f115b5d331fec73fd8c01ea1bd Mon Sep 17 00:00:00 2001
> From: Paul B Mahol <onemda@gmail.com>
> Date: Sat, 6 May 2023 22:52:47 +0200
> Subject: [PATCH] avfilter: add zoneplate video test source
> 
> Signed-off-by: Paul B Mahol <onemda@gmail.com>
> ---
>  doc/filters.texi           |  96 +++++++++++++++++++++++
>  libavfilter/Makefile       |   1 +
>  libavfilter/allfilters.c   |   1 +
>  libavfilter/vsrc_testsrc.c | 157 +++++++++++++++++++++++++++++++++++++
>  4 files changed, 255 insertions(+)

missing Changelog entry?

> diff --git a/doc/filters.texi b/doc/filters.texi
> index 8b443c24e9..d7c828d119 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -28080,6 +28080,102 @@ Set max jump for single pan destination. Allowed range is from 1 to 10000.
>  Set fractal type, can be default @code{carpet} or @code{triangle}.
>  @end table
>  
> +@section zoneplate
> +Generates a zoneplate test video pattern.

Nit: impersonal form (Generate) for the first sentence.

> +Numerous options for signal controls output phase in all three axis.

This source accepts ...

> +
> +This source accepts the following options:
> +
> +@table @option
> +@item size, s
> +Set frame size. For the syntax of this option, check the @ref{video size syntax,,"Video
> +size" section in the ffmpeg-utils manual,ffmpeg-utils}. Default value is "320x240".
> +
> +@item rate, r
> +Set frame rate, expressed as number of frames per second. Default
> +value is "25".
> +
> +@item duration, d
> +Set the duration of the sourced video. See
> +@ref{time duration syntax,,the Time duration section in the ffmpeg-utils(1) manual,ffmpeg-utils}
> +for the accepted syntax.
> +
> +If not specified, or the expressed duration is negative, the video is
> +supposed to be generated forever.
> +
> +@item sar
> +Set the sample aspect ratio of the sourced video.
> +
> +@item precision
> +Set precision in bits for look-up table for sine calculations. Default value is 10.
> +Allowed range is from 4 to 16.
> +
> +@item xo
> +Set horizontal axis offset for output signal. Default value is 0.
> +
> +@item yo
> +Set vertical axis offset for output signal. Default value is 0.
> +
> +@item k0
> +Set 0-order, constant added to signal phase. Default value is 0.
> +
> +@item kx
> +Set 1-order, phase factor multiplier for horizontal axis. Default value is 0.
> +
> +@item ky
> +Set 1-order, phase factor multiplier for vertical axis. Default value is 0.
> +
> +@item kt
> +Set 1-order, phase factor multiplier for time axis. Default value is 0.
> +
> +@item kxt, kyt, kxy
> +Set phase factor multipliers for combination of spatial and temporal axis.
> +Default value is 0.
> +
> +@item kx2
> +Set 2-order, phase factor multiplier for horizontal axis. Default value is 0.
> +
> +@item ky2
> +Set 2-order, phase factor multiplier for vertical axis. Default value is 0.
> +
> +@item kt2
> +Set 2-order, phase factor multiplier for time axis. Default value is 0.
> +
> +@item ku
> +Set the constant added to final phase to produce chroma-blue component of signal.
> +Default value is 0.
> +
> +@item kv
> +Set the constant added to final phase to produce chroma-red component of signal.
> +Default value is 0.
> +@end table
> +
> +@subsection Commands
> +

> +Filter supports the some above options as @ref{commands}.

Nit: This source supports the above options as ...

> +
> +@subsection Examples
> +
> +@itemize
> +@item
> +Generate horizontal color sine sweep:
> +@example
> +zoneplate=ku=512:kv=0:kt2=0:kx2=256:s=wvga:xo=-426:kt=11
> +@end example
> +
> +@item
> +Generate vertical color sine sweep:
> +@example
> +zoneplate=ku=512:kv=0:kt2=0:ky2=156:s=wvga:yo=-240:kt=11
> +@end example
> +
> +@item
> +Generate circular zone-plate:
> +@example
> +zoneplate=ku=512:kv=100:kt2=0:ky2=256:kx2=556:s=wvga:yo=0:kt=11
> +@end example
> +@end itemize
> +
>  @c man end VIDEO SOURCES
>  
>  @chapter Video Sinks
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index 93c614eeb7..9fb9a1095f 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -598,6 +598,7 @@ OBJS-$(CONFIG_SMPTEHDBARS_FILTER)            += vsrc_testsrc.o
>  OBJS-$(CONFIG_TESTSRC_FILTER)                += vsrc_testsrc.o
>  OBJS-$(CONFIG_TESTSRC2_FILTER)               += vsrc_testsrc.o
>  OBJS-$(CONFIG_YUVTESTSRC_FILTER)             += vsrc_testsrc.o
> +OBJS-$(CONFIG_ZONEPLATE_FILTER)              += vsrc_testsrc.o
>  
>  OBJS-$(CONFIG_NULLSINK_FILTER)               += vsink_nullsink.o
>  
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index 1c0bc12a92..025966dc45 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -563,6 +563,7 @@ extern const AVFilter ff_vsrc_smptehdbars;
>  extern const AVFilter ff_vsrc_testsrc;
>  extern const AVFilter ff_vsrc_testsrc2;
>  extern const AVFilter ff_vsrc_yuvtestsrc;
> +extern const AVFilter ff_vsrc_zoneplate;
>  
>  extern const AVFilter ff_vsink_nullsink;
>  
> diff --git a/libavfilter/vsrc_testsrc.c b/libavfilter/vsrc_testsrc.c
> index f391ac02e0..1e74a0e42e 100644
> --- a/libavfilter/vsrc_testsrc.c
> +++ b/libavfilter/vsrc_testsrc.c
> @@ -88,6 +88,14 @@ typedef struct TestSourceContext {
>  
>      /* only used by haldclut */
>      int level;
> +
> +    /* only used by zoneplate */
> +    int k0, kx, ky, kt;
> +    int kxt, kyt, kxy;
> +    int kx2, ky2, kt2;
> +    int xo, yo, kU, kV;
> +    int lut_precision;
> +    uint8_t *lut;
>  } TestSourceContext;
>  
>  #define OFFSET(x) offsetof(TestSourceContext, x)
> @@ -135,6 +143,7 @@ static av_cold void uninit(AVFilterContext *ctx)
>      TestSourceContext *test = ctx->priv;
>  
>      av_frame_free(&test->picref);
> +    av_freep(&test->lut);
>  }
>  
>  static int config_props(AVFilterLink *outlink)
> @@ -2049,3 +2058,151 @@ const AVFilter ff_vsrc_colorchart = {
>  };
>  
>  #endif /* CONFIG_COLORCHART_FILTER */
> +

> +#if CONFIG_COLORCHART_FILTER

ZONEPLATE?

> +
> +static const AVOption zoneplate_options[] = {
> +    COMMON_OPTIONS
> +    { "precision", "set LUT precision", OFFSET(lut_precision), AV_OPT_TYPE_INT, {.i64=10}, 4, 16, FLAGS },
> +    { "xo", "set x-axis offset", OFFSET(xo), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "yo", "set y-axis offset", OFFSET(yo), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "k0", "set 0-order phase", OFFSET(k0), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "kx", "set 1-order X-axis phase", OFFSET(kx), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "ky", "set 1-order Y-axis phase", OFFSET(ky), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "kt", "set 1-order T-axis phase", OFFSET(kt), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "kxt", "set X-axis*T-axis product phase", OFFSET(kxt), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "kyt", "set Y-axis*T-axis product phase", OFFSET(kyt), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "kxy", "set X-axis*Y-axis product phase", OFFSET(kxy), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "kx2", "set 2-order X-axis phase", OFFSET(kx2), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "ky2", "set 2-order Y-axis phase", OFFSET(ky2), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "kt2", "set 2-order T-axis phase", OFFSET(kt2), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "ku", "set 0-order U-color phase", OFFSET(kU), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { "kv", "set 0-order V-color phase", OFFSET(kV), AV_OPT_TYPE_INT, {.i64=0}, INT_MIN, INT_MAX, FLAGSR },
> +    { NULL }
> +};
> +
> +AVFILTER_DEFINE_CLASS(zoneplate);
> +
> +static int zoneplate_config_props(AVFilterLink *inlink)
> +{
> +    AVFilterContext *ctx = inlink->src;
> +    TestSourceContext *s = ctx->priv;
> +
> +    if (av_image_check_size(s->w, s->h, 0, ctx) < 0)
> +        return AVERROR(EINVAL);
> +    return config_props(inlink);
> +}
> +
> +static void zoneplate_fill_slice(AVFilterContext *ctx, void *arg, int job, int nb_jobs)
> +{
> +    TestSourceContext *test = ctx->priv;
> +    AVFrame *frame = arg;
> +    const int w = frame->width;
> +    const int h = frame->height;
> +    const int kxt = test->kxt, kyt = test->kyt, kx2 = test->kx2;
> +    const int t = test->pts, k0 = test->k0;
> +    const int kt = test->kt, kt2 = test->kt2, ky2 = test->ky2;
> +    const int ky = test->ky, kx = test->kx, kxy = test->kxy;
> +    const int lut_mask = (1 << test->lut_precision) - 1;
> +    const int nkt2t = kt2 * t * t, nktt = kt * t;
> +    const int start = (h *  job   ) / nb_jobs;
> +    const int end   = (h * (job+1)) / nb_jobs;
> +    const int ylinesize = frame->linesize[0];
> +    const int ulinesize = frame->linesize[1];
> +    const int vlinesize = frame->linesize[2];
> +    const int xreset = -(w / 2) - test->xo;
> +    const int yreset = -(h / 2) - test->yo + start;
> +    const int kU = test->kU, kV = test->kV;
> +    const int skxy = 0xffff / (w / 2);
> +    const int skx2 = 0xffff / w;
> +    const int dkxt = kxt * t;
> +    int akx, akxt, aky, akyt;
> +    uint8_t *ydst = frame->data[0] + start * ylinesize;
> +    uint8_t *udst = frame->data[1] + start * ulinesize;
> +    uint8_t *vdst = frame->data[2] + start * vlinesize;
> +    const uint8_t *lut = test->lut;
> +
> +    aky = start * ky;
> +    akyt = start * kyt * t;
> +
> +    for (int j = start, y = yreset; j < end; j++, y++) {
> +        const int dkxy = kxy * y * skxy;
> +        const int nky2kt2 = (ky2 * y * y) / h + (nkt2t >> 1);
> +        int akxy = dkxy * xreset;
> +
> +        akx = 0;
> +        akxt = 0;
> +        aky += ky;
> +        akyt += kyt * t;
> +        for (int i = 0, x = xreset; i < w; i++, x++) {
> +            int phase = k0, uphase = kU, vphase = kV;
> +
> +            akx += kx;
> +            phase += akx + aky + nktt;
> +
> +            akxt += dkxt;
> +            akxy += dkxy;
> +            phase += akxt + akyt;
> +            phase += akxy >> 16;
> +            phase += ((kx2 * x * x * skx2) >> 16) + nky2kt2;
> +            uphase += phase;
> +            vphase += phase;
> +
> +            ydst[i] = lut[phase  & lut_mask];
> +            udst[i] = lut[uphase & lut_mask];
> +            vdst[i] = lut[vphase & lut_mask];
> +        }
> +
> +        ydst += ylinesize;
> +        udst += ulinesize;
> +        vdst += vlinesize;
> +    }
> +}
> +
> +static void zoneplate_fill_picture(AVFilterContext *ctx, AVFrame *frame)
> +{
> +    ff_filter_execute(ctx, zoneplate_fill_slice, frame, NULL,
> +                      FFMIN(frame->height, ff_filter_get_nb_threads(ctx)));
> +}
> +
> +static av_cold int zoneplate_init(AVFilterContext *ctx)
> +{
> +    TestSourceContext *test = ctx->priv;
> +    const int lut_size = 1 << test->lut_precision;
> +
> +    test->lut = av_calloc(lut_size, sizeof(*test->lut));
> +    if (!test->lut)
> +        return AVERROR(ENOMEM);
> +
> +    for (int i = 0; i < lut_size; i++)
> +        test->lut[i] = lrintf(255.f * (0.5f + 0.5f * sinf((2.f * M_PI * i) / lut_size)));
> +
> +    test->draw_once = 0;
> +    test->fill_picture_fn = zoneplate_fill_picture;
> +    return init(ctx);
> +}
> +
> +static const AVFilterPad avfilter_vsrc_zoneplate_outputs[] = {
> +    {
> +        .name          = "default",
> +        .type          = AVMEDIA_TYPE_VIDEO,
> +        .config_props  = zoneplate_config_props,
> +    },
> +};
> +
> +const AVFilter ff_vsrc_zoneplate = {
> +    .name          = "zoneplate",
> +    .description   = NULL_IF_CONFIG_SMALL("Generate zone-plate."),
> +    .priv_size     = sizeof(TestSourceContext),
> +    .priv_class    = &zoneplate_class,
> +    .init          = zoneplate_init,
> +    .uninit        = uninit,
> +    .activate      = activate,
> +    .inputs        = NULL,
> +    FILTER_OUTPUTS(avfilter_vsrc_zoneplate_outputs),
> +    FILTER_SINGLE_PIXFMT(AV_PIX_FMT_YUV444P),
> +    .flags         = AVFILTER_FLAG_SLICE_THREADS,
> +    .process_command = ff_filter_process_command,
> +};
> +

> +#endif /* CONFIG_COLORCHART_FILTER */

ZONEPLATE ?

LGTM otherwise, thanks.

Isn't it better to use cos() instead of sin()? With sin() there is a 
discontinuity in the center.

On Wed, May 10, 2023 at 1:58 PM Michael Koch <astroelectronic@t-online.de>
wrote:

> Isn't it better to use cos() instead of sin()? With sin() there is a
> discontinuity in the center.
>

Where you see discontinuity in the filter output?

ffmpeg -f lavfi -i zoneplate=kx2=320:ky2=240,format=gray -frames 1 -y 
out.png

When you draw a line through the center, you have
black - gray - white - gray - white - gray -black
which means the sine wave changes its sign in the center.
That's why zoneplates are normally calculated with cos() function.
See also: https://en.wikipedia.org/wiki/Zone_plate#Continuous_zone_plates

On Wed, May 10, 2023 at 2:32 PM Michael Koch <astroelectronic@t-online.de>
wrote:

> ffmpeg -f lavfi -i zoneplate=kx2=320:ky2=240,format=gray -frames 1 -y
> out.png
>
> When you draw a line through the center, you have
> black - gray - white - gray - white - gray -black
> which means the sine wave changes its sign in the center.
> That's why zoneplates are normally calculated with cos() function.
> See also: https://en.wikipedia.org/wiki/Zone_plate#Continuous_zone_plates


I replaced sinf with cosf, and output is same, just different phase.

Well, I can't see how it looks now, but the sequence white - gray - 
white (which is impossible in a sine wave) should no longer be there, 
because cos(x) = cos(-x).

On Wed, May 10, 2023 at 2:50 PM Michael Koch <astroelectronic@t-online.de>
wrote:

> Well, I can't see how it looks now, but the sequence white - gray -
> white (which is impossible in a sine wave) should no longer be there,
> because cos(x) = cos(-x).
>

This is not caused by sin() but because video is full range yuv but
interpreted as limited.