[FFmpeg-devel,1/3] avcodec/libaomenc: Add parameter for avif single image encoding

Add a parameter to libaom-av1 encoder to enforce some of the single
image constraints in the AV1 encoder. Setting this flag will limit
the encoder to producing exactly one frame and the sequence header
that is produced by the encoder will be conformant to the AVIF
specification [1].

Part of Fixing Trac ticket #7621

[1] https://aomediacodec.github.io/av1-avif

Signed-off-by:: Vignesh Venkatasubramanian <vigneshv@google.com>
---
 libavcodec/libaomenc.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

On Wed, Feb 16, 2022 at 9:51 PM Vignesh Venkatasubramanian
<vigneshv-at-google.com@ffmpeg.org> wrote:
>
> Add a parameter to libaom-av1 encoder to enforce some of the single
> image constraints in the AV1 encoder. Setting this flag will limit
> the encoder to producing exactly one frame and the sequence header
> that is produced by the encoder will be conformant to the AVIF
> specification [1].
>
> Part of Fixing Trac ticket #7621
>
> [1] https://aomediacodec.github.io/av1-avif
>
> Signed-off-by:: Vignesh Venkatasubramanian <vigneshv@google.com>
> ---
>  libavcodec/libaomenc.c | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
>
> [...]
> @@ -1290,6 +1303,7 @@ static const AVOption options[] = {
>      { "psnr",            NULL,         0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_PSNR}, 0, 0, VE, "tune"},
>      { "ssim",            NULL,         0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_SSIM}, 0, 0, VE, "tune"},
>      FF_AV1_PROFILE_OPTS
> +    { "avif-image", "Encode in single frame mode for still AVIF images.", OFFSET(is_avif), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE },

Can this instead be mapped to something that can be sent via aom-params?

On Thu, Feb 17, 2022 at 10:09 AM James Zern <jzern@google.com> wrote:
>
> On Wed, Feb 16, 2022 at 9:51 PM Vignesh Venkatasubramanian
> <vigneshv-at-google.com@ffmpeg.org> wrote:
> >
> > Add a parameter to libaom-av1 encoder to enforce some of the single
> > image constraints in the AV1 encoder. Setting this flag will limit
> > the encoder to producing exactly one frame and the sequence header
> > that is produced by the encoder will be conformant to the AVIF
> > specification [1].
> >
> > Part of Fixing Trac ticket #7621
> >
> > [1] https://aomediacodec.github.io/av1-avif
> >
> > Signed-off-by:: Vignesh Venkatasubramanian <vigneshv@google.com>
> > ---
> >  libavcodec/libaomenc.c | 14 ++++++++++++++
> >  1 file changed, 14 insertions(+)
> >
> > [...]
> > @@ -1290,6 +1303,7 @@ static const AVOption options[] = {
> >      { "psnr",            NULL,         0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_PSNR}, 0, 0, VE, "tune"},
> >      { "ssim",            NULL,         0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_SSIM}, 0, 0, VE, "tune"},
> >      FF_AV1_PROFILE_OPTS
> > +    { "avif-image", "Encode in single frame mode for still AVIF images.", OFFSET(is_avif), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE },
>
> Can this instead be mapped to something that can be sent via aom-params?


From what i understand, aom-params can only be used to set key-value
pairs that are set using the aom_codec_set_option() function call. For
single image AVIF encodes, we are modifying some of the
aom_codec_enc_cfg entries. There is no way to do that in a generic way
other than to introduce a specific option like this.

On 2/17/2022 2:51 AM, Vignesh Venkatasubramanian wrote:
> Add a parameter to libaom-av1 encoder to enforce some of the single
> image constraints in the AV1 encoder. Setting this flag will limit
> the encoder to producing exactly one frame and the sequence header
> that is produced by the encoder will be conformant to the AVIF
> specification [1].
> 
> Part of Fixing Trac ticket #7621
> 
> [1] https://aomediacodec.github.io/av1-avif
> 
> Signed-off-by:: Vignesh Venkatasubramanian <vigneshv@google.com>
> ---
>   libavcodec/libaomenc.c | 14 ++++++++++++++
>   1 file changed, 14 insertions(+)
> 
> diff --git a/libavcodec/libaomenc.c b/libavcodec/libaomenc.c
> index 963cc1bcbc..0398060a2f 100644
> --- a/libavcodec/libaomenc.c
> +++ b/libavcodec/libaomenc.c
> @@ -99,6 +99,7 @@ typedef struct AOMEncoderContext {
>       int enable_restoration;
>       int usage;
>       int tune;
> +    int is_avif;
>       int enable_rect_partitions;
>       int enable_1to4_partitions;
>       int enable_ab_partitions;
> @@ -746,6 +747,18 @@ static av_cold int aom_init(AVCodecContext *avctx,
>       if (res < 0)
>           return res;
>   
> +    if (ctx->is_avif) {
> +        // Set the maximum number of frames to 1. This will let libaom set
> +        // still_picture and reduced_still_picture_header to 1 in the Sequence
> +        // Header as required by AVIF still images.
> +        enccfg.g_limit = 1;

How will libaom react if you feed it more than 1 frame? Will it reject 
the input and return an error?

> +        // Reduce memory usage for still images.
> +        enccfg.g_lag_in_frames = 0;
> +        // All frames will be key frames.
> +        enccfg.kf_max_dist = 0;
> +        enccfg.kf_mode = AOM_KF_DISABLED;
> +    }
> +
>       /* Construct Encoder Context */
>       res = aom_codec_enc_init(&ctx->encoder, iface, &enccfg, flags);
>       if (res != AOM_CODEC_OK) {
> @@ -1290,6 +1303,7 @@ static const AVOption options[] = {
>       { "psnr",            NULL,         0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_PSNR}, 0, 0, VE, "tune"},
>       { "ssim",            NULL,         0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_SSIM}, 0, 0, VE, "tune"},
>       FF_AV1_PROFILE_OPTS
> +    { "avif-image", "Encode in single frame mode for still AVIF images.", OFFSET(is_avif), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE },

Judging by the above settings, this option should be called 
"still_picture". The description can mention it's required for AVIF output.

An alternative is to add new AVProfiles, giving them values outside the 
range that would be written into the sequence header. Like 
FF_PROFILE_AV1_MAIN_STILL_PICTURE with a value of 0x80, 
FF_PROFILE_AV1_HIGH_STILL_PICTURE with a value of 0x81, etc.
Then you just look for avctx->profile & 0x80 to know it's a still picture.

Of course considerations would need to be done to avoid problems, like 
the line "enccfg.g_profile = avctx->profile;" being changed to 
"enccfg.g_profile = avctx->profile & 0x3;" (We can define a constant for 
that 0x3 mask called profile_bits or whatever).

>       { "enable-rect-partitions", "Enable rectangular partitions", OFFSET(enable_rect_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE},
>       { "enable-1to4-partitions", "Enable 1:4/4:1 partitions",     OFFSET(enable_1to4_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE},
>       { "enable-ab-partitions",   "Enable ab shape partitions",    OFFSET(enable_ab_partitions),   AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE},

On Thu, Feb 17, 2022 at 1:00 PM James Almer <jamrial@gmail.com> wrote:
>
>
>
> On 2/17/2022 2:51 AM, Vignesh Venkatasubramanian wrote:
> > Add a parameter to libaom-av1 encoder to enforce some of the single
> > image constraints in the AV1 encoder. Setting this flag will limit
> > the encoder to producing exactly one frame and the sequence header
> > that is produced by the encoder will be conformant to the AVIF
> > specification [1].
> >
> > Part of Fixing Trac ticket #7621
> >
> > [1] https://aomediacodec.github.io/av1-avif
> >
> > Signed-off-by:: Vignesh Venkatasubramanian <vigneshv@google.com>
> > ---
> >   libavcodec/libaomenc.c | 14 ++++++++++++++
> >   1 file changed, 14 insertions(+)
> >
> > diff --git a/libavcodec/libaomenc.c b/libavcodec/libaomenc.c
> > index 963cc1bcbc..0398060a2f 100644
> > --- a/libavcodec/libaomenc.c
> > +++ b/libavcodec/libaomenc.c
> > @@ -99,6 +99,7 @@ typedef struct AOMEncoderContext {
> >       int enable_restoration;
> >       int usage;
> >       int tune;
> > +    int is_avif;
> >       int enable_rect_partitions;
> >       int enable_1to4_partitions;
> >       int enable_ab_partitions;
> > @@ -746,6 +747,18 @@ static av_cold int aom_init(AVCodecContext *avctx,
> >       if (res < 0)
> >           return res;
> >
> > +    if (ctx->is_avif) {
> > +        // Set the maximum number of frames to 1. This will let libaom set
> > +        // still_picture and reduced_still_picture_header to 1 in the Sequence
> > +        // Header as required by AVIF still images.
> > +        enccfg.g_limit = 1;
>
> How will libaom react if you feed it more than 1 frame? Will it reject
> the input and return an error?
>

Yes, trying to encode a second frame after setting g_limit to 1 will
result in aom_codec_encode() returning an error.

> > +        // Reduce memory usage for still images.
> > +        enccfg.g_lag_in_frames = 0;
> > +        // All frames will be key frames.
> > +        enccfg.kf_max_dist = 0;
> > +        enccfg.kf_mode = AOM_KF_DISABLED;
> > +    }
> > +
> >       /* Construct Encoder Context */
> >       res = aom_codec_enc_init(&ctx->encoder, iface, &enccfg, flags);
> >       if (res != AOM_CODEC_OK) {
> > @@ -1290,6 +1303,7 @@ static const AVOption options[] = {
> >       { "psnr",            NULL,         0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_PSNR}, 0, 0, VE, "tune"},
> >       { "ssim",            NULL,         0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_SSIM}, 0, 0, VE, "tune"},
> >       FF_AV1_PROFILE_OPTS
> > +    { "avif-image", "Encode in single frame mode for still AVIF images.", OFFSET(is_avif), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE },
>
> Judging by the above settings, this option should be called
> "still_picture". The description can mention it's required for AVIF output.
>

Renamed the variable and the option name to "still_picture".

> An alternative is to add new AVProfiles, giving them values outside the
> range that would be written into the sequence header. Like
> FF_PROFILE_AV1_MAIN_STILL_PICTURE with a value of 0x80,
> FF_PROFILE_AV1_HIGH_STILL_PICTURE with a value of 0x81, etc.
> Then you just look for avctx->profile & 0x80 to know it's a still picture.
>
> Of course considerations would need to be done to avoid problems, like
> the line "enccfg.g_profile = avctx->profile;" being changed to
> "enccfg.g_profile = avctx->profile & 0x3;" (We can define a constant for
> that 0x3 mask called profile_bits or whatever).
>
> >       { "enable-rect-partitions", "Enable rectangular partitions", OFFSET(enable_rect_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE},
> >       { "enable-1to4-partitions", "Enable 1:4/4:1 partitions",     OFFSET(enable_1to4_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE},
> >       { "enable-ab-partitions",   "Enable ab shape partitions",    OFFSET(enable_ab_partitions),   AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE},
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".