[FFmpeg-devel] liavcodec: add bit-rate support to RoQ video encoder

The bitrate option (-b:v) can now be used to specify the bit rate
of the video stream of the RoQ encoder.

Original patch by Joseph Fenton aka Chilly Willy

Signed-off-by: Victor Luchits <vluchits@gmail.com>
---
  Changelog                |   1 +
  libavcodec/roqvideo.h    |   1 +
  libavcodec/roqvideodec.c |  16 ++++++
  libavcodec/roqvideoenc.c | 107 +++++++++++++++++++++++++++++++++++----
  libavcodec/version.h     |   2 +-
  5 files changed, 117 insertions(+), 10 deletions(-)

mån 2024-01-22 klockan 22:14 +0300 skrev Victor Luchits:
> The bitrate option (-b:v) can now be used to specify the bit rate
> of the video stream of the RoQ encoder.
> 
> Original patch by Joseph Fenton aka Chilly Willy
> 
> Signed-off-by: Victor Luchits <vluchits@gmail.com>

Still doesn't apply.

> +    /* Keyframe when no MOT or FCC codes in frame */
> +    if (s->key_frame) {
> +        av_log(avctx, AV_LOG_VERBOSE, "\nFound keyframe!\n");
> +        rframe->pict_type = AV_PICTURE_TYPE_I;
> +        avpkt->flags |= AV_PKT_FLAG_KEY;
> +    } else {
> +        rframe->pict_type = AV_PICTURE_TYPE_P;
> +        avpkt->flags &= ~AV_PKT_FLAG_KEY;
> +    }

Looks like framesSinceKeyframe still doesn't get reset

> +    if (avctx->bit_rate) {
> +        /* no specific bit rate desired, use frame quality */
> +        if (frame->quality)
> +            enc->lambda = frame->quality - 1;
> +        else
> +            enc->lambda = 2*ROQ_LAMBDA_SCALE;
> +    }

Looks like you got this backwards

/Tomas

On Tue, Jan 23, 2024 at 12:12 AM Tomas Härdin <git@haerdin.se> wrote:

> mån 2024-01-22 klockan 22:14 +0300 skrev Victor Luchits:
> > The bitrate option (-b:v) can now be used to specify the bit rate
> > of the video stream of the RoQ encoder.
> >
> > Original patch by Joseph Fenton aka Chilly Willy
> >
> > Signed-off-by: Victor Luchits <vluchits@gmail.com>
>
> Still doesn't apply.
>

I have no clue as to what's going on there.. The patch applies perfectly
fine
without all the eml stuff.

root@banana:~/ffmpeg# git reset --hard origin/master
HEAD is now at d2eb6f4d44 fftools/ffmpeg_mux_init: don't free the
AVDictionaryEntry until after it's been used
root@ banana:~/ffmpeg# git apply
0001-liavcodec-add-bit-rate-support-to-RoQ-video-encoder.patch
root@ banana:~/ffmpeg# echo $?
0

Any help would be highly appreciated.


> > +    /* Keyframe when no MOT or FCC codes in frame */
> > +    if (s->key_frame) {
> > +        av_log(avctx, AV_LOG_VERBOSE, "\nFound keyframe!\n");
> > +        rframe->pict_type = AV_PICTURE_TYPE_I;
> > +        avpkt->flags |= AV_PKT_FLAG_KEY;
> > +    } else {
> > +        rframe->pict_type = AV_PICTURE_TYPE_P;
> > +        avpkt->flags &= ~AV_PKT_FLAG_KEY;
> > +    }
>
> Looks like framesSinceKeyframe still doesn't get reset
>

framesSinceKeyframe wasn't introduced in this patch and I'm a bit scared
to mess with it as part of this changeset as that would introduce some new
changes to the core of the original implementation.


> > +    if (avctx->bit_rate) {
> > +        /* no specific bit rate desired, use frame quality */
> > +        if (frame->quality)
> > +            enc->lambda = frame->quality - 1;
> > +        else
> > +            enc->lambda = 2*ROQ_LAMBDA_SCALE;
> > +    }
>
> Looks like you got this backwards
>

You're totally right, thanks!

I'll resubmit the patch once we figure out what's the deal with git failing
to apply the patch for you.


>
> /Tomas
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>

Typo in commit message: liavcodec should be libavcodec

On 22 Jan 2024, at 20:14, Victor Luchits wrote:

> The bitrate option (-b:v) can now be used to specify the bit rate
> of the video stream of the RoQ encoder.
>
> Original patch by Joseph Fenton aka Chilly Willy
>
> Signed-off-by: Victor Luchits <vluchits@gmail.com>
> ---
>  Changelog                |   1 +
>  libavcodec/roqvideo.h    |   1 +
>  libavcodec/roqvideodec.c |  16 ++++++
>  libavcodec/roqvideoenc.c | 107 +++++++++++++++++++++++++++++++++++----
>  libavcodec/version.h     |   2 +-
>  5 files changed, 117 insertions(+), 10 deletions(-)
>
> diff --git a/Changelog b/Changelog
> index c40b6d08fd..6974312f9d 100644
> --- a/Changelog
> +++ b/Changelog
> @@ -22,6 +22,7 @@ version <next>:
>  - ffmpeg CLI -bsf option may now be used for input as well as output
>  - ffmpeg CLI options may now be used as -/opt <path>, which is equivalent
>    to -opt <contents of file <path>>
> +- RoQ video bit rate option support
>   version 6.1:
>  - libaribcaption decoder
> diff --git a/libavcodec/roqvideo.h b/libavcodec/roqvideo.h
> index 2c2e42884d..6d30bcaada 100644
> --- a/libavcodec/roqvideo.h
> +++ b/libavcodec/roqvideo.h
> @@ -43,6 +43,7 @@ typedef struct RoqContext {
>      AVFrame *last_frame;
>      AVFrame *current_frame;
>      int width, height;
> +    int key_frame;
>       roq_cell cb2x2[256];
>      roq_qcell cb4x4[256];
> diff --git a/libavcodec/roqvideodec.c b/libavcodec/roqvideodec.c
> index bfc69a65c9..b4ade3a43b 100644
> --- a/libavcodec/roqvideodec.c
> +++ b/libavcodec/roqvideodec.c
> @@ -70,6 +70,7 @@ static void roqvideo_decode_frame(RoqContext *ri, GetByteContext *gb)
>       chunk_start = bytestream2_tell(gb);
>      xpos = ypos = 0;
> +    ri->key_frame = 1;
>       if (chunk_size > bytestream2_get_bytes_left(gb)) {
>          av_log(ri->logctx, AV_LOG_ERROR, "Chunk does not fit in input buffer\n");
> @@ -92,12 +93,14 @@ static void roqvideo_decode_frame(RoqContext *ri, GetByteContext *gb)
>                   switch(vqid) {
>                  case RoQ_ID_MOT:
> +                    ri->key_frame = 0;
>                      break;
>                  case RoQ_ID_FCC: {
>                      int byte = bytestream2_get_byte(gb);
>                      mx = 8 - (byte >> 4) - ((signed char) (chunk_arg >> 8));
>                      my = 8 - (byte & 0xf) - ((signed char) chunk_arg);
>                      ff_apply_motion_8x8(ri, xp, yp, mx, my);
> +                    ri->key_frame = 0;
>                      break;
>                  }
>                  case RoQ_ID_SLD:
> @@ -125,12 +128,14 @@ static void roqvideo_decode_frame(RoqContext *ri, GetByteContext *gb)
>                          vqflg_pos--;
>                          switch(vqid) {
>                          case RoQ_ID_MOT:
> +                            ri->key_frame = 0;
>                              break;
>                          case RoQ_ID_FCC: {
>                              int byte = bytestream2_get_byte(gb);
>                              mx = 8 - (byte >> 4) - ((signed char) (chunk_arg >> 8));
>                              my = 8 - (byte & 0xf) - ((signed char) chunk_arg);
>                              ff_apply_motion_4x4(ri, x, y, mx, my);
> +                            ri->key_frame = 0;
>                              break;
>                          }
>                          case RoQ_ID_SLD:
> @@ -214,6 +219,17 @@ static int roq_decode_frame(AVCodecContext *avctx, AVFrame *rframe,
>       if ((ret = av_frame_ref(rframe, s->current_frame)) < 0)
>          return ret;
> +
> +    /* Keyframe when no MOT or FCC codes in frame */
> +    if (s->key_frame) {
> +        av_log(avctx, AV_LOG_VERBOSE, "\nFound keyframe!\n");
> +        rframe->pict_type = AV_PICTURE_TYPE_I;
> +        avpkt->flags |= AV_PKT_FLAG_KEY;
> +    } else {
> +        rframe->pict_type = AV_PICTURE_TYPE_P;
> +        avpkt->flags &= ~AV_PKT_FLAG_KEY;
> +    }
> +
>      *got_frame      = 1;
>       /* shuffle frames */
> diff --git a/libavcodec/roqvideoenc.c b/libavcodec/roqvideoenc.c
> index 0933abf4f9..68ec9ec238 100644
> --- a/libavcodec/roqvideoenc.c
> +++ b/libavcodec/roqvideoenc.c
> @@ -136,6 +136,8 @@ typedef struct RoqEncContext {
>      struct ELBGContext *elbg;
>      AVLFG randctx;
>      uint64_t lambda;
> +    uint64_t last_lambda;
> +    int lambda_delta;
>       motion_vect *this_motion4;
>      motion_vect *last_motion4;
> @@ -887,8 +889,9 @@ static int generate_new_codebooks(RoqEncContext *enc)
>      return 0;
>  }
>  -static int roq_encode_video(RoqEncContext *enc)
> +static int roq_encode_video(AVCodecContext *avctx)
>  {
> +    RoqEncContext *const enc = avctx->priv_data;
>      RoqTempData *const tempData = &enc->tmp_data;
>      RoqContext *const roq = &enc->common;
>      int ret;
> @@ -910,14 +913,14 @@ static int roq_encode_video(RoqEncContext *enc)
>       /* Quake 3 can't handle chunks bigger than 65535 bytes */
>      if (tempData->mainChunkSize/8 > 65535 && enc->quake3_compat) {
> -        if (enc->lambda > 100000) {
> +        if (enc->lambda > 100000000) {
>              av_log(roq->logctx, AV_LOG_ERROR, "Cannot encode video in Quake compatible form\n");
>              return AVERROR(EINVAL);
>          }
>          av_log(roq->logctx, AV_LOG_ERROR,
>                 "Warning, generated a frame too big for Quake (%d > 65535), "
> -               "now switching to a bigger qscale value.\n",
> -               tempData->mainChunkSize/8);
> +               "now switching to a bigger qscale value (%d).\n",
> +               tempData->mainChunkSize/8, (int)enc->lambda);
>          enc->lambda *= 1.5;
>          tempData->mainChunkSize = 0;
>          memset(tempData->used_option, 0, sizeof(tempData->used_option));
> @@ -931,6 +934,80 @@ static int roq_encode_video(RoqEncContext *enc)
>       remap_codebooks(enc);
>  +    /* bit rate control code - could make encoding very slow */
> +    if (avctx->bit_rate) {
> +        /* a bit rate has been specified - try to match it */
> +        int ftotal = (tempData->mainChunkSize / 8 + tempData->numCB2*6 + tempData->numCB4*4) * avctx->time_base.den * 8;
> +        int64_t tol = avctx->bit_rate_tolerance;
> +
> +        /* tolerance > bit rate, set to 5% of the bit rate */
> +        if (tol > avctx->bit_rate)
> +            tol = avctx->bit_rate / 20;
> +
> +        av_log(roq->logctx, AV_LOG_VERBOSE,
> +               "\nDesired bit rate (%d kbps), "
> +               "Bit rate tolerance (%d), "
> +               "Frame rate (%d)\n",
> +               (int)avctx->bit_rate, (int)tol, avctx->time_base.den);
> +
> +        if (ftotal > (avctx->bit_rate + tol)) {
> +            /* frame is too big - increase qscale */
> +            if (enc->lambda > 100000000) {
> +                av_log(roq->logctx, AV_LOG_ERROR, "\nCannot encode video at desired bitrate\n");
> +                return AVERROR(EINVAL);
> +            }
> +            enc->lambda_delta = enc->lambda_delta <= 0 ? 1 : enc->lambda_delta < 65536 ? enc->lambda_delta*2 : 65536;
> +            enc->last_lambda = enc->lambda;
> +            enc->lambda += enc->lambda_delta;
> +            av_log(roq->logctx, AV_LOG_INFO,
> +                   "\nGenerated a frame too big for desired bit rate (%d kbps), "
> +                   "now switching to a bigger qscale value (%d).\n",
> +                   ftotal / 1000, (int)enc->lambda);
> +            tempData->mainChunkSize = 0;
> +            memset(tempData->used_option, 0, sizeof(tempData->used_option));
> +            memset(tempData->codebooks.usedCB4, 0,
> +                   sizeof(tempData->codebooks.usedCB4));
> +            memset(tempData->codebooks.usedCB2, 0,
> +                   sizeof(tempData->codebooks.usedCB2));
> +
> +            goto retry_encode;
> +        } else if (ftotal < (avctx->bit_rate - tol)) {
> +            /* frame is too small - decrease qscale */
> +            if (enc->lambda <= 1) {
> +                av_log(roq->logctx, AV_LOG_WARNING,
> +                       "\nGenerated a frame too small for desired bit rate (%d kbps), "
> +                       "qscale value cannot be lowered any further (%d).\n",
> +                       ftotal / 1000, (int)enc->lambda);
> +            } else if ((enc->lambda - enc->last_lambda) == 1) {
> +                av_log(roq->logctx, AV_LOG_WARNING,
> +                       "\nCannot find qscale that gives desired bit rate within desired tolerance, "
> +                       "using lower bitrate (%d kbps) with higher qscale value (%d).\n",
> +                       ftotal / 1000, (int)enc->lambda);
> +            } else {
> +                enc->lambda_delta = 0;
> +                if (enc->lambda == enc->last_lambda) {
> +                    enc->lambda >>= 1;
> +                    enc->last_lambda = enc->lambda;
> +                } else {
> +                    enc->lambda = enc->last_lambda;
> +                    //enc->lambda *= (float)(tempData->mainChunkSize * avctx->time_base.den) / avctx->bit_rate;
> +                    av_log(roq->logctx, AV_LOG_INFO,
> +                           "\nGenerated a frame too small for desired bit rate (%d kbps), "
> +                           "reverting lambda and using smaller inc on qscale (%d).\n",
> +                           ftotal / 1000, (int)enc->lambda);
> +                }
> +                tempData->mainChunkSize = 0;
> +                memset(tempData->used_option, 0, sizeof(tempData->used_option));
> +                memset(tempData->codebooks.usedCB4, 0,
> +                       sizeof(tempData->codebooks.usedCB4));
> +                memset(tempData->codebooks.usedCB2, 0,
> +                       sizeof(tempData->codebooks.usedCB2));
> +
> +                goto retry_encode;
> +            }
> +        }
> +    }
> +
>      write_codebooks(enc);
>       reconstruct_and_encode_image(enc, roq->width, roq->height,
> @@ -991,8 +1068,11 @@ static av_cold int roq_encode_init(AVCodecContext *avctx)
>      roq->width  = avctx->width;
>      roq->height = avctx->height;
>  +    enc->lambda = 2*ROQ_LAMBDA_SCALE;
>      enc->framesSinceKeyframe = 0;
>      enc->first_frame = 1;
> +    enc->last_lambda = 1;
> +    enc->lambda_delta = 0;
>       roq->last_frame    = av_frame_alloc();
>      roq->current_frame = av_frame_alloc();
> @@ -1059,10 +1139,13 @@ static int roq_encode_frame(AVCodecContext *avctx, AVPacket *pkt,
>       enc->frame_to_enc = frame;
>  -    if (frame->quality)
> -        enc->lambda = frame->quality - 1;
> -    else
> -        enc->lambda = 2*ROQ_LAMBDA_SCALE;
> +    if (avctx->bit_rate) {
> +        /* no specific bit rate desired, use frame quality */
> +        if (frame->quality)
> +            enc->lambda = frame->quality - 1;
> +        else
> +            enc->lambda = 2*ROQ_LAMBDA_SCALE;
> +    }
>       /* 138 bits max per 8x8 block +
>       *     256 codebooks*(6 bytes 2x2 + 4 bytes 4x4) + 8 bytes frame header */
> @@ -1089,7 +1172,7 @@ static int roq_encode_frame(AVCodecContext *avctx, AVPacket *pkt,
>      }
>       /* Encode the actual frame */
> -    ret = roq_encode_video(enc);
> +    ret = roq_encode_video(avctx);
>      if (ret < 0)
>          return ret;
>  @@ -1115,6 +1198,11 @@ static const AVClass roq_class = {
>      .version    = LIBAVUTIL_VERSION_INT,
>  };
>  +static const FFCodecDefault roq_defaults[] = {
> +    { "b",                "0" },
> +    { NULL },
> +};
> +
>  const FFCodec ff_roq_encoder = {
>      .p.name               = "roqvideo",
>      CODEC_LONG_NAME("id RoQ video"),
> @@ -1129,4 +1217,5 @@ const FFCodec ff_roq_encoder = {
>                                                          AV_PIX_FMT_NONE },
>      .p.priv_class   = &roq_class,
>      .caps_internal        = FF_CODEC_CAP_INIT_CLEANUP,
> +    .defaults             = roq_defaults,
>  };
> diff --git a/libavcodec/version.h b/libavcodec/version.h
> index 376388c5bb..581151cda7 100644
> --- a/libavcodec/version.h
> +++ b/libavcodec/version.h
> @@ -30,7 +30,7 @@
>  #include "version_major.h"
>   #define LIBAVCODEC_VERSION_MINOR  37
> -#define LIBAVCODEC_VERSION_MICRO 100
> +#define LIBAVCODEC_VERSION_MICRO 101
>   #define LIBAVCODEC_VERSION_INT  AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \
>                                                 LIBAVCODEC_VERSION_MINOR, \
> -- 
> 2.25.1
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

tis 2024-01-23 klockan 00:40 +0300 skrev Victor Luchitz:
> On Tue, Jan 23, 2024 at 12:12 AM Tomas Härdin <git@haerdin.se> wrote:
> 
> > mån 2024-01-22 klockan 22:14 +0300 skrev Victor Luchits:
> > > The bitrate option (-b:v) can now be used to specify the bit rate
> > > of the video stream of the RoQ encoder.
> > > 
> > > Original patch by Joseph Fenton aka Chilly Willy
> > > 
> > > Signed-off-by: Victor Luchits <vluchits@gmail.com>
> > 
> > Still doesn't apply.
> > 
> 
> I have no clue as to what's going on there.. The patch applies
> perfectly
> fine
> without all the eml stuff.
> 
> root@banana:~/ffmpeg# git reset --hard origin/master
> HEAD is now at d2eb6f4d44 fftools/ffmpeg_mux_init: don't free the
> AVDictionaryEntry until after it's been used
> root@ banana:~/ffmpeg# git apply
> 0001-liavcodec-add-bit-rate-support-to-RoQ-video-encoder.patch
> root@ banana:~/ffmpeg# echo $?
> 0
> 
> Any help would be highly appreciated.
> 
> 
> > > +    /* Keyframe when no MOT or FCC codes in frame */
> > > +    if (s->key_frame) {
> > > +        av_log(avctx, AV_LOG_VERBOSE, "\nFound keyframe!\n");
> > > +        rframe->pict_type = AV_PICTURE_TYPE_I;
> > > +        avpkt->flags |= AV_PKT_FLAG_KEY;
> > > +    } else {
> > > +        rframe->pict_type = AV_PICTURE_TYPE_P;
> > > +        avpkt->flags &= ~AV_PKT_FLAG_KEY;
> > > +    }
> > 
> > Looks like framesSinceKeyframe still doesn't get reset
> > 
> 
> framesSinceKeyframe wasn't introduced in this patch and I'm a bit
> scared
> to mess with it as part of this changeset as that would introduce
> some new
> changes to the core of the original implementation.

It doesn't have to hold this patch up I guess. Doesn't look terribly
complicated either way

> > > +    if (avctx->bit_rate) {
> > > +        /* no specific bit rate desired, use frame quality */
> > > +        if (frame->quality)
> > > +            enc->lambda = frame->quality - 1;
> > > +        else
> > > +            enc->lambda = 2*ROQ_LAMBDA_SCALE;
> > > +    }
> > 
> > Looks like you got this backwards
> > 
> 
> You're totally right, thanks!
> 
> I'll resubmit the patch once we figure out what's the deal with git
> failing
> to apply the patch for you.

Try attaching the patch rather than pasting it

/Tomas