diff mbox series

[FFmpeg-devel] avcodec/videotoolboxenc: add low-latency encoding

Message ID tencent_B0D12DCB0981661FC678313A81AF67A7E60A@qq.com
State New
Headers show
Series [FFmpeg-devel] avcodec/videotoolboxenc: add low-latency encoding | expand

Checks

Context Check Description
yinshiyou/make_loongarch64 success Make finished
yinshiyou/make_fate_loongarch64 success Make fate finished

Commit Message

徐福隆 May 19, 2023, 3:17 a.m. UTC
When using low-latency mode, it eliminates frame reordering
and follows a one-in-one-out encoding mode

Signed-off-by: xufuji456 <839789740@qq.com>
---
 libavcodec/videotoolboxenc.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

Comments

Zhao Zhili May 19, 2023, 3:34 a.m. UTC | #1
> On May 19, 2023, at 11:17, xufuji456 <839789740@qq.com> wrote:
> 
> When using low-latency mode, it eliminates frame reordering
> and follows a one-in-one-out encoding mode
> 
> Signed-off-by: xufuji456 <839789740@qq.com>
> ---
> libavcodec/videotoolboxenc.c | 11 +++++++++++
> 1 file changed, 11 insertions(+)
> 
> diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
> index c6f22723d6..ae1ef32b2d 100644
> --- a/libavcodec/videotoolboxenc.c
> +++ b/libavcodec/videotoolboxenc.c
> @@ -1441,6 +1441,17 @@ static int vtenc_create_encoder(AVCodecContext   *avctx,
>         }
>     }
> 
> +    // low-latency mode: eliminate frame reordering, follow a one-in-one-out encoding mode
> +    if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) && avctx->codec_id == AV_CODEC_ID_H264) {
> +        status = VTSessionSetProperty(vtctx->session,
> +                                      kVTVideoEncoderSpecification_EnableLowLatencyRateControl,
> +                                      kCFBooleanTrue);

Please use compat_keys in case of kVTVideoEncoderSpecification_EnableLowLatencyRateControl
isn’t available.

> +
> +        if (status) {
> +            av_log(avctx, AV_LOG_ERROR, "Error setting low latency property: %d\n", status);
> +        }
> +    }
> +
>     status = VTCompressionSessionPrepareToEncodeFrames(vtctx->session);
>     if (status) {
>         av_log(avctx, AV_LOG_ERROR, "Error: cannot prepare encoder: %d\n", status);
> -- 
> 2.32.0 (Apple Git-132)
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
徐福隆 May 19, 2023, 5:55 a.m. UTC | #2
Thank you for your review, and point out the details.
Actually, I have also considered this issue.
I see that some keys are used directly, some are used by compatibility.
I will use compat_key instead, and submit a new patch.


------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        ""zhilizhao(赵志立)""                                                                                    <quinkblack@foxmail.com&gt;;
发送时间:&nbsp;2023年5月19日(星期五) 中午11:34
收件人:&nbsp;"FFmpeg development discussions and patches"<ffmpeg-devel@ffmpeg.org&gt;;
抄送:&nbsp;"徐福隆"<839789740@qq.com&gt;;
主题:&nbsp;Re: [FFmpeg-devel] [PATCH] avcodec/videotoolboxenc: add low-latency encoding





&gt; On May 19, 2023, at 11:17, xufuji456 <839789740@qq.com&gt; wrote:
&gt; 
&gt; When using low-latency mode, it eliminates frame reordering
&gt; and follows a one-in-one-out encoding mode
&gt; 
&gt; Signed-off-by: xufuji456 <839789740@qq.com&gt;
&gt; ---
&gt; libavcodec/videotoolboxenc.c | 11 +++++++++++
&gt; 1 file changed, 11 insertions(+)
&gt; 
&gt; diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
&gt; index c6f22723d6..ae1ef32b2d 100644
&gt; --- a/libavcodec/videotoolboxenc.c
&gt; +++ b/libavcodec/videotoolboxenc.c
&gt; @@ -1441,6 +1441,17 @@ static int vtenc_create_encoder(AVCodecContext&nbsp;&nbsp; *avctx,
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
&gt;&nbsp;&nbsp;&nbsp;&nbsp; }
&gt; 
&gt; +&nbsp;&nbsp;&nbsp; // low-latency mode: eliminate frame reordering, follow a one-in-one-out encoding mode
&gt; +&nbsp;&nbsp;&nbsp; if ((avctx-&gt;flags &amp; AV_CODEC_FLAG_LOW_DELAY) &amp;&amp; avctx-&gt;codec_id == AV_CODEC_ID_H264) {
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; status = VTSessionSetProperty(vtctx-&gt;session,
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; kVTVideoEncoderSpecification_EnableLowLatencyRateControl,
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; kCFBooleanTrue);

Please use compat_keys in case of kVTVideoEncoderSpecification_EnableLowLatencyRateControl
isn’t available.

&gt; +
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (status) {
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; av_log(avctx, AV_LOG_ERROR, "Error setting low latency property: %d\n", status);
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
&gt; +&nbsp;&nbsp;&nbsp; }
&gt; +
&gt;&nbsp;&nbsp;&nbsp;&nbsp; status = VTCompressionSessionPrepareToEncodeFrames(vtctx-&gt;session);
&gt;&nbsp;&nbsp;&nbsp;&nbsp; if (status) {
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; av_log(avctx, AV_LOG_ERROR, "Error: cannot prepare encoder: %d\n", status);
&gt; -- 
&gt; 2.32.0 (Apple Git-132)
&gt; 
&gt; _______________________________________________
&gt; ffmpeg-devel mailing list
&gt; ffmpeg-devel@ffmpeg.org
&gt; https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
&gt; 
&gt; To unsubscribe, visit link above, or email
&gt; ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Rick Kern May 19, 2023, 1:36 p.m. UTC | #3
On Fri, May 19, 2023 at 1:56 AM 徐福隆 <839789740@qq.com> wrote:

> Thank you for your review, and point out the details.
> Actually, I have also considered this issue.
> I see that some keys are used directly, some are used by compatibility.
>

The compat_keys are used for symbols that might not exist. For constants
available in the earliest version of VideoToolbox, compat_keys doesn't need
to be used.

Your patch looks good - I'll push it shortly.


> I will use compat_key instead, and submit a new patch.
>
>
> ------------------&nbsp;原始邮件&nbsp;------------------
> 发件人:
>                                                   ""zhilizhao(赵志立)""
>
> <quinkblack@foxmail.com&gt;;
> 发送时间:&nbsp;2023年5月19日(星期五) 中午11:34
> 收件人:&nbsp;"FFmpeg development discussions and patches"<
> ffmpeg-devel@ffmpeg.org&gt;;
> 抄送:&nbsp;"徐福隆"<839789740@qq.com&gt;;
> 主题:&nbsp;Re: [FFmpeg-devel] [PATCH] avcodec/videotoolboxenc: add
> low-latency encoding
>
>
>
>
>
> &gt; On May 19, 2023, at 11:17, xufuji456 <839789740@qq.com&gt; wrote:
> &gt;
> &gt; When using low-latency mode, it eliminates frame reordering
> &gt; and follows a one-in-one-out encoding mode
> &gt;
> &gt; Signed-off-by: xufuji456 <839789740@qq.com&gt;
> &gt; ---
> &gt; libavcodec/videotoolboxenc.c | 11 +++++++++++
> &gt; 1 file changed, 11 insertions(+)
> &gt;
> &gt; diff --git a/libavcodec/videotoolboxenc.c
> b/libavcodec/videotoolboxenc.c
> &gt; index c6f22723d6..ae1ef32b2d 100644
> &gt; --- a/libavcodec/videotoolboxenc.c
> &gt; +++ b/libavcodec/videotoolboxenc.c
> &gt; @@ -1441,6 +1441,17 @@ static int
> vtenc_create_encoder(AVCodecContext&nbsp;&nbsp; *avctx,
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
> &gt;&nbsp;&nbsp;&nbsp;&nbsp; }
> &gt;
> &gt; +&nbsp;&nbsp;&nbsp; // low-latency mode: eliminate frame reordering,
> follow a one-in-one-out encoding mode
> &gt; +&nbsp;&nbsp;&nbsp; if ((avctx-&gt;flags &amp;
> AV_CODEC_FLAG_LOW_DELAY) &amp;&amp; avctx-&gt;codec_id == AV_CODEC_ID_H264)
> {
> &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; status =
> VTSessionSetProperty(vtctx-&gt;session,
> &gt;
> +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> kVTVideoEncoderSpecification_EnableLowLatencyRateControl,
> &gt;
> +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> kCFBooleanTrue);
>
> Please use compat_keys in case of
> kVTVideoEncoderSpecification_EnableLowLatencyRateControl
> isn’t available.
>
> &gt; +
> &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (status) {
> &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> av_log(avctx, AV_LOG_ERROR, "Error setting low latency property: %d\n",
> status);
> &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
> &gt; +&nbsp;&nbsp;&nbsp; }
> &gt; +
> &gt;&nbsp;&nbsp;&nbsp;&nbsp; status =
> VTCompressionSessionPrepareToEncodeFrames(vtctx-&gt;session);
> &gt;&nbsp;&nbsp;&nbsp;&nbsp; if (status) {
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; av_log(avctx,
> AV_LOG_ERROR, "Error: cannot prepare encoder: %d\n", status);
> &gt; --
> &gt; 2.32.0 (Apple Git-132)
> &gt;
> &gt; _______________________________________________
> &gt; ffmpeg-devel mailing list
> &gt; ffmpeg-devel@ffmpeg.org
> &gt; https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> &gt;
> &gt; To unsubscribe, visit link above, or email
> &gt; ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
Zhao Zhili May 19, 2023, 6:05 p.m. UTC | #4
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Rick Kern
> Sent: 2023年5月19日 21:37
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Cc: zhilizhao(赵志立) <quinkblack@foxmail.com>
> Subject: Re: [FFmpeg-devel] 回复: [PATCH] avcodec/videotoolboxenc: add low-latency encoding
> 
> On Fri, May 19, 2023 at 1:56 AM 徐福隆 <839789740@qq.com> wrote:
> 
> > Thank you for your review, and point out the details.
> > Actually, I have also considered this issue.
> > I see that some keys are used directly, some are used by compatibility.
> >
> 
> The compat_keys are used for symbols that might not exist. For constants
> available in the earliest version of VideoToolbox, compat_keys doesn't need
> to be used.

I curious to know what's the minimum supported version of macOS which
doesn't need dynamic loading symbols. I can't find such information. The question
applies to other OS too. It's not clear from configure, and I can't find the relevant
documentation.

> 
> Your patch looks good - I'll push it shortly.
> 
> 
> > I will use compat_key instead, and submit a new patch.
> >
> >
> > ------------------&nbsp;原始邮件&nbsp;------------------
> > 发件人:
> >                                                   ""zhilizhao(赵志立)""
> >
> > <quinkblack@foxmail.com&gt;;
> > 发送时间:&nbsp;2023年5月19日(星期五) 中午11:34
> > 收件人:&nbsp;"FFmpeg development discussions and patches"<
> > ffmpeg-devel@ffmpeg.org&gt;;
> > 抄送:&nbsp;"徐福隆"<839789740@qq.com&gt;;
> > 主题:&nbsp;Re: [FFmpeg-devel] [PATCH] avcodec/videotoolboxenc: add
> > low-latency encoding
> >
> >
> >
> >
> >
> > &gt; On May 19, 2023, at 11:17, xufuji456 <839789740@qq.com&gt; wrote:
> > &gt;
> > &gt; When using low-latency mode, it eliminates frame reordering
> > &gt; and follows a one-in-one-out encoding mode
> > &gt;
> > &gt; Signed-off-by: xufuji456 <839789740@qq.com&gt;
> > &gt; ---
> > &gt; libavcodec/videotoolboxenc.c | 11 +++++++++++
> > &gt; 1 file changed, 11 insertions(+)
> > &gt;
> > &gt; diff --git a/libavcodec/videotoolboxenc.c
> > b/libavcodec/videotoolboxenc.c
> > &gt; index c6f22723d6..ae1ef32b2d 100644
> > &gt; --- a/libavcodec/videotoolboxenc.c
> > &gt; +++ b/libavcodec/videotoolboxenc.c
> > &gt; @@ -1441,6 +1441,17 @@ static int
> > vtenc_create_encoder(AVCodecContext&nbsp;&nbsp; *avctx,
> > &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
> > &gt;&nbsp;&nbsp;&nbsp;&nbsp; }
> > &gt;
> > &gt; +&nbsp;&nbsp;&nbsp; // low-latency mode: eliminate frame reordering,
> > follow a one-in-one-out encoding mode
> > &gt; +&nbsp;&nbsp;&nbsp; if ((avctx-&gt;flags &amp;
> > AV_CODEC_FLAG_LOW_DELAY) &amp;&amp; avctx-&gt;codec_id == AV_CODEC_ID_H264)
> > {
> > &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; status =
> > VTSessionSetProperty(vtctx-&gt;session,
> > &gt;
> >
> +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp
> ;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > kVTVideoEncoderSpecification_EnableLowLatencyRateControl,
> > &gt;
> >
> +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp
> ;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > kCFBooleanTrue);
> >
> > Please use compat_keys in case of
> > kVTVideoEncoderSpecification_EnableLowLatencyRateControl
> > isn’t available.
> >
> > &gt; +
> > &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (status) {
> > &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > av_log(avctx, AV_LOG_ERROR, "Error setting low latency property: %d\n",
> > status);
> > &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
> > &gt; +&nbsp;&nbsp;&nbsp; }
> > &gt; +
> > &gt;&nbsp;&nbsp;&nbsp;&nbsp; status =
> > VTCompressionSessionPrepareToEncodeFrames(vtctx-&gt;session);
> > &gt;&nbsp;&nbsp;&nbsp;&nbsp; if (status) {
> > &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; av_log(avctx,
> > AV_LOG_ERROR, "Error: cannot prepare encoder: %d\n", status);
> > &gt; --
> > &gt; 2.32.0 (Apple Git-132)
> > &gt;
> > &gt; _______________________________________________
> > &gt; ffmpeg-devel mailing list
> > &gt; ffmpeg-devel@ffmpeg.org
> > &gt; https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > &gt;
> > &gt; To unsubscribe, visit link above, or email
> > &gt; ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Rick Kern May 20, 2023, 3 p.m. UTC | #5
On Fri, May 19, 2023 at 2:06 PM Zhao Zhili <quinkblack@foxmail.com> wrote:

>
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Rick
> Kern
> > Sent: 2023年5月19日 21:37
> > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > Cc: zhilizhao(赵志立) <quinkblack@foxmail.com>
> > Subject: Re: [FFmpeg-devel] 回复: [PATCH] avcodec/videotoolboxenc: add
> low-latency encoding
> >
> > On Fri, May 19, 2023 at 1:56 AM 徐福隆 <839789740@qq.com> wrote:
> >
> > > Thank you for your review, and point out the details.
> > > Actually, I have also considered this issue.
> > > I see that some keys are used directly, some are used by compatibility.
> > >
> >
> > The compat_keys are used for symbols that might not exist. For constants
> > available in the earliest version of VideoToolbox, compat_keys doesn't
> need
> > to be used.
>
> I curious to know what's the minimum supported version of macOS which
> doesn't need dynamic loading symbols. I can't find such information. The
> question
> applies to other OS too. It's not clear from configure, and I can't find
> the relevant
> documentation.
>
VideoToolbox was released in MacOS 10.8 and iOS 8.0. I'll update the
compat_keys docs with this info.


> >
> > Your patch looks good - I'll push it shortly.
> >
> >
> > > I will use compat_key instead, and submit a new patch.
> > >
> > >
> > > ------------------&nbsp;原始邮件&nbsp;------------------
> > > 发件人:
> > >                                                   ""zhilizhao(赵志立)""
> > >
> > > <quinkblack@foxmail.com&gt;;
> > > 发送时间:&nbsp;2023年5月19日(星期五) 中午11:34
> > > 收件人:&nbsp;"FFmpeg development discussions and patches"<
> > > ffmpeg-devel@ffmpeg.org&gt;;
> > > 抄送:&nbsp;"徐福隆"<839789740@qq.com&gt;;
> > > 主题:&nbsp;Re: [FFmpeg-devel] [PATCH] avcodec/videotoolboxenc: add
> > > low-latency encoding
> > >
> > >
> > >
> > >
> > >
> > > &gt; On May 19, 2023, at 11:17, xufuji456 <839789740@qq.com&gt; wrote:
> > > &gt;
> > > &gt; When using low-latency mode, it eliminates frame reordering
> > > &gt; and follows a one-in-one-out encoding mode
> > > &gt;
> > > &gt; Signed-off-by: xufuji456 <839789740@qq.com&gt;
> > > &gt; ---
> > > &gt; libavcodec/videotoolboxenc.c | 11 +++++++++++
> > > &gt; 1 file changed, 11 insertions(+)
> > > &gt;
> > > &gt; diff --git a/libavcodec/videotoolboxenc.c
> > > b/libavcodec/videotoolboxenc.c
> > > &gt; index c6f22723d6..ae1ef32b2d 100644
> > > &gt; --- a/libavcodec/videotoolboxenc.c
> > > &gt; +++ b/libavcodec/videotoolboxenc.c
> > > &gt; @@ -1441,6 +1441,17 @@ static int
> > > vtenc_create_encoder(AVCodecContext&nbsp;&nbsp; *avctx,
> > > &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
> > > &gt;&nbsp;&nbsp;&nbsp;&nbsp; }
> > > &gt;
> > > &gt; +&nbsp;&nbsp;&nbsp; // low-latency mode: eliminate frame
> reordering,
> > > follow a one-in-one-out encoding mode
> > > &gt; +&nbsp;&nbsp;&nbsp; if ((avctx-&gt;flags &amp;
> > > AV_CODEC_FLAG_LOW_DELAY) &amp;&amp; avctx-&gt;codec_id ==
> AV_CODEC_ID_H264)
> > > {
> > > &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; status =
> > > VTSessionSetProperty(vtctx-&gt;session,
> > > &gt;
> > >
> >
> +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp
> >
> ;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > kVTVideoEncoderSpecification_EnableLowLatencyRateControl,
> > > &gt;
> > >
> >
> +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp
> >
> ;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > kCFBooleanTrue);
> > >
> > > Please use compat_keys in case of
> > > kVTVideoEncoderSpecification_EnableLowLatencyRateControl
> > > isn’t available.
> > >
> > > &gt; +
> > > &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (status) {
> > > &gt;
> +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > av_log(avctx, AV_LOG_ERROR, "Error setting low latency property: %d\n",
> > > status);
> > > &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
> > > &gt; +&nbsp;&nbsp;&nbsp; }
> > > &gt; +
> > > &gt;&nbsp;&nbsp;&nbsp;&nbsp; status =
> > > VTCompressionSessionPrepareToEncodeFrames(vtctx-&gt;session);
> > > &gt;&nbsp;&nbsp;&nbsp;&nbsp; if (status) {
> > > &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; av_log(avctx,
> > > AV_LOG_ERROR, "Error: cannot prepare encoder: %d\n", status);
> > > &gt; --
> > > &gt; 2.32.0 (Apple Git-132)
> > > &gt;
> > > &gt; _______________________________________________
> > > &gt; ffmpeg-devel mailing list
> > > &gt; ffmpeg-devel@ffmpeg.org
> > > &gt; https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > &gt;
> > > &gt; To unsubscribe, visit link above, or email
> > > &gt; ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > _______________________________________________
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
Zhao Zhili May 20, 2023, 4:54 p.m. UTC | #6
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Rick Kern
> Sent: 2023年5月20日 23:01
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] 回复: [PATCH] avcodec/videotoolboxenc: add low-latency encoding
> 
> On Fri, May 19, 2023 at 2:06 PM Zhao Zhili <quinkblack@foxmail.com> wrote:
> 
> >
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Rick
> > Kern
> > > Sent: 2023年5月19日 21:37
> > > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > > Cc: zhilizhao(赵志立) <quinkblack@foxmail.com>
> > > Subject: Re: [FFmpeg-devel] 回复: [PATCH] avcodec/videotoolboxenc: add
> > low-latency encoding
> > >
> > > On Fri, May 19, 2023 at 1:56 AM 徐福隆 <839789740@qq.com> wrote:
> > >
> > > > Thank you for your review, and point out the details.
> > > > Actually, I have also considered this issue.
> > > > I see that some keys are used directly, some are used by compatibility.
> > > >
> > >
> > > The compat_keys are used for symbols that might not exist. For constants
> > > available in the earliest version of VideoToolbox, compat_keys doesn't
> > need
> > > to be used.
> >
> > I curious to know what's the minimum supported version of macOS which
> > doesn't need dynamic loading symbols. I can't find such information. The
> > question
> > applies to other OS too. It's not clear from configure, and I can't find
> > the relevant
> > documentation.
> >
> VideoToolbox was released in MacOS 10.8 and iOS 8.0. I'll update the
> compat_keys docs with this info.

Maybe it's off-topic, but besides than the first version of MacOS which has
videotoolbox supports, I'm considering the strategy from FFmpeg's side: how
to decide when to bump the minimum supported version of OS, like
"[RFC] [Vote] Drop Windows XP support".

https://ffmpeg.org/pipermail/ffmpeg-devel/2017-December/222354.html

I'm not saying it's time to bump the required version of macOS now, but as
time goes by, we might need to do that in the future for other OS, to relief the
maintaining burden.

> 
> 
> > >
> > > Your patch looks good - I'll push it shortly.
> > >
> > >
> > > > I will use compat_key instead, and submit a new patch.
> > > >
> > > >
> > > > ------------------&nbsp;原始邮件&nbsp;------------------
> > > > 发件人:
> > > >                                                   ""zhilizhao(赵志立)""
> > > >
> > > > <quinkblack@foxmail.com&gt;;
> > > > 发送时间:&nbsp;2023年5月19日(星期五) 中午11:34
> > > > 收件人:&nbsp;"FFmpeg development discussions and patches"<
> > > > ffmpeg-devel@ffmpeg.org&gt;;
> > > > 抄送:&nbsp;"徐福隆"<839789740@qq.com&gt;;
> > > > 主题:&nbsp;Re: [FFmpeg-devel] [PATCH] avcodec/videotoolboxenc: add
> > > > low-latency encoding
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > &gt; On May 19, 2023, at 11:17, xufuji456 <839789740@qq.com&gt; wrote:
> > > > &gt;
> > > > &gt; When using low-latency mode, it eliminates frame reordering
> > > > &gt; and follows a one-in-one-out encoding mode
> > > > &gt;
> > > > &gt; Signed-off-by: xufuji456 <839789740@qq.com&gt;
> > > > &gt; ---
> > > > &gt; libavcodec/videotoolboxenc.c | 11 +++++++++++
> > > > &gt; 1 file changed, 11 insertions(+)
> > > > &gt;
> > > > &gt; diff --git a/libavcodec/videotoolboxenc.c
> > > > b/libavcodec/videotoolboxenc.c
> > > > &gt; index c6f22723d6..ae1ef32b2d 100644
> > > > &gt; --- a/libavcodec/videotoolboxenc.c
> > > > &gt; +++ b/libavcodec/videotoolboxenc.c
> > > > &gt; @@ -1441,6 +1441,17 @@ static int
> > > > vtenc_create_encoder(AVCodecContext&nbsp;&nbsp; *avctx,
> > > > &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
> > > > &gt;&nbsp;&nbsp;&nbsp;&nbsp; }
> > > > &gt;
> > > > &gt; +&nbsp;&nbsp;&nbsp; // low-latency mode: eliminate frame
> > reordering,
> > > > follow a one-in-one-out encoding mode
> > > > &gt; +&nbsp;&nbsp;&nbsp; if ((avctx-&gt;flags &amp;
> > > > AV_CODEC_FLAG_LOW_DELAY) &amp;&amp; avctx-&gt;codec_id ==
> > AV_CODEC_ID_H264)
> > > > {
> > > > &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; status =
> > > > VTSessionSetProperty(vtctx-&gt;session,
> > > > &gt;
> > > >
> > >
> >
> +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp
> > >
> > ;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > > kVTVideoEncoderSpecification_EnableLowLatencyRateControl,
> > > > &gt;
> > > >
> > >
> >
> +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp
> > >
> > ;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > > kCFBooleanTrue);
> > > >
> > > > Please use compat_keys in case of
> > > > kVTVideoEncoderSpecification_EnableLowLatencyRateControl
> > > > isn’t available.
> > > >
> > > > &gt; +
> > > > &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (status) {
> > > > &gt;
> > +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > > av_log(avctx, AV_LOG_ERROR, "Error setting low latency property: %d\n",
> > > > status);
> > > > &gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
> > > > &gt; +&nbsp;&nbsp;&nbsp; }
> > > > &gt; +
> > > > &gt;&nbsp;&nbsp;&nbsp;&nbsp; status =
> > > > VTCompressionSessionPrepareToEncodeFrames(vtctx-&gt;session);
> > > > &gt;&nbsp;&nbsp;&nbsp;&nbsp; if (status) {
> > > > &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; av_log(avctx,
> > > > AV_LOG_ERROR, "Error: cannot prepare encoder: %d\n", status);
> > > > &gt; --
> > > > &gt; 2.32.0 (Apple Git-132)
> > > > &gt;
> > > > &gt; _______________________________________________
> > > > &gt; ffmpeg-devel mailing list
> > > > &gt; ffmpeg-devel@ffmpeg.org
> > > > &gt; https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > > &gt;
> > > > &gt; To unsubscribe, visit link above, or email
> > > > &gt; ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > > _______________________________________________
> > > > ffmpeg-devel mailing list
> > > > ffmpeg-devel@ffmpeg.org
> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > >
> > > > To unsubscribe, visit link above, or email
> > > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> > > >
> > > _______________________________________________
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff mbox series

Patch

diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
index c6f22723d6..ae1ef32b2d 100644
--- a/libavcodec/videotoolboxenc.c
+++ b/libavcodec/videotoolboxenc.c
@@ -1441,6 +1441,17 @@  static int vtenc_create_encoder(AVCodecContext   *avctx,
         }
     }
 
+    // low-latency mode: eliminate frame reordering, follow a one-in-one-out encoding mode
+    if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) && avctx->codec_id == AV_CODEC_ID_H264) {
+        status = VTSessionSetProperty(vtctx->session,
+                                      kVTVideoEncoderSpecification_EnableLowLatencyRateControl,
+                                      kCFBooleanTrue);
+
+        if (status) {
+            av_log(avctx, AV_LOG_ERROR, "Error setting low latency property: %d\n", status);
+        }
+    }
+
     status = VTCompressionSessionPrepareToEncodeFrames(vtctx->session);
     if (status) {
         av_log(avctx, AV_LOG_ERROR, "Error: cannot prepare encoder: %d\n", status);