[FFmpeg-devel,3/3] avcodec/videotoolboxenc: fix invalid session on iOS

Submitted by Thomas Guillem on June 11, 2018, 2:21 p.m.

Details

Message ID 20180611142119.12001-3-thomas@gllm.fr
State New
Headers show

Commit Message

Thomas Guillem June 11, 2018, 2:21 p.m.
Cf. comment. Restart the VT session when the APP goes from foreground to
background and vice versa.
---
 libavcodec/videotoolboxenc.c | 25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)

Comments

Aman Gupta June 13, 2018, 6:45 p.m.
On Mon, Jun 11, 2018 at 7:21 AM Thomas Guillem <thomas@gllm.fr> wrote:

> Cf. comment. Restart the VT session when the APP goes from foreground to
> background and vice versa.
> ---
>  libavcodec/videotoolboxenc.c | 25 +++++++++++++++++++++++--
>  1 file changed, 23 insertions(+), 2 deletions(-)
>
> diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
> index 1060055ba5..4b79fca45e 100644
> --- a/libavcodec/videotoolboxenc.c
> +++ b/libavcodec/videotoolboxenc.c
> @@ -2175,8 +2175,29 @@ static int create_cv_pixel_buffer(AVCodecContext
>  *avctx,
>  #if TARGET_OS_IPHONE
>      pix_buf_pool = VTCompressionSessionGetPixelBufferPool(vtctx->session);
>      if (!pix_buf_pool) {
> -        av_log(avctx, AV_LOG_ERROR, "Could not get pixel buffer pool.\n");
> -        return AVERROR_EXTERNAL;
> +        /* On iOS, the VT session is invalidated when the APP switches
> from
> +         * foreground to background and vice versa. Fetch the actual
> error code
> +         * of the VT session to detect that case and restart the VT
> session
> +         * accordingly. */
> +        OSStatus vtstatus =
> +            VTCompressionSessionPrepareToEncodeFrames(vtctx->session);
> +
> +        if (vtstatus == kVTInvalidSessionErr)
> +        {
>

Please copy the style of the surrounding code.


> +            CFRelease(vtctx->session);
> +            vtctx->session = NULL;
> +            status = vtenc_configure_encoder(avctx);
> +            if (status == 0)
> +                pix_buf_pool =
> VTCompressionSessionGetPixelBufferPool(vtctx->session);
> +        }
> +        if (!pix_buf_pool)
> +        {
> +            av_log(avctx, AV_LOG_ERROR, "Could not get pixel buffer
> pool.\n");
> +            return AVERROR_EXTERNAL;
> +        }
> +        else
> +            av_log(avctx, AV_LOG_WARNING, "VT session restarted because
> of a "
> +                   "kVTInvalidSessionErr error.\n");
>      }
>
>      status = CVPixelBufferPoolCreatePixelBuffer(NULL,
>

Patchset diff looks good to me. Did not test on device, but the changes
look reasonable.

I can commit this if you fix the style issues above.

Aman


> --
> 2.17.1
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>

Patch hide | download patch | download mbox

diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
index 1060055ba5..4b79fca45e 100644
--- a/libavcodec/videotoolboxenc.c
+++ b/libavcodec/videotoolboxenc.c
@@ -2175,8 +2175,29 @@  static int create_cv_pixel_buffer(AVCodecContext   *avctx,
 #if TARGET_OS_IPHONE
     pix_buf_pool = VTCompressionSessionGetPixelBufferPool(vtctx->session);
     if (!pix_buf_pool) {
-        av_log(avctx, AV_LOG_ERROR, "Could not get pixel buffer pool.\n");
-        return AVERROR_EXTERNAL;
+        /* On iOS, the VT session is invalidated when the APP switches from
+         * foreground to background and vice versa. Fetch the actual error code
+         * of the VT session to detect that case and restart the VT session
+         * accordingly. */
+        OSStatus vtstatus =
+            VTCompressionSessionPrepareToEncodeFrames(vtctx->session);
+
+        if (vtstatus == kVTInvalidSessionErr)
+        {
+            CFRelease(vtctx->session);
+            vtctx->session = NULL;
+            status = vtenc_configure_encoder(avctx);
+            if (status == 0)
+                pix_buf_pool = VTCompressionSessionGetPixelBufferPool(vtctx->session);
+        }
+        if (!pix_buf_pool)
+        {
+            av_log(avctx, AV_LOG_ERROR, "Could not get pixel buffer pool.\n");
+            return AVERROR_EXTERNAL;
+        }
+        else
+            av_log(avctx, AV_LOG_WARNING, "VT session restarted because of a "
+                   "kVTInvalidSessionErr error.\n");
     }
 
     status = CVPixelBufferPoolCreatePixelBuffer(NULL,