diff mbox

[FFmpeg-devel,3/4] avcodec/nvdec: add support for 12 bit formats

Message ID 20171111152139.14394-3-timo@rothenpieler.org
State New
Headers show

Commit Message

Timo Rothenpieler Nov. 11, 2017, 3:21 p.m. UTC 
---
 libavcodec/nvdec.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

Comments

Carl Eugen Hoyos Nov. 12, 2017, 1:23 a.m. UTC | #1 
2017-11-11 16:21 GMT+01:00 Timo Rothenpieler <timo@rothenpieler.org>:
> ---
>  libavcodec/nvdec.c | 18 +++++++++++++++---
>  1 file changed, 15 insertions(+), 3 deletions(-)
>
> diff --git a/libavcodec/nvdec.c b/libavcodec/nvdec.c
> index 780c2e67cf..d3620f0d1f 100644
> --- a/libavcodec/nvdec.c
> +++ b/libavcodec/nvdec.c
> @@ -234,9 +234,21 @@ int ff_nvdec_decode_init(AVCodecContext *avctx, unsigned int dpb_size)
>          frames_ctx->format            = AV_PIX_FMT_CUDA;
>          frames_ctx->width             = (avctx->width + 1) & ~1;
>          frames_ctx->height            = (avctx->height + 1) & ~1;
> -        frames_ctx->sw_format         = AV_PIX_FMT_NV12;
> -        frames_ctx->sw_format         = sw_desc->comp[0].depth > 8 ?
> -                                        AV_PIX_FMT_P010 : AV_PIX_FMT_NV12;
> +
> +        switch (sw_desc->comp[0].depth) {
> +        case 8:
> +            frames_ctx->sw_format = AV_PIX_FMT_NV12;
> +            break;
> +        case 10:
> +            frames_ctx->sw_format = AV_PIX_FMT_P010;
> +            break;
> +        case 12:
> +            frames_ctx->sw_format = AV_PIX_FMT_P016;

Ideally, this should also set avctx->bits_per_raw_sample
or an equivalent, if it exists.

Carl Eugen
Timo Rothenpieler Nov. 12, 2017, 11:04 a.m. UTC | #2 
> Ideally, this should also set avctx->bits_per_raw_sample
> or an equivalent, if it exists.

I would assume that the surrounding codec parser already sets that?
A hwaccel does not usually set fields in avctx.
diff mbox

Patch

diff --git a/libavcodec/nvdec.c b/libavcodec/nvdec.c
index 780c2e67cf..d3620f0d1f 100644
--- a/libavcodec/nvdec.c
+++ b/libavcodec/nvdec.c
@@ -234,9 +234,21 @@  int ff_nvdec_decode_init(AVCodecContext *avctx, unsigned int dpb_size)
         frames_ctx->format            = AV_PIX_FMT_CUDA;
         frames_ctx->width             = (avctx->width + 1) & ~1;
         frames_ctx->height            = (avctx->height + 1) & ~1;
-        frames_ctx->sw_format         = AV_PIX_FMT_NV12;
-        frames_ctx->sw_format         = sw_desc->comp[0].depth > 8 ?
-                                        AV_PIX_FMT_P010 : AV_PIX_FMT_NV12;
+
+        switch (sw_desc->comp[0].depth) {
+        case 8:
+            frames_ctx->sw_format = AV_PIX_FMT_NV12;
+            break;
+        case 10:
+            frames_ctx->sw_format = AV_PIX_FMT_P010;
+            break;
+        case 12:
+            frames_ctx->sw_format = AV_PIX_FMT_P016;
+            break;
+        default:
+            return AVERROR(EINVAL);
+        }
+
         frames_ctx->initial_pool_size = dpb_size;
 
         ret = av_hwframe_ctx_init(avctx->hw_frames_ctx);