| Message ID | 20210322063150.2269270-1-wenbin.chen@intel.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel] libavcodec/qsvdec: use the param from decodeHeader to configure surface | expand |
| Context | Check | Description |
|---|---|---|
| andriy/x86_make | success | Make finished |
| andriy/x86_make_fate | success | Make fate finished |
| andriy/PPC64_make | success | Make finished |
| andriy/PPC64_make_fate | success | Make fate finished |
On Mon, 2021-03-22 at 14:31 +0800, wenbin.chen@intel.com wrote: > From: "Chen,Wenbin" <wenbin.chen@intel.com> > > MSDK recognizes both yuv420p10 and yuv420p9 as MFX_FOURCC_P010, but param > are different. When decode yuv420p9 video, ffmpeg-qsv will use > yuv420p10le to configure surface which is different with param from > DecoderHeader and this will lead to error. Now change it use > param from decoderHeader to configure surface. Both yuv420p10 and yuv420p9 have 3 planes ( https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/pixdesc.c#L1359-L1406) , but MFX_FOURCC_P010 has 2 planes only. MSDK doesn't recognizes yuv420p10 and yuv420p9 as MFX_FOURCC_P010. FFmpeg-qsv uses p010le instead of yuv420p10le to configure surface for 10bit video. > > Signed-off-by Wenbin Chen <wenbin.chen@intel.com> > --- > libavcodec/qsvdec.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c > index 569ccd4fba..3ab48ea7a2 100644 > --- a/libavcodec/qsvdec.c > +++ b/libavcodec/qsvdec.c > @@ -309,13 +309,13 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext > *q, QSVFrame *frame) > if (frame->frame->format == AV_PIX_FMT_QSV) { > frame->surface = *(mfxFrameSurface1*)frame->frame->data[3]; > } else { > - frame->surface.Info = q->frame_info; > - > frame->surface.Data.PitchLow = frame->frame->linesize[0]; > frame->surface.Data.Y = frame->frame->data[0]; > frame->surface.Data.UV = frame->frame->data[1]; > } > > + frame->surface.Info = q->frame_info; > + > if (q->frames_ctx.mids) { > ret = ff_qsv_find_surface_idx(&q->frames_ctx, frame); > if (ret < 0)
> -----Original Message----- > From: Xiang, Haihao <haihao.xiang@intel.com> > Sent: Tuesday, March 23, 2021 12:10 PM > To: ffmpeg-devel@ffmpeg.org > Cc: Chen, Wenbin <wenbin.chen@intel.com> > Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/qsvdec: use the param from > decodeHeader to configure surface > > On Mon, 2021-03-22 at 14:31 +0800, wenbin.chen@intel.com wrote: > > From: "Chen,Wenbin" <wenbin.chen@intel.com> > > > > MSDK recognizes both yuv420p10 and yuv420p9 as MFX_FOURCC_P010, > but param > > are different. When decode yuv420p9 video, ffmpeg-qsv will use > > yuv420p10le to configure surface which is different with param from > > DecoderHeader and this will lead to error. Now change it use > > param from decoderHeader to configure surface. > > > Both yuv420p10 and yuv420p9 have 3 planes ( > https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/pixdesc.c#L1359- > L1406) , > but MFX_FOURCC_P010 has 2 planes only. MSDK doesn't recognizes > yuv420p10 and > yuv420p9 as MFX_FOURCC_P010. FFmpeg-qsv uses p010le instead of > yuv420p10le to > configure surface for 10bit video. > Sorry, I didn't describe the problem well. I mean when MSDK decodeHeader a 9bit video It will return MFX_FOURCC_P010, but its frameInfo is 9bit. However FFmpeg-qsv will use P010's parameter to configure surface which will be 10bit, and if ffmpeg-qsv send 10bit surface to decode 9bit video MSDK will report an error. > > > > > Signed-off-by Wenbin Chen <wenbin.chen@intel.com> > > --- > > libavcodec/qsvdec.c | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c > > index 569ccd4fba..3ab48ea7a2 100644 > > --- a/libavcodec/qsvdec.c > > +++ b/libavcodec/qsvdec.c > > @@ -309,13 +309,13 @@ static int alloc_frame(AVCodecContext *avctx, > QSVContext > > *q, QSVFrame *frame) > > if (frame->frame->format == AV_PIX_FMT_QSV) { > > frame->surface = *(mfxFrameSurface1*)frame->frame->data[3]; > > } else { > > - frame->surface.Info = q->frame_info; > > - > > frame->surface.Data.PitchLow = frame->frame->linesize[0]; > > frame->surface.Data.Y = frame->frame->data[0]; > > frame->surface.Data.UV = frame->frame->data[1]; > > } > > > > + frame->surface.Info = q->frame_info; > > + > > if (q->frames_ctx.mids) { > > ret = ff_qsv_find_surface_idx(&q->frames_ctx, frame); > > if (ret < 0)
On Tue, 2021-03-23 at 06:45 +0000, Chen, Wenbin wrote: > > -----Original Message----- > > From: Xiang, Haihao <haihao.xiang@intel.com> > > Sent: Tuesday, March 23, 2021 12:10 PM > > To: ffmpeg-devel@ffmpeg.org > > Cc: Chen, Wenbin <wenbin.chen@intel.com> > > Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/qsvdec: use the param from > > decodeHeader to configure surface > > > > On Mon, 2021-03-22 at 14:31 +0800, wenbin.chen@intel.com wrote: > > > From: "Chen,Wenbin" <wenbin.chen@intel.com> > > > > > > MSDK recognizes both yuv420p10 and yuv420p9 as MFX_FOURCC_P010, > > > > but param > > > are different. When decode yuv420p9 video, ffmpeg-qsv will use > > > yuv420p10le to configure surface which is different with param from > > > DecoderHeader and this will lead to error. Now change it use > > > param from decoderHeader to configure surface. > > > > > > Both yuv420p10 and yuv420p9 have 3 planes ( > > https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/pixdesc.c#L1359- > > L1406) , > > but MFX_FOURCC_P010 has 2 planes only. MSDK doesn't recognizes > > yuv420p10 and > > yuv420p9 as MFX_FOURCC_P010. FFmpeg-qsv uses p010le instead of > > yuv420p10le to > > configure surface for 10bit video. > > > > Sorry, I didn't describe the problem well. I mean when MSDK decodeHeader a > 9bit video > It will return MFX_FOURCC_P010, but its frameInfo is 9bit. However FFmpeg-qsv > will use P010's parameter > to configure surface which will be 10bit, and if ffmpeg-qsv send 10bit surface > to decode 9bit video MSDK will report an error. > So it will use yuv420p10le for a 9bit video, right? A potential issue here is the depth info will be lost in the pipeline, a filter after qsv decoder will take it as 10bit frame instead of 9bit frame. Thanks Haihao > > > > > > > > Signed-off-by Wenbin Chen <wenbin.chen@intel.com> > > > --- > > > libavcodec/qsvdec.c | 4 ++-- > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c > > > index 569ccd4fba..3ab48ea7a2 100644 > > > --- a/libavcodec/qsvdec.c > > > +++ b/libavcodec/qsvdec.c > > > @@ -309,13 +309,13 @@ static int alloc_frame(AVCodecContext *avctx, > > > > QSVContext > > > *q, QSVFrame *frame) > > > if (frame->frame->format == AV_PIX_FMT_QSV) { > > > frame->surface = *(mfxFrameSurface1*)frame->frame->data[3]; > > > } else { > > > - frame->surface.Info = q->frame_info; > > > - > > > frame->surface.Data.PitchLow = frame->frame->linesize[0]; > > > frame->surface.Data.Y = frame->frame->data[0]; > > > frame->surface.Data.UV = frame->frame->data[1]; > > > } > > > > > > + frame->surface.Info = q->frame_info; > > > + > > > if (q->frames_ctx.mids) { > > > ret = ff_qsv_find_surface_idx(&q->frames_ctx, frame); > > > if (ret < 0)
> -----Original Message----- > From: Xiang, Haihao <haihao.xiang@intel.com> > Sent: Tuesday, April 6, 2021 2:24 PM > To: Chen, Wenbin <wenbin.chen@intel.com>; ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/qsvdec: use the param from > decodeHeader to configure surface > > On Tue, 2021-03-23 at 06:45 +0000, Chen, Wenbin wrote: > > > -----Original Message----- > > > From: Xiang, Haihao <haihao.xiang@intel.com> > > > Sent: Tuesday, March 23, 2021 12:10 PM > > > To: ffmpeg-devel@ffmpeg.org > > > Cc: Chen, Wenbin <wenbin.chen@intel.com> > > > Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/qsvdec: use the param > from > > > decodeHeader to configure surface > > > > > > On Mon, 2021-03-22 at 14:31 +0800, wenbin.chen@intel.com wrote: > > > > From: "Chen,Wenbin" <wenbin.chen@intel.com> > > > > > > > > MSDK recognizes both yuv420p10 and yuv420p9 as MFX_FOURCC_P010, > > > > > > but param > > > > are different. When decode yuv420p9 video, ffmpeg-qsv will use > > > > yuv420p10le to configure surface which is different with param from > > > > DecoderHeader and this will lead to error. Now change it use > > > > param from decoderHeader to configure surface. > > > > > > > > > Both yuv420p10 and yuv420p9 have 3 planes ( > > > > https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/pixdesc.c#L1359- > > > L1406) , > > > but MFX_FOURCC_P010 has 2 planes only. MSDK doesn't recognizes > > > yuv420p10 and > > > yuv420p9 as MFX_FOURCC_P010. FFmpeg-qsv uses p010le instead of > > > yuv420p10le to > > > configure surface for 10bit video. > > > > > > > Sorry, I didn't describe the problem well. I mean when MSDK > decodeHeader a > > 9bit video > > It will return MFX_FOURCC_P010, but its frameInfo is 9bit. However > FFmpeg-qsv > > will use P010's parameter > > to configure surface which will be 10bit, and if ffmpeg-qsv send 10bit > surface > > to decode 9bit video MSDK will report an error. > > > > So it will use yuv420p10le for a 9bit video, right? A potential issue here is > the depth info will be lost in the pipeline, a filter after qsv decoder will > take it as 10bit frame instead of 9bit frame. > > Thanks > Haihao > > Yes, there is a risk of losing depth information. Function ff_qsv_map_fourcc() should also takes "FrameInfo.BitDepth" into consideration. This issue is not brought by this patch, we may fix it in a new patch. Thanks wenbin > > > > > > > > > > > Signed-off-by Wenbin Chen <wenbin.chen@intel.com> > > > > --- > > > > libavcodec/qsvdec.c | 4 ++-- > > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c > > > > index 569ccd4fba..3ab48ea7a2 100644 > > > > --- a/libavcodec/qsvdec.c > > > > +++ b/libavcodec/qsvdec.c > > > > @@ -309,13 +309,13 @@ static int alloc_frame(AVCodecContext *avctx, > > > > > > QSVContext > > > > *q, QSVFrame *frame) > > > > if (frame->frame->format == AV_PIX_FMT_QSV) { > > > > frame->surface = *(mfxFrameSurface1*)frame->frame->data[3]; > > > > } else { > > > > - frame->surface.Info = q->frame_info; > > > > - > > > > frame->surface.Data.PitchLow = frame->frame->linesize[0]; > > > > frame->surface.Data.Y = frame->frame->data[0]; > > > > frame->surface.Data.UV = frame->frame->data[1]; > > > > } > > > > > > > > + frame->surface.Info = q->frame_info; > > > > + > > > > if (q->frames_ctx.mids) { > > > > ret = ff_qsv_find_surface_idx(&q->frames_ctx, frame); > > > > if (ret < 0)
On Tue, 2021-04-06 at 07:54 +0000, Chen, Wenbin wrote: > > -----Original Message----- > > From: Xiang, Haihao <haihao.xiang@intel.com> > > Sent: Tuesday, April 6, 2021 2:24 PM > > To: Chen, Wenbin <wenbin.chen@intel.com>; ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/qsvdec: use the param from > > decodeHeader to configure surface > > > > On Tue, 2021-03-23 at 06:45 +0000, Chen, Wenbin wrote: > > > > -----Original Message----- > > > > From: Xiang, Haihao <haihao.xiang@intel.com> > > > > Sent: Tuesday, March 23, 2021 12:10 PM > > > > To: ffmpeg-devel@ffmpeg.org > > > > Cc: Chen, Wenbin <wenbin.chen@intel.com> > > > > Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/qsvdec: use the param > > > > from > > > > decodeHeader to configure surface > > > > > > > > On Mon, 2021-03-22 at 14:31 +0800, wenbin.chen@intel.com wrote: > > > > > From: "Chen,Wenbin" <wenbin.chen@intel.com> > > > > > > > > > > MSDK recognizes both yuv420p10 and yuv420p9 as MFX_FOURCC_P010, > > > > > > > > but param > > > > > are different. When decode yuv420p9 video, ffmpeg-qsv will use > > > > > yuv420p10le to configure surface which is different with param from > > > > > DecoderHeader and this will lead to error. Now change it use > > > > > param from decoderHeader to configure surface. > > > > > > > > > > > > Both yuv420p10 and yuv420p9 have 3 planes ( > > > > > > > > https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/pixdesc.c#L1359- > > > > L1406) , > > > > but MFX_FOURCC_P010 has 2 planes only. MSDK doesn't recognizes > > > > yuv420p10 and > > > > yuv420p9 as MFX_FOURCC_P010. FFmpeg-qsv uses p010le instead of > > > > yuv420p10le to > > > > configure surface for 10bit video. > > > > > > > > > > Sorry, I didn't describe the problem well. I mean when MSDK > > > > decodeHeader a > > > 9bit video > > > It will return MFX_FOURCC_P010, but its frameInfo is 9bit. However > > > > FFmpeg-qsv > > > will use P010's parameter > > > to configure surface which will be 10bit, and if ffmpeg-qsv send 10bit > > > > surface > > > to decode 9bit video MSDK will report an error. > > > > > > > So it will use yuv420p10le for a 9bit video, right? A potential issue here > > is > > the depth info will be lost in the pipeline, a filter after qsv decoder will > > take it as 10bit frame instead of 9bit frame. > > > > Thanks > > Haihao > > > > > > Yes, there is a risk of losing depth information. Function ff_qsv_map_fourcc() > should also takes > "FrameInfo.BitDepth" into consideration. This issue is not brought by this > patch, we may fix it in a new patch. FFmpeg has already used pixfmt for both memory layout and the used bits in a frame. I think you should add a new pixfmt for your case, otherwise we won't know the bit depth when handling a yuv420p10le frame in a filter if using yuv420p10le for a 9bit video. > > Thanks > wenbin > > > > > > > > > > > > > > > Signed-off-by Wenbin Chen <wenbin.chen@intel.com> > > > > > --- > > > > > libavcodec/qsvdec.c | 4 ++-- > > > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c > > > > > index 569ccd4fba..3ab48ea7a2 100644 > > > > > --- a/libavcodec/qsvdec.c > > > > > +++ b/libavcodec/qsvdec.c > > > > > @@ -309,13 +309,13 @@ static int alloc_frame(AVCodecContext *avctx, > > > > > > > > QSVContext > > > > > *q, QSVFrame *frame) > > > > > if (frame->frame->format == AV_PIX_FMT_QSV) { > > > > > frame->surface = *(mfxFrameSurface1*)frame->frame->data[3]; > > > > > } else { > > > > > - frame->surface.Info = q->frame_info; > > > > > - > > > > > frame->surface.Data.PitchLow = frame->frame->linesize[0]; > > > > > frame->surface.Data.Y = frame->frame->data[0]; > > > > > frame->surface.Data.UV = frame->frame->data[1]; > > > > > } > > > > > > > > > > + frame->surface.Info = q->frame_info; > > > > > + > > > > > if (q->frames_ctx.mids) { > > > > > ret = ff_qsv_find_surface_idx(&q->frames_ctx, frame); > > > > > if (ret < 0)
> -----Original Message----- > From: Xiang, Haihao <haihao.xiang@intel.com> > Sent: Tuesday, April 6, 2021 11:08 PM > To: Chen, Wenbin <wenbin.chen@intel.com>; ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/qsvdec: use the param from > decodeHeader to configure surface > > On Tue, 2021-04-06 at 07:54 +0000, Chen, Wenbin wrote: > > > -----Original Message----- > > > From: Xiang, Haihao <haihao.xiang@intel.com> > > > Sent: Tuesday, April 6, 2021 2:24 PM > > > To: Chen, Wenbin <wenbin.chen@intel.com>; ffmpeg-devel@ffmpeg.org > > > Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/qsvdec: use the param > from > > > decodeHeader to configure surface > > > > > > On Tue, 2021-03-23 at 06:45 +0000, Chen, Wenbin wrote: > > > > > -----Original Message----- > > > > > From: Xiang, Haihao <haihao.xiang@intel.com> > > > > > Sent: Tuesday, March 23, 2021 12:10 PM > > > > > To: ffmpeg-devel@ffmpeg.org > > > > > Cc: Chen, Wenbin <wenbin.chen@intel.com> > > > > > Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/qsvdec: use the > param > > > > > > from > > > > > decodeHeader to configure surface > > > > > > > > > > On Mon, 2021-03-22 at 14:31 +0800, wenbin.chen@intel.com wrote: > > > > > > From: "Chen,Wenbin" <wenbin.chen@intel.com> > > > > > > > > > > > > MSDK recognizes both yuv420p10 and yuv420p9 as > MFX_FOURCC_P010, > > > > > > > > > > but param > > > > > > are different. When decode yuv420p9 video, ffmpeg-qsv will use > > > > > > yuv420p10le to configure surface which is different with param from > > > > > > DecoderHeader and this will lead to error. Now change it use > > > > > > param from decoderHeader to configure surface. > > > > > > > > > > > > > > > Both yuv420p10 and yuv420p9 have 3 planes ( > > > > > > > > > > > > https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/pixdesc.c#L1359- > > > > > L1406) , > > > > > but MFX_FOURCC_P010 has 2 planes only. MSDK doesn't recognizes > > > > > yuv420p10 and > > > > > yuv420p9 as MFX_FOURCC_P010. FFmpeg-qsv uses p010le instead of > > > > > yuv420p10le to > > > > > configure surface for 10bit video. > > > > > > > > > > > > > Sorry, I didn't describe the problem well. I mean when MSDK > > > > > > decodeHeader a > > > > 9bit video > > > > It will return MFX_FOURCC_P010, but its frameInfo is 9bit. However > > > > > > FFmpeg-qsv > > > > will use P010's parameter > > > > to configure surface which will be 10bit, and if ffmpeg-qsv send 10bit > > > > > > surface > > > > to decode 9bit video MSDK will report an error. > > > > > > > > > > So it will use yuv420p10le for a 9bit video, right? A potential issue here > > > is > > > the depth info will be lost in the pipeline, a filter after qsv decoder will > > > take it as 10bit frame instead of 9bit frame. > > > > > > Thanks > > > Haihao > > > > > > > > > > Yes, there is a risk of losing depth information. Function > ff_qsv_map_fourcc() > > should also takes > > "FrameInfo.BitDepth" into consideration. This issue is not brought by this > > patch, we may fix it in a new patch. > > > FFmpeg has already used pixfmt for both memory layout and the used bits in > a > frame. I think you should add a new pixfmt for your case, otherwise we > won't > know the bit depth when handling a yuv420p10le frame in a filter if using > yuv420p10le for a 9bit video. > Ok, I will try to add a new pixel format. > > > > > Thanks > > wenbin > > > > > > > > > > > > > > > > > > > Signed-off-by Wenbin Chen <wenbin.chen@intel.com> > > > > > > --- > > > > > > libavcodec/qsvdec.c | 4 ++-- > > > > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > > > > > > > diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c > > > > > > index 569ccd4fba..3ab48ea7a2 100644 > > > > > > --- a/libavcodec/qsvdec.c > > > > > > +++ b/libavcodec/qsvdec.c > > > > > > @@ -309,13 +309,13 @@ static int alloc_frame(AVCodecContext > *avctx, > > > > > > > > > > QSVContext > > > > > > *q, QSVFrame *frame) > > > > > > if (frame->frame->format == AV_PIX_FMT_QSV) { > > > > > > frame->surface = *(mfxFrameSurface1*)frame->frame- > >data[3]; > > > > > > } else { > > > > > > - frame->surface.Info = q->frame_info; > > > > > > - > > > > > > frame->surface.Data.PitchLow = frame->frame->linesize[0]; > > > > > > frame->surface.Data.Y = frame->frame->data[0]; > > > > > > frame->surface.Data.UV = frame->frame->data[1]; > > > > > > } > > > > > > > > > > > > + frame->surface.Info = q->frame_info; > > > > > > + > > > > > > if (q->frames_ctx.mids) { > > > > > > ret = ff_qsv_find_surface_idx(&q->frames_ctx, frame); > > > > > > if (ret < 0)
diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 569ccd4fba..3ab48ea7a2 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -309,13 +309,13 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext *q, QSVFrame *frame) if (frame->frame->format == AV_PIX_FMT_QSV) { frame->surface = *(mfxFrameSurface1*)frame->frame->data[3]; } else { - frame->surface.Info = q->frame_info; - frame->surface.Data.PitchLow = frame->frame->linesize[0]; frame->surface.Data.Y = frame->frame->data[0]; frame->surface.Data.UV = frame->frame->data[1]; } + frame->surface.Info = q->frame_info; + if (q->frames_ctx.mids) { ret = ff_qsv_find_surface_idx(&q->frames_ctx, frame); if (ret < 0)
From: "Chen,Wenbin" <wenbin.chen@intel.com> MSDK recognizes both yuv420p10 and yuv420p9 as MFX_FOURCC_P010, but param are different. When decode yuv420p9 video, ffmpeg-qsv will use yuv420p10le to configure surface which is different with param from DecoderHeader and this will lead to error. Now change it use param from decoderHeader to configure surface. Signed-off-by Wenbin Chen <wenbin.chen@intel.com> --- libavcodec/qsvdec.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)