| Message ID | 20231027170446.63684-1-ffmpeg@haasn.xyz |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel,1/8] swscale: fix sws_setColorspaceDetails after sws_init_context | expand |
| Context | Check | Description |
|---|---|---|
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | success | Make fate finished |
| yinshiyou/make_loongarch64 | success | Make finished |
| yinshiyou/make_fate_loongarch64 | success | Make fate finished |
On Fri, Oct 27, 2023 at 07:04:39PM +0200, Niklas Haas wrote: > From: Niklas Haas <git@haasn.dev> > > More commonly, this fixes the case of sws_setColorspaceDetails after > sws_getContext, since the latter implies sws_init_context. > > The problem here is that sws_init_context sets up the range conversion > and fast path tables based on the values of srcRange/dstRange at init > time. This may result in locking in a "wrong" path (either using > unscaled fast path when range conversion later required, or using > scaled slow path when range conversion becomes no longer required). > > There are two way outs: > > 1. Always initialize range conversion and unscaled converters, even if > they will be unused, and extend the runtime check. > 2. Re-do initialization if the values change after > sws_setColorspaceDetails. > > I opted for approach 1 because it was simpler and easier to reason > about. > --- > libswscale/swscale.c | 2 +- > libswscale/utils.c | 5 +---- > 2 files changed, 2 insertions(+), 5 deletions(-) > > diff --git a/libswscale/swscale.c b/libswscale/swscale.c > index 90e5b299ab..46ba68fe6a 100644 > --- a/libswscale/swscale.c > +++ b/libswscale/swscale.c > @@ -1016,7 +1016,7 @@ static int scale_internal(SwsContext *c, > reset_ptr(src2, c->srcFormat); > reset_ptr((void*)dst2, c->dstFormat); > > - if (c->convert_unscaled) { > + if (c->convert_unscaled && !c->lumConvertRange && !c->chrConvertRange) { > int offset = srcSliceY_internal; > int slice_h = srcSliceH; > > diff --git a/libswscale/utils.c b/libswscale/utils.c > index e1ad685972..455955e907 100644 > --- a/libswscale/utils.c > +++ b/libswscale/utils.c > @@ -1728,9 +1728,7 @@ static av_cold int sws_init_single_context(SwsContext *c, SwsFilter *srcFilter, > } > > /* unscaled special cases */ > - if (unscaled && !usesHFilter && !usesVFilter && > - (c->srcRange == c->dstRange || isAnyRGB(dstFormat) || > - isFloat(srcFormat) || isFloat(dstFormat))){ > + if (unscaled && !usesHFilter && !usesVFilter) { > ff_get_unscaled_swscale(c); > > if (c->convert_unscaled) { > @@ -1738,7 +1736,6 @@ static av_cold int sws_init_single_context(SwsContext *c, SwsFilter *srcFilter, > av_log(c, AV_LOG_INFO, > "using unscaled %s -> %s special converter\n", > av_get_pix_fmt_name(srcFormat), av_get_pix_fmt_name(dstFormat)); > - return 0; > } > } the av_log() message will be wrong if this path is unused also this ties convert_unscaled to never do range conversion, if thats intended, i guess thats ok. Otherwise that maybe is a restriction we dont want to add. thx [...]
On Fri, 27 Oct 2023 23:42:41 +0200 Michael Niedermayer <michael@niedermayer.cc> wrote: > On Fri, Oct 27, 2023 at 07:04:39PM +0200, Niklas Haas wrote: > > From: Niklas Haas <git@haasn.dev> > > > > More commonly, this fixes the case of sws_setColorspaceDetails after > > sws_getContext, since the latter implies sws_init_context. > > > > The problem here is that sws_init_context sets up the range conversion > > and fast path tables based on the values of srcRange/dstRange at init > > time. This may result in locking in a "wrong" path (either using > > unscaled fast path when range conversion later required, or using > > scaled slow path when range conversion becomes no longer required). > > > > There are two way outs: > > > > 1. Always initialize range conversion and unscaled converters, even if > > they will be unused, and extend the runtime check. > > 2. Re-do initialization if the values change after > > sws_setColorspaceDetails. > > > > I opted for approach 1 because it was simpler and easier to reason > > about. > > --- > > libswscale/swscale.c | 2 +- > > libswscale/utils.c | 5 +---- > > 2 files changed, 2 insertions(+), 5 deletions(-) > > > > diff --git a/libswscale/swscale.c b/libswscale/swscale.c > > index 90e5b299ab..46ba68fe6a 100644 > > --- a/libswscale/swscale.c > > +++ b/libswscale/swscale.c > > @@ -1016,7 +1016,7 @@ static int scale_internal(SwsContext *c, > > reset_ptr(src2, c->srcFormat); > > reset_ptr((void*)dst2, c->dstFormat); > > > > - if (c->convert_unscaled) { > > + if (c->convert_unscaled && !c->lumConvertRange && !c->chrConvertRange) { > > int offset = srcSliceY_internal; > > int slice_h = srcSliceH; > > > > diff --git a/libswscale/utils.c b/libswscale/utils.c > > index e1ad685972..455955e907 100644 > > --- a/libswscale/utils.c > > +++ b/libswscale/utils.c > > @@ -1728,9 +1728,7 @@ static av_cold int sws_init_single_context(SwsContext *c, SwsFilter *srcFilter, > > } > > > > /* unscaled special cases */ > > - if (unscaled && !usesHFilter && !usesVFilter && > > - (c->srcRange == c->dstRange || isAnyRGB(dstFormat) || > > - isFloat(srcFormat) || isFloat(dstFormat))){ > > + if (unscaled && !usesHFilter && !usesVFilter) { > > ff_get_unscaled_swscale(c); > > > > if (c->convert_unscaled) { > > @@ -1738,7 +1736,6 @@ static av_cold int sws_init_single_context(SwsContext *c, SwsFilter *srcFilter, > > av_log(c, AV_LOG_INFO, > > "using unscaled %s -> %s special converter\n", > > av_get_pix_fmt_name(srcFormat), av_get_pix_fmt_name(dstFormat)); > > - return 0; > > } > > } > > the av_log() message will be wrong if this path is unused Indeed. Perhaps the only way to fix that would be to instead try approach 2 and go through full reinit if any params change after setColorspaceDetails. > also this ties convert_unscaled to never do range conversion, if thats > intended, i guess thats ok. Otherwise that maybe is a restriction > we dont want to add. In the original code, c->convert_unscaled is set behind a `c->srcRange == c->dstRange` check, so how is that a change in behavior?
On Fri, 27 Oct 2023 23:42:41 +0200 Michael Niedermayer <michael@niedermayer.cc> wrote:
> the av_log() message will be wrong if this path is unused
Well, this is a problem even if we move the log printing to context
reinit after sws_setColorspaceDetails, because it will still get printed
on the initial init (before range is known), unless we move it to
sws_scale_frame() itself. But then we would need to keep track also of
whether or not we already printed this.
Since this is just a developer-facing debug message, I'll probably just
reword it.
On Sat, Oct 28, 2023 at 12:46:38PM +0200, Niklas Haas wrote: > On Fri, 27 Oct 2023 23:42:41 +0200 Michael Niedermayer <michael@niedermayer.cc> wrote: > > On Fri, Oct 27, 2023 at 07:04:39PM +0200, Niklas Haas wrote: > > > From: Niklas Haas <git@haasn.dev> > > > > > > More commonly, this fixes the case of sws_setColorspaceDetails after > > > sws_getContext, since the latter implies sws_init_context. > > > > > > The problem here is that sws_init_context sets up the range conversion > > > and fast path tables based on the values of srcRange/dstRange at init > > > time. This may result in locking in a "wrong" path (either using > > > unscaled fast path when range conversion later required, or using > > > scaled slow path when range conversion becomes no longer required). > > > > > > There are two way outs: > > > > > > 1. Always initialize range conversion and unscaled converters, even if > > > they will be unused, and extend the runtime check. > > > 2. Re-do initialization if the values change after > > > sws_setColorspaceDetails. > > > > > > I opted for approach 1 because it was simpler and easier to reason > > > about. > > > --- > > > libswscale/swscale.c | 2 +- > > > libswscale/utils.c | 5 +---- > > > 2 files changed, 2 insertions(+), 5 deletions(-) > > > > > > diff --git a/libswscale/swscale.c b/libswscale/swscale.c > > > index 90e5b299ab..46ba68fe6a 100644 > > > --- a/libswscale/swscale.c > > > +++ b/libswscale/swscale.c > > > @@ -1016,7 +1016,7 @@ static int scale_internal(SwsContext *c, > > > reset_ptr(src2, c->srcFormat); > > > reset_ptr((void*)dst2, c->dstFormat); > > > > > > - if (c->convert_unscaled) { > > > + if (c->convert_unscaled && !c->lumConvertRange && !c->chrConvertRange) { > > > int offset = srcSliceY_internal; > > > int slice_h = srcSliceH; > > > > > > diff --git a/libswscale/utils.c b/libswscale/utils.c > > > index e1ad685972..455955e907 100644 > > > --- a/libswscale/utils.c > > > +++ b/libswscale/utils.c > > > @@ -1728,9 +1728,7 @@ static av_cold int sws_init_single_context(SwsContext *c, SwsFilter *srcFilter, > > > } > > > > > > /* unscaled special cases */ > > > - if (unscaled && !usesHFilter && !usesVFilter && > > > - (c->srcRange == c->dstRange || isAnyRGB(dstFormat) || > > > - isFloat(srcFormat) || isFloat(dstFormat))){ > > > + if (unscaled && !usesHFilter && !usesVFilter) { > > > ff_get_unscaled_swscale(c); > > > > > > if (c->convert_unscaled) { > > > @@ -1738,7 +1736,6 @@ static av_cold int sws_init_single_context(SwsContext *c, SwsFilter *srcFilter, > > > av_log(c, AV_LOG_INFO, > > > "using unscaled %s -> %s special converter\n", > > > av_get_pix_fmt_name(srcFormat), av_get_pix_fmt_name(dstFormat)); > > > - return 0; > > > } > > > } > > > > the av_log() message will be wrong if this path is unused > > Indeed. Perhaps the only way to fix that would be to instead try > approach 2 and go through full reinit if any params change after > setColorspaceDetails. > > > also this ties convert_unscaled to never do range conversion, if thats > > intended, i guess thats ok. Otherwise that maybe is a restriction > > we dont want to add. > > In the original code, c->convert_unscaled is set behind a `c->srcRange > == c->dstRange` check, so how is that a change in behavior? not "change in behavior" but it may be more complex if a yuv->yuv unscaled convert is added that supports range convert because before convert_unscaled is set only when it can be used afterwards convert_unscaled is always set but sometimes it then can be used with differing yuv ranges sometimes not. what i meant is that the conditions related to convert_unscaled could become more dispersed and duplciated _IF_ specific future code is added also its not just c->srcRange == c->dstRange, the condition is more complex thx [...]
diff --git a/libswscale/swscale.c b/libswscale/swscale.c index 90e5b299ab..46ba68fe6a 100644 --- a/libswscale/swscale.c +++ b/libswscale/swscale.c @@ -1016,7 +1016,7 @@ static int scale_internal(SwsContext *c, reset_ptr(src2, c->srcFormat); reset_ptr((void*)dst2, c->dstFormat); - if (c->convert_unscaled) { + if (c->convert_unscaled && !c->lumConvertRange && !c->chrConvertRange) { int offset = srcSliceY_internal; int slice_h = srcSliceH; diff --git a/libswscale/utils.c b/libswscale/utils.c index e1ad685972..455955e907 100644 --- a/libswscale/utils.c +++ b/libswscale/utils.c @@ -1728,9 +1728,7 @@ static av_cold int sws_init_single_context(SwsContext *c, SwsFilter *srcFilter, } /* unscaled special cases */ - if (unscaled && !usesHFilter && !usesVFilter && - (c->srcRange == c->dstRange || isAnyRGB(dstFormat) || - isFloat(srcFormat) || isFloat(dstFormat))){ + if (unscaled && !usesHFilter && !usesVFilter) { ff_get_unscaled_swscale(c); if (c->convert_unscaled) { @@ -1738,7 +1736,6 @@ static av_cold int sws_init_single_context(SwsContext *c, SwsFilter *srcFilter, av_log(c, AV_LOG_INFO, "using unscaled %s -> %s special converter\n", av_get_pix_fmt_name(srcFormat), av_get_pix_fmt_name(dstFormat)); - return 0; } }
From: Niklas Haas <git@haasn.dev> More commonly, this fixes the case of sws_setColorspaceDetails after sws_getContext, since the latter implies sws_init_context. The problem here is that sws_init_context sets up the range conversion and fast path tables based on the values of srcRange/dstRange at init time. This may result in locking in a "wrong" path (either using unscaled fast path when range conversion later required, or using scaled slow path when range conversion becomes no longer required). There are two way outs: 1. Always initialize range conversion and unscaled converters, even if they will be unused, and extend the runtime check. 2. Re-do initialization if the values change after sws_setColorspaceDetails. I opted for approach 1 because it was simpler and easier to reason about. --- libswscale/swscale.c | 2 +- libswscale/utils.c | 5 +---- 2 files changed, 2 insertions(+), 5 deletions(-)