[FFmpeg-devel,v13,2/4] libavformat/avisynth.c: Remove MAX_PATH limit

---
 libavformat/avisynth.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Nil
> Admirari
> Sent: Thursday, June 9, 2022 9:04 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH v13 2/4] libavformat/avisynth.c:
> Remove MAX_PATH limit
> 
> ---
>  libavformat/avisynth.c | 12 +++++++-----
>  1 file changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/libavformat/avisynth.c b/libavformat/avisynth.c
> index 8ba2bde..f7bea8c 100644
> --- a/libavformat/avisynth.c
> +++ b/libavformat/avisynth.c
> @@ -34,6 +34,7 @@
>  /* Platform-specific directives. */
>  #ifdef _WIN32
>    #include "compat/w32dlfcn.h"
> +  #include "libavutil/wchar_filename.h"
>    #undef EXTERN_C
>    #define AVISYNTH_LIB "avisynth"
>  #else
> @@ -810,8 +811,7 @@ static int avisynth_open_file(AVFormatContext *s)
>      AVS_Value arg, val;
>      int ret;
>  #ifdef _WIN32
> -    char filename_ansi[MAX_PATH * 4];
> -    wchar_t filename_wc[MAX_PATH * 4];
> +    char *filename_ansi = NULL;
>  #endif
> 
>      if (ret = avisynth_context_create(s))
> @@ -819,10 +819,12 @@ static int avisynth_open_file(AVFormatContext
> *s)
> 
>  #ifdef _WIN32
>      /* Convert UTF-8 to ANSI code page */
> -    MultiByteToWideChar(CP_UTF8, 0, s->url, -1, filename_wc,
> MAX_PATH * 4);
> -    WideCharToMultiByte(CP_THREAD_ACP, 0, filename_wc, -1,
> filename_ansi,
> -                        MAX_PATH * 4, NULL, NULL);
> +    if (utf8toansi(s->url, &filename_ansi)) {
> +        ret = AVERROR_UNKNOWN;
> +        goto fail;
> +    }

Why not use the AviSynth mechanism that allows to supply a UTF-8 string?

https://github.com/AviSynth/AviSynthPlus/blob/c377916aa4146d2f4386852d91dc177d49103c16/avs_core/core/parser/script.cpp#L477-L481

Best,
sw

> Why not use the AviSynth mechanism that allows to supply a UTF-8 string?
>
> https://github.com/AviSynth/AviSynthPlus/blob/c377916aa4146d2f4386852d91dc177d49103c16/avs_core/core/parser/script.cpp#L477-L481

Was not aware such a mechanism exists.

Commit dates back to 10 April 2017, first release supporting it is, apparently, Avisynth+ r2487-MT: https://github.com/pinterf/AviSynthPlus/releases/tag/r2489-MT.

A remark in https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/avisynth.c#L844 says:

/* On Windows, FFmpeg supports AviSynth interface version 6 or higher.
 * This includes AviSynth 2.6 RC1 or higher, and AviSynth+ r1718 or higher,
 * and excludes 2.5 and the 2.6 alphas. */

Support for plain AviSynth will have to be dropped.

On the other hand, configure checks for https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/avisynth.c#L844

> die "ERROR: AviSynth+ header version must be >= 3.7.1"

so probably plain AviSynth and AviSynth+ below r2489-MT are already unsupported.

On 6/11/22 1:01 PM, nil-admirari@mailo.com wrote:
>> Why not use the AviSynth mechanism that allows to supply a UTF-8 string?
>>
>> https://github.com/AviSynth/AviSynthPlus/blob/c377916aa4146d2f4386852d91dc177d49103c16/avs_core/core/parser/script.cpp#L477-L481
> 
> Was not aware such a mechanism exists.
> 
> Commit dates back to 10 April 2017, first release supporting it is, apparently, Avisynth+ r2487-MT: https://github.com/pinterf/AviSynthPlus/releases/tag/r2489-MT.
> 
> A remark in https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/avisynth.c#L844 says:
> 
> /* On Windows, FFmpeg supports AviSynth interface version 6 or higher.
>   * This includes AviSynth 2.6 RC1 or higher, and AviSynth+ r1718 or higher,
>   * and excludes 2.5 and the 2.6 alphas. */
> 
> Support for plain AviSynth will have to be dropped.

Presumably, the original manifest idea, parsed down to only using it to 
force FFmpeg into UTF-8, would be sufficient for this, right?  As long 
as AviSynth inherits that from FFmpeg, UTF-8 strings would be pervasive 
and both A) the utf8 parameter would not need to be used and B) 2.6 
would work just fine with it, transparently.

The Windows API does have a SetConsoleCP function.  If that accomplishes 
the same effect as the manifest idea, that would be simpler, but it 
probably would need to be located somewhere *other* than the AviSynth 
demuxer.  And while it might work for the fftools themselves, does it 
also work for usage of the libraries directly in applications that may 
not be console apps?

Barring that, if/else checks to ensure that
A) IsWindowsVersionOrGreater is at least 1903
A1) If yes, go to B
A2) If no, use the existing logic

B) If yes, GetACP to check that it's UTF8
B1) If yes, the Import call stays the same as it is now, no utf8 parameter
B2) If no, that's where things get complicated:

C1) Use the no result to tell it to then force UTF-8 mode with 
SetConsoleCP, if that actually works for what we need it to do.  Then 
don't use the utf8 parameter.
C2) Use avs_get_version to detect an incompatible version of AviSynth 
and gracefully exit with a message about upgrading to a supported 
version of AviSynth+, before then using the utf8 parameter for real.
C3) Use avs_get_version, but if it's not a new enough version, just fall 
back to the logic that exists now, where 2.6 may or may not work just 
because the system may or may not be already set to UTF-8.

C2 should really be considered a last resort IMO, because it's an 
artificial limit and doesn't actually have anything to do with the 
AviSynth API.

The reason is that the utf8 parameter being discussed here is not part 
of the AviSynth API, it's an option handed to one of the script-level 
functions that avs_invoke (which is the actual API call there) is using.

> On the other hand, configure checks for https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/avisynth.c#L844
> 
>> die "ERROR: AviSynth+ header version must be >= 3.7.1"
> 
> so probably plain AviSynth and AviSynth+ below r2489-MT are already unsupported.
> 

The header version check there isn't because of old versions of 
AviSynth(+) being unsupported (as far as the demuxer is concerned, 
anyway).  3.7.1 is still API compatible with 2.6 in all the functions 
the demuxer uses that are shared between them. The additional 
Plus-specific functionality is enabled with runtime checks, so if you 
don't use the newer header, it will fail to build, but you can run 2.6 
without problems even when using the newer header to compile the demuxer.

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Stephen Hutchinson
> Sent: Sunday, June 12, 2022 4:15 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v13 2/4] libavformat/avisynth.c:
> Remove MAX_PATH limit
> 
> On 6/11/22 1:01 PM, nil-admirari@mailo.com wrote:
> >> Why not use the AviSynth mechanism that allows to supply a UTF-8
> string?
> >>
> >>
> https://github.com/AviSynth/AviSynthPlus/blob/c377916aa4146d2f4386852
> d91dc177d49103c16/avs_core/core/parser/script.cpp#L477-L481
> >
> > Was not aware such a mechanism exists.
> >
> > Commit dates back to 10 April 2017, first release supporting it is,
> apparently, Avisynth+ r2487-MT:
> https://github.com/pinterf/AviSynthPlus/releases/tag/r2489-MT.
> >
> > A remark in
> https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/avisynth.c#L
> 844 says:
> >
> > /* On Windows, FFmpeg supports AviSynth interface version 6 or
> higher.
> >   * This includes AviSynth 2.6 RC1 or higher, and AviSynth+ r1718
> or higher,
> >   * and excludes 2.5 and the 2.6 alphas. */
> >
> > Support for plain AviSynth will have to be dropped.
> 
> Presumably, the original manifest idea, parsed down to only using it
> to
> force FFmpeg into UTF-8, would be sufficient for this, right?  As

This is a change that would affect ffmpeg behavior at a global level,
just for the sake of accommodating for a single 3rd party library 
(and even: only some ancient versions of it).

> as AviSynth inherits that from FFmpeg, UTF-8 strings would be
> pervasive
> and both A) the utf8 parameter would not need to be used and B) 2.6
> would work just fine with it, transparently.
> The Windows API does have a SetConsoleCP function.  If that
> accomplishes
> the same effect as the manifest idea, that would be simpler, but it
> probably would need to be located somewhere *other* than the AviSynth
> demuxer.  

ffmpeg does not interact with AviSynth via console interface.
AFAIU, it uses AviSynth in-process loading it via an API instead:

    val = avs_library.avs_invoke(avs->env, "Import", arg, 0);

Those functions like SetConsoleCP and SetConsoleOutputCP, have no effect
on the current process, it's only about console pipe communication with 
child (cli) processes.
The manifest approach is too invasive IMO, as laid out before.


At least with regards to AviSynthPlus versions since two years ago, 
we're not talking about long paths anymore.
AviSynthPlus is using the same prefixing approach for long paths
that we have employed in ffmpeg as well, now.

The only question is whether we supply the script/path argument to
AviSynthPlus as Ansi or UTF-8 string.
It will handle long paths in both cases. The only difference is that
when we're converting a UTF-8 path to an Ansi codepage, it might
become an invalid path when the projection would be ambiguous. 
It's been like that all the time before - nothing new about it.

There are functions available to check the version:
avs_get_version, avs_check_version,

So - in case that requiring AviSynthPlus from 2020 as a minimum
would be undesirable, it should be possible to find out at
runtime whether the loaded AviSynth supports the UTF8 parameter
or not and set the invoke parameters accordingly.

Best regards,
softworkz

> Presumably, the original manifest idea, parsed down to only using it to 
> force FFmpeg into UTF-8, would be sufficient for this, right?

UTF-8 in manifest was rejected as too far-reaching, and then longs paths
were reimplemented with \\?\ prefixes, so now there is no manifest at all.

> The Windows API does have a SetConsoleCP function.

It does not change process code page.

> C3) Use avs_get_version, but if it's not a new enough version, just fall 
> back to the logic that exists now

Done in https://ffmpeg.org/pipermail/ffmpeg-devel/2022-June/297494.html.