diff mbox series

[FFmpeg-devel,2/6] avfilter/vf_sr: add options for backend configs

Message ID 1619434136-17760-2-git-send-email-lance.lmwang@gmail.com
State New
Headers show
Series [FFmpeg-devel,1/6] doc/filters: correct http link
Related show

Checks

Context Check Description
andriy/x86_make success Make finished
andriy/x86_make_fate success Make fate finished
andriy/PPC64_make success Make finished
andriy/PPC64_make_fate success Make fate finished

Commit Message

Limin Wang April 26, 2021, 10:48 a.m. UTC
From: Limin Wang <lance.lmwang@gmail.com>

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
---
 libavfilter/vf_sr.c | 2 ++
 1 file changed, 2 insertions(+)

Comments

Guo, Yejun April 27, 2021, 2:27 a.m. UTC | #1
> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> lance.lmwang@gmail.com
> Sent: 2021年4月26日 18:49
> To: ffmpeg-devel@ffmpeg.org
> Cc: Limin Wang <lance.lmwang@gmail.com>
> Subject: [FFmpeg-devel] [PATCH 2/6] avfilter/vf_sr: add options for backend
> configs
> 
> From: Limin Wang <lance.lmwang@gmail.com>
> 
> Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> ---
>  libavfilter/vf_sr.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/libavfilter/vf_sr.c b/libavfilter/vf_sr.c
> index 45f941a..282c468 100644
> --- a/libavfilter/vf_sr.c
> +++ b/libavfilter/vf_sr.c
> @@ -55,6 +55,8 @@ static const AVOption sr_options[] = {
>      { "model", "path to model file specifying network architecture and its
> parameters", OFFSET(dnnctx.model_filename), AV_OPT_TYPE_STRING,
> {.str=NULL}, 0, 0, FLAGS },
>      { "input",       "input name of the model",
> OFFSET(dnnctx.model_inputname),  AV_OPT_TYPE_STRING,    { .str =
> "x" },  0, 0, FLAGS },
>      { "output",      "output name of the model",
> OFFSET(dnnctx.model_outputname), AV_OPT_TYPE_STRING,    { .str =
> "y" },  0, 0, FLAGS },
> +    { "backend_configs",    "backend configs",
> OFFSET(dnnctx.backend_options),  AV_OPT_TYPE_STRING,    { .str =
> NULL }, 0, 0, FLAGS },
> +    { "options",            "backend configs",
> OFFSET(dnnctx.backend_options),  AV_OPT_TYPE_STRING,    { .str =
> NULL }, 0, 0, FLAGS },
>      { NULL }
>  };
> 

vf_dnn_processing.c is designed to cover all the cases for image/video
processing with M inputs and N outputs (currently only M=1 and N=1 is
supported), covering the features of vf_sr.c and vf_derain.c. To avoid
duplicate/similar code in different filters, we don't add more feature in 
vf_sr.c/vf_derain.c, we just keep supporting their current feature.
Limin Wang April 27, 2021, 3:29 a.m. UTC | #2
On Tue, Apr 27, 2021 at 02:27:48AM +0000, Guo, Yejun wrote:
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > lance.lmwang@gmail.com
> > Sent: 2021年4月26日 18:49
> > To: ffmpeg-devel@ffmpeg.org
> > Cc: Limin Wang <lance.lmwang@gmail.com>
> > Subject: [FFmpeg-devel] [PATCH 2/6] avfilter/vf_sr: add options for backend
> > configs
> > 
> > From: Limin Wang <lance.lmwang@gmail.com>
> > 
> > Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> > ---
> >  libavfilter/vf_sr.c | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/libavfilter/vf_sr.c b/libavfilter/vf_sr.c
> > index 45f941a..282c468 100644
> > --- a/libavfilter/vf_sr.c
> > +++ b/libavfilter/vf_sr.c
> > @@ -55,6 +55,8 @@ static const AVOption sr_options[] = {
> >      { "model", "path to model file specifying network architecture and its
> > parameters", OFFSET(dnnctx.model_filename), AV_OPT_TYPE_STRING,
> > {.str=NULL}, 0, 0, FLAGS },
> >      { "input",       "input name of the model",
> > OFFSET(dnnctx.model_inputname),  AV_OPT_TYPE_STRING,    { .str =
> > "x" },  0, 0, FLAGS },
> >      { "output",      "output name of the model",
> > OFFSET(dnnctx.model_outputname), AV_OPT_TYPE_STRING,    { .str =
> > "y" },  0, 0, FLAGS },
> > +    { "backend_configs",    "backend configs",
> > OFFSET(dnnctx.backend_options),  AV_OPT_TYPE_STRING,    { .str =
> > NULL }, 0, 0, FLAGS },
> > +    { "options",            "backend configs",
> > OFFSET(dnnctx.backend_options),  AV_OPT_TYPE_STRING,    { .str =
> > NULL }, 0, 0, FLAGS },
> >      { NULL }
> >  };
> > 
> 
> vf_dnn_processing.c is designed to cover all the cases for image/video
> processing with M inputs and N outputs (currently only M=1 and N=1 is
> supported), covering the features of vf_sr.c and vf_derain.c. To avoid
> duplicate/similar code in different filters, we don't add more feature in 
> vf_sr.c/vf_derain.c, we just keep supporting their current feature.

The problem is when trying with SRCNN model with vf_sr, I can't find scale_factor
for dnn_processing, so I had to use the old vf_sr still. That's why I had to
make the options be configured for vf_sr like dnn_processing. Or it's easy to
get "out of memory" with default sess_config for TF. 


> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Guo, Yejun April 27, 2021, 3:56 a.m. UTC | #3
> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> lance.lmwang@gmail.com
> Sent: 2021年4月27日 11:29
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH 2/6] avfilter/vf_sr: add options for
> backend configs
> 
> On Tue, Apr 27, 2021 at 02:27:48AM +0000, Guo, Yejun wrote:
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > > lance.lmwang@gmail.com
> > > Sent: 2021年4月26日 18:49
> > > To: ffmpeg-devel@ffmpeg.org
> > > Cc: Limin Wang <lance.lmwang@gmail.com>
> > > Subject: [FFmpeg-devel] [PATCH 2/6] avfilter/vf_sr: add options for
> backend
> > > configs
> > >
> > > From: Limin Wang <lance.lmwang@gmail.com>
> > >
> > > Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> > > ---
> > >  libavfilter/vf_sr.c | 2 ++
> > >  1 file changed, 2 insertions(+)
> > >
> > > diff --git a/libavfilter/vf_sr.c b/libavfilter/vf_sr.c
> > > index 45f941a..282c468 100644
> > > --- a/libavfilter/vf_sr.c
> > > +++ b/libavfilter/vf_sr.c
> > > @@ -55,6 +55,8 @@ static const AVOption sr_options[] = {
> > >      { "model", "path to model file specifying network architecture
> and its
> > > parameters", OFFSET(dnnctx.model_filename), AV_OPT_TYPE_STRING,
> > > {.str=NULL}, 0, 0, FLAGS },
> > >      { "input",       "input name of the model",
> > > OFFSET(dnnctx.model_inputname),  AV_OPT_TYPE_STRING,    { .str
> =
> > > "x" },  0, 0, FLAGS },
> > >      { "output",      "output name of the model",
> > > OFFSET(dnnctx.model_outputname), AV_OPT_TYPE_STRING,    { .str =
> > > "y" },  0, 0, FLAGS },
> > > +    { "backend_configs",    "backend configs",
> > > OFFSET(dnnctx.backend_options),  AV_OPT_TYPE_STRING,    { .str =
> > > NULL }, 0, 0, FLAGS },
> > > +    { "options",            "backend configs",
> > > OFFSET(dnnctx.backend_options),  AV_OPT_TYPE_STRING,    { .str =
> > > NULL }, 0, 0, FLAGS },
> > >      { NULL }
> > >  };
> > >
> >
> > vf_dnn_processing.c is designed to cover all the cases for image/video
> > processing with M inputs and N outputs (currently only M=1 and N=1 is
> > supported), covering the features of vf_sr.c and vf_derain.c. To avoid
> > duplicate/similar code in different filters, we don't add more feature in
> > vf_sr.c/vf_derain.c, we just keep supporting their current feature.
> 
> The problem is when trying with SRCNN model with vf_sr, I can't find
> scale_factor
> for dnn_processing, so I had to use the old vf_sr still. 

For SRCNN, we can use dnn_processing like:
scale=w=iw*2:h=ih*2,dnn_processing=...
Limin Wang April 27, 2021, 5:47 a.m. UTC | #4
On Tue, Apr 27, 2021 at 03:56:22AM +0000, Guo, Yejun wrote:
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > lance.lmwang@gmail.com
> > Sent: 2021年4月27日 11:29
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH 2/6] avfilter/vf_sr: add options for
> > backend configs
> > 
> > On Tue, Apr 27, 2021 at 02:27:48AM +0000, Guo, Yejun wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > > > lance.lmwang@gmail.com
> > > > Sent: 2021年4月26日 18:49
> > > > To: ffmpeg-devel@ffmpeg.org
> > > > Cc: Limin Wang <lance.lmwang@gmail.com>
> > > > Subject: [FFmpeg-devel] [PATCH 2/6] avfilter/vf_sr: add options for
> > backend
> > > > configs
> > > >
> > > > From: Limin Wang <lance.lmwang@gmail.com>
> > > >
> > > > Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> > > > ---
> > > >  libavfilter/vf_sr.c | 2 ++
> > > >  1 file changed, 2 insertions(+)
> > > >
> > > > diff --git a/libavfilter/vf_sr.c b/libavfilter/vf_sr.c
> > > > index 45f941a..282c468 100644
> > > > --- a/libavfilter/vf_sr.c
> > > > +++ b/libavfilter/vf_sr.c
> > > > @@ -55,6 +55,8 @@ static const AVOption sr_options[] = {
> > > >      { "model", "path to model file specifying network architecture
> > and its
> > > > parameters", OFFSET(dnnctx.model_filename), AV_OPT_TYPE_STRING,
> > > > {.str=NULL}, 0, 0, FLAGS },
> > > >      { "input",       "input name of the model",
> > > > OFFSET(dnnctx.model_inputname),  AV_OPT_TYPE_STRING,    { .str
> > =
> > > > "x" },  0, 0, FLAGS },
> > > >      { "output",      "output name of the model",
> > > > OFFSET(dnnctx.model_outputname), AV_OPT_TYPE_STRING,    { .str =
> > > > "y" },  0, 0, FLAGS },
> > > > +    { "backend_configs",    "backend configs",
> > > > OFFSET(dnnctx.backend_options),  AV_OPT_TYPE_STRING,    { .str =
> > > > NULL }, 0, 0, FLAGS },
> > > > +    { "options",            "backend configs",
> > > > OFFSET(dnnctx.backend_options),  AV_OPT_TYPE_STRING,    { .str =
> > > > NULL }, 0, 0, FLAGS },
> > > >      { NULL }
> > > >  };
> > > >
> > >
> > > vf_dnn_processing.c is designed to cover all the cases for image/video
> > > processing with M inputs and N outputs (currently only M=1 and N=1 is
> > > supported), covering the features of vf_sr.c and vf_derain.c. To avoid
> > > duplicate/similar code in different filters, we don't add more feature in
> > > vf_sr.c/vf_derain.c, we just keep supporting their current feature.
> > 
> > The problem is when trying with SRCNN model with vf_sr, I can't find
> > scale_factor
> > for dnn_processing, so I had to use the old vf_sr still. 
> 
> For SRCNN, we can use dnn_processing like:
> scale=w=iw*2:h=ih*2,dnn_processing=...

thanks, I'm glad to hear about that, will try with this way.

> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff mbox series

Patch

diff --git a/libavfilter/vf_sr.c b/libavfilter/vf_sr.c
index 45f941a..282c468 100644
--- a/libavfilter/vf_sr.c
+++ b/libavfilter/vf_sr.c
@@ -55,6 +55,8 @@  static const AVOption sr_options[] = {
     { "model", "path to model file specifying network architecture and its parameters", OFFSET(dnnctx.model_filename), AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, FLAGS },
     { "input",       "input name of the model",     OFFSET(dnnctx.model_inputname),  AV_OPT_TYPE_STRING,    { .str = "x" },  0, 0, FLAGS },
     { "output",      "output name of the model",    OFFSET(dnnctx.model_outputname), AV_OPT_TYPE_STRING,    { .str = "y" },  0, 0, FLAGS },
+    { "backend_configs",    "backend configs",      OFFSET(dnnctx.backend_options),  AV_OPT_TYPE_STRING,    { .str = NULL }, 0, 0, FLAGS },
+    { "options",            "backend configs",      OFFSET(dnnctx.backend_options),  AV_OPT_TYPE_STRING,    { .str = NULL }, 0, 0, FLAGS },
     { NULL }
 };