diff mbox series

[FFmpeg-devel,v3,2/2] dnn_backend_native_layer_conv2d.c: refine code.

Message ID 20200916100717.3142217-2-xujunzz@sjtu.edu.cn
State Accepted
Commit 7d3cd9f9566ef5fb0c2222f64be90473152c68dc
Headers show
Series [FFmpeg-devel,v3,1/2] dnn_backend_native_layer_conv2d.c: fix memory allocation bug in multithread function.
Related show

Checks

Context Check Description
andriy/default pending
andriy/make success Make finished
andriy/make_fate success Make fate finished

Commit Message

Xu Jun Sept. 16, 2020, 10:07 a.m. UTC
From: Xu Jun <xujunzz@sjtu.edu.cn>

Move thread area allocate out of thread function into
main thread.

Signed-off-by: Xu Jun <xujunzz@sjtu.edu.cn>
---
 .../dnn/dnn_backend_native_layer_conv2d.c     | 30 +++++++++----------
 1 file changed, 14 insertions(+), 16 deletions(-)

Comments

Guo, Yejun Sept. 17, 2020, 2:07 a.m. UTC | #1
> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> xujunzz@sjtu.edu.cn
> Sent: 2020年9月16日 18:07
> To: ffmpeg-devel@ffmpeg.org
> Cc: xujunzz@sjtu.edu.cn
> Subject: [FFmpeg-devel] [PATCH v3 2/2] dnn_backend_native_layer_conv2d.c:
> refine code.
> 
> From: Xu Jun <xujunzz@sjtu.edu.cn>
> 
> Move thread area allocate out of thread function into main thread.
> 
> Signed-off-by: Xu Jun <xujunzz@sjtu.edu.cn>
> ---
>  .../dnn/dnn_backend_native_layer_conv2d.c     | 30 +++++++++----------
>  1 file changed, 14 insertions(+), 16 deletions(-)

LGTM, will push soon, thanks.
Hendrik Leppkes Sept. 17, 2020, 11:21 a.m. UTC | #2
On Thu, Sep 17, 2020 at 4:07 AM Guo, Yejun <yejun.guo@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > xujunzz@sjtu.edu.cn
> > Sent: 2020年9月16日 18:07
> > To: ffmpeg-devel@ffmpeg.org
> > Cc: xujunzz@sjtu.edu.cn
> > Subject: [FFmpeg-devel] [PATCH v3 2/2] dnn_backend_native_layer_conv2d.c:
> > refine code.
> >
> > From: Xu Jun <xujunzz@sjtu.edu.cn>
> >
> > Move thread area allocate out of thread function into main thread.
> >
> > Signed-off-by: Xu Jun <xujunzz@sjtu.edu.cn>
> > ---
> >  .../dnn/dnn_backend_native_layer_conv2d.c     | 30 +++++++++----------
> >  1 file changed, 14 insertions(+), 16 deletions(-)
>
> LGTM, will push soon, thanks.

The dnn-layer-conv2d test fails after the recent changes on some
systems, namely:

Windows mingw 64-bit (eg.
http://fate.ffmpeg.org/report.cgi?time=20200917051623&slot=x86_64-mingw-w64-windows-native)
MSVC 32-bit static build (eg.
http://fate.ffmpeg.org/report.cgi?time=20200917102627&slot=x86_32-msvc16-windows-native)
and maybe more windows combination, some of my boxes are still running FATE.

There is no output, just a wrong return value apparently. Any ideas/fixes?

- Hendrik
Guo, Yejun Sept. 17, 2020, noon UTC | #3
> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Hendrik
> Leppkes
> Sent: 2020年9月17日 19:21
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v3 2/2]
> dnn_backend_native_layer_conv2d.c: refine code.
> 
> On Thu, Sep 17, 2020 at 4:07 AM Guo, Yejun <yejun.guo@intel.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > > xujunzz@sjtu.edu.cn
> > > Sent: 2020年9月16日 18:07
> > > To: ffmpeg-devel@ffmpeg.org
> > > Cc: xujunzz@sjtu.edu.cn
> > > Subject: [FFmpeg-devel] [PATCH v3 2/2]
> dnn_backend_native_layer_conv2d.c:
> > > refine code.
> > >
> > > From: Xu Jun <xujunzz@sjtu.edu.cn>
> > >
> > > Move thread area allocate out of thread function into main thread.
> > >
> > > Signed-off-by: Xu Jun <xujunzz@sjtu.edu.cn>
> > > ---
> > >  .../dnn/dnn_backend_native_layer_conv2d.c     | 30 +++++++++----------
> > >  1 file changed, 14 insertions(+), 16 deletions(-)
> >
> > LGTM, will push soon, thanks.
> 
> The dnn-layer-conv2d test fails after the recent changes on some systems,
> namely:
> 
> Windows mingw 64-bit (eg.
> http://fate.ffmpeg.org/report.cgi?time=20200917051623&slot=x86_64-mingw-
> w64-windows-native)
> MSVC 32-bit static build (eg.
> http://fate.ffmpeg.org/report.cgi?time=20200917102627&slot=x86_32-msvc16
> -windows-native)
> and maybe more windows combination, some of my boxes are still running FATE.
> 
> There is no output, just a wrong return value apparently. Any ideas/fixes?
> 
thanks for pointing out.

I think the output is in file dnn-layer-conv2d.err
"Test dnn-layer-conv2d failed. Look at tests/data/fate/dnn-layer-conv2d.err for details."

Does anyone have such system and can lend me for a while? I can quickly have a debug, thanks.
Hendrik Leppkes Sept. 17, 2020, 12:27 p.m. UTC | #4
On Thu, Sep 17, 2020 at 2:02 PM Guo, Yejun <yejun.guo@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Hendrik
> > Leppkes
> > Sent: 2020年9月17日 19:21
> > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH v3 2/2]
> > dnn_backend_native_layer_conv2d.c: refine code.
> >
> > On Thu, Sep 17, 2020 at 4:07 AM Guo, Yejun <yejun.guo@intel.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > > > xujunzz@sjtu.edu.cn
> > > > Sent: 2020年9月16日 18:07
> > > > To: ffmpeg-devel@ffmpeg.org
> > > > Cc: xujunzz@sjtu.edu.cn
> > > > Subject: [FFmpeg-devel] [PATCH v3 2/2]
> > dnn_backend_native_layer_conv2d.c:
> > > > refine code.
> > > >
> > > > From: Xu Jun <xujunzz@sjtu.edu.cn>
> > > >
> > > > Move thread area allocate out of thread function into main thread.
> > > >
> > > > Signed-off-by: Xu Jun <xujunzz@sjtu.edu.cn>
> > > > ---
> > > >  .../dnn/dnn_backend_native_layer_conv2d.c     | 30 +++++++++----------
> > > >  1 file changed, 14 insertions(+), 16 deletions(-)
> > >
> > > LGTM, will push soon, thanks.
> >
> > The dnn-layer-conv2d test fails after the recent changes on some systems,
> > namely:
> >
> > Windows mingw 64-bit (eg.
> > http://fate.ffmpeg.org/report.cgi?time=20200917051623&slot=x86_64-mingw-
> > w64-windows-native)
> > MSVC 32-bit static build (eg.
> > http://fate.ffmpeg.org/report.cgi?time=20200917102627&slot=x86_32-msvc16
> > -windows-native)
> > and maybe more windows combination, some of my boxes are still running FATE.
> >
> > There is no output, just a wrong return value apparently. Any ideas/fixes?
> >
> thanks for pointing out.
>
> I think the output is in file dnn-layer-conv2d.err
> "Test dnn-layer-conv2d failed. Look at tests/data/fate/dnn-layer-conv2d.err for details."
>

There is no output, all these files are empty.

- Hendrik
Guo, Yejun Sept. 17, 2020, 1:30 p.m. UTC | #5
> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Hendrik
> Leppkes
> Sent: 2020年9月17日 20:28
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v3 2/2]
> dnn_backend_native_layer_conv2d.c: refine code.
> 
> On Thu, Sep 17, 2020 at 2:02 PM Guo, Yejun <yejun.guo@intel.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > > Hendrik Leppkes
> > > Sent: 2020年9月17日 19:21
> > > To: FFmpeg development discussions and patches
> > > <ffmpeg-devel@ffmpeg.org>
> > > Subject: Re: [FFmpeg-devel] [PATCH v3 2/2]
> > > dnn_backend_native_layer_conv2d.c: refine code.
> > >
> > > On Thu, Sep 17, 2020 at 4:07 AM Guo, Yejun <yejun.guo@intel.com> wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf
> > > > > Of xujunzz@sjtu.edu.cn
> > > > > Sent: 2020年9月16日 18:07
> > > > > To: ffmpeg-devel@ffmpeg.org
> > > > > Cc: xujunzz@sjtu.edu.cn
> > > > > Subject: [FFmpeg-devel] [PATCH v3 2/2]
> > > dnn_backend_native_layer_conv2d.c:
> > > > > refine code.
> > > > >
> > > > > From: Xu Jun <xujunzz@sjtu.edu.cn>
> > > > >
> > > > > Move thread area allocate out of thread function into main thread.
> > > > >
> > > > > Signed-off-by: Xu Jun <xujunzz@sjtu.edu.cn>
> > > > > ---
> > > > >  .../dnn/dnn_backend_native_layer_conv2d.c     | 30
> +++++++++----------
> > > > >  1 file changed, 14 insertions(+), 16 deletions(-)
> > > >
> > > > LGTM, will push soon, thanks.
> > >
> > > The dnn-layer-conv2d test fails after the recent changes on some
> > > systems,
> > > namely:
> > >
> > > Windows mingw 64-bit (eg.
> > > http://fate.ffmpeg.org/report.cgi?time=20200917051623&slot=x86_64-mi
> > > ngw-
> > > w64-windows-native)
> > > MSVC 32-bit static build (eg.
> > > http://fate.ffmpeg.org/report.cgi?time=20200917102627&slot=x86_32-ms
> > > vc16
> > > -windows-native)
> > > and maybe more windows combination, some of my boxes are still running
> FATE.
> > >
> > > There is no output, just a wrong return value apparently. Any ideas/fixes?
> > >
> > thanks for pointing out.
> >
> > I think the output is in file dnn-layer-conv2d.err "Test
> > dnn-layer-conv2d failed. Look at tests/data/fate/dnn-layer-conv2d.err for
> details."
> >
> 
> There is no output, all these files are empty.
> 
I see, and we are setting up the system locally to reproduce the issue.
Xu Jun Sept. 18, 2020, 6:50 p.m. UTC | #6
----- Original Message -----
> From: "Yejun Guo" <yejun.guo@intel.com>
> To: "FFmpeg development discussions and patches" <ffmpeg-devel@ffmpeg.org>
> Cc: "xujunzz" <xujunzz@sjtu.edu.cn>
> Sent: Thursday, September 17, 2020 9:30:08 PM
> Subject: RE: [FFmpeg-devel] [PATCH v3 2/2] dnn_backend_native_layer_conv2d.c: refine code.

>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Hendrik
>> Leppkes
>> Sent: 2020年9月17日 20:28
>> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] [PATCH v3 2/2]
>> dnn_backend_native_layer_conv2d.c: refine code.
>> 
>> On Thu, Sep 17, 2020 at 2:02 PM Guo, Yejun <yejun.guo@intel.com> wrote:
>> >
>> >
>> >
>> > > -----Original Message-----
>> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
>> > > Hendrik Leppkes
>> > > Sent: 2020年9月17日 19:21
>> > > To: FFmpeg development discussions and patches
>> > > <ffmpeg-devel@ffmpeg.org>
>> > > Subject: Re: [FFmpeg-devel] [PATCH v3 2/2]
>> > > dnn_backend_native_layer_conv2d.c: refine code.
>> > >
>> > > On Thu, Sep 17, 2020 at 4:07 AM Guo, Yejun <yejun.guo@intel.com> wrote:
>> > > >
>> > > >
>> > > >
>> > > > > -----Original Message-----
>> > > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf
>> > > > > Of xujunzz@sjtu.edu.cn
>> > > > > Sent: 2020年9月16日 18:07
>> > > > > To: ffmpeg-devel@ffmpeg.org
>> > > > > Cc: xujunzz@sjtu.edu.cn
>> > > > > Subject: [FFmpeg-devel] [PATCH v3 2/2]
>> > > dnn_backend_native_layer_conv2d.c:
>> > > > > refine code.
>> > > > >
>> > > > > From: Xu Jun <xujunzz@sjtu.edu.cn>
>> > > > >
>> > > > > Move thread area allocate out of thread function into main thread.
>> > > > >
>> > > > > Signed-off-by: Xu Jun <xujunzz@sjtu.edu.cn>
>> > > > > ---
>> > > > >  .../dnn/dnn_backend_native_layer_conv2d.c     | 30
>> +++++++++----------
>> > > > >  1 file changed, 14 insertions(+), 16 deletions(-)
>> > > >
>> > > > LGTM, will push soon, thanks.
>> > >
>> > > The dnn-layer-conv2d test fails after the recent changes on some
>> > > systems,
>> > > namely:
>> > >
>> > > Windows mingw 64-bit (eg.
>> > > http://fate.ffmpeg.org/report.cgi?time=20200917051623&slot=x86_64-mi
>> > > ngw-
>> > > w64-windows-native)
>> > > MSVC 32-bit static build (eg.
>> > > http://fate.ffmpeg.org/report.cgi?time=20200917102627&slot=x86_32-ms
>> > > vc16
>> > > -windows-native)
>> > > and maybe more windows combination, some of my boxes are still running
>> FATE.
>> > >
>> > > There is no output, just a wrong return value apparently. Any ideas/fixes?
>> > >
>> > thanks for pointing out.
>> >
>> > I think the output is in file dnn-layer-conv2d.err "Test
>> > dnn-layer-conv2d failed. Look at tests/data/fate/dnn-layer-conv2d.err for
>> details."
>> >
>> 
>> There is no output, all these files are empty.
>> 
> I see, and we are setting up the system locally to reproduce the issue.

I have build my mingw win10 environment and have reproduce the issue. 
I'm going to look for the bug and try to fix it.

- Xu Jun
diff mbox series

Patch

diff --git a/libavfilter/dnn/dnn_backend_native_layer_conv2d.c b/libavfilter/dnn/dnn_backend_native_layer_conv2d.c
index 5c313454f7..2aaa4162df 100644
--- a/libavfilter/dnn/dnn_backend_native_layer_conv2d.c
+++ b/libavfilter/dnn/dnn_backend_native_layer_conv2d.c
@@ -33,12 +33,11 @@  typedef struct thread_common_param{
     const void *parameters;
     NativeContext *ctx;
     float *output_data;
-    int thread_num;
 } thread_common_param;
 
 typedef struct thread_param{
     thread_common_param *thread_common_param;
-    int thread_index;
+    int thread_start, thread_end;
 } thread_param;
 
 int dnn_load_layer_conv2d(Layer *layer, AVIOContext *model_file_context, int file_size, int operands_num)
@@ -125,16 +124,12 @@  static void * dnn_execute_layer_conv2d_thread(void *threadarg)
     int filter_size = conv_params->kernel_size * filter_linesize;
     int pad_size = (conv_params->padding_method == VALID) ? (conv_params->kernel_size - 1) / 2 * conv_params->dilation : 0;
 
-    int thread_stride = (height - pad_size * 2) / thread_common_param->thread_num;
-    int thread_start = thread_stride * thread_param->thread_index + pad_size;
-    int thread_end = (thread_param->thread_index == thread_common_param->thread_num - 1) ? (height - pad_size) : (thread_start + thread_stride);
-
     float *output = thread_common_param->output_data;
-    output += (conv_params->output_num) * (width - 2 * pad_size) * (thread_start - pad_size);
+    output += (conv_params->output_num) * (width - 2 * pad_size) * (thread_param->thread_start - pad_size);
 
     av_assert0(channel == conv_params->input_num);
 
-    for (int y = thread_start; y < thread_end; ++y) {
+    for (int y = thread_param->thread_start; y < thread_param->thread_end; ++y) {
         for (int x = pad_size; x < width - pad_size; ++x) {
             for (int n_filter = 0; n_filter < conv_params->output_num; ++n_filter) {
                 if (conv_params->has_bias)
@@ -193,16 +188,19 @@  int dnn_execute_layer_conv2d(DnnOperand *operands, const int32_t *input_operand_
         ? (av_cpu_count() + 1) : (ctx->options.conv2d_threads);
 #if HAVE_PTHREAD_CANCEL
     pthread_t *thread_id = av_malloc(thread_num * sizeof(pthread_t));
+    int thread_stride;
 #endif
     thread_param **thread_param = av_malloc(thread_num * sizeof(*thread_param));
     thread_common_param thread_common_param;
     const ConvolutionalParams *conv_params = (const ConvolutionalParams *)(parameters);
+    int height = operands[input_operand_indexes[0]].dims[1];
+    int width = operands[input_operand_indexes[0]].dims[2];
     int pad_size = (conv_params->padding_method == VALID) ? (conv_params->kernel_size - 1) / 2 * conv_params->dilation : 0;
     DnnOperand *output_operand = &operands[output_operand_index];
 
     output_operand->dims[0] = operands[input_operand_indexes[0]].dims[0];
-    output_operand->dims[1] = operands[input_operand_indexes[0]].dims[1] - pad_size * 2;
-    output_operand->dims[2] = operands[input_operand_indexes[0]].dims[2] - pad_size * 2;
+    output_operand->dims[1] = height - pad_size * 2;
+    output_operand->dims[2] = width - pad_size * 2;
     output_operand->dims[3] = conv_params->output_num;
     output_operand->data_type = operands[input_operand_indexes[0]].data_type;
     output_operand->length = calculate_operand_data_length(output_operand);
@@ -223,13 +221,13 @@  int dnn_execute_layer_conv2d(DnnOperand *operands, const int32_t *input_operand_
     thread_common_param.ctx = ctx;
 
 #if HAVE_PTHREAD_CANCEL
-    thread_common_param.thread_num = thread_num;
-
+    thread_stride = (height - pad_size * 2) / thread_num;
     //create threads
     for (int i = 0; i < thread_num; i++){
         thread_param[i] = av_malloc(sizeof(**thread_param));
         thread_param[i]->thread_common_param = &thread_common_param;
-        thread_param[i]->thread_index = i;
+        thread_param[i]->thread_start = thread_stride * i + pad_size;
+        thread_param[i]->thread_end = (i == thread_num - 1) ? (height - pad_size) : (thread_param[i]->thread_start + thread_stride);
         pthread_create(&thread_id[i], NULL, dnn_execute_layer_conv2d_thread, (void *)thread_param[i]);
     }
 
@@ -245,10 +243,10 @@  int dnn_execute_layer_conv2d(DnnOperand *operands, const int32_t *input_operand_
         av_free(thread_param[i]);
     }
 #else
-    thread_common_param.thread_num = 1;
-    thread_param[0] = av_malloc(sizeof(thread_param));
+    thread_param[0] = av_malloc(sizeof(**thread_param));
     thread_param[0]->thread_common_param = &thread_common_param;
-    thread_param[0]->thread_index = 0;
+    thread_param[0]->thread_start = 0;
+    thread_param[0]->thread_end = height - pad_size;
     dnn_execute_layer_conv2d_thread((void *)thread_param[0]);
     av_free(thread_param[0]);
 #endif