mbox series

[FFmpeg-devel,v2,0/7] HEVC native support for Screen content coding

Message ID 20200910064245.1996343-1-haihao.xiang@intel.com
Headers show
Series HEVC native support for Screen content coding
Related show

Message

Xiang, Haihao Sept. 10, 2020, 6:42 a.m. UTC
Resend Linjie's patchset with the updates.

As a part of the support for VA-API HEVC SCC decoding (gen12, Tiger
Lake+).

The full support could be accessed in:
https://github.com/intel-media-ci/ffmpeg/pull/231

The VAAPI part would be provided later once the implementations of
native parsing and reference management are all decent.

Linjie Fu (7):
  lavc/avcodec: Add FF_PROFILE_HEVC_SCC for screen content coding
  lavc/hevc_ps: Add sps parse support for HEVC SCC extension syntax
  lavc/hevc_ps: Add pps parse support for HEVC SCC extension
  lavc/hevc_ps: Add slice parse support for HEVC SCC extension
  lavc/hevcdec: Fix the parsing for use_integer_mv_flag
  lavc/hevcdec: Set max_num_merge_cand to uint8_t
  lavc/hevc: Update reference list for SCC

 libavcodec/avcodec.h   |   1 +
 libavcodec/hevc.h      |   3 ++
 libavcodec/hevc_ps.c   | 118 +++++++++++++++++++++++++++++++++++++++--
 libavcodec/hevc_ps.h   |  32 +++++++++++
 libavcodec/hevc_refs.c |  27 +++++++++-
 libavcodec/hevcdec.c   |  17 +++++-
 libavcodec/hevcdec.h   |   7 ++-
 libavcodec/profiles.c  |   1 +
 8 files changed, 197 insertions(+), 9 deletions(-)

Comments

Xiang, Haihao Sept. 11, 2020, 1:54 a.m. UTC | #1
Hi James,

Your concerns about SCC PPS were addressed in the new version. 
http://ffmpeg.org/pipermail/ffmpeg-devel/2020-July/266037.html
http://ffmpeg.org/pipermail/ffmpeg-devel/2020-July/266038.html

Could you please take a look when you have time?

Thanks
Haihao


> Resend Linjie's patchset with the updates.
> 
> As a part of the support for VA-API HEVC SCC decoding (gen12, Tiger
> Lake+).
> 
> The full support could be accessed in:
> https://github.com/intel-media-ci/ffmpeg/pull/231
> 
> The VAAPI part would be provided later once the implementations of
> native parsing and reference management are all decent.
> 
> Linjie Fu (7):
>   lavc/avcodec: Add FF_PROFILE_HEVC_SCC for screen content coding
>   lavc/hevc_ps: Add sps parse support for HEVC SCC extension syntax
>   lavc/hevc_ps: Add pps parse support for HEVC SCC extension
>   lavc/hevc_ps: Add slice parse support for HEVC SCC extension
>   lavc/hevcdec: Fix the parsing for use_integer_mv_flag
>   lavc/hevcdec: Set max_num_merge_cand to uint8_t
>   lavc/hevc: Update reference list for SCC
> 
>  libavcodec/avcodec.h   |   1 +
>  libavcodec/hevc.h      |   3 ++
>  libavcodec/hevc_ps.c   | 118 +++++++++++++++++++++++++++++++++++++++--
>  libavcodec/hevc_ps.h   |  32 +++++++++++
>  libavcodec/hevc_refs.c |  27 +++++++++-
>  libavcodec/hevcdec.c   |  17 +++++-
>  libavcodec/hevcdec.h   |   7 ++-
>  libavcodec/profiles.c  |   1 +
>  8 files changed, 197 insertions(+), 9 deletions(-)
>
Mark Thompson Sept. 29, 2020, 1:56 p.m. UTC | #2
On 10/09/2020 07:42, Haihao Xiang wrote:
> Resend Linjie's patchset with the updates.
> 
> As a part of the support for VA-API HEVC SCC decoding (gen12, Tiger
> Lake+).
> 
> The full support could be accessed in:
> https://github.com/intel-media-ci/ffmpeg/pull/231
> 
> The VAAPI part would be provided later once the implementations of
> native parsing and reference management are all decent.
> 
> Linjie Fu (7):
>    lavc/avcodec: Add FF_PROFILE_HEVC_SCC for screen content coding
>    lavc/hevc_ps: Add sps parse support for HEVC SCC extension syntax
>    lavc/hevc_ps: Add pps parse support for HEVC SCC extension
>    lavc/hevc_ps: Add slice parse support for HEVC SCC extension
>    lavc/hevcdec: Fix the parsing for use_integer_mv_flag
>    lavc/hevcdec: Set max_num_merge_cand to uint8_t
>    lavc/hevc: Update reference list for SCC
> 
>   libavcodec/avcodec.h   |   1 +
>   libavcodec/hevc.h      |   3 ++
>   libavcodec/hevc_ps.c   | 118 +++++++++++++++++++++++++++++++++++++++--
>   libavcodec/hevc_ps.h   |  32 +++++++++++
>   libavcodec/hevc_refs.c |  27 +++++++++-
>   libavcodec/hevcdec.c   |  17 +++++-
>   libavcodec/hevcdec.h   |   7 ++-
>   libavcodec/profiles.c  |   1 +
>   8 files changed, 197 insertions(+), 9 deletions(-)

This looks generally pretty good, but the lack of a software implementation is kindof unfortunate - is there any plan to do anything about that?

If not, I think you need to make sure that all of the newly-added flags give a suitable error message to say that it isn't supported.  (Especially in cases like use_integer_mv_flag where it can return incorrect output while saying that everything is fine.)


It seems to always hang with all threads waiting for each other when given a non-software-decodable SCC stream with threads enabled (it hangs forever entirely repeatably before the SIGINT):

$ ./ffmpeg_g -threads 2 -i PPI_A_InterDigital_2.bit -f null -
...
     Stream #0:0: Video: hevc (Scc), yuv444p(tv), 1280x720, 25 fps, 25 tbr, 1200k tbn, 25 tbc
[hevc @ 0x555557b3a680] high_precision_offsets_enabled_flag not yet implemented
[hevc @ 0x555557b3a680] Overread PPS by 8 bits
[New Thread 0x7ffff344f700 (LWP 1954956)]
[New Thread 0x7ffff2c4e700 (LWP 1954957)]
Stream mapping:

   Stream #0:0 -> #0:0 (hevc (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[hevc @ 0x555557b3a680] high_precision_offsets_enabled_flag not yet implemented
[hevc @ 0x555557b3a680] Overread PPS by 8 bits
[hevc @ 0x555557b3a680] PPS id out of range: 0
[hevc @ 0x555557b3a680] Error parsing NAL unit #3.
Error while decoding stream #0:0: Invalid data found when processing input

Thread 1 "ffmpeg_g" received signal SIGINT, Interrupt.
futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cd90) at ../sysdeps/nptl/futex-internal.h:183
183     ../sysdeps/nptl/futex-internal.h: No such file or directory.
(gdb) i thr
   Id   Target Id                                      Frame
* 1    Thread 0x7ffff34561c0 (LWP 1954952) "ffmpeg_g" futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cd90) at ../sysdeps/nptl/futex-internal.h:183
   2    Thread 0x7ffff344f700 (LWP 1954956) "ffmpeg_g" futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cb80) at ../sysdeps/nptl/futex-internal.h:183
   3    Thread 0x7ffff2c4e700 (LWP 1954957) "ffmpeg_g" futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cd64) at ../sysdeps/nptl/futex-internal.h:183
(gdb) thr ap al bt

Thread 3 (Thread 0x7ffff2c4e700 (LWP 1954957)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cd64) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x555557b4cdc0, cond=0x555557b4cd38) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x555557b4cd38, mutex=0x555557b4cdc0) at pthread_cond_wait.c:638
#3  0x00005555560eaed8 in strict_pthread_cond_wait (cond=0x555557b4cd38, mutex=0x555557b4cdc0) at src/libavutil/thread.h:112
#4  0x00005555560ec49c in ff_thread_await_progress (f=0x555557b6b0a8, n=41, field=0) at src/libavcodec/pthread_frame.c:601
#5  0x0000555555e8e73b in hevc_await_progress (s=0x555557b6a140, ref=0x555557b6b0a0, mv=0x7ffff2c4da00, y0=0, height=32) at src/libavcodec/hevcdec.c:1796
#6  0x0000555555e8eeb3 in hls_prediction_unit (s=0x555557b6a140, x0=32, y0=0, nPbW=32, nPbH=32, log2_cb_size=5, partIdx=0, idx=3) at src/libavcodec/hevcdec.c:1900
#7  0x0000555555e902f5 in hls_coding_unit (s=0x555557b6a140, x0=32, y0=0, log2_cb_size=5) at src/libavcodec/hevcdec.c:2206
#8  0x0000555555e9114c in hls_coding_quadtree (s=0x555557b6a140, x0=32, y0=0, log2_cb_size=5, cb_depth=1) at src/libavcodec/hevcdec.c:2388
#9  0x0000555555e90fce in hls_coding_quadtree (s=0x555557b6a140, x0=0, y0=0, log2_cb_size=6, cb_depth=0) at src/libavcodec/hevcdec.c:2362
#10 0x0000555555e91bc4 in hls_decode_entry (avctxt=0x555557b77580, isFilterThread=0x7ffff2c4dce8) at src/libavcodec/hevcdec.c:2498
#11 0x00005555562059d3 in avcodec_default_execute (c=0x555557b77580, func=0x555555e9188d <hls_decode_entry>, arg=0x7ffff2c4dce8, ret=0x7ffff2c4dce0, count=1, size=4) at src/libavcodec/utils.c:448
#12 0x0000555555e91ce2 in hls_slice_data (s=0x555557b6a140) at src/libavcodec/hevcdec.c:2525
#13 0x0000555555e93f5b in decode_nal_unit (s=0x555557b6a140, nal=0x7fffe4006470) at src/libavcodec/hevcdec.c:3098
#14 0x0000555555e9428d in decode_nal_units (s=0x555557b6a140, buf=0x555557b85590 "", length=12760) at src/libavcodec/hevcdec.c:3171
#15 0x0000555555e948a0 in hevc_decode_frame (avctx=0x555557b77580, data=0x555557b4a9c0, got_output=0x555557b4ce50, avpkt=0x555557b4cdf0) at src/libavcodec/hevcdec.c:3314
#16 0x00005555560eb205 in frame_worker_thread (arg=0x555557b4ccf0) at src/libavcodec/pthread_frame.c:201
#17 0x00007ffff6d25ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
#18 0x00007ffff4ad7eaf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 2 (Thread 0x7ffff344f700 (LWP 1954956)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cb80) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x555557b4cbe8, cond=0x555557b4cb58) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x555557b4cb58, mutex=0x555557b4cbe8) at pthread_cond_wait.c:638
#3  0x00005555560eaed8 in strict_pthread_cond_wait (cond=0x555557b4cb58, mutex=0x555557b4cbe8) at src/libavutil/thread.h:112
#4  0x00005555560eb0bf in frame_worker_thread (arg=0x555557b4cb40) at src/libavcodec/pthread_frame.c:177
#5  0x00007ffff6d25ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
#6  0x00007ffff4ad7eaf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7ffff34561c0 (LWP 1954952)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cd90) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x555557b4cdc0, cond=0x555557b4cd68) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x555557b4cd68, mutex=0x555557b4cdc0) at pthread_cond_wait.c:638
#3  0x00005555560eaed8 in strict_pthread_cond_wait (cond=0x555557b4cd68, mutex=0x555557b4cdc0) at src/libavutil/thread.h:112
#4  0x00005555560ec0ef in ff_thread_decode_frame (avctx=0x555557b39840, picture=0x555557b39d80, got_picture_ptr=0x7fffffffd5e4, avpkt=0x555557b39600) at src/libavcodec/pthread_frame.c:526
#5  0x0000555555d884f6 in decode_simple_internal (avctx=0x555557b39840, frame=0x555557b39d80) at src/libavcodec/decode.c:350
#6  0x0000555555d89132 in decode_simple_receive_frame (avctx=0x555557b39840, frame=0x555557b39d80) at src/libavcodec/decode.c:549
#7  0x0000555555d89218 in decode_receive_frame_internal (avctx=0x555557b39840, frame=0x555557b39d80) at src/libavcodec/decode.c:569
#8  0x0000555555d89475 in avcodec_send_packet (avctx=0x555557b39840, avpkt=0x7fffffffd7f0) at src/libavcodec/decode.c:627
#9  0x0000555555695517 in decode (avctx=0x555557b39840, frame=0x555557b61d80, got_frame=0x7fffffffd964, pkt=0x7fffffffd7f0) at src/fftools/ffmpeg.c:2218
#10 0x0000555555695d86 in decode_video (ist=0x555557b34f40, pkt=0x7fffffffd970, got_output=0x7fffffffd964, duration_pts=0x7fffffffd968, eof=0, decode_failed=0x7fffffffd960) at src/fftools/ffmpeg.c:2360
#11 0x0000555555696d79 in process_input_packet (ist=0x555557b34f40, pkt=0x7fffffffdc40, no_eof=0) at src/fftools/ffmpeg.c:2601
#12 0x000055555569e06f in process_input (file_index=0) at src/fftools/ffmpeg.c:4494
#13 0x000055555569e594 in transcode_step () at src/fftools/ffmpeg.c:4614
#14 0x000055555569e6be in transcode () at src/fftools/ffmpeg.c:4668
#15 0x000055555569ef4b in main (argc=8, argv=0x7fffffffe3b8) at src/fftools/ffmpeg.c:4873
(gdb)

That PPS overread looks like something which shouldn't happen, too.

- Mark
Linjie Fu Sept. 29, 2020, 2:55 p.m. UTC | #3
On Tue, Sep 29, 2020 at 22:03 Mark Thompson <sw@jkqxz.net> wrote:

> On 10/09/2020 07:42, Haihao Xiang wrote:
>
> > Resend Linjie's patchset with the updates.
>
> >
>
> > As a part of the support for VA-API HEVC SCC decoding (gen12, Tiger
>
> > Lake+).
>
> >
>
> > The full support could be accessed in:
>
> > https://github.com/intel-media-ci/ffmpeg/pull/231
>
> >
>
> > The VAAPI part would be provided later once the implementations of
>
> > native parsing and reference management are all decent.
>
> >
>
> > Linjie Fu (7):
>
> >    lavc/avcodec: Add FF_PROFILE_HEVC_SCC for screen content coding
>
> >    lavc/hevc_ps: Add sps parse support for HEVC SCC extension syntax
>
> >    lavc/hevc_ps: Add pps parse support for HEVC SCC extension
>
> >    lavc/hevc_ps: Add slice parse support for HEVC SCC extension
>
> >    lavc/hevcdec: Fix the parsing for use_integer_mv_flag
>
> >    lavc/hevcdec: Set max_num_merge_cand to uint8_t
>
> >    lavc/hevc: Update reference list for SCC
>
> >
>
> >   libavcodec/avcodec.h   |   1 +
>
> >   libavcodec/hevc.h      |   3 ++
>
> >   libavcodec/hevc_ps.c   | 118 +++++++++++++++++++++++++++++++++++++++--
>
> >   libavcodec/hevc_ps.h   |  32 +++++++++++
>
> >   libavcodec/hevc_refs.c |  27 +++++++++-
>
> >   libavcodec/hevcdec.c   |  17 +++++-
>
> >   libavcodec/hevcdec.h   |   7 ++-
>
> >   libavcodec/profiles.c  |   1 +
>
> >   8 files changed, 197 insertions(+), 9 deletions(-)
>
>
>
> This looks generally pretty good, but the lack of a software
> implementation is kindof unfortunate - is there any plan to do anything
> about that?


>
> If not, I think you need to make sure that all of the newly-added flags
> give a suitable error message to say that it isn't supported.  (Especially
> in cases like use_integer_mv_flag where it can return incorrect output
> while saying that everything is fine.)


I didn’t see such plans for now, hence adding sufficient error message
seems to be a proper way.

>
> It seems to always hang with all threads waiting for each other when given
> a non-software-decodable SCC stream with threads enabled (it hangs forever
> entirely repeatably before the SIGINT):
>
>
>
> $ ./ffmpeg_g -threads 2 -i PPI_A_InterDigital_2.bit -f null -
>
> ...
>
>      Stream #0:0: Video: hevc (Scc), yuv444p(tv), 1280x720, 25 fps, 25
> tbr, 1200k tbn, 25 tbc
>
> [hevc @ 0x555557b3a680] high_precision_offsets_enabled_flag not yet
> implemented
>
> [hevc @ 0x555557b3a680] Overread PPS by 8 bits
>
> [New Thread 0x7ffff344f700 (LWP 1954956)]
>
> [New Thread 0x7ffff2c4e700 (LWP 1954957)]
>
> Stream mapping:
>
>
>
>    Stream #0:0 -> #0:0 (hevc (native) -> wrapped_avframe (native))
>
> Press [q] to stop, [?] for help
>
> [hevc @ 0x555557b3a680] high_precision_offsets_enabled_flag not yet
> implemented
>
> [hevc @ 0x555557b3a680] Overread PPS by 8 bits
>
> [hevc @ 0x555557b3a680] PPS id out of range: 0
>
> [hevc @ 0x555557b3a680] Error parsing NAL unit #3.
>
> Error while decoding stream #0:0: Invalid data found when processing input
>
>
>
> Thread 1 "ffmpeg_g" received signal SIGINT, Interrupt.
>
> futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cd90)
> at ../sysdeps/nptl/futex-internal.h:183
>
> 183     ../sysdeps/nptl/futex-internal.h: No such file or directory.
>
> (gdb) i thr
>
>    Id   Target Id                                      Frame
>
> * 1    Thread 0x7ffff34561c0 (LWP 1954952) "ffmpeg_g"
> futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cd90) at
> ../sysdeps/nptl/futex-internal.h:183
>
>    2    Thread 0x7ffff344f700 (LWP 1954956) "ffmpeg_g"
> futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cb80) at
> ../sysdeps/nptl/futex-internal.h:183
>
>    3    Thread 0x7ffff2c4e700 (LWP 1954957) "ffmpeg_g"
> futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cd64) at
> ../sysdeps/nptl/futex-internal.h:183
>
> (gdb) thr ap al bt
>
>
>
> Thread 3 (Thread 0x7ffff2c4e700 (LWP 1954957)):
>
> #0  futex_wait_cancelable (private=0, expected=0,
> futex_word=0x555557b4cd64) at ../sysdeps/nptl/futex-internal.h:183
>
> #1  __pthread_cond_wait_common (abstime=0x0, clockid=0,
> mutex=0x555557b4cdc0, cond=0x555557b4cd38) at pthread_cond_wait.c:508
>
> #2  __pthread_cond_wait (cond=0x555557b4cd38, mutex=0x555557b4cdc0) at
> pthread_cond_wait.c:638
>
> #3  0x00005555560eaed8 in strict_pthread_cond_wait (cond=0x555557b4cd38,
> mutex=0x555557b4cdc0) at src/libavutil/thread.h:112
>
> #4  0x00005555560ec49c in ff_thread_await_progress (f=0x555557b6b0a8,
> n=41, field=0) at src/libavcodec/pthread_frame.c:601
>
> #5  0x0000555555e8e73b in hevc_await_progress (s=0x555557b6a140,
> ref=0x555557b6b0a0, mv=0x7ffff2c4da00, y0=0, height=32) at
> src/libavcodec/hevcdec.c:1796
>
> #6  0x0000555555e8eeb3 in hls_prediction_unit (s=0x555557b6a140, x0=32,
> y0=0, nPbW=32, nPbH=32, log2_cb_size=5, partIdx=0, idx=3) at
> src/libavcodec/hevcdec.c:1900
>
> #7  0x0000555555e902f5 in hls_coding_unit (s=0x555557b6a140, x0=32, y0=0,
> log2_cb_size=5) at src/libavcodec/hevcdec.c:2206
>
> #8  0x0000555555e9114c in hls_coding_quadtree (s=0x555557b6a140, x0=32,
> y0=0, log2_cb_size=5, cb_depth=1) at src/libavcodec/hevcdec.c:2388
>
> #9  0x0000555555e90fce in hls_coding_quadtree (s=0x555557b6a140, x0=0,
> y0=0, log2_cb_size=6, cb_depth=0) at src/libavcodec/hevcdec.c:2362
>
> #10 0x0000555555e91bc4 in hls_decode_entry (avctxt=0x555557b77580,
> isFilterThread=0x7ffff2c4dce8) at src/libavcodec/hevcdec.c:2498
>
> #11 0x00005555562059d3 in avcodec_default_execute (c=0x555557b77580,
> func=0x555555e9188d <hls_decode_entry>, arg=0x7ffff2c4dce8,
> ret=0x7ffff2c4dce0, count=1, size=4) at src/libavcodec/utils.c:448
>
> #12 0x0000555555e91ce2 in hls_slice_data (s=0x555557b6a140) at
> src/libavcodec/hevcdec.c:2525
>
> #13 0x0000555555e93f5b in decode_nal_unit (s=0x555557b6a140,
> nal=0x7fffe4006470) at src/libavcodec/hevcdec.c:3098
>
> #14 0x0000555555e9428d in decode_nal_units (s=0x555557b6a140,
> buf=0x555557b85590 "", length=12760) at src/libavcodec/hevcdec.c:3171
>
> #15 0x0000555555e948a0 in hevc_decode_frame (avctx=0x555557b77580,
> data=0x555557b4a9c0, got_output=0x555557b4ce50, avpkt=0x555557b4cdf0) at
> src/libavcodec/hevcdec.c:3314
>
> #16 0x00005555560eb205 in frame_worker_thread (arg=0x555557b4ccf0) at
> src/libavcodec/pthread_frame.c:201
>
> #17 0x00007ffff6d25ea7 in start_thread (arg=<optimized out>) at
> pthread_create.c:477
>
> #18 0x00007ffff4ad7eaf in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
>
>
>
> Thread 2 (Thread 0x7ffff344f700 (LWP 1954956)):
>
> #0  futex_wait_cancelable (private=0, expected=0,
> futex_word=0x555557b4cb80) at ../sysdeps/nptl/futex-internal.h:183
>
> #1  __pthread_cond_wait_common (abstime=0x0, clockid=0,
> mutex=0x555557b4cbe8, cond=0x555557b4cb58) at pthread_cond_wait.c:508
>
> #2  __pthread_cond_wait (cond=0x555557b4cb58, mutex=0x555557b4cbe8) at
> pthread_cond_wait.c:638
>
> #3  0x00005555560eaed8 in strict_pthread_cond_wait (cond=0x555557b4cb58,
> mutex=0x555557b4cbe8) at src/libavutil/thread.h:112
>
> #4  0x00005555560eb0bf in frame_worker_thread (arg=0x555557b4cb40) at
> src/libavcodec/pthread_frame.c:177
>
> #5  0x00007ffff6d25ea7 in start_thread (arg=<optimized out>) at
> pthread_create.c:477
>
> #6  0x00007ffff4ad7eaf in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
>
>
>
> Thread 1 (Thread 0x7ffff34561c0 (LWP 1954952)):
>
> #0  futex_wait_cancelable (private=0, expected=0,
> futex_word=0x555557b4cd90) at ../sysdeps/nptl/futex-internal.h:183
>
> #1  __pthread_cond_wait_common (abstime=0x0, clockid=0,
> mutex=0x555557b4cdc0, cond=0x555557b4cd68) at pthread_cond_wait.c:508
>
> #2  __pthread_cond_wait (cond=0x555557b4cd68, mutex=0x555557b4cdc0) at
> pthread_cond_wait.c:638
>
> #3  0x00005555560eaed8 in strict_pthread_cond_wait (cond=0x555557b4cd68,
> mutex=0x555557b4cdc0) at src/libavutil/thread.h:112
>
> #4  0x00005555560ec0ef in ff_thread_decode_frame (avctx=0x555557b39840,
> picture=0x555557b39d80, got_picture_ptr=0x7fffffffd5e4,
> avpkt=0x555557b39600) at src/libavcodec/pthread_frame.c:526
>
> #5  0x0000555555d884f6 in decode_simple_internal (avctx=0x555557b39840,
> frame=0x555557b39d80) at src/libavcodec/decode.c:350
>
> #6  0x0000555555d89132 in decode_simple_receive_frame
> (avctx=0x555557b39840, frame=0x555557b39d80) at src/libavcodec/decode.c:549
>
> #7  0x0000555555d89218 in decode_receive_frame_internal
> (avctx=0x555557b39840, frame=0x555557b39d80) at src/libavcodec/decode.c:569
>
> #8  0x0000555555d89475 in avcodec_send_packet (avctx=0x555557b39840,
> avpkt=0x7fffffffd7f0) at src/libavcodec/decode.c:627
>
> #9  0x0000555555695517 in decode (avctx=0x555557b39840,
> frame=0x555557b61d80, got_frame=0x7fffffffd964, pkt=0x7fffffffd7f0) at
> src/fftools/ffmpeg.c:2218
>
> #10 0x0000555555695d86 in decode_video (ist=0x555557b34f40,
> pkt=0x7fffffffd970, got_output=0x7fffffffd964, duration_pts=0x7fffffffd968,
> eof=0, decode_failed=0x7fffffffd960) at src/fftools/ffmpeg.c:2360
>
> #11 0x0000555555696d79 in process_input_packet (ist=0x555557b34f40,
> pkt=0x7fffffffdc40, no_eof=0) at src/fftools/ffmpeg.c:2601
>
> #12 0x000055555569e06f in process_input (file_index=0) at
> src/fftools/ffmpeg.c:4494
>
> #13 0x000055555569e594 in transcode_step () at src/fftools/ffmpeg.c:4614
>
> #14 0x000055555569e6be in transcode () at src/fftools/ffmpeg.c:4668
>
> #15 0x000055555569ef4b in main (argc=8, argv=0x7fffffffe3b8) at
> src/fftools/ffmpeg.c:4873
>
> (gdb)
>
>
>
> That PPS overread looks like something which shouldn't happen, too.
>

Ahhh, will trace into this to get more info, thx.

- Linjie
Guangxin Xu Oct. 2, 2020, 3:09 p.m. UTC | #4
On Tue, Sep 29, 2020 at 10:03 PM Mark Thompson <sw@jkqxz.net> wrote:

> On 10/09/2020 07:42, Haihao Xiang wrote:
> > Resend Linjie's patchset with the updates.
> >
> > As a part of the support for VA-API HEVC SCC decoding (gen12, Tiger
> > Lake+).
> >
> > The full support could be accessed in:
> > https://github.com/intel-media-ci/ffmpeg/pull/231
> >
> > The VAAPI part would be provided later once the implementations of
> > native parsing and reference management are all decent.
> >
> > Linjie Fu (7):
> >    lavc/avcodec: Add FF_PROFILE_HEVC_SCC for screen content coding
> >    lavc/hevc_ps: Add sps parse support for HEVC SCC extension syntax
> >    lavc/hevc_ps: Add pps parse support for HEVC SCC extension
> >    lavc/hevc_ps: Add slice parse support for HEVC SCC extension
> >    lavc/hevcdec: Fix the parsing for use_integer_mv_flag
> >    lavc/hevcdec: Set max_num_merge_cand to uint8_t
> >    lavc/hevc: Update reference list for SCC
> >
> >   libavcodec/avcodec.h   |   1 +
> >   libavcodec/hevc.h      |   3 ++
> >   libavcodec/hevc_ps.c   | 118 +++++++++++++++++++++++++++++++++++++++--
> >   libavcodec/hevc_ps.h   |  32 +++++++++++
> >   libavcodec/hevc_refs.c |  27 +++++++++-
> >   libavcodec/hevcdec.c   |  17 +++++-
> >   libavcodec/hevcdec.h   |   7 ++-
> >   libavcodec/profiles.c  |   1 +
> >   8 files changed, 197 insertions(+), 9 deletions(-)
>
> This looks generally pretty good, but the lack of a software
> implementation is kindof unfortunate - is there any plan to do anything
> about that?
>
Most of scc conformance clip has tiles.
But currently, the hevc software decoder has many issues for tile cabac
saving and loading.
We'd better fix them before starting implement scc tool.

I have queue up some patches to address the cabac issue at [1] and send the
first one to review at [2]
but, no one responded to me yet. Do you know who can help review the patch?
thanks

[1] https://github.com/oddstone/FFmpeg/commits/rext1
[2]
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200829055218.32261-1-oddstone@gmail.com/

>
> If not, I think you need to make sure that all of the newly-added flags
> give a suitable error message to say that it isn't supported.  (Especially
> in cases like use_integer_mv_flag where it can return incorrect output
> while saying that everything is fine.)
>
>
> It seems to always hang with all threads waiting for each other when given
> a non-software-decodable SCC stream with threads enabled (it hangs forever
> entirely repeatably before the SIGINT):
>
> $ ./ffmpeg_g -threads 2 -i PPI_A_InterDigital_2.bit -f null -
> ...
>      Stream #0:0: Video: hevc (Scc), yuv444p(tv), 1280x720, 25 fps, 25
> tbr, 1200k tbn, 25 tbc
> [hevc @ 0x555557b3a680] high_precision_offsets_enabled_flag not yet
> implemented
> [hevc @ 0x555557b3a680] Overread PPS by 8 bits
> [New Thread 0x7ffff344f700 (LWP 1954956)]
> [New Thread 0x7ffff2c4e700 (LWP 1954957)]
> Stream mapping:
>
>    Stream #0:0 -> #0:0 (hevc (native) -> wrapped_avframe (native))
> Press [q] to stop, [?] for help
> [hevc @ 0x555557b3a680] high_precision_offsets_enabled_flag not yet
> implemented
> [hevc @ 0x555557b3a680] Overread PPS by 8 bits
> [hevc @ 0x555557b3a680] PPS id out of range: 0
> [hevc @ 0x555557b3a680] Error parsing NAL unit #3.
> Error while decoding stream #0:0: Invalid data found when processing input
>
> Thread 1 "ffmpeg_g" received signal SIGINT, Interrupt.
> futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cd90)
> at ../sysdeps/nptl/futex-internal.h:183
> 183     ../sysdeps/nptl/futex-internal.h: No such file or directory.
> (gdb) i thr
>    Id   Target Id                                      Frame
> * 1    Thread 0x7ffff34561c0 (LWP 1954952) "ffmpeg_g"
> futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cd90) at
> ../sysdeps/nptl/futex-internal.h:183
>    2    Thread 0x7ffff344f700 (LWP 1954956) "ffmpeg_g"
> futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cb80) at
> ../sysdeps/nptl/futex-internal.h:183
>    3    Thread 0x7ffff2c4e700 (LWP 1954957) "ffmpeg_g"
> futex_wait_cancelable (private=0, expected=0, futex_word=0x555557b4cd64) at
> ../sysdeps/nptl/futex-internal.h:183
> (gdb) thr ap al bt
>
> Thread 3 (Thread 0x7ffff2c4e700 (LWP 1954957)):
> #0  futex_wait_cancelable (private=0, expected=0,
> futex_word=0x555557b4cd64) at ../sysdeps/nptl/futex-internal.h:183
> #1  __pthread_cond_wait_common (abstime=0x0, clockid=0,
> mutex=0x555557b4cdc0, cond=0x555557b4cd38) at pthread_cond_wait.c:508
> #2  __pthread_cond_wait (cond=0x555557b4cd38, mutex=0x555557b4cdc0) at
> pthread_cond_wait.c:638
> #3  0x00005555560eaed8 in strict_pthread_cond_wait (cond=0x555557b4cd38,
> mutex=0x555557b4cdc0) at src/libavutil/thread.h:112
> #4  0x00005555560ec49c in ff_thread_await_progress (f=0x555557b6b0a8,
> n=41, field=0) at src/libavcodec/pthread_frame.c:601
> #5  0x0000555555e8e73b in hevc_await_progress (s=0x555557b6a140,
> ref=0x555557b6b0a0, mv=0x7ffff2c4da00, y0=0, height=32) at
> src/libavcodec/hevcdec.c:1796
> #6  0x0000555555e8eeb3 in hls_prediction_unit (s=0x555557b6a140, x0=32,
> y0=0, nPbW=32, nPbH=32, log2_cb_size=5, partIdx=0, idx=3) at
> src/libavcodec/hevcdec.c:1900
> #7  0x0000555555e902f5 in hls_coding_unit (s=0x555557b6a140, x0=32, y0=0,
> log2_cb_size=5) at src/libavcodec/hevcdec.c:2206
> #8  0x0000555555e9114c in hls_coding_quadtree (s=0x555557b6a140, x0=32,
> y0=0, log2_cb_size=5, cb_depth=1) at src/libavcodec/hevcdec.c:2388
> #9  0x0000555555e90fce in hls_coding_quadtree (s=0x555557b6a140, x0=0,
> y0=0, log2_cb_size=6, cb_depth=0) at src/libavcodec/hevcdec.c:2362
> #10 0x0000555555e91bc4 in hls_decode_entry (avctxt=0x555557b77580,
> isFilterThread=0x7ffff2c4dce8) at src/libavcodec/hevcdec.c:2498
> #11 0x00005555562059d3 in avcodec_default_execute (c=0x555557b77580,
> func=0x555555e9188d <hls_decode_entry>, arg=0x7ffff2c4dce8,
> ret=0x7ffff2c4dce0, count=1, size=4) at src/libavcodec/utils.c:448
> #12 0x0000555555e91ce2 in hls_slice_data (s=0x555557b6a140) at
> src/libavcodec/hevcdec.c:2525
> #13 0x0000555555e93f5b in decode_nal_unit (s=0x555557b6a140,
> nal=0x7fffe4006470) at src/libavcodec/hevcdec.c:3098
> #14 0x0000555555e9428d in decode_nal_units (s=0x555557b6a140,
> buf=0x555557b85590 "", length=12760) at src/libavcodec/hevcdec.c:3171
> #15 0x0000555555e948a0 in hevc_decode_frame (avctx=0x555557b77580,
> data=0x555557b4a9c0, got_output=0x555557b4ce50, avpkt=0x555557b4cdf0) at
> src/libavcodec/hevcdec.c:3314
> #16 0x00005555560eb205 in frame_worker_thread (arg=0x555557b4ccf0) at
> src/libavcodec/pthread_frame.c:201
> #17 0x00007ffff6d25ea7 in start_thread (arg=<optimized out>) at
> pthread_create.c:477
> #18 0x00007ffff4ad7eaf in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
>
> Thread 2 (Thread 0x7ffff344f700 (LWP 1954956)):
> #0  futex_wait_cancelable (private=0, expected=0,
> futex_word=0x555557b4cb80) at ../sysdeps/nptl/futex-internal.h:183
> #1  __pthread_cond_wait_common (abstime=0x0, clockid=0,
> mutex=0x555557b4cbe8, cond=0x555557b4cb58) at pthread_cond_wait.c:508
> #2  __pthread_cond_wait (cond=0x555557b4cb58, mutex=0x555557b4cbe8) at
> pthread_cond_wait.c:638
> #3  0x00005555560eaed8 in strict_pthread_cond_wait (cond=0x555557b4cb58,
> mutex=0x555557b4cbe8) at src/libavutil/thread.h:112
> #4  0x00005555560eb0bf in frame_worker_thread (arg=0x555557b4cb40) at
> src/libavcodec/pthread_frame.c:177
> #5  0x00007ffff6d25ea7 in start_thread (arg=<optimized out>) at
> pthread_create.c:477
> #6  0x00007ffff4ad7eaf in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
>
> Thread 1 (Thread 0x7ffff34561c0 (LWP 1954952)):
> #0  futex_wait_cancelable (private=0, expected=0,
> futex_word=0x555557b4cd90) at ../sysdeps/nptl/futex-internal.h:183
> #1  __pthread_cond_wait_common (abstime=0x0, clockid=0,
> mutex=0x555557b4cdc0, cond=0x555557b4cd68) at pthread_cond_wait.c:508
> #2  __pthread_cond_wait (cond=0x555557b4cd68, mutex=0x555557b4cdc0) at
> pthread_cond_wait.c:638
> #3  0x00005555560eaed8 in strict_pthread_cond_wait (cond=0x555557b4cd68,
> mutex=0x555557b4cdc0) at src/libavutil/thread.h:112
> #4  0x00005555560ec0ef in ff_thread_decode_frame (avctx=0x555557b39840,
> picture=0x555557b39d80, got_picture_ptr=0x7fffffffd5e4,
> avpkt=0x555557b39600) at src/libavcodec/pthread_frame.c:526
> #5  0x0000555555d884f6 in decode_simple_internal (avctx=0x555557b39840,
> frame=0x555557b39d80) at src/libavcodec/decode.c:350
> #6  0x0000555555d89132 in decode_simple_receive_frame
> (avctx=0x555557b39840, frame=0x555557b39d80) at src/libavcodec/decode.c:549
> #7  0x0000555555d89218 in decode_receive_frame_internal
> (avctx=0x555557b39840, frame=0x555557b39d80) at src/libavcodec/decode.c:569
> #8  0x0000555555d89475 in avcodec_send_packet (avctx=0x555557b39840,
> avpkt=0x7fffffffd7f0) at src/libavcodec/decode.c:627
> #9  0x0000555555695517 in decode (avctx=0x555557b39840,
> frame=0x555557b61d80, got_frame=0x7fffffffd964, pkt=0x7fffffffd7f0) at
> src/fftools/ffmpeg.c:2218
> #10 0x0000555555695d86 in decode_video (ist=0x555557b34f40,
> pkt=0x7fffffffd970, got_output=0x7fffffffd964, duration_pts=0x7fffffffd968,
> eof=0, decode_failed=0x7fffffffd960) at src/fftools/ffmpeg.c:2360
> #11 0x0000555555696d79 in process_input_packet (ist=0x555557b34f40,
> pkt=0x7fffffffdc40, no_eof=0) at src/fftools/ffmpeg.c:2601
> #12 0x000055555569e06f in process_input (file_index=0) at
> src/fftools/ffmpeg.c:4494
> #13 0x000055555569e594 in transcode_step () at src/fftools/ffmpeg.c:4614
> #14 0x000055555569e6be in transcode () at src/fftools/ffmpeg.c:4668
> #15 0x000055555569ef4b in main (argc=8, argv=0x7fffffffe3b8) at
> src/fftools/ffmpeg.c:4873
> (gdb)
>
> That PPS overread looks like something which shouldn't happen, too.
>
> - Mark
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Christophe Gisquet Oct. 29, 2020, 1:51 p.m. UTC | #5
Hi,

Le ven. 2 oct. 2020 à 18:12, Guangxin Xu <oddstone@gmail.com> a écrit :
> Most of scc conformance clip has tiles.
> But currently, the hevc software decoder has many issues for tile cabac
> saving and loading.
> We'd better fix them before starting implement scc tool.
>
> I have queue up some patches to address the cabac issue at [1] and send the
> first one to review at [2]
> but, no one responded to me yet. Do you know who can help review the patch?
> thanks
>
> [1] https://github.com/oddstone/FFmpeg/commits/rext1

This has additional fixes (which looks good, haven't really delved
into it) that unfortunately doesn't fix:

> [2]
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200829055218.32261-1-oddstone@gmail.com/

this patch being run under fate with THREADS=12 THREAD_TYPE=slice

--
Christophe
Christophe Gisquet Oct. 29, 2020, 1:57 p.m. UTC | #6
Hi,

Le mar. 29 sept. 2020 à 17:55, Linjie Fu <linjie.justin.fu@gmail.com> a écrit :
> I didn’t see such plans for now, hence adding sufficient error message
> seems to be a proper way.

Hi, as you are the only one active on this decoder, this shouldn't matter, but:
down the line, the ffmpeg project has no way of testing if someone
breaks even the basic parsing of these extensions in the future.
To test, the hardware you mention is needed, as well as maybe specific tests.

At some point, fate lacks some support for verifying h/w decoding. It
would be really nice if some of these companies with all this new
awesome hardware would consider this, and for instance contribute fate
instances to perform such test for the ffmpeg project.
Christophe Gisquet Oct. 29, 2020, 2 p.m. UTC | #7
Forgot to add this:

Le jeu. 29 oct. 2020 à 14:51, Christophe Gisquet
<christophe.gisquet@gmail.com> a écrit :
> > [1] https://github.com/oddstone/FFmpeg/commits/rext1
>
> This has additional fixes (which looks good, haven't really delved
> into it) that unfortunately doesn't fix:

And I suspect you need these entropy state fixes (and other ones) to
be before adding any such tools, because these tools will not pass
some fate testing without them.
Christophe Gisquet Nov. 18, 2020, 10:29 a.m. UTC | #8
Hi,

Le jeu. 29 oct. 2020 à 14:57, Christophe Gisquet
<christophe.gisquet@gmail.com> a écrit :
> Hi, as you are the only one active on this decoder, this shouldn't matter, but:
> down the line, the ffmpeg project has no way of testing if someone
> breaks even the basic parsing of these extensions in the future.
> To test, the hardware you mention is needed, as well as maybe specific tests.
>
> At some point, fate lacks some support for verifying h/w decoding. It
> would be really nice if some of these companies with all this new
> awesome hardware would consider this, and for instance contribute fate
> instances to perform such test for the ffmpeg project.

Ping? This is irrespective of the patchset being accepted or not. I
just wish people are aware of the stakes in the long term.

Thanks anyway for investing the time for what you've already done.
Michael Niedermayer Nov. 18, 2020, 9:50 p.m. UTC | #9
On Thu, Oct 29, 2020 at 02:57:13PM +0100, Christophe Gisquet wrote:
> Hi,
> 
> Le mar. 29 sept. 2020 à 17:55, Linjie Fu <linjie.justin.fu@gmail.com> a écrit :
> > I didn’t see such plans for now, hence adding sufficient error message
> > seems to be a proper way.
> 
> Hi, as you are the only one active on this decoder, this shouldn't matter, but:
> down the line, the ffmpeg project has no way of testing if someone
> breaks even the basic parsing of these extensions in the future.
> To test, the hardware you mention is needed, as well as maybe specific tests.
> 

> At some point, fate lacks some support for verifying h/w decoding. It
> would be really nice if some of these companies with all this new
> awesome hardware would consider this, and for instance contribute fate
> instances to perform such test for the ffmpeg project.

I agree!

thx

[...]
Linjie Fu Nov. 19, 2020, 3:48 p.m. UTC | #10
On Thu, Nov 19, 2020 at 5:50 AM Michael Niedermayer <michael@niedermayer.cc>
wrote:

> On Thu, Oct 29, 2020 at 02:57:13PM +0100, Christophe Gisquet wrote:
> > Hi,
> >
> > Le mar. 29 sept. 2020 à 17:55, Linjie Fu <linjie.justin.fu@gmail.com> a
> écrit :
> > > I didn’t see such plans for now, hence adding sufficient error message
> > > seems to be a proper way.
> >
> > Hi, as you are the only one active on this decoder, this shouldn't
> matter, but:
> > down the line, the ffmpeg project has no way of testing if someone
> > breaks even the basic parsing of these extensions in the future.
> > To test, the hardware you mention is needed, as well as maybe specific
> tests.
> >
>
> > At some point, fate lacks some support for verifying h/w decoding. It
> > would be really nice if some of these companies with all this new
> > awesome hardware would consider this, and for instance contribute fate
> > instances to perform such test for the ffmpeg project.
>
> I agree!
>

Update on mail list for guys who may be interested too:
Just discussed with Chris about this through IRC, including how to bridge
and get this step further.

Regards,
Linjie