| Message ID | NZMWg6Y--3-9@lynne.ee |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel] lavfi/bwdif: remove interpolated sample clipping | expand |
| Context | Check | Description |
|---|---|---|
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | fail | Make fate failed |
Jul 2, 2023, 18:54 by dev@lynne.ee: > The issue is that clipping the interpolated temporal sample against > the spatially predicted sample causes artifacts to appear. > > Discovered while writing the Vulkan version (where I omitted the > same check). > > The clipping in the code is carried over from yadif. Removing the > same code in yadif does not make any difference to the output. > I think that the check was simply ill-adapted to the new prediction > code and does more harm. > > I tested replacing the range clip with only an FFMAX, and only an > FFMIN, but in both cases, artifacts still appeared. > > Test sample 1: https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2: https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv > > Command line: > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v rawvideo -y <OUTPUT>.nut > Make sure to disable the assembly. > > Comparisons: > https://files.lynne.ee/bwdif_01_before.png > https://files.lynne.ee/bwdif_01_after.png > Generated from sample 1 via: > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png > > https://files.lynne.ee/bwdif_02_before.png > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png > Corrected links for the second sample: https://files.lynne.ee/bwdif_02_before.png https://files.lynne.ee/bwdif_02_after.png ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png I'm sure I hit a newline. The artifacts are a lot more noticeable in the second sample.
Am So., 2. Juli 2023 um 18:57 Uhr schrieb Lynne <dev@lynne.ee>: > Jul 2, 2023, 18:54 by dev@lynne.ee: > > > The issue is that clipping the interpolated temporal sample against > > the spatially predicted sample causes artifacts to appear. > > > > Discovered while writing the Vulkan version (where I omitted the > > same check). > > > > The clipping in the code is carried over from yadif. Removing the > > same code in yadif does not make any difference to the output. > > I think that the check was simply ill-adapted to the new prediction > > code and does more harm. > > > > I tested replacing the range clip with only an FFMAX, and only an > > FFMIN, but in both cases, artifacts still appeared. > > > > Test sample 1: > https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2: > https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv > > > > Command line: > > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v > rawvideo -y <OUTPUT>.nut > > Make sure to disable the assembly. > > > > Comparisons: > > https://files.lynne.ee/bwdif_01_before.png > > https://files.lynne.ee/bwdif_01_after.png > > Generated from sample 1 via: > > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf > crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png > > > > https://files.lynne.ee/bwdif_02_before.png > > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i > <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y > <OUTPUT>.png > > > > Corrected links for the second sample: > > https://files.lynne.ee/bwdif_02_before.png > https://files.lynne.ee/bwdif_02_after.png > ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf > crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png > > I'm sure I hit a newline. The artifacts are a lot more noticeable in the > second sample. > I developed the bwdif to achieve the best possible balance between speed and quality of all different image contents from the broadcast point of view. This includes moving video as well as moving and static graphic elements. Unfortunately, the improvement of one image content often leads to the degradation of another. The code you removed fundamentally stabilizes the static graphic elements. This outweighs the slightly more frequent artifacts in moving video considering the general purpose of the filter. For noisy pure motion video content, for example, the w3fdif is better suited, since it does not make if/else decisions and thus does not produce artifacts. Regards, Thomas
On Sun, Jul 2, 2023 at 6:57 PM Lynne <dev@lynne.ee> wrote: > Jul 2, 2023, 18:54 by dev@lynne.ee: > > > The issue is that clipping the interpolated temporal sample against > > the spatially predicted sample causes artifacts to appear. > > > > Discovered while writing the Vulkan version (where I omitted the > > same check). > > > > The clipping in the code is carried over from yadif. Removing the > > same code in yadif does not make any difference to the output. > > I think that the check was simply ill-adapted to the new prediction > > code and does more harm. > > > > I tested replacing the range clip with only an FFMAX, and only an > > FFMIN, but in both cases, artifacts still appeared. > > > > Test sample 1: > https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2: > https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv > > > > Command line: > > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v > rawvideo -y <OUTPUT>.nut > > Make sure to disable the assembly. > > > > Comparisons: > > https://files.lynne.ee/bwdif_01_before.png > > https://files.lynne.ee/bwdif_01_after.png > > Generated from sample 1 via: > > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf > crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png > > > > https://files.lynne.ee/bwdif_02_before.png > > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i > <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y > <OUTPUT>.png > > > > Corrected links for the second sample: > > https://files.lynne.ee/bwdif_02_before.png > https://files.lynne.ee/bwdif_02_after.png > ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf > crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png > > I'm sure I hit a newline. The artifacts are a lot more noticeable in the > second sample. > Single png images are not way to prove something. Please provide videos and not just single file that exhibit this issue. (Keep showing same file over and over is not going to help show that it helps) Also how PSNR/SSIM/VMAF changes before after not just in single sample but in more samples. > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Jul 2, 2023, 20:41 by tmundt75@gmail.com: > Am So., 2. Juli 2023 um 18:57 Uhr schrieb Lynne <dev@lynne.ee>: > >> Jul 2, 2023, 18:54 by dev@lynne.ee: >> >> > The issue is that clipping the interpolated temporal sample against >> > the spatially predicted sample causes artifacts to appear. >> > >> > Discovered while writing the Vulkan version (where I omitted the >> > same check). >> > >> > The clipping in the code is carried over from yadif. Removing the >> > same code in yadif does not make any difference to the output. >> > I think that the check was simply ill-adapted to the new prediction >> > code and does more harm. >> > >> > I tested replacing the range clip with only an FFMAX, and only an >> > FFMIN, but in both cases, artifacts still appeared. >> > >> > Test sample 1: >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2: >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv >> > >> > Command line: >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v >> rawvideo -y <OUTPUT>.nut >> > Make sure to disable the assembly. >> > >> > Comparisons: >> > https://files.lynne.ee/bwdif_01_before.png >> > https://files.lynne.ee/bwdif_01_after.png >> > Generated from sample 1 via: >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png >> > >> > https://files.lynne.ee/bwdif_02_before.png >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y >> <OUTPUT>.png >> > >> >> Corrected links for the second sample: >> >> https://files.lynne.ee/bwdif_02_before.png >> https://files.lynne.ee/bwdif_02_after.png >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png >> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in the >> second sample. >> > > I developed the bwdif to achieve the best possible balance between speed > and quality of all different image contents from the broadcast point of > view. This includes moving video as well as moving and static graphic > elements. Unfortunately, the improvement of one image content often leads > to the degradation of another. The code you removed fundamentally > stabilizes the static graphic elements. This outweighs the slightly more > frequent artifacts in moving video considering the general purpose of the > filter. > Could you post examples? I've been unable to find any that look worse with the patch.
Jul 2, 2023, 20:48 by onemda@gmail.com: > On Sun, Jul 2, 2023 at 6:57 PM Lynne <dev@lynne.ee> wrote: > >> Jul 2, 2023, 18:54 by dev@lynne.ee: >> >> > The issue is that clipping the interpolated temporal sample against >> > the spatially predicted sample causes artifacts to appear. >> > >> > Discovered while writing the Vulkan version (where I omitted the >> > same check). >> > >> > The clipping in the code is carried over from yadif. Removing the >> > same code in yadif does not make any difference to the output. >> > I think that the check was simply ill-adapted to the new prediction >> > code and does more harm. >> > >> > I tested replacing the range clip with only an FFMAX, and only an >> > FFMIN, but in both cases, artifacts still appeared. >> > >> > Test sample 1: >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2: >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv >> > >> > Command line: >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v >> rawvideo -y <OUTPUT>.nut >> > Make sure to disable the assembly. >> > >> > Comparisons: >> > https://files.lynne.ee/bwdif_01_before.png >> > https://files.lynne.ee/bwdif_01_after.png >> > Generated from sample 1 via: >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png >> > >> > https://files.lynne.ee/bwdif_02_before.png >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y >> <OUTPUT>.png >> > >> >> Corrected links for the second sample: >> >> https://files.lynne.ee/bwdif_02_before.png >> https://files.lynne.ee/bwdif_02_after.png >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png >> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in the >> second sample. >> > > Single png images are not way to prove something. > > Please provide videos and not just single file that exhibit this issue. > (Keep showing same file over and over is not going to help show that it > helps) > > Also how PSNR/SSIM/VMAF changes before after not just in single sample but > in more samples. > I posted samples and instructions. Enough for a discussion. I'm not posting gigabytes of uncompressed samples. PSNR is irrelevant if there are visible artifacts.
On Sun, Jul 2, 2023 at 9:13 PM Lynne <dev@lynne.ee> wrote: > Jul 2, 2023, 20:48 by onemda@gmail.com: > > > On Sun, Jul 2, 2023 at 6:57 PM Lynne <dev@lynne.ee> wrote: > > > >> Jul 2, 2023, 18:54 by dev@lynne.ee: > >> > >> > The issue is that clipping the interpolated temporal sample against > >> > the spatially predicted sample causes artifacts to appear. > >> > > >> > Discovered while writing the Vulkan version (where I omitted the > >> > same check). > >> > > >> > The clipping in the code is carried over from yadif. Removing the > >> > same code in yadif does not make any difference to the output. > >> > I think that the check was simply ill-adapted to the new prediction > >> > code and does more harm. > >> > > >> > I tested replacing the range clip with only an FFMAX, and only an > >> > FFMIN, but in both cases, artifacts still appeared. > >> > > >> > Test sample 1: > >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2: > >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv > >> > > >> > Command line: > >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v > >> rawvideo -y <OUTPUT>.nut > >> > Make sure to disable the assembly. > >> > > >> > Comparisons: > >> > https://files.lynne.ee/bwdif_01_before.png > >> > https://files.lynne.ee/bwdif_01_after.png > >> > Generated from sample 1 via: > >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf > >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png > >> > > >> > https://files.lynne.ee/bwdif_02_before.png > >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i > >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y > >> <OUTPUT>.png > >> > > >> > >> Corrected links for the second sample: > >> > >> https://files.lynne.ee/bwdif_02_before.png > >> https://files.lynne.ee/bwdif_02_after.png > >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf > >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png > >> > >> I'm sure I hit a newline. The artifacts are a lot more noticeable in the > >> second sample. > >> > > > > Single png images are not way to prove something. > > > > Please provide videos and not just single file that exhibit this issue. > > (Keep showing same file over and over is not going to help show that it > > helps) > > > > Also how PSNR/SSIM/VMAF changes before after not just in single sample > but > > in more samples. > > > > I posted samples and instructions. Enough for a discussion. > I'm not posting gigabytes of uncompressed samples. > PSNR is irrelevant if there are visible artifacts. > Nope, you havent. Some strange samples. PSNR is relevant as also SSIM and VMAF. Ignoring this does not help project, but just force tyrannic behavior.
Jul 2, 2023, 21:38 by onemda@gmail.com: > On Sun, Jul 2, 2023 at 9:13 PM Lynne <dev@lynne.ee> wrote: > >> Jul 2, 2023, 20:48 by onemda@gmail.com: >> >> > On Sun, Jul 2, 2023 at 6:57 PM Lynne <dev@lynne.ee> wrote: >> > >> >> Jul 2, 2023, 18:54 by dev@lynne.ee: >> >> >> >> > The issue is that clipping the interpolated temporal sample against >> >> > the spatially predicted sample causes artifacts to appear. >> >> > >> >> > Discovered while writing the Vulkan version (where I omitted the >> >> > same check). >> >> > >> >> > The clipping in the code is carried over from yadif. Removing the >> >> > same code in yadif does not make any difference to the output. >> >> > I think that the check was simply ill-adapted to the new prediction >> >> > code and does more harm. >> >> > >> >> > I tested replacing the range clip with only an FFMAX, and only an >> >> > FFMIN, but in both cases, artifacts still appeared. >> >> > >> >> > Test sample 1: >> >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2: >> >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv >> >> > >> >> > Command line: >> >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v >> >> rawvideo -y <OUTPUT>.nut >> >> > Make sure to disable the assembly. >> >> > >> >> > Comparisons: >> >> > https://files.lynne.ee/bwdif_01_before.png >> >> > https://files.lynne.ee/bwdif_01_after.png >> >> > Generated from sample 1 via: >> >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf >> >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png >> >> > >> >> > https://files.lynne.ee/bwdif_02_before.png >> >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i >> >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y >> >> <OUTPUT>.png >> >> > >> >> >> >> Corrected links for the second sample: >> >> >> >> https://files.lynne.ee/bwdif_02_before.png >> >> https://files.lynne.ee/bwdif_02_after.png >> >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf >> >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png >> >> >> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in the >> >> second sample. >> >> >> > >> > Single png images are not way to prove something. >> > >> > Please provide videos and not just single file that exhibit this issue. >> > (Keep showing same file over and over is not going to help show that it >> > helps) >> > >> > Also how PSNR/SSIM/VMAF changes before after not just in single sample >> but >> > in more samples. >> > >> >> I posted samples and instructions. Enough for a discussion. >> I'm not posting gigabytes of uncompressed samples. >> PSNR is irrelevant if there are visible artifacts. >> > > Nope, you havent. > > Some strange samples. > PSNR is relevant as also SSIM and VMAF. > Fair enough, on a 4k60 game recording, with lots of text and movement, ffmpeg -i game_4k60test.mkv -filter_complex "[0:0] split [t1] [t2] ; [t2] interlace [t2] ; [t2] bwdif=mode=send_field [t2] ; [t1] [t2] ssim [t3]" -map "[t3]" -f null - Before: 0.990397 (20.175775) After: 0.990417 (20.184970) Slightly higher, but not really significant. PSNR (average): 32.711758 vs 32.704465 Slightly lower, but not really significant. > Ignoring this does not help project, but just force tyrannic behavior. > Relax, I'm not going to make you write a stablediffusion filter source :)
On Sun, Jul 2, 2023 at 10:25 PM Lynne <dev@lynne.ee> wrote: > Jul 2, 2023, 21:38 by onemda@gmail.com: > > > On Sun, Jul 2, 2023 at 9:13 PM Lynne <dev@lynne.ee> wrote: > > > >> Jul 2, 2023, 20:48 by onemda@gmail.com: > >> > >> > On Sun, Jul 2, 2023 at 6:57 PM Lynne <dev@lynne.ee> wrote: > >> > > >> >> Jul 2, 2023, 18:54 by dev@lynne.ee: > >> >> > >> >> > The issue is that clipping the interpolated temporal sample against > >> >> > the spatially predicted sample causes artifacts to appear. > >> >> > > >> >> > Discovered while writing the Vulkan version (where I omitted the > >> >> > same check). > >> >> > > >> >> > The clipping in the code is carried over from yadif. Removing the > >> >> > same code in yadif does not make any difference to the output. > >> >> > I think that the check was simply ill-adapted to the new prediction > >> >> > code and does more harm. > >> >> > > >> >> > I tested replacing the range clip with only an FFMAX, and only an > >> >> > FFMIN, but in both cases, artifacts still appeared. > >> >> > > >> >> > Test sample 1: > >> >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample > 2: > >> >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv > >> >> > > >> >> > Command line: > >> >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v > >> >> rawvideo -y <OUTPUT>.nut > >> >> > Make sure to disable the assembly. > >> >> > > >> >> > Comparisons: > >> >> > https://files.lynne.ee/bwdif_01_before.png > >> >> > https://files.lynne.ee/bwdif_01_after.png > >> >> > Generated from sample 1 via: > >> >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf > >> >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png > >> >> > > >> >> > https://files.lynne.ee/bwdif_02_before.png > >> >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 > -i > >> >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y > >> >> <OUTPUT>.png > >> >> > > >> >> > >> >> Corrected links for the second sample: > >> >> > >> >> https://files.lynne.ee/bwdif_02_before.png > >> >> https://files.lynne.ee/bwdif_02_after.png > >> >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf > >> >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png > >> >> > >> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in > the > >> >> second sample. > >> >> > >> > > >> > Single png images are not way to prove something. > >> > > >> > Please provide videos and not just single file that exhibit this > issue. > >> > (Keep showing same file over and over is not going to help show that > it > >> > helps) > >> > > >> > Also how PSNR/SSIM/VMAF changes before after not just in single sample > >> but > >> > in more samples. > >> > > >> > >> I posted samples and instructions. Enough for a discussion. > >> I'm not posting gigabytes of uncompressed samples. > >> PSNR is irrelevant if there are visible artifacts. > >> > > > > Nope, you havent. > > > > Some strange samples. > > PSNR is relevant as also SSIM and VMAF. > > > > Fair enough, on a 4k60 game recording, with lots of text and movement, > ffmpeg -i game_4k60test.mkv -filter_complex "[0:0] split [t1] [t2] ; [t2] > interlace [t2] ; [t2] bwdif=mode=send_field [t2] ; [t1] [t2] ssim [t3]" > -map "[t3]" -f null - > > Before: > 0.990397 (20.175775) > After: > 0.990417 (20.184970) > Slightly higher, but not really significant. > > PSNR (average): > 32.711758 > vs > 32.704465 > Slightly lower, but not really significant. > > Not very conclusive, I guess I need to test it myself on this sample visually: https://media.xiph.org/video/derf/y4m/west_wind_easy_1080p.y4m > > > Ignoring this does not help project, but just force tyrannic behavior. > > > > Relax, I'm not going to make you write a stablediffusion filter source :) > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >
Am So., 2. Juli 2023 um 20:58 Uhr schrieb Lynne <dev@lynne.ee>: > Jul 2, 2023, 20:41 by tmundt75@gmail.com: > > > Am So., 2. Juli 2023 um 18:57 Uhr schrieb Lynne <dev@lynne.ee>: > > > >> Jul 2, 2023, 18:54 by dev@lynne.ee: > >> > >> > The issue is that clipping the interpolated temporal sample against > >> > the spatially predicted sample causes artifacts to appear. > >> > > >> > Discovered while writing the Vulkan version (where I omitted the > >> > same check). > >> > > >> > The clipping in the code is carried over from yadif. Removing the > >> > same code in yadif does not make any difference to the output. > >> > I think that the check was simply ill-adapted to the new prediction > >> > code and does more harm. > >> > > >> > I tested replacing the range clip with only an FFMAX, and only an > >> > FFMIN, but in both cases, artifacts still appeared. > >> > > >> > Test sample 1: > >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2: > >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv > >> > > >> > Command line: > >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v > >> rawvideo -y <OUTPUT>.nut > >> > Make sure to disable the assembly. > >> > > >> > Comparisons: > >> > https://files.lynne.ee/bwdif_01_before.png > >> > https://files.lynne.ee/bwdif_01_after.png > >> > Generated from sample 1 via: > >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf > >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png > >> > > >> > https://files.lynne.ee/bwdif_02_before.png > >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i > >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y > >> <OUTPUT>.png > >> > > >> > >> Corrected links for the second sample: > >> > >> https://files.lynne.ee/bwdif_02_before.png > >> https://files.lynne.ee/bwdif_02_after.png > >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf > >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png > >> > >> I'm sure I hit a newline. The artifacts are a lot more noticeable in the > >> second sample. > >> > > > > I developed the bwdif to achieve the best possible balance between speed > > and quality of all different image contents from the broadcast point of > > view. This includes moving video as well as moving and static graphic > > elements. Unfortunately, the improvement of one image content often leads > > to the degradation of another. The code you removed fundamentally > > stabilizes the static graphic elements. This outweighs the slightly more > > frequent artifacts in moving video considering the general purpose of the > > filter. > > > > Could you post examples? I've been unable to find any that look worse > with the patch. > Unfortunately, I no longer have most of the test material that I used years ago at the development of the bwdif. But on the quick I have this clip with an "Archiv" insert. With your patch the letters are jumping. Without your patch they stay static. https://www.dropbox.com/s/jzoezjbi3ho9nja/bwdif-test.mov?dl=1 ffmpeg.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1, scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21 "bwdif_original.mp4" ffmpeg_lynne_patch.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1, scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21 "bwdif_lynne_patch.mp4" https://www.dropbox.com/s/tonsomtkhyaha91/bwdif_original.mp4?dl=1 https://www.dropbox.com/s/aaj8o5yzlocu55z/bwdif_lynne_patch.mp4?dl=1 Regards, Thomas
Jul 4, 2023, 00:08 by tmundt75@gmail.com: > Am So., 2. Juli 2023 um 20:58 Uhr schrieb Lynne <dev@lynne.ee>: > >> Jul 2, 2023, 20:41 by tmundt75@gmail.com: >> >> > Am So., 2. Juli 2023 um 18:57 Uhr schrieb Lynne <dev@lynne.ee>: >> > >> >> Jul 2, 2023, 18:54 by dev@lynne.ee: >> >> >> >> > The issue is that clipping the interpolated temporal sample against >> >> > the spatially predicted sample causes artifacts to appear. >> >> > >> >> > Discovered while writing the Vulkan version (where I omitted the >> >> > same check). >> >> > >> >> > The clipping in the code is carried over from yadif. Removing the >> >> > same code in yadif does not make any difference to the output. >> >> > I think that the check was simply ill-adapted to the new prediction >> >> > code and does more harm. >> >> > >> >> > I tested replacing the range clip with only an FFMAX, and only an >> >> > FFMIN, but in both cases, artifacts still appeared. >> >> > >> >> > Test sample 1: >> >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2: >> >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv >> >> > >> >> > Command line: >> >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v >> >> rawvideo -y <OUTPUT>.nut >> >> > Make sure to disable the assembly. >> >> > >> >> > Comparisons: >> >> > https://files.lynne.ee/bwdif_01_before.png >> >> > https://files.lynne.ee/bwdif_01_after.png >> >> > Generated from sample 1 via: >> >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf >> >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png >> >> > >> >> > https://files.lynne.ee/bwdif_02_before.png >> >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i >> >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y >> >> <OUTPUT>.png >> >> > >> >> >> >> Corrected links for the second sample: >> >> >> >> https://files.lynne.ee/bwdif_02_before.png >> >> https://files.lynne.ee/bwdif_02_after.png >> >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf >> >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png >> >> >> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in the >> >> second sample. >> >> >> > >> > I developed the bwdif to achieve the best possible balance between speed >> > and quality of all different image contents from the broadcast point of >> > view. This includes moving video as well as moving and static graphic >> > elements. Unfortunately, the improvement of one image content often leads >> > to the degradation of another. The code you removed fundamentally >> > stabilizes the static graphic elements. This outweighs the slightly more >> > frequent artifacts in moving video considering the general purpose of the >> > filter. >> > >> >> Could you post examples? I've been unable to find any that look worse >> with the patch. >> > > Unfortunately, I no longer have most of the test material that I used years > ago at the development of the bwdif. > But on the quick I have this clip with an "Archiv" insert. With your patch > the letters are jumping. Without your patch they stay static. > https://www.dropbox.com/s/jzoezjbi3ho9nja/bwdif-test.mov?dl=1 > ffmpeg.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1, > scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21 > "bwdif_original.mp4" > ffmpeg_lynne_patch.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1, > scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21 > "bwdif_lynne_patch.mp4" > https://www.dropbox.com/s/tonsomtkhyaha91/bwdif_original.mp4?dl=1 > https://www.dropbox.com/s/aaj8o5yzlocu55z/bwdif_lynne_patch.mp4?dl=1 > Can confirm the letters are jumping with my patch. Fair enough, consider this patch dropped. I've added the check in Vulkan to make that version exact to C. Thanks for testing and writing the filter!
Lynne <dev@lynne.ee> schrieb am Di., 4. Juli 2023, 00:54: > Jul 4, 2023, 00:08 by tmundt75@gmail.com: > > > Am So., 2. Juli 2023 um 20:58 Uhr schrieb Lynne <dev@lynne.ee>: > > > >> Jul 2, 2023, 20:41 by tmundt75@gmail.com: > >> > >> > Am So., 2. Juli 2023 um 18:57 Uhr schrieb Lynne <dev@lynne.ee>: > >> > > >> >> Jul 2, 2023, 18:54 by dev@lynne.ee: > >> >> > >> >> > The issue is that clipping the interpolated temporal sample against > >> >> > the spatially predicted sample causes artifacts to appear. > >> >> > > >> >> > Discovered while writing the Vulkan version (where I omitted the > >> >> > same check). > >> >> > > >> >> > The clipping in the code is carried over from yadif. Removing the > >> >> > same code in yadif does not make any difference to the output. > >> >> > I think that the check was simply ill-adapted to the new prediction > >> >> > code and does more harm. > >> >> > > >> >> > I tested replacing the range clip with only an FFMAX, and only an > >> >> > FFMIN, but in both cases, artifacts still appeared. > >> >> > > >> >> > Test sample 1: > >> >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample > 2: > >> >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv > >> >> > > >> >> > Command line: > >> >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v > >> >> rawvideo -y <OUTPUT>.nut > >> >> > Make sure to disable the assembly. > >> >> > > >> >> > Comparisons: > >> >> > https://files.lynne.ee/bwdif_01_before.png > >> >> > https://files.lynne.ee/bwdif_01_after.png > >> >> > Generated from sample 1 via: > >> >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf > >> >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png > >> >> > > >> >> > https://files.lynne.ee/bwdif_02_before.png > >> >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 > -i > >> >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y > >> >> <OUTPUT>.png > >> >> > > >> >> > >> >> Corrected links for the second sample: > >> >> > >> >> https://files.lynne.ee/bwdif_02_before.png > >> >> https://files.lynne.ee/bwdif_02_after.png > >> >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf > >> >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png > >> >> > >> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in > the > >> >> second sample. > >> >> > >> > > >> > I developed the bwdif to achieve the best possible balance between > speed > >> > and quality of all different image contents from the broadcast point > of > >> > view. This includes moving video as well as moving and static graphic > >> > elements. Unfortunately, the improvement of one image content often > leads > >> > to the degradation of another. The code you removed fundamentally > >> > stabilizes the static graphic elements. This outweighs the slightly > more > >> > frequent artifacts in moving video considering the general purpose of > the > >> > filter. > >> > > >> > >> Could you post examples? I've been unable to find any that look worse > >> with the patch. > >> > > > > Unfortunately, I no longer have most of the test material that I used > years > > ago at the development of the bwdif. > > But on the quick I have this clip with an "Archiv" insert. With your > patch > > the letters are jumping. Without your patch they stay static. > > https://www.dropbox.com/s/jzoezjbi3ho9nja/bwdif-test.mov?dl=1 > > ffmpeg.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1, > > scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21 > > "bwdif_original.mp4" > > ffmpeg_lynne_patch.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1, > > scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21 > > "bwdif_lynne_patch.mp4" > > https://www.dropbox.com/s/tonsomtkhyaha91/bwdif_original.mp4?dl=1 > > https://www.dropbox.com/s/aaj8o5yzlocu55z/bwdif_lynne_patch.mp4?dl=1 > > > > Can confirm the letters are jumping with my patch. > Fair enough, consider this patch dropped. I've added the check > in Vulkan to make that version exact to C. > Thanks for testing and writing the filter! > Thanks >
From abda922dc040ce7237ad3048f55ab20b03f96a27 Mon Sep 17 00:00:00 2001 From: Lynne <dev@lynne.ee> Date: Sun, 2 Jul 2023 18:10:47 +0200 Subject: [PATCH] lavfi/bwdif: remove interpolated sample clipping The issue is that clipping the interpolated temporal sample against the spatially predicted sample causes artifacts. --- libavfilter/vf_bwdif.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index e278cf1217..55dec526bf 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -106,10 +106,6 @@ typedef struct ThreadData { interpol = (c + e) >> 1; #define FILTER2() \ - if (interpol > d + diff) \ - interpol = d + diff; \ - else if (interpol < d - diff) \ - interpol = d - diff; \ \ dst[0] = av_clip(interpol, 0, clip_max); \ } \ -- 2.40.1