| Message ID | 1520850268-32246-1-git-send-email-hazem.s.ashmawy@gmail.com |
|---|---|
| State | New |
| Headers | show |
On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: > Specifications can be found here: > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md > > Signed-off-by: Hazem Ashmawy <hazem.s.ashmawy@gmail.com> > --- > libavfilter/vf_panorama.c | 32 ++++++++++++++++---------------- > 1 file changed, 16 insertions(+), 16 deletions(-) > Please post whole patch, not just recent changes. Or just link to github branch.
On Mon, 12 Mar 2018 11:26:26 +0100 Paul B Mahol <onemda@gmail.com> wrote: > On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: > > Specifications can be found here: > > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md > > > > Signed-off-by: Hazem Ashmawy <hazem.s.ashmawy@gmail.com> > > --- > > libavfilter/vf_panorama.c | 32 ++++++++++++++++---------------- > > 1 file changed, 16 insertions(+), 16 deletions(-) > > > > Please post whole patch, not just recent changes. > > Or just link to github branch. Also, in comment on this whole issue: this is why it should use the libavutil spherical metadata, because then we have something rigorously defined. Just saying.
On 3/12/18, wm4 <nfxjfg@googlemail.com> wrote: > On Mon, 12 Mar 2018 11:26:26 +0100 > Paul B Mahol <onemda@gmail.com> wrote: > >> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >> > Specifications can be found here: >> > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md >> > >> > Signed-off-by: Hazem Ashmawy <hazem.s.ashmawy@gmail.com> >> > --- >> > libavfilter/vf_panorama.c | 32 ++++++++++++++++---------------- >> > 1 file changed, 16 insertions(+), 16 deletions(-) >> > >> >> Please post whole patch, not just recent changes. >> >> Or just link to github branch. > > Also, in comment on this whole issue: this is why it should use the > libavutil spherical metadata, because then we have something rigorously > defined. Just saying. I already wrote that can not be used, similar to stereo3d frame side data for another filter.
Sorry about that! Here is github branch https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube On 3/12/18, Paul B Mahol <onemda@gmail.com> wrote: > On 3/12/18, wm4 <nfxjfg@googlemail.com> wrote: >> On Mon, 12 Mar 2018 11:26:26 +0100 >> Paul B Mahol <onemda@gmail.com> wrote: >> >>> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >>> > Specifications can be found here: >>> > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md >>> > >>> > Signed-off-by: Hazem Ashmawy <hazem.s.ashmawy@gmail.com> >>> > --- >>> > libavfilter/vf_panorama.c | 32 ++++++++++++++++---------------- >>> > 1 file changed, 16 insertions(+), 16 deletions(-) >>> > >>> >>> Please post whole patch, not just recent changes. >>> >>> Or just link to github branch. >> >> Also, in comment on this whole issue: this is why it should use the >> libavutil spherical metadata, because then we have something rigorously >> defined. Just saying. > > I already wrote that can not be used, similar to stereo3d frame side > data for another filter. > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >
On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: > Sorry about that! Here is github branch > > https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube Good, now can you look at how to use bilinear instead of nearest interpolation for cubemap to equirectangular conversion. Nearest is extremly ugly. > > On 3/12/18, Paul B Mahol <onemda@gmail.com> wrote: >> On 3/12/18, wm4 <nfxjfg@googlemail.com> wrote: >>> On Mon, 12 Mar 2018 11:26:26 +0100 >>> Paul B Mahol <onemda@gmail.com> wrote: >>> >>>> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >>>> > Specifications can be found here: >>>> > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md >>>> > >>>> > Signed-off-by: Hazem Ashmawy <hazem.s.ashmawy@gmail.com> >>>> > --- >>>> > libavfilter/vf_panorama.c | 32 ++++++++++++++++---------------- >>>> > 1 file changed, 16 insertions(+), 16 deletions(-) >>>> > >>>> >>>> Please post whole patch, not just recent changes. >>>> >>>> Or just link to github branch. >>> >>> Also, in comment on this whole issue: this is why it should use the >>> libavutil spherical metadata, because then we have something rigorously >>> defined. Just saying. >> >> I already wrote that can not be used, similar to stereo3d frame side >> data for another filter. >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >
On 3/12/2018 7:48 AM, Paul B Mahol wrote: > On 3/12/18, wm4 <nfxjfg@googlemail.com> wrote: >> On Mon, 12 Mar 2018 11:26:26 +0100 >> Paul B Mahol <onemda@gmail.com> wrote: >> >>> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >>>> Specifications can be found here: >>>> https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md >>>> >>>> Signed-off-by: Hazem Ashmawy <hazem.s.ashmawy@gmail.com> >>>> --- >>>> libavfilter/vf_panorama.c | 32 ++++++++++++++++---------------- >>>> 1 file changed, 16 insertions(+), 16 deletions(-) >>>> >>> >>> Please post whole patch, not just recent changes. >>> >>> Or just link to github branch. >> >> Also, in comment on this whole issue: this is why it should use the >> libavutil spherical metadata, because then we have something rigorously >> defined. Just saying. > > I already wrote that can not be used, similar to stereo3d frame side > data for another filter. What are these avfilter limitations you speak of? The API for that matter follows the Google spec to the letter for Cubemap and Equirectangular projections. If they came up with some new projection definition not currently in the spec then they should be asked to make it official in said document, so we can implement it. Case in point the contents of this "ytmp" box in recent spherical mp4 streams they seem to be serving. In any case, please add a big TODO message to make sure this filter and the framework are eventually adapted to use the frame side data when available.
On 3/12/18, James Almer <jamrial@gmail.com> wrote: > On 3/12/2018 7:48 AM, Paul B Mahol wrote: >> On 3/12/18, wm4 <nfxjfg@googlemail.com> wrote: >>> On Mon, 12 Mar 2018 11:26:26 +0100 >>> Paul B Mahol <onemda@gmail.com> wrote: >>> >>>> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >>>>> Specifications can be found here: >>>>> https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md >>>>> >>>>> Signed-off-by: Hazem Ashmawy <hazem.s.ashmawy@gmail.com> >>>>> --- >>>>> libavfilter/vf_panorama.c | 32 ++++++++++++++++---------------- >>>>> 1 file changed, 16 insertions(+), 16 deletions(-) >>>>> >>>> >>>> Please post whole patch, not just recent changes. >>>> >>>> Or just link to github branch. >>> >>> Also, in comment on this whole issue: this is why it should use the >>> libavutil spherical metadata, because then we have something rigorously >>> defined. Just saying. >> >> I already wrote that can not be used, similar to stereo3d frame side >> data for another filter. > > What are these avfilter limitations you speak of? > > The API for that matter follows the Google spec to the letter for > Cubemap and Equirectangular projections. If they came up with some new > projection definition not currently in the spec then they should be > asked to make it official in said document, so we can implement it. Case > in point the contents of this "ytmp" box in recent spherical mp4 streams > they seem to be serving. > > In any case, please add a big TODO message to make sure this filter and > the framework are eventually adapted to use the frame side data when > available. Sure.
Paul B Mahol (2018-03-12): > > What are these avfilter limitations you speak of? > Sure. Is "sure" supposed to be an answer to that question? Regards,
On 3/12/18, Nicolas George <george@nsup.org> wrote: > Paul B Mahol (2018-03-12): >> > What are these avfilter limitations you speak of? > >> Sure. > > Is "sure" supposed to be an answer to that question? Look at thread for stereo3d video filter patch.
On 3/12/18, Paul B Mahol <onemda@gmail.com> wrote: > On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >> Sorry about that! Here is github branch >> >> https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube > > Good, now can you look at how to use bilinear instead of nearest > interpolation > for cubemap to equirectangular conversion. > > Nearest is extremly ugly. > Sure, will look into that. Just to take that into my consideration, why didn't you use it at the first place? Were there any obstacles? >> >> On 3/12/18, Paul B Mahol <onemda@gmail.com> wrote: >>> On 3/12/18, wm4 <nfxjfg@googlemail.com> wrote: >>>> On Mon, 12 Mar 2018 11:26:26 +0100 >>>> Paul B Mahol <onemda@gmail.com> wrote: >>>> >>>>> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >>>>> > Specifications can be found here: >>>>> > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md >>>>> > >>>>> > Signed-off-by: Hazem Ashmawy <hazem.s.ashmawy@gmail.com> >>>>> > --- >>>>> > libavfilter/vf_panorama.c | 32 ++++++++++++++++---------------- >>>>> > 1 file changed, 16 insertions(+), 16 deletions(-) >>>>> > >>>>> >>>>> Please post whole patch, not just recent changes. >>>>> >>>>> Or just link to github branch. >>>> >>>> Also, in comment on this whole issue: this is why it should use the >>>> libavutil spherical metadata, because then we have something rigorously >>>> defined. Just saying. >>> >>> I already wrote that can not be used, similar to stereo3d frame side >>> data for another filter. >>> _______________________________________________ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >
On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: > On 3/12/18, Paul B Mahol <onemda@gmail.com> wrote: >> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >>> Sorry about that! Here is github branch >>> >>> https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube >> >> Good, now can you look at how to use bilinear instead of nearest >> interpolation >> for cubemap to equirectangular conversion. >> >> Nearest is extremly ugly. >> > Sure, will look into that. Just to take that into my consideration, > why didn't you use it at the first place? Were there any obstacles? It was non-trivial to do.
On Mon, 12 Mar 2018 17:05:30 +0100 Paul B Mahol <onemda@gmail.com> wrote: > On 3/12/18, Nicolas George <george@nsup.org> wrote: > > Paul B Mahol (2018-03-12): > >> > What are these avfilter limitations you speak of? > > > >> Sure. > > > > Is "sure" supposed to be an answer to that question? > > Look at thread for stereo3d video filter patch. Right, it can't know the output format until it sees an AVFrame with the side data. But in libavfilter it needs to report the output format before any frame is input. Fixing it requires either adding side data to the input/output filter pad configuration, or dynamic reconfiguration at runtime.
So I'm still looking into using bilinear instead of nearest. Most of the online discussions are in the context of computer graphics and using things like openGL shaders. One solution I found and may try to implement is to add padding pixels around each face tile in the input frame. This should help in handling sampling near cube edges and corners. However, I'm not sure about the complexity of this or its effect on the performance. WDYT? Meanwhile this is a hacky/lazy way to use bilinear, but I can barely notice any improvements over using nearest by my bare eyes, are there any systematic way to evaluate the quality? https://github.com/HazemSamir/FFmpeg/commit/fcd82b9782b5f32072d9b0516b275fe3d4f4a163 On 3/12/18, Paul B Mahol <onemda@gmail.com> wrote: > On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >> On 3/12/18, Paul B Mahol <onemda@gmail.com> wrote: >>> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >>>> Sorry about that! Here is github branch >>>> >>>> https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube >>> >>> Good, now can you look at how to use bilinear instead of nearest >>> interpolation >>> for cubemap to equirectangular conversion. >>> >>> Nearest is extremly ugly. >>> >> Sure, will look into that. Just to take that into my consideration, >> why didn't you use it at the first place? Were there any obstacles? > > It was non-trivial to do. > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >
On 3/14/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: > So I'm still looking into using bilinear instead of nearest. Most of > the online discussions are in the context of computer graphics and > using things like openGL shaders. > > One solution I found and may try to implement is to add padding pixels > around each face tile in the input frame. This should help in handling > sampling near cube edges and corners. However, I'm not sure about the > complexity of this or its effect on the performance. WDYT? Either that or doing it hard way, by picking different pixels for each face. Anyway that (padding) solution doesn't sound that bad, and is much simpler than above mentioned alternative. > > Meanwhile this is a hacky/lazy way to use bilinear, but I can barely > notice any improvements > over using nearest by my bare eyes, are there any systematic way to > evaluate the quality? Not realy. One could compare with another projection conversion utility, there are at least two freely available AFAIK. > https://github.com/HazemSamir/FFmpeg/commit/fcd82b9782b5f32072d9b0516b275fe3d4f4a163 > > On 3/12/18, Paul B Mahol <onemda@gmail.com> wrote: >> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >>> On 3/12/18, Paul B Mahol <onemda@gmail.com> wrote: >>>> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >>>>> Sorry about that! Here is github branch >>>>> >>>>> https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube >>>> >>>> Good, now can you look at how to use bilinear instead of nearest >>>> interpolation >>>> for cubemap to equirectangular conversion. >>>> >>>> Nearest is extremly ugly. >>>> >>> Sure, will look into that. Just to take that into my consideration, >>> why didn't you use it at the first place? Were there any obstacles? >> >> It was non-trivial to do. >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >
Please find another implementation for bilinear https://github.com/HazemSamir/FFmpeg/commit/6a62f6db30bed37323d70eb7d66d6c33fa8f00c3 Please let me know your thoughts. On 3/14/18, Paul B Mahol <onemda@gmail.com> wrote: > On 3/14/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >> So I'm still looking into using bilinear instead of nearest. Most of >> the online discussions are in the context of computer graphics and >> using things like openGL shaders. >> >> One solution I found and may try to implement is to add padding pixels >> around each face tile in the input frame. This should help in handling >> sampling near cube edges and corners. However, I'm not sure about the >> complexity of this or its effect on the performance. WDYT? > > Either that or doing it hard way, by picking different pixels for each face. > Anyway that (padding) solution doesn't sound that bad, and is much simpler > than > above mentioned alternative. > >> >> Meanwhile this is a hacky/lazy way to use bilinear, but I can barely >> notice any improvements >> over using nearest by my bare eyes, are there any systematic way to >> evaluate the quality? > > Not realy. One could compare with another projection conversion utility, > there are at least two freely available AFAIK. > >> https://github.com/HazemSamir/FFmpeg/commit/fcd82b9782b5f32072d9b0516b275fe3d4f4a163 >> >> On 3/12/18, Paul B Mahol <onemda@gmail.com> wrote: >>> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >>>> On 3/12/18, Paul B Mahol <onemda@gmail.com> wrote: >>>>> On 3/12/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: >>>>>> Sorry about that! Here is github branch >>>>>> >>>>>> https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube >>>>> >>>>> Good, now can you look at how to use bilinear instead of nearest >>>>> interpolation >>>>> for cubemap to equirectangular conversion. >>>>> >>>>> Nearest is extremly ugly. >>>>> >>>> Sure, will look into that. Just to take that into my consideration, >>>> why didn't you use it at the first place? Were there any obstacles? >>> >>> It was non-trivial to do. >>> _______________________________________________ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >
On 4/1/18, Hazem Ashmawy <hazem.s.ashmawy@gmail.com> wrote: > Please find another implementation for bilinear > https://github.com/HazemSamir/FFmpeg/commit/6a62f6db30bed37323d70eb7d66d6c33fa8f00c3 > > Please let me know your thoughts. > Should be fine if it works.
diff --git a/libavfilter/vf_panorama.c b/libavfilter/vf_panorama.c index de08ef4..2fb30a1 100644 --- a/libavfilter/vf_panorama.c +++ b/libavfilter/vf_panorama.c @@ -34,12 +34,12 @@ enum Projections { }; enum Faces { - LEFT, - FRONT, RIGHT, + LEFT, TOP, - BACK, DOWN, + FRONT, + BACK, }; struct XYRemap { @@ -152,27 +152,27 @@ static void to_cube3x2_xyz(int i, int j, int face, double ew, double eh, if (face == BACK) { *x = -1 ; - *y = 3. - a; + *y = 5. - a; *z = 3. - b; } else if (face == LEFT) { - *x = a - 1; + *x = a - 3; *y = -1 ; *z = 1. - b; } else if (face == FRONT) { *x = 1 ; *y = a - 3; - *z = 1. - b; + *z = 3. - b; } else if (face == RIGHT) { - *x = 5. - a; + *x = 1. - a; *y = 1 ; *z = 1. - b; } else if (face == TOP) { - *x = b - 3; - *y = a - 1; + *x = b - 1; + *y = a - 5; *z = 1 ; } else if (face == DOWN) { *x = -b + 3; - *y = a - 5; + *y = a - 1; *z = -1 ; } } @@ -185,27 +185,27 @@ static void to_cube6x1_xyz(int i, int j, int face, double ew, double eh, if (face == BACK) { *x = -1 ; - *y = 9. - a; + *y = 11. - a; *z = 1. - b; } else if (face == LEFT) { - *x = a - 1; + *x = a - 3; *y = -1 ; *z = 1. - b; } else if (face == FRONT) { *x = 1 ; - *y = a - 3; + *y = a - 9; *z = 1. - b; } else if (face == RIGHT) { - *x = 5. - a; + *x = 1. - a; *y = 1 ; *z = 1. - b; } else if (face == TOP) { *x = b - 1; - *y = a - 7; + *y = a - 5; *z = 1 ; } else if (face == DOWN) { *x = -b + 1; - *y = a - 11; + *y = a - 7; *z = -1 ; } }
Specifications can be found here: https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md Signed-off-by: Hazem Ashmawy <hazem.s.ashmawy@gmail.com> --- libavfilter/vf_panorama.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-)