| Message ID | 20240605114702.71176-1-martin@martin.st |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel,v2] movenc: Add an option for hiding fragments at the end | expand |
| Context | Check | Description |
|---|---|---|
| andriy/configure_x86 | warning | Failed to apply patch |
On Wed, 5 Jun 2024, Martin Storsjö wrote: > This allows ending up with a normal, non-fragmented file when > the file is finished, while keeping the file readable if writing > is aborted abruptly at any point. (Normally when writing a > mov/mp4 file, the unfinished file is completely useless unless it > is finished properly.) > > This results in a file where the mdat atom contains (and hides) > all the moof atoms that were part of the fragmented file structure > initially. > --- > v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes > no sense without it). > > Updated the description of the flag to "Write a fragmented file that > is converted to non-fragmented at the end". > > Kept the flag named "hide_fragments", but I'm also pondering if we > maybe should go for a name like "hybrid_fragmented" or so, as a > better description of _what_ it produces, as opposed to _how_ it > does things. (One could also consider "hybrid_mp4", but even if mp4 > is the main thing, the same also goes for mov and a bunch of other > related formats.) I'd otherwise want to push this, but I'm not entirely satisfied with the option name quite yet. I'm pondering if we should call it "hybrid_fragmented" - any opinions, Dennis or Timo? // Martin
On 13/06/2024 14:50, Martin Storsjö wrote: > On Wed, 5 Jun 2024, Martin Storsjö wrote: > >> This allows ending up with a normal, non-fragmented file when >> the file is finished, while keeping the file readable if writing >> is aborted abruptly at any point. (Normally when writing a >> mov/mp4 file, the unfinished file is completely useless unless it >> is finished properly.) >> >> This results in a file where the mdat atom contains (and hides) >> all the moof atoms that were part of the fragmented file structure >> initially. >> --- >> v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes >> no sense without it). >> >> Updated the description of the flag to "Write a fragmented file that >> is converted to non-fragmented at the end". >> >> Kept the flag named "hide_fragments", but I'm also pondering if we >> maybe should go for a name like "hybrid_fragmented" or so, as a >> better description of _what_ it produces, as opposed to _how_ it >> does things. (One could also consider "hybrid_mp4", but even if mp4 >> is the main thing, the same also goes for mov and a bunch of other >> related formats.) > > I'd otherwise want to push this, but I'm not entirely satisfied with the > option name quite yet. I'm pondering if we should call it > "hybrid_fragmented" - any opinions, Dennis or Timo? It's at least in line with what OBS dubbed the same feature, so would allow consistent Search-Results. So it seems like a good compromise to me.
On 2024-06-13 06:20 pm, Martin Storsjö wrote: > On Wed, 5 Jun 2024, Martin Storsjö wrote: > >> This allows ending up with a normal, non-fragmented file when >> the file is finished, while keeping the file readable if writing >> is aborted abruptly at any point. (Normally when writing a >> mov/mp4 file, the unfinished file is completely useless unless it >> is finished properly.) >> >> This results in a file where the mdat atom contains (and hides) >> all the moof atoms that were part of the fragmented file structure >> initially. >> --- >> v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes >> no sense without it). >> >> Updated the description of the flag to "Write a fragmented file that >> is converted to non-fragmented at the end". >> >> Kept the flag named "hide_fragments", but I'm also pondering if we >> maybe should go for a name like "hybrid_fragmented" or so, as a >> better description of _what_ it produces, as opposed to _how_ it >> does things. (One could also consider "hybrid_mp4", but even if mp4 >> is the main thing, the same also goes for mov and a bunch of other >> related formats.) > > I'd otherwise want to push this, but I'm not entirely satisfied with > the option name quite yet. I'm pondering if we should call it > "hybrid_fragmented" - any opinions, Dennis or Timo? How about `resilient_mode` or `recoverable`? I agree that the how is secondary. Regards, Gyan
On 2024-06-13 14:50, Martin Storsjö wrote: > On Wed, 5 Jun 2024, Martin Storsjö wrote: > >> This allows ending up with a normal, non-fragmented file when >> the file is finished, while keeping the file readable if writing >> is aborted abruptly at any point. (Normally when writing a >> mov/mp4 file, the unfinished file is completely useless unless it >> is finished properly.) >> >> This results in a file where the mdat atom contains (and hides) >> all the moof atoms that were part of the fragmented file structure >> initially. >> --- >> v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes >> no sense without it). >> >> Updated the description of the flag to "Write a fragmented file that >> is converted to non-fragmented at the end". >> >> Kept the flag named "hide_fragments", but I'm also pondering if we >> maybe should go for a name like "hybrid_fragmented" or so, as a >> better description of _what_ it produces, as opposed to _how_ it >> does things. (One could also consider "hybrid_mp4", but even if mp4 >> is the main thing, the same also goes for mov and a bunch of other >> related formats.) > > I'd otherwise want to push this, but I'm not entirely satisfied with > the option name quite yet. I'm pondering if we should call it > "hybrid_fragmented" - any opinions, Dennis or Timo? > > // Martin That seems fine to me, but I might be biased considering that's what I called it :P ~Dennis
On Thu, 13 Jun 2024, Gyan Doshi wrote: > On 2024-06-13 06:20 pm, Martin Storsjö wrote: >> On Wed, 5 Jun 2024, Martin Storsjö wrote: >> >>> This allows ending up with a normal, non-fragmented file when >>> the file is finished, while keeping the file readable if writing >>> is aborted abruptly at any point. (Normally when writing a >>> mov/mp4 file, the unfinished file is completely useless unless it >>> is finished properly.) >>> >>> This results in a file where the mdat atom contains (and hides) >>> all the moof atoms that were part of the fragmented file structure >>> initially. >>> --- >>> v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes >>> no sense without it). >>> >>> Updated the description of the flag to "Write a fragmented file that >>> is converted to non-fragmented at the end". >>> >>> Kept the flag named "hide_fragments", but I'm also pondering if we >>> maybe should go for a name like "hybrid_fragmented" or so, as a >>> better description of _what_ it produces, as opposed to _how_ it >>> does things. (One could also consider "hybrid_mp4", but even if mp4 >>> is the main thing, the same also goes for mov and a bunch of other >>> related formats.) >> >> I'd otherwise want to push this, but I'm not entirely satisfied with >> the option name quite yet. I'm pondering if we should call it >> "hybrid_fragmented" - any opinions, Dennis or Timo? > > How about `resilient_mode` or `recoverable`? > I agree that the how is secondary. Those are good suggestions as well - but I think I prefer "hybrid_fragmented" still. In theory, I guess one could implement resilient writing in a number of different ways, whereas the hybrid fragmented/non-fragmented only is one. So with a couple other voices agreeing with the name "hybrid_fragmented", I'll post a new patch with the option in that form - hopefully you don't object to it. // Martin
On 2024-06-14 02:18 am, Martin Storsjö wrote: > On Thu, 13 Jun 2024, Gyan Doshi wrote: > >> On 2024-06-13 06:20 pm, Martin Storsjö wrote: >>> On Wed, 5 Jun 2024, Martin Storsjö wrote: >>> >>>> This allows ending up with a normal, non-fragmented file when >>>> the file is finished, while keeping the file readable if writing >>>> is aborted abruptly at any point. (Normally when writing a >>>> mov/mp4 file, the unfinished file is completely useless unless it >>>> is finished properly.) >>>> >>>> This results in a file where the mdat atom contains (and hides) >>>> all the moof atoms that were part of the fragmented file structure >>>> initially. >>>> --- >>>> v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes >>>> no sense without it). >>>> >>>> Updated the description of the flag to "Write a fragmented file that >>>> is converted to non-fragmented at the end". >>>> >>>> Kept the flag named "hide_fragments", but I'm also pondering if we >>>> maybe should go for a name like "hybrid_fragmented" or so, as a >>>> better description of _what_ it produces, as opposed to _how_ it >>>> does things. (One could also consider "hybrid_mp4", but even if mp4 >>>> is the main thing, the same also goes for mov and a bunch of other >>>> related formats.) >>> >>> I'd otherwise want to push this, but I'm not entirely satisfied with >>> the option name quite yet. I'm pondering if we should call it >>> "hybrid_fragmented" - any opinions, Dennis or Timo? >> >> How about `resilient_mode` or `recoverable`? >> I agree that the how is secondary. > > Those are good suggestions as well - but I think I prefer > "hybrid_fragmented" still. > > In theory, I guess one could implement resilient writing in a number > of different ways, whereas the hybrid fragmented/non-fragmented only > is one. > > So with a couple other voices agreeing with the name > "hybrid_fragmented", I'll post a new patch with the option in that > form - hopefully you don't object to it. The term hybrid is not applicable here. The fragmented state is transient during writing and contingent in the finished artifact depending on how the writing process concluded. Hybrid implies both modes available e.g.. a hybrid vehicle can use both types of energy sources. The artifact here will be one _or_ the other. Regards, Gyan
On Fri, 14 Jun 2024, Gyan Doshi wrote: > On 2024-06-14 02:18 am, Martin Storsjö wrote: >> On Thu, 13 Jun 2024, Gyan Doshi wrote: >> >>> On 2024-06-13 06:20 pm, Martin Storsjö wrote: >>>> >>>> I'd otherwise want to push this, but I'm not entirely satisfied with >>>> the option name quite yet. I'm pondering if we should call it >>>> "hybrid_fragmented" - any opinions, Dennis or Timo? >>> >>> How about `resilient_mode` or `recoverable`? >>> I agree that the how is secondary. >> >> Those are good suggestions as well - but I think I prefer >> "hybrid_fragmented" still. >> >> In theory, I guess one could implement resilient writing in a number >> of different ways, whereas the hybrid fragmented/non-fragmented only >> is one. >> >> So with a couple other voices agreeing with the name >> "hybrid_fragmented", I'll post a new patch with the option in that >> form - hopefully you don't object to it. > > The term hybrid is not applicable here. The fragmented state is > transient during writing and contingent in the finished artifact > depending on how the writing process concluded. > Hybrid implies both modes available e.g.. a hybrid vehicle can use both > types of energy sources. The artifact here will be one _or_ the other. Sure, the file itself is either or, but the process of writing will have utilized both. TBH, I don't see it as such a black-or-white thing. What do the others who have chimed in on the thread think, compared to calling it "recoverable" or "resilient_mode"? // Martin
On 14/06/2024 12:44, Martin Storsjö wrote: > On Fri, 14 Jun 2024, Gyan Doshi wrote: > >> On 2024-06-14 02:18 am, Martin Storsjö wrote: >>> On Thu, 13 Jun 2024, Gyan Doshi wrote: >>> >>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote: >>>>> >>>>> I'd otherwise want to push this, but I'm not entirely satisfied >>>>> with the option name quite yet. I'm pondering if we should call it >>>>> "hybrid_fragmented" - any opinions, Dennis or Timo? >>>> >>>> How about `resilient_mode` or `recoverable`? >>>> I agree that the how is secondary. >>> >>> Those are good suggestions as well - but I think I prefer >>> "hybrid_fragmented" still. >>> >>> In theory, I guess one could implement resilient writing in a number >>> of different ways, whereas the hybrid fragmented/non-fragmented only >>> is one. >>> >>> So with a couple other voices agreeing with the name >>> "hybrid_fragmented", I'll post a new patch with the option in that >>> form - hopefully you don't object to it. >> >> The term hybrid is not applicable here. The fragmented state is >> transient during writing and contingent in the finished artifact >> depending on how the writing process concluded. >> Hybrid implies both modes available e.g.. a hybrid vehicle can use >> both types of energy sources. The artifact here will be one _or_ the >> other. > > Sure, the file itself is either or, but the process of writing will have > utilized both. TBH, I don't see it as such a black-or-white thing. > > What do the others who have chimed in on the thread think, compared to > calling it "recoverable" or "resilient_mode"? I don't have a super strong opinion on it, but out of the options provided, I'd prefer the hybrid_ one, since there's a good chance it'll become an established term now that OBS presents it quite publicly visible.
On Fri, 14 Jun 2024, Timo Rothenpieler wrote: > On 14/06/2024 12:44, Martin Storsjö wrote: >> On Fri, 14 Jun 2024, Gyan Doshi wrote: >> >>> On 2024-06-14 02:18 am, Martin Storsjö wrote: >>>> On Thu, 13 Jun 2024, Gyan Doshi wrote: >>>> >>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote: >>>>>> >>>>>> I'd otherwise want to push this, but I'm not entirely satisfied >>>>>> with the option name quite yet. I'm pondering if we should call it >>>>>> "hybrid_fragmented" - any opinions, Dennis or Timo? >>>>> >>>>> How about `resilient_mode` or `recoverable`? >>>>> I agree that the how is secondary. >>>> >>>> Those are good suggestions as well - but I think I prefer >>>> "hybrid_fragmented" still. >>>> >>>> In theory, I guess one could implement resilient writing in a number >>>> of different ways, whereas the hybrid fragmented/non-fragmented only >>>> is one. >>>> >>>> So with a couple other voices agreeing with the name >>>> "hybrid_fragmented", I'll post a new patch with the option in that >>>> form - hopefully you don't object to it. >>> >>> The term hybrid is not applicable here. The fragmented state is >>> transient during writing and contingent in the finished artifact >>> depending on how the writing process concluded. >>> Hybrid implies both modes available e.g.. a hybrid vehicle can use >>> both types of energy sources. The artifact here will be one _or_ the >>> other. >> >> Sure, the file itself is either or, but the process of writing will have >> utilized both. TBH, I don't see it as such a black-or-white thing. >> >> What do the others who have chimed in on the thread think, compared to >> calling it "recoverable" or "resilient_mode"? > > I don't have a super strong opinion on it, but out of the options > provided, I'd prefer the hybrid_ one, since there's a good chance it'll > become an established term now that OBS presents it quite publicly visible. For context, the same feature in OBS, is documented at e.g. https://obsproject.com/kb/hybrid-mp4. // Martin
On 2024-06-14 04:35 pm, Timo Rothenpieler wrote: > On 14/06/2024 12:44, Martin Storsjö wrote: >> On Fri, 14 Jun 2024, Gyan Doshi wrote: >> >>> On 2024-06-14 02:18 am, Martin Storsjö wrote: >>>> On Thu, 13 Jun 2024, Gyan Doshi wrote: >>>> >>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote: >>>>>> >>>>>> I'd otherwise want to push this, but I'm not entirely satisfied >>>>>> with the option name quite yet. I'm pondering if we should call >>>>>> it "hybrid_fragmented" - any opinions, Dennis or Timo? >>>>> >>>>> How about `resilient_mode` or `recoverable`? >>>>> I agree that the how is secondary. >>>> >>>> Those are good suggestions as well - but I think I prefer >>>> "hybrid_fragmented" still. >>>> >>>> In theory, I guess one could implement resilient writing in a >>>> number of different ways, whereas the hybrid >>>> fragmented/non-fragmented only is one. >>>> >>>> So with a couple other voices agreeing with the name >>>> "hybrid_fragmented", I'll post a new patch with the option in that >>>> form - hopefully you don't object to it. >>> >>> The term hybrid is not applicable here. The fragmented state is >>> transient during writing and contingent in the finished artifact >>> depending on how the writing process concluded. >>> Hybrid implies both modes available e.g.. a hybrid vehicle can use >>> both types of energy sources. The artifact here will be one _or_ the >>> other. >> >> Sure, the file itself is either or, but the process of writing will >> have utilized both. TBH, I don't see it as such a black-or-white thing. >> >> What do the others who have chimed in on the thread think, compared >> to calling it "recoverable" or "resilient_mode"? > > I don't have a super strong opinion on it, but out of the options > provided, I'd prefer the hybrid_ one, since there's a good chance > it'll become an established term now that OBS presents it quite > publicly visible. The OBS dev intends to change the term: "Come up with a better name than "Hybrid MP4" that hopefully won't confuse users" https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024 Regards, Gyan
On 2024-06-14 13:23, Gyan Doshi wrote: > > > On 2024-06-14 04:35 pm, Timo Rothenpieler wrote: >> On 14/06/2024 12:44, Martin Storsjö wrote: >>> On Fri, 14 Jun 2024, Gyan Doshi wrote: >>> >>>> On 2024-06-14 02:18 am, Martin Storsjö wrote: >>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote: >>>>> >>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote: >>>>>>> >>>>>>> I'd otherwise want to push this, but I'm not entirely satisfied >>>>>>> with the option name quite yet. I'm pondering if we should call >>>>>>> it "hybrid_fragmented" - any opinions, Dennis or Timo? >>>>>> >>>>>> How about `resilient_mode` or `recoverable`? >>>>>> I agree that the how is secondary. >>>>> >>>>> Those are good suggestions as well - but I think I prefer >>>>> "hybrid_fragmented" still. >>>>> >>>>> In theory, I guess one could implement resilient writing in a >>>>> number of different ways, whereas the hybrid >>>>> fragmented/non-fragmented only is one. >>>>> >>>>> So with a couple other voices agreeing with the name >>>>> "hybrid_fragmented", I'll post a new patch with the option in that >>>>> form - hopefully you don't object to it. >>>> >>>> The term hybrid is not applicable here. The fragmented state is >>>> transient during writing and contingent in the finished artifact >>>> depending on how the writing process concluded. >>>> Hybrid implies both modes available e.g.. a hybrid vehicle can use >>>> both types of energy sources. The artifact here will be one _or_ >>>> the other. >>> >>> Sure, the file itself is either or, but the process of writing will >>> have utilized both. TBH, I don't see it as such a black-or-white thing. >>> >>> What do the others who have chimed in on the thread think, compared >>> to calling it "recoverable" or "resilient_mode"? >> >> I don't have a super strong opinion on it, but out of the options >> provided, I'd prefer the hybrid_ one, since there's a good chance >> it'll become an established term now that OBS presents it quite >> publicly visible. > > The OBS dev intends to change the term: > > "Come up with a better name than "Hybrid MP4" that hopefully won't > confuse users" > https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024 > > > Regards, > Gyan Now that it's merged and in the hands of users I don't have any intention of changing the name any more. We had some chats about about it, but nobody suggested anything that people agreed was better, so it stuck. While "resilient" certainly fits, it could equally apply to regular fragmented MP4 (e.g. vMix uses that terminology for fMP4 if I'm not mistaken). The important attribute with this approach is that it's resilient *and* compatible, and I'm still not sure how to get that across in name alone. Obviously FFmpeg has a different target demographic and can use something different than OBS, "Hybrid MP4" is just what I came up with as user-facing name for the feature at the time. ~Dennis
On 2024-06-15 03:54 am, Dennis Sädtler via ffmpeg-devel wrote: > On 2024-06-14 13:23, Gyan Doshi wrote: >> >> >> On 2024-06-14 04:35 pm, Timo Rothenpieler wrote: >>> On 14/06/2024 12:44, Martin Storsjö wrote: >>>> On Fri, 14 Jun 2024, Gyan Doshi wrote: >>>> >>>>> On 2024-06-14 02:18 am, Martin Storsjö wrote: >>>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote: >>>>>> >>>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote: >>>>>>>> >>>>>>>> I'd otherwise want to push this, but I'm not entirely satisfied >>>>>>>> with the option name quite yet. I'm pondering if we should call >>>>>>>> it "hybrid_fragmented" - any opinions, Dennis or Timo? >>>>>>> >>>>>>> How about `resilient_mode` or `recoverable`? >>>>>>> I agree that the how is secondary. >>>>>> >>>>>> Those are good suggestions as well - but I think I prefer >>>>>> "hybrid_fragmented" still. >>>>>> >>>>>> In theory, I guess one could implement resilient writing in a >>>>>> number of different ways, whereas the hybrid >>>>>> fragmented/non-fragmented only is one. >>>>>> >>>>>> So with a couple other voices agreeing with the name >>>>>> "hybrid_fragmented", I'll post a new patch with the option in >>>>>> that form - hopefully you don't object to it. >>>>> >>>>> The term hybrid is not applicable here. The fragmented state is >>>>> transient during writing and contingent in the finished artifact >>>>> depending on how the writing process concluded. >>>>> Hybrid implies both modes available e.g.. a hybrid vehicle can use >>>>> both types of energy sources. The artifact here will be one _or_ >>>>> the other. >>>> >>>> Sure, the file itself is either or, but the process of writing will >>>> have utilized both. TBH, I don't see it as such a black-or-white >>>> thing. >>>> >>>> What do the others who have chimed in on the thread think, compared >>>> to calling it "recoverable" or "resilient_mode"? >>> >>> I don't have a super strong opinion on it, but out of the options >>> provided, I'd prefer the hybrid_ one, since there's a good chance >>> it'll become an established term now that OBS presents it quite >>> publicly visible. >> >> The OBS dev intends to change the term: >> >> "Come up with a better name than "Hybrid MP4" that hopefully won't >> confuse users" >> https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024 >> >> >> Regards, >> Gyan > > Now that it's merged and in the hands of users I don't have any > intention of changing the name any more. > We had some chats about about it, but nobody suggested anything that > people agreed was better, so it stuck. > > While "resilient" certainly fits, it could equally apply to regular > fragmented MP4 (e.g. vMix uses that terminology for fMP4 if I'm not > mistaken). > The important attribute with this approach is that it's resilient > *and* compatible, and I'm still not sure how to get that across in > name alone. How about `failsafe`? Regards, Gyan
On Sat, 15 Jun 2024, Gyan Doshi wrote: > On 2024-06-15 03:54 am, Dennis Sädtler via ffmpeg-devel wrote: >> On 2024-06-14 13:23, Gyan Doshi wrote: >>> >>> On 2024-06-14 04:35 pm, Timo Rothenpieler wrote: >>>> On 14/06/2024 12:44, Martin Storsjö wrote: >>>>> On Fri, 14 Jun 2024, Gyan Doshi wrote: >>>>> >>>>>> On 2024-06-14 02:18 am, Martin Storsjö wrote: >>>>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote: >>>>>>> >>>>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote: >>>>>>>>> >>>>>>>>> I'd otherwise want to push this, but I'm not entirely satisfied >>>>>>>>> with the option name quite yet. I'm pondering if we should call >>>>>>>>> it "hybrid_fragmented" - any opinions, Dennis or Timo? >>>>>>>> >>>>>>>> How about `resilient_mode` or `recoverable`? >>>>>>>> I agree that the how is secondary. >>>>>>> >>>>>>> Those are good suggestions as well - but I think I prefer >>>>>>> "hybrid_fragmented" still. >>>>>>> >>>>>>> In theory, I guess one could implement resilient writing in a >>>>>>> number of different ways, whereas the hybrid >>>>>>> fragmented/non-fragmented only is one. >>>>>>> >>>>>>> So with a couple other voices agreeing with the name >>>>>>> "hybrid_fragmented", I'll post a new patch with the option in >>>>>>> that form - hopefully you don't object to it. >>>>>> >>>>>> The term hybrid is not applicable here. The fragmented state is >>>>>> transient during writing and contingent in the finished artifact >>>>>> depending on how the writing process concluded. >>>>>> Hybrid implies both modes available e.g.. a hybrid vehicle can use >>>>>> both types of energy sources. The artifact here will be one _or_ >>>>>> the other. >>>>> >>>>> Sure, the file itself is either or, but the process of writing will >>>>> have utilized both. TBH, I don't see it as such a black-or-white >>>>> thing. >>>>> >>>>> What do the others who have chimed in on the thread think, compared >>>>> to calling it "recoverable" or "resilient_mode"? >>>> >>>> I don't have a super strong opinion on it, but out of the options >>>> provided, I'd prefer the hybrid_ one, since there's a good chance >>>> it'll become an established term now that OBS presents it quite >>>> publicly visible. >>> >>> The OBS dev intends to change the term: >>> >>> "Come up with a better name than "Hybrid MP4" that hopefully won't >>> confuse users" >>> > https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024 >>> >>> >>> Regards, >>> Gyan >> >> Now that it's merged and in the hands of users I don't have any >> intention of changing the name any more. >> We had some chats about about it, but nobody suggested anything that >> people agreed was better, so it stuck. >> >> While "resilient" certainly fits, it could equally apply to regular >> fragmented MP4 (e.g. vMix uses that terminology for fMP4 if I'm not >> mistaken). >> The important attribute with this approach is that it's resilient >> *and* compatible, and I'm still not sure how to get that across in >> name alone. > > How about `failsafe`? I don't see how that differs from "resilient", as a regular fragmented file also is failsafe (or resilient) in the same way - while the special thing here is that it's both fragmented and not. // Martin
On 2024-06-17 04:08 pm, Martin Storsjö wrote: > On Sat, 15 Jun 2024, Gyan Doshi wrote: > >> On 2024-06-15 03:54 am, Dennis Sädtler via ffmpeg-devel wrote: >>> On 2024-06-14 13:23, Gyan Doshi wrote: >>>> >>>> On 2024-06-14 04:35 pm, Timo Rothenpieler wrote: >>>>> On 14/06/2024 12:44, Martin Storsjö wrote: >>>>>> On Fri, 14 Jun 2024, Gyan Doshi wrote: >>>>>> >>>>>>> On 2024-06-14 02:18 am, Martin Storsjö wrote: >>>>>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote: >>>>>>>> >>>>>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote: >>>>>>>>>> >>>>>>>>>> I'd otherwise want to push this, but I'm not entirely >>>>>>>>>> satisfied with the option name quite yet. I'm pondering if we >>>>>>>>>> should call it "hybrid_fragmented" - any opinions, Dennis or >>>>>>>>>> Timo? >>>>>>>>> >>>>>>>>> How about `resilient_mode` or `recoverable`? >>>>>>>>> I agree that the how is secondary. >>>>>>>> >>>>>>>> Those are good suggestions as well - but I think I prefer >>>>>>>> "hybrid_fragmented" still. >>>>>>>> >>>>>>>> In theory, I guess one could implement resilient writing in a >>>>>>>> number of different ways, whereas the hybrid >>>>>>>> fragmented/non-fragmented only is one. >>>>>>>> >>>>>>>> So with a couple other voices agreeing with the name >>>>>>>> "hybrid_fragmented", I'll post a new patch with the option in >>>>>>>> that form - hopefully you don't object to it. >>>>>>> >>>>>>> The term hybrid is not applicable here. The fragmented state is >>>>>>> transient during writing and contingent in the finished artifact >>>>>>> depending on how the writing process concluded. >>>>>>> Hybrid implies both modes available e.g.. a hybrid vehicle can >>>>>>> use both types of energy sources. The artifact here will be one >>>>>>> _or_ the other. >>>>>> >>>>>> Sure, the file itself is either or, but the process of writing >>>>>> will have utilized both. TBH, I don't see it as such a >>>>>> black-or-white thing. >>>>>> >>>>>> What do the others who have chimed in on the thread think, >>>>>> compared to calling it "recoverable" or "resilient_mode"? >>>>> >>>>> I don't have a super strong opinion on it, but out of the options >>>>> provided, I'd prefer the hybrid_ one, since there's a good chance >>>>> it'll become an established term now that OBS presents it quite >>>>> publicly visible. >>>> >>>> The OBS dev intends to change the term: >>>> >>>> "Come up with a better name than "Hybrid MP4" that hopefully won't >>>> confuse users" >>>> >> https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024 >> >>>> >>>> >>>> Regards, >>>> Gyan >>> >>> Now that it's merged and in the hands of users I don't have any >>> intention of changing the name any more. >>> We had some chats about about it, but nobody suggested anything that >>> people agreed was better, so it stuck. >>> >>> While "resilient" certainly fits, it could equally apply to regular >>> fragmented MP4 (e.g. vMix uses that terminology for fMP4 if I'm not >>> mistaken). >>> The important attribute with this approach is that it's resilient >>> *and* compatible, and I'm still not sure how to get that across in >>> name alone. >> >> How about `failsafe`? > > I don't see how that differs from "resilient", as a regular fragmented > file also is failsafe (or resilient) in the same way - while the > special thing here is that it's both fragmented and not. The expert user already knows to save a fragmented file if they want resilience. This option saves them a remux step if the original writing ends gracefully. For all other users, the value proposition _is_ the resilience. If the muxing ends normally, they just have a normal file. If it ends prematurely, they just want to be able to convert to a regular seekable MP4. The fact that it is saved in fragmented or any other mode is irrelevant - an academic detail at best. Ultimately, as long as the doc is clear about what the use of this option is, and what to do next if the muxing does abort, it should not matter too much what the option is called. But just like the faststart flag name identifies the purpose instead of being called something like moov_in_front, hopefully so will the name here. Regards, Gyan
On Mon, 17 Jun 2024, Gyan Doshi via ffmpeg-devel wrote: > On 2024-06-17 04:08 pm, Martin Storsjö wrote: >> On Sat, 15 Jun 2024, Gyan Doshi wrote: >> >>> On 2024-06-15 03:54 am, Dennis Sädtler via ffmpeg-devel wrote: >>>> On 2024-06-14 13:23, Gyan Doshi wrote: >>>>> >>>>> On 2024-06-14 04:35 pm, Timo Rothenpieler wrote: >>>>>> On 14/06/2024 12:44, Martin Storsjö wrote: >>>>>>> On Fri, 14 Jun 2024, Gyan Doshi wrote: >>>>>>> >>>>>>>> On 2024-06-14 02:18 am, Martin Storsjö wrote: >>>>>>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote: >>>>>>>>> >>>>>>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote: >>>>>>>>>>> >>>>>>>>>>> I'd otherwise want to push this, but I'm not entirely >>>>>>>>>>> satisfied with the option name quite yet. I'm pondering if we >>>>>>>>>>> should call it "hybrid_fragmented" - any opinions, Dennis or >>>>>>>>>>> Timo? >>>>>>>>>> >>>>>>>>>> How about `resilient_mode` or `recoverable`? >>>>>>>>>> I agree that the how is secondary. >>>>>>>>> >>>>>>>>> Those are good suggestions as well - but I think I prefer >>>>>>>>> "hybrid_fragmented" still. >>>>>>>>> >>>>>>>>> In theory, I guess one could implement resilient writing in a >>>>>>>>> number of different ways, whereas the hybrid >>>>>>>>> fragmented/non-fragmented only is one. >>>>>>>>> >>>>>>>>> So with a couple other voices agreeing with the name >>>>>>>>> "hybrid_fragmented", I'll post a new patch with the option in >>>>>>>>> that form - hopefully you don't object to it. >>>>>>>> >>>>>>>> The term hybrid is not applicable here. The fragmented state is >>>>>>>> transient during writing and contingent in the finished artifact >>>>>>>> depending on how the writing process concluded. >>>>>>>> Hybrid implies both modes available e.g.. a hybrid vehicle can >>>>>>>> use both types of energy sources. The artifact here will be one >>>>>>>> _or_ the other. >>>>>>> >>>>>>> Sure, the file itself is either or, but the process of writing >>>>>>> will have utilized both. TBH, I don't see it as such a >>>>>>> black-or-white thing. >>>>>>> >>>>>>> What do the others who have chimed in on the thread think, >>>>>>> compared to calling it "recoverable" or "resilient_mode"? >>>>>> >>>>>> I don't have a super strong opinion on it, but out of the options >>>>>> provided, I'd prefer the hybrid_ one, since there's a good chance >>>>>> it'll become an established term now that OBS presents it quite >>>>>> publicly visible. >>>>> >>>>> The OBS dev intends to change the term: >>>>> >>>>> "Come up with a better name than "Hybrid MP4" that hopefully won't >>>>> confuse users" >>>>> >>> > https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024 >>> >>>>> >>>>> >>>>> Regards, >>>>> Gyan >>>> >>>> Now that it's merged and in the hands of users I don't have any >>>> intention of changing the name any more. >>>> We had some chats about about it, but nobody suggested anything that >>>> people agreed was better, so it stuck. >>>> >>>> While "resilient" certainly fits, it could equally apply to regular >>>> fragmented MP4 (e.g. vMix uses that terminology for fMP4 if I'm not >>>> mistaken). >>>> The important attribute with this approach is that it's resilient >>>> *and* compatible, and I'm still not sure how to get that across in >>>> name alone. >>> >>> How about `failsafe`? >> >> I don't see how that differs from "resilient", as a regular fragmented >> file also is failsafe (or resilient) in the same way - while the >> special thing here is that it's both fragmented and not. > > The expert user already knows to save a fragmented file if they want > resilience. This option saves them a remux step if the original writing > ends gracefully. > For all other users, the value proposition _is_ the resilience. If the > muxing ends normally, they just have a normal file. If it ends > prematurely, they just want to be able to convert to a regular seekable > MP4. The fact that it is saved in fragmented or any other mode is > irrelevant - an academic detail at best. Sure > Ultimately, as long as the doc is clear about what the use of this > option is, and what to do next if the muxing does abort, it should not > matter too much what the option is called. So, are you saying you'd accept the hybrid_fragmented name, as long as the docs explain this correctly? > But just like the faststart flag name identifies the purpose instead of > being called something like moov_in_front, hopefully so will the name > here. TBH, the "faststart" name isn't necessarily the best either - although it's clearly better than moov_in_front though. But here, the name "faststart" was already established for the concept of this kind of operation, from the "qt-faststart" tool before it was integrated into the muxer. And here, we seem to begin to have some sort of similar establishing of "hybrid" as name for this kind of operation. But if this really is inacceptable to you, I guess all others who support that name, will have to figure out which of the alternate names we prefer. // Martin
On 2024-06-19 06:04 pm, Martin Storsjö wrote: > On Mon, 17 Jun 2024, Gyan Doshi via ffmpeg-devel wrote: > > >> Ultimately, as long as the doc is clear about what the use of this >> option is, and what to do next if the muxing does abort, it should >> not matter too much what the option is called. > > So, are you saying you'd accept the hybrid_fragmented name, as long as > the docs explain this correctly? Yes, Regards, Gyan
diff --git a/doc/muxers.texi b/doc/muxers.texi index 6340c8e54d..e313b5e631 100644 --- a/doc/muxers.texi +++ b/doc/muxers.texi @@ -569,6 +569,13 @@ experimental, may be renamed or changed, do not use from scripts. @item write_gama write deprecated gama atom + +@item hide_fragments +After writing a fragmented file, convert it to a regular, non-fragmented +file at the end. This keeps the file readable while it is being +written, and makes it recoverable if the process of writing the file +gets aborted uncleanly, while still producing an easily seekable +and widely compatible non-fragmented file in the end. @end table @item movie_timescale @var{scale} diff --git a/libavformat/movenc.c b/libavformat/movenc.c index d1517870fc..da16a9dae5 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -110,6 +110,7 @@ static const AVOption options[] = { { "use_metadata_tags", "Use mdta atom for metadata.", 0, AV_OPT_TYPE_CONST, {.i64 = FF_MOV_FLAG_USE_MDTA}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, .unit = "movflags" }, { "write_colr", "Write colr atom even if the color info is unspecified (Experimental, may be renamed or changed, do not use from scripts)", 0, AV_OPT_TYPE_CONST, {.i64 = FF_MOV_FLAG_WRITE_COLR}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, .unit = "movflags" }, { "write_gama", "Write deprecated gama atom", 0, AV_OPT_TYPE_CONST, {.i64 = FF_MOV_FLAG_WRITE_GAMA}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, .unit = "movflags" }, + { "hide_fragments", "Write a fragmented file that is converted to non-fragmented at the end", 0, AV_OPT_TYPE_CONST, {.i64 = FF_MOV_FLAG_HIDE_FRAGMENTS}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, .unit = "movflags" }, { "min_frag_duration", "Minimum fragment duration", offsetof(MOVMuxContext, min_fragment_duration), AV_OPT_TYPE_INT, {.i64 = 0}, 0, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM}, { "mov_gamma", "gamma value for gama atom", offsetof(MOVMuxContext, gamma), AV_OPT_TYPE_FLOAT, {.dbl = 0.0 }, 0.0, 10, AV_OPT_FLAG_ENCODING_PARAM}, { "movie_timescale", "set movie timescale", offsetof(MOVMuxContext, movie_timescale), AV_OPT_TYPE_INT, {.i64 = MOV_TIMESCALE}, 1, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM}, @@ -5993,10 +5994,30 @@ static int mov_write_squashed_packets(AVFormatContext *s) return 0; } -static int mov_finish_fragment(MOVTrack *track) +static int mov_finish_fragment(MOVMuxContext *mov, MOVTrack *track, + int64_t ref_pos) { + int i; if (!track->entry) return 0; + if (mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS) { + for (i = 0; i < track->entry; i++) + track->cluster[i].pos += ref_pos + track->data_offset; + if (track->cluster_written == 0 && !(mov->flags & FF_MOV_FLAG_EMPTY_MOOV)) { + // First flush. If this was a case of not using empty moov, reset chunking. + for (i = 0; i < track->entry; i++) { + track->cluster[i].chunkNum = 0; + track->cluster[i].samples_in_chunk = track->cluster[i].entries; + } + } + if (av_reallocp_array(&track->cluster_written, + track->entry_written + track->entry, + sizeof(*track->cluster))) + return AVERROR(ENOMEM); + memcpy(&track->cluster_written[track->entry_written], + track->cluster, track->entry * sizeof(*track->cluster)); + track->entry_written += track->entry; + } track->entry = 0; track->entries_flushed = 0; track->end_reliable = 0; @@ -6007,7 +6028,7 @@ static int mov_flush_fragment(AVFormatContext *s, int force) { MOVMuxContext *mov = s->priv_data; int i, first_track = -1; - int64_t mdat_size = 0; + int64_t mdat_size = 0, mdat_start = 0; int ret; int has_video = 0, starts_with_key = 0, first_video_track = 1; @@ -6113,7 +6134,7 @@ static int mov_flush_fragment(AVFormatContext *s, int force) mov->moov_written = 1; mov->mdat_size = 0; for (i = 0; i < mov->nb_tracks; i++) - mov_finish_fragment(&mov->tracks[i]); + mov_finish_fragment(mov, &mov->tracks[i], 0); avio_write_marker(s->pb, AV_NOPTS_VALUE, AVIO_DATA_MARKER_FLUSH_POINT); return 0; } @@ -6182,9 +6203,10 @@ static int mov_flush_fragment(AVFormatContext *s, int force) avio_wb32(s->pb, mdat_size + 8); ffio_wfourcc(s->pb, "mdat"); + mdat_start = avio_tell(s->pb); } - mov_finish_fragment(&mov->tracks[i]); + mov_finish_fragment(mov, &mov->tracks[i], mdat_start); if (!mov->frag_interleave) { if (!track->mdat_buf) continue; @@ -7169,6 +7191,7 @@ static void mov_free(AVFormatContext *s) else if (track->tag == MKTAG('t','m','c','d') && mov->nb_meta_tmcd) av_freep(&track->par); av_freep(&track->cluster); + av_freep(&track->cluster_written); av_freep(&track->frag_info); av_packet_free(&track->cover_image); @@ -7363,6 +7386,9 @@ static int mov_init(AVFormatContext *s) mov->flags |= FF_MOV_FLAG_FRAGMENT; /* Set other implicit flags immediately */ + if (mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS) + mov->flags |= FF_MOV_FLAG_FRAGMENT; + if (mov->mode == MODE_ISM) mov->flags |= FF_MOV_FLAG_EMPTY_MOOV | FF_MOV_FLAG_SEPARATE_MOOF | FF_MOV_FLAG_FRAGMENT | FF_MOV_FLAG_NEGATIVE_CTS_OFFSETS; @@ -7887,6 +7913,11 @@ static int mov_write_header(AVFormatContext *s) FF_MOV_FLAG_FRAG_EVERY_FRAME)) && !mov->max_fragment_duration && !mov->max_fragment_size) mov->flags |= FF_MOV_FLAG_FRAG_KEYFRAME; + if (mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS) { + avio_wb32(pb, 8); // placeholder for extended size field (64 bit) + ffio_wfourcc(pb, mov->mode == MODE_MOV ? "wide" : "free"); + mov->mdat_pos = avio_tell(pb); + } } else if (mov->mode != MODE_AVIF) { if (mov->flags & FF_MOV_FLAG_FASTSTART) mov->reserved_header_pos = avio_tell(pb); @@ -8090,13 +8121,34 @@ static int mov_write_trailer(AVFormatContext *s) } } - if (!(mov->flags & FF_MOV_FLAG_FRAGMENT)) { + if (!(mov->flags & FF_MOV_FLAG_FRAGMENT) || + mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS) { + if (mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS) { + mov_flush_fragment(s, 1); + mov->mdat_size = avio_tell(pb) - mov->mdat_pos - 8; + for (i = 0; i < mov->nb_tracks; i++) { + MOVTrack *track = &mov->tracks[i]; + track->data_offset = 0; + av_free(track->cluster); + track->cluster = track->cluster_written; + track->entry = track->entry_written; + track->cluster_written = NULL; + track->entry_written = 0; + track->chunkCount = 0; // Force build_chunks to rebuild the list of chunks + } + // Clear the empty_moov flag, as we do want the moov to include + // all the samples at this point. + mov->flags &= ~FF_MOV_FLAG_EMPTY_MOOV; + } + moov_pos = avio_tell(pb); /* Write size of mdat tag */ if (mov->mdat_size + 8 <= UINT32_MAX) { avio_seek(pb, mov->mdat_pos, SEEK_SET); avio_wb32(pb, mov->mdat_size + 8); + if (mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS) + ffio_wfourcc(pb, "mdat"); // overwrite the original moov into a mdat } else { /* overwrite 'wide' placeholder atom */ avio_seek(pb, mov->mdat_pos - 8, SEEK_SET); diff --git a/libavformat/movenc.h b/libavformat/movenc.h index 08d580594d..59daf8946b 100644 --- a/libavformat/movenc.h +++ b/libavformat/movenc.h @@ -85,7 +85,7 @@ typedef struct MOVFragmentInfo { typedef struct MOVTrack { int mode; - int entry; + int entry, entry_written; unsigned timescale; uint64_t time; int64_t track_duration; @@ -114,6 +114,7 @@ typedef struct MOVTrack { int vos_len; uint8_t *vos_data; MOVIentry *cluster; + MOVIentry *cluster_written; unsigned cluster_capacity; int audio_vbr; int height; ///< active picture (w/o VBI) height for D-10/IMX @@ -282,6 +283,7 @@ typedef struct MOVMuxContext { #define FF_MOV_FLAG_SKIP_SIDX (1 << 21) #define FF_MOV_FLAG_CMAF (1 << 22) #define FF_MOV_FLAG_PREFER_ICC (1 << 23) +#define FF_MOV_FLAG_HIDE_FRAGMENTS (1 << 24) int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt); diff --git a/libavformat/version.h b/libavformat/version.h index 4687cd857c..af7d0a1024 100644 --- a/libavformat/version.h +++ b/libavformat/version.h @@ -31,8 +31,8 @@ #include "version_major.h" -#define LIBAVFORMAT_VERSION_MINOR 3 -#define LIBAVFORMAT_VERSION_MICRO 104 +#define LIBAVFORMAT_VERSION_MINOR 4 +#define LIBAVFORMAT_VERSION_MICRO 100 #define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \ LIBAVFORMAT_VERSION_MINOR, \ diff --git a/tests/fate/lavf-container.mak b/tests/fate/lavf-container.mak index d84117c50f..035c394c09 100644 --- a/tests/fate/lavf-container.mak +++ b/tests/fate/lavf-container.mak @@ -5,7 +5,7 @@ FATE_LAVF_CONTAINER-$(call ENCDEC, FLV, FLV) + FATE_LAVF_CONTAINER-$(call ENCDEC, RAWVIDEO, FILMSTRIP) += flm FATE_LAVF_CONTAINER-$(call ENCDEC2, MPEG2VIDEO, PCM_S16LE, GXF) += gxf gxf_pal gxf_ntsc FATE_LAVF_CONTAINER-$(call ENCDEC2, MPEG4, MP2, MATROSKA) += mkv mkv_attachment -FATE_LAVF_CONTAINER-$(call ENCDEC2, MPEG4, PCM_ALAW, MOV) += mov mov_rtphint ismv +FATE_LAVF_CONTAINER-$(call ENCDEC2, MPEG4, PCM_ALAW, MOV) += mov mov_rtphint mov_hide_frag ismv FATE_LAVF_CONTAINER-$(call ENCDEC, MPEG4, MOV) += mp4 FATE_LAVF_CONTAINER-$(call ENCDEC2, MPEG1VIDEO, MP2, MPEG1SYSTEM MPEGPS) += mpg FATE_LAVF_CONTAINER-$(call ENCDEC , FFV1, MXF) += mxf_ffv1 @@ -51,6 +51,7 @@ fate-lavf-mkv: CMD = lavf_container "" "-c:a mp2 -c:v mpeg4 -ar 44100 -threads 1 fate-lavf-mkv_attachment: CMD = lavf_container_attach "-c:a mp2 -c:v mpeg4 -threads 1 -f matroska" fate-lavf-mov: CMD = lavf_container_timecode "-movflags +faststart -c:a pcm_alaw -c:v mpeg4 -threads 1" fate-lavf-mov_rtphint: CMD = lavf_container "" "-movflags +rtphint -c:a pcm_alaw -c:v mpeg4 -threads 1 -f mov" +fate-lavf-mov_hide_frag: CMD = lavf_container "" "-movflags +frag_keyframe+hide_fragments -c:a pcm_alaw -c:v mpeg4 -threads 1 -f mov" fate-lavf-mp4: CMD = lavf_container_timecode "-c:v mpeg4 -an -threads 1" fate-lavf-mpg: CMD = lavf_container_timecode "-ar 44100 -threads 1" fate-lavf-mxf: CMD = lavf_container_timecode "-af aresample=48000:tsf=s16p -bf 2 -threads 1" diff --git a/tests/ref/lavf/mov_hide_frag b/tests/ref/lavf/mov_hide_frag new file mode 100644 index 0000000000..af3af5b171 --- /dev/null +++ b/tests/ref/lavf/mov_hide_frag @@ -0,0 +1,3 @@ +4871796f41234350f1b050317d0288a3 *tests/data/lavf/lavf.mov_hide_frag +358508 tests/data/lavf/lavf.mov_hide_frag +tests/data/lavf/lavf.mov_hide_frag CRC=0xbb2b949b