diff mbox series

[FFmpeg-devel,v2] movenc: Add an option for hiding fragments at the end

Message ID 20240605114702.71176-1-martin@martin.st
State Superseded
Headers show
Series [FFmpeg-devel,v2] movenc: Add an option for hiding fragments at the end | expand

Checks

Context Check Description
andriy/configure_x86 warning Failed to apply patch

Commit Message

Martin Storsjö June 5, 2024, 11:47 a.m. UTC
This allows ending up with a normal, non-fragmented file when
the file is finished, while keeping the file readable if writing
is aborted abruptly at any point. (Normally when writing a
mov/mp4 file, the unfinished file is completely useless unless it
is finished properly.)

This results in a file where the mdat atom contains (and hides)
all the moof atoms that were part of the fragmented file structure
initially.
---
v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes
no sense without it).

Updated the description of the flag to "Write a fragmented file that
is converted to non-fragmented at the end".

Kept the flag named "hide_fragments", but I'm also pondering if we
maybe should go for a name like "hybrid_fragmented" or so, as a
better description of _what_ it produces, as opposed to _how_ it
does things. (One could also consider "hybrid_mp4", but even if mp4
is the main thing, the same also goes for mov and a bunch of other
related formats.)
---
 doc/muxers.texi               |  7 ++++
 libavformat/movenc.c          | 62 ++++++++++++++++++++++++++++++++---
 libavformat/movenc.h          |  4 ++-
 libavformat/version.h         |  4 +--
 tests/fate/lavf-container.mak |  3 +-
 tests/ref/lavf/mov_hide_frag  |  3 ++
 6 files changed, 74 insertions(+), 9 deletions(-)
 create mode 100644 tests/ref/lavf/mov_hide_frag

Comments

Martin Storsjö June 13, 2024, 12:50 p.m. UTC | #1
On Wed, 5 Jun 2024, Martin Storsjö wrote:

> This allows ending up with a normal, non-fragmented file when
> the file is finished, while keeping the file readable if writing
> is aborted abruptly at any point. (Normally when writing a
> mov/mp4 file, the unfinished file is completely useless unless it
> is finished properly.)
>
> This results in a file where the mdat atom contains (and hides)
> all the moof atoms that were part of the fragmented file structure
> initially.
> ---
> v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes
> no sense without it).
>
> Updated the description of the flag to "Write a fragmented file that
> is converted to non-fragmented at the end".
>
> Kept the flag named "hide_fragments", but I'm also pondering if we
> maybe should go for a name like "hybrid_fragmented" or so, as a
> better description of _what_ it produces, as opposed to _how_ it
> does things. (One could also consider "hybrid_mp4", but even if mp4
> is the main thing, the same also goes for mov and a bunch of other
> related formats.)

I'd otherwise want to push this, but I'm not entirely satisfied with the 
option name quite yet. I'm pondering if we should call it 
"hybrid_fragmented" - any opinions, Dennis or Timo?

// Martin
Timo Rothenpieler June 13, 2024, 1:44 p.m. UTC | #2
On 13/06/2024 14:50, Martin Storsjö wrote:
> On Wed, 5 Jun 2024, Martin Storsjö wrote:
> 
>> This allows ending up with a normal, non-fragmented file when
>> the file is finished, while keeping the file readable if writing
>> is aborted abruptly at any point. (Normally when writing a
>> mov/mp4 file, the unfinished file is completely useless unless it
>> is finished properly.)
>>
>> This results in a file where the mdat atom contains (and hides)
>> all the moof atoms that were part of the fragmented file structure
>> initially.
>> ---
>> v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes
>> no sense without it).
>>
>> Updated the description of the flag to "Write a fragmented file that
>> is converted to non-fragmented at the end".
>>
>> Kept the flag named "hide_fragments", but I'm also pondering if we
>> maybe should go for a name like "hybrid_fragmented" or so, as a
>> better description of _what_ it produces, as opposed to _how_ it
>> does things. (One could also consider "hybrid_mp4", but even if mp4
>> is the main thing, the same also goes for mov and a bunch of other
>> related formats.)
> 
> I'd otherwise want to push this, but I'm not entirely satisfied with the 
> option name quite yet. I'm pondering if we should call it 
> "hybrid_fragmented" - any opinions, Dennis or Timo?

It's at least in line with what OBS dubbed the same feature, so would 
allow consistent Search-Results.
So it seems like a good compromise to me.
Gyan Doshi June 13, 2024, 1:45 p.m. UTC | #3
On 2024-06-13 06:20 pm, Martin Storsjö wrote:
> On Wed, 5 Jun 2024, Martin Storsjö wrote:
>
>> This allows ending up with a normal, non-fragmented file when
>> the file is finished, while keeping the file readable if writing
>> is aborted abruptly at any point. (Normally when writing a
>> mov/mp4 file, the unfinished file is completely useless unless it
>> is finished properly.)
>>
>> This results in a file where the mdat atom contains (and hides)
>> all the moof atoms that were part of the fragmented file structure
>> initially.
>> ---
>> v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes
>> no sense without it).
>>
>> Updated the description of the flag to "Write a fragmented file that
>> is converted to non-fragmented at the end".
>>
>> Kept the flag named "hide_fragments", but I'm also pondering if we
>> maybe should go for a name like "hybrid_fragmented" or so, as a
>> better description of _what_ it produces, as opposed to _how_ it
>> does things. (One could also consider "hybrid_mp4", but even if mp4
>> is the main thing, the same also goes for mov and a bunch of other
>> related formats.)
>
> I'd otherwise want to push this, but I'm not entirely satisfied with 
> the option name quite yet. I'm pondering if we should call it 
> "hybrid_fragmented" - any opinions, Dennis or Timo?

How about `resilient_mode` or `recoverable`?
I agree that the how is secondary.

Regards,
Gyan
Dennis Sädtler June 13, 2024, 7:41 p.m. UTC | #4
On 2024-06-13 14:50, Martin Storsjö wrote:
> On Wed, 5 Jun 2024, Martin Storsjö wrote:
>
>> This allows ending up with a normal, non-fragmented file when
>> the file is finished, while keeping the file readable if writing
>> is aborted abruptly at any point. (Normally when writing a
>> mov/mp4 file, the unfinished file is completely useless unless it
>> is finished properly.)
>>
>> This results in a file where the mdat atom contains (and hides)
>> all the moof atoms that were part of the fragmented file structure
>> initially.
>> ---
>> v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes
>> no sense without it).
>>
>> Updated the description of the flag to "Write a fragmented file that
>> is converted to non-fragmented at the end".
>>
>> Kept the flag named "hide_fragments", but I'm also pondering if we
>> maybe should go for a name like "hybrid_fragmented" or so, as a
>> better description of _what_ it produces, as opposed to _how_ it
>> does things. (One could also consider "hybrid_mp4", but even if mp4
>> is the main thing, the same also goes for mov and a bunch of other
>> related formats.)
>
> I'd otherwise want to push this, but I'm not entirely satisfied with 
> the option name quite yet. I'm pondering if we should call it 
> "hybrid_fragmented" - any opinions, Dennis or Timo?
>
> // Martin

That seems fine to me, but I might be biased considering that's what I 
called it :P

~Dennis
Martin Storsjö June 13, 2024, 8:48 p.m. UTC | #5
On Thu, 13 Jun 2024, Gyan Doshi wrote:

> On 2024-06-13 06:20 pm, Martin Storsjö wrote:
>> On Wed, 5 Jun 2024, Martin Storsjö wrote:
>>
>>> This allows ending up with a normal, non-fragmented file when
>>> the file is finished, while keeping the file readable if writing
>>> is aborted abruptly at any point. (Normally when writing a
>>> mov/mp4 file, the unfinished file is completely useless unless it
>>> is finished properly.)
>>>
>>> This results in a file where the mdat atom contains (and hides)
>>> all the moof atoms that were part of the fragmented file structure
>>> initially.
>>> ---
>>> v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes
>>> no sense without it).
>>>
>>> Updated the description of the flag to "Write a fragmented file that
>>> is converted to non-fragmented at the end".
>>>
>>> Kept the flag named "hide_fragments", but I'm also pondering if we
>>> maybe should go for a name like "hybrid_fragmented" or so, as a
>>> better description of _what_ it produces, as opposed to _how_ it
>>> does things. (One could also consider "hybrid_mp4", but even if mp4
>>> is the main thing, the same also goes for mov and a bunch of other
>>> related formats.)
>>
>> I'd otherwise want to push this, but I'm not entirely satisfied with 
>> the option name quite yet. I'm pondering if we should call it 
>> "hybrid_fragmented" - any opinions, Dennis or Timo?
>
> How about `resilient_mode` or `recoverable`?
> I agree that the how is secondary.

Those are good suggestions as well - but I think I prefer 
"hybrid_fragmented" still.

In theory, I guess one could implement resilient writing in a number of 
different ways, whereas the hybrid fragmented/non-fragmented only is one.

So with a couple other voices agreeing with the name "hybrid_fragmented", 
I'll post a new patch with the option in that form - hopefully you don't 
object to it.

// Martin
Gyan Doshi June 14, 2024, 4:27 a.m. UTC | #6
On 2024-06-14 02:18 am, Martin Storsjö wrote:
> On Thu, 13 Jun 2024, Gyan Doshi wrote:
>
>> On 2024-06-13 06:20 pm, Martin Storsjö wrote:
>>> On Wed, 5 Jun 2024, Martin Storsjö wrote:
>>>
>>>> This allows ending up with a normal, non-fragmented file when
>>>> the file is finished, while keeping the file readable if writing
>>>> is aborted abruptly at any point. (Normally when writing a
>>>> mov/mp4 file, the unfinished file is completely useless unless it
>>>> is finished properly.)
>>>>
>>>> This results in a file where the mdat atom contains (and hides)
>>>> all the moof atoms that were part of the fragmented file structure
>>>> initially.
>>>> ---
>>>> v2: Made the flag implicitly set FF_MOV_FLAG_FRAGMENT (as it makes
>>>> no sense without it).
>>>>
>>>> Updated the description of the flag to "Write a fragmented file that
>>>> is converted to non-fragmented at the end".
>>>>
>>>> Kept the flag named "hide_fragments", but I'm also pondering if we
>>>> maybe should go for a name like "hybrid_fragmented" or so, as a
>>>> better description of _what_ it produces, as opposed to _how_ it
>>>> does things. (One could also consider "hybrid_mp4", but even if mp4
>>>> is the main thing, the same also goes for mov and a bunch of other
>>>> related formats.)
>>>
>>> I'd otherwise want to push this, but I'm not entirely satisfied with 
>>> the option name quite yet. I'm pondering if we should call it 
>>> "hybrid_fragmented" - any opinions, Dennis or Timo?
>>
>> How about `resilient_mode` or `recoverable`?
>> I agree that the how is secondary.
>
> Those are good suggestions as well - but I think I prefer 
> "hybrid_fragmented" still.
>
> In theory, I guess one could implement resilient writing in a number 
> of different ways, whereas the hybrid fragmented/non-fragmented only 
> is one.
>
> So with a couple other voices agreeing with the name 
> "hybrid_fragmented", I'll post a new patch with the option in that 
> form - hopefully you don't object to it.

The term hybrid is not applicable here. The fragmented state is 
transient during writing and contingent in the finished artifact 
depending on how the writing process concluded.
Hybrid implies both modes available e.g.. a hybrid vehicle can use both 
types of energy sources. The artifact here will be one _or_ the other.

Regards,
Gyan
Martin Storsjö June 14, 2024, 10:44 a.m. UTC | #7
On Fri, 14 Jun 2024, Gyan Doshi wrote:

> On 2024-06-14 02:18 am, Martin Storsjö wrote:
>> On Thu, 13 Jun 2024, Gyan Doshi wrote:
>>
>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote:
>>>>
>>>> I'd otherwise want to push this, but I'm not entirely satisfied with 
>>>> the option name quite yet. I'm pondering if we should call it 
>>>> "hybrid_fragmented" - any opinions, Dennis or Timo?
>>>
>>> How about `resilient_mode` or `recoverable`?
>>> I agree that the how is secondary.
>>
>> Those are good suggestions as well - but I think I prefer 
>> "hybrid_fragmented" still.
>>
>> In theory, I guess one could implement resilient writing in a number 
>> of different ways, whereas the hybrid fragmented/non-fragmented only 
>> is one.
>>
>> So with a couple other voices agreeing with the name 
>> "hybrid_fragmented", I'll post a new patch with the option in that 
>> form - hopefully you don't object to it.
>
> The term hybrid is not applicable here. The fragmented state is 
> transient during writing and contingent in the finished artifact 
> depending on how the writing process concluded.
> Hybrid implies both modes available e.g.. a hybrid vehicle can use both 
> types of energy sources. The artifact here will be one _or_ the other.

Sure, the file itself is either or, but the process of writing will have 
utilized both. TBH, I don't see it as such a black-or-white thing.

What do the others who have chimed in on the thread think, compared to 
calling it "recoverable" or "resilient_mode"?

// Martin
Timo Rothenpieler June 14, 2024, 11:05 a.m. UTC | #8
On 14/06/2024 12:44, Martin Storsjö wrote:
> On Fri, 14 Jun 2024, Gyan Doshi wrote:
> 
>> On 2024-06-14 02:18 am, Martin Storsjö wrote:
>>> On Thu, 13 Jun 2024, Gyan Doshi wrote:
>>>
>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote:
>>>>>
>>>>> I'd otherwise want to push this, but I'm not entirely satisfied 
>>>>> with the option name quite yet. I'm pondering if we should call it 
>>>>> "hybrid_fragmented" - any opinions, Dennis or Timo?
>>>>
>>>> How about `resilient_mode` or `recoverable`?
>>>> I agree that the how is secondary.
>>>
>>> Those are good suggestions as well - but I think I prefer 
>>> "hybrid_fragmented" still.
>>>
>>> In theory, I guess one could implement resilient writing in a number 
>>> of different ways, whereas the hybrid fragmented/non-fragmented only 
>>> is one.
>>>
>>> So with a couple other voices agreeing with the name 
>>> "hybrid_fragmented", I'll post a new patch with the option in that 
>>> form - hopefully you don't object to it.
>>
>> The term hybrid is not applicable here. The fragmented state is 
>> transient during writing and contingent in the finished artifact 
>> depending on how the writing process concluded.
>> Hybrid implies both modes available e.g.. a hybrid vehicle can use 
>> both types of energy sources. The artifact here will be one _or_ the 
>> other.
> 
> Sure, the file itself is either or, but the process of writing will have 
> utilized both. TBH, I don't see it as such a black-or-white thing.
> 
> What do the others who have chimed in on the thread think, compared to 
> calling it "recoverable" or "resilient_mode"?

I don't have a super strong opinion on it, but out of the options 
provided, I'd prefer the hybrid_ one, since there's a good chance it'll 
become an established term now that OBS presents it quite publicly visible.
Martin Storsjö June 14, 2024, 11:08 a.m. UTC | #9
On Fri, 14 Jun 2024, Timo Rothenpieler wrote:

> On 14/06/2024 12:44, Martin Storsjö wrote:
>> On Fri, 14 Jun 2024, Gyan Doshi wrote:
>> 
>>> On 2024-06-14 02:18 am, Martin Storsjö wrote:
>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote:
>>>>
>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote:
>>>>>>
>>>>>> I'd otherwise want to push this, but I'm not entirely satisfied 
>>>>>> with the option name quite yet. I'm pondering if we should call it 
>>>>>> "hybrid_fragmented" - any opinions, Dennis or Timo?
>>>>>
>>>>> How about `resilient_mode` or `recoverable`?
>>>>> I agree that the how is secondary.
>>>>
>>>> Those are good suggestions as well - but I think I prefer 
>>>> "hybrid_fragmented" still.
>>>>
>>>> In theory, I guess one could implement resilient writing in a number 
>>>> of different ways, whereas the hybrid fragmented/non-fragmented only 
>>>> is one.
>>>>
>>>> So with a couple other voices agreeing with the name 
>>>> "hybrid_fragmented", I'll post a new patch with the option in that 
>>>> form - hopefully you don't object to it.
>>>
>>> The term hybrid is not applicable here. The fragmented state is 
>>> transient during writing and contingent in the finished artifact 
>>> depending on how the writing process concluded.
>>> Hybrid implies both modes available e.g.. a hybrid vehicle can use 
>>> both types of energy sources. The artifact here will be one _or_ the 
>>> other.
>> 
>> Sure, the file itself is either or, but the process of writing will have 
>> utilized both. TBH, I don't see it as such a black-or-white thing.
>> 
>> What do the others who have chimed in on the thread think, compared to 
>> calling it "recoverable" or "resilient_mode"?
>
> I don't have a super strong opinion on it, but out of the options 
> provided, I'd prefer the hybrid_ one, since there's a good chance it'll 
> become an established term now that OBS presents it quite publicly visible.

For context, the same feature in OBS, is documented at e.g. 
https://obsproject.com/kb/hybrid-mp4.

// Martin
Gyan Doshi June 14, 2024, 11:23 a.m. UTC | #10
On 2024-06-14 04:35 pm, Timo Rothenpieler wrote:
> On 14/06/2024 12:44, Martin Storsjö wrote:
>> On Fri, 14 Jun 2024, Gyan Doshi wrote:
>>
>>> On 2024-06-14 02:18 am, Martin Storsjö wrote:
>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote:
>>>>
>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote:
>>>>>>
>>>>>> I'd otherwise want to push this, but I'm not entirely satisfied 
>>>>>> with the option name quite yet. I'm pondering if we should call 
>>>>>> it "hybrid_fragmented" - any opinions, Dennis or Timo?
>>>>>
>>>>> How about `resilient_mode` or `recoverable`?
>>>>> I agree that the how is secondary.
>>>>
>>>> Those are good suggestions as well - but I think I prefer 
>>>> "hybrid_fragmented" still.
>>>>
>>>> In theory, I guess one could implement resilient writing in a 
>>>> number of different ways, whereas the hybrid 
>>>> fragmented/non-fragmented only is one.
>>>>
>>>> So with a couple other voices agreeing with the name 
>>>> "hybrid_fragmented", I'll post a new patch with the option in that 
>>>> form - hopefully you don't object to it.
>>>
>>> The term hybrid is not applicable here. The fragmented state is 
>>> transient during writing and contingent in the finished artifact 
>>> depending on how the writing process concluded.
>>> Hybrid implies both modes available e.g.. a hybrid vehicle can use 
>>> both types of energy sources. The artifact here will be one _or_ the 
>>> other.
>>
>> Sure, the file itself is either or, but the process of writing will 
>> have utilized both. TBH, I don't see it as such a black-or-white thing.
>>
>> What do the others who have chimed in on the thread think, compared 
>> to calling it "recoverable" or "resilient_mode"?
>
> I don't have a super strong opinion on it, but out of the options 
> provided, I'd prefer the hybrid_ one, since there's a good chance 
> it'll become an established term now that OBS presents it quite 
> publicly visible.

The OBS dev intends to change the term:

"Come up with a better name than "Hybrid MP4" that hopefully won't 
confuse users"
https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024

Regards,
Gyan
Dennis Sädtler June 14, 2024, 10:24 p.m. UTC | #11
On 2024-06-14 13:23, Gyan Doshi wrote:
>
>
> On 2024-06-14 04:35 pm, Timo Rothenpieler wrote:
>> On 14/06/2024 12:44, Martin Storsjö wrote:
>>> On Fri, 14 Jun 2024, Gyan Doshi wrote:
>>>
>>>> On 2024-06-14 02:18 am, Martin Storsjö wrote:
>>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote:
>>>>>
>>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote:
>>>>>>>
>>>>>>> I'd otherwise want to push this, but I'm not entirely satisfied 
>>>>>>> with the option name quite yet. I'm pondering if we should call 
>>>>>>> it "hybrid_fragmented" - any opinions, Dennis or Timo?
>>>>>>
>>>>>> How about `resilient_mode` or `recoverable`?
>>>>>> I agree that the how is secondary.
>>>>>
>>>>> Those are good suggestions as well - but I think I prefer 
>>>>> "hybrid_fragmented" still.
>>>>>
>>>>> In theory, I guess one could implement resilient writing in a 
>>>>> number of different ways, whereas the hybrid 
>>>>> fragmented/non-fragmented only is one.
>>>>>
>>>>> So with a couple other voices agreeing with the name 
>>>>> "hybrid_fragmented", I'll post a new patch with the option in that 
>>>>> form - hopefully you don't object to it.
>>>>
>>>> The term hybrid is not applicable here. The fragmented state is 
>>>> transient during writing and contingent in the finished artifact 
>>>> depending on how the writing process concluded.
>>>> Hybrid implies both modes available e.g.. a hybrid vehicle can use 
>>>> both types of energy sources. The artifact here will be one _or_ 
>>>> the other.
>>>
>>> Sure, the file itself is either or, but the process of writing will 
>>> have utilized both. TBH, I don't see it as such a black-or-white thing.
>>>
>>> What do the others who have chimed in on the thread think, compared 
>>> to calling it "recoverable" or "resilient_mode"?
>>
>> I don't have a super strong opinion on it, but out of the options 
>> provided, I'd prefer the hybrid_ one, since there's a good chance 
>> it'll become an established term now that OBS presents it quite 
>> publicly visible.
>
> The OBS dev intends to change the term:
>
> "Come up with a better name than "Hybrid MP4" that hopefully won't 
> confuse users"
> https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024 
>
>
> Regards,
> Gyan

Now that it's merged and in the hands of users I don't have any 
intention of changing the name any more.
We had some chats about about it, but nobody suggested anything that 
people agreed was better, so it stuck.

While "resilient" certainly fits, it could equally apply to regular 
fragmented MP4 (e.g. vMix uses that terminology for fMP4 if I'm not 
mistaken).
The important attribute with this approach is that it's resilient *and* 
compatible, and I'm still not sure how to get that across in name alone.

Obviously FFmpeg has a different target demographic and can use 
something different than OBS, "Hybrid MP4" is just what I came up with 
as user-facing name for the feature at the time.

~Dennis
Gyan Doshi June 15, 2024, 9:15 a.m. UTC | #12
On 2024-06-15 03:54 am, Dennis Sädtler via ffmpeg-devel wrote:
> On 2024-06-14 13:23, Gyan Doshi wrote:
>>
>>
>> On 2024-06-14 04:35 pm, Timo Rothenpieler wrote:
>>> On 14/06/2024 12:44, Martin Storsjö wrote:
>>>> On Fri, 14 Jun 2024, Gyan Doshi wrote:
>>>>
>>>>> On 2024-06-14 02:18 am, Martin Storsjö wrote:
>>>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote:
>>>>>>
>>>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote:
>>>>>>>>
>>>>>>>> I'd otherwise want to push this, but I'm not entirely satisfied 
>>>>>>>> with the option name quite yet. I'm pondering if we should call 
>>>>>>>> it "hybrid_fragmented" - any opinions, Dennis or Timo?
>>>>>>>
>>>>>>> How about `resilient_mode` or `recoverable`?
>>>>>>> I agree that the how is secondary.
>>>>>>
>>>>>> Those are good suggestions as well - but I think I prefer 
>>>>>> "hybrid_fragmented" still.
>>>>>>
>>>>>> In theory, I guess one could implement resilient writing in a 
>>>>>> number of different ways, whereas the hybrid 
>>>>>> fragmented/non-fragmented only is one.
>>>>>>
>>>>>> So with a couple other voices agreeing with the name 
>>>>>> "hybrid_fragmented", I'll post a new patch with the option in 
>>>>>> that form - hopefully you don't object to it.
>>>>>
>>>>> The term hybrid is not applicable here. The fragmented state is 
>>>>> transient during writing and contingent in the finished artifact 
>>>>> depending on how the writing process concluded.
>>>>> Hybrid implies both modes available e.g.. a hybrid vehicle can use 
>>>>> both types of energy sources. The artifact here will be one _or_ 
>>>>> the other.
>>>>
>>>> Sure, the file itself is either or, but the process of writing will 
>>>> have utilized both. TBH, I don't see it as such a black-or-white 
>>>> thing.
>>>>
>>>> What do the others who have chimed in on the thread think, compared 
>>>> to calling it "recoverable" or "resilient_mode"?
>>>
>>> I don't have a super strong opinion on it, but out of the options 
>>> provided, I'd prefer the hybrid_ one, since there's a good chance 
>>> it'll become an established term now that OBS presents it quite 
>>> publicly visible.
>>
>> The OBS dev intends to change the term:
>>
>> "Come up with a better name than "Hybrid MP4" that hopefully won't 
>> confuse users"
>> https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024 
>>
>>
>> Regards,
>> Gyan
>
> Now that it's merged and in the hands of users I don't have any 
> intention of changing the name any more.
> We had some chats about about it, but nobody suggested anything that 
> people agreed was better, so it stuck.
>
> While "resilient" certainly fits, it could equally apply to regular 
> fragmented MP4 (e.g. vMix uses that terminology for fMP4 if I'm not 
> mistaken).
> The important attribute with this approach is that it's resilient 
> *and* compatible, and I'm still not sure how to get that across in 
> name alone.

How about `failsafe`?

Regards,
Gyan
Martin Storsjö June 17, 2024, 10:38 a.m. UTC | #13
On Sat, 15 Jun 2024, Gyan Doshi wrote:

> On 2024-06-15 03:54 am, Dennis Sädtler via ffmpeg-devel wrote:
>> On 2024-06-14 13:23, Gyan Doshi wrote:
>>>
>>> On 2024-06-14 04:35 pm, Timo Rothenpieler wrote:
>>>> On 14/06/2024 12:44, Martin Storsjö wrote:
>>>>> On Fri, 14 Jun 2024, Gyan Doshi wrote:
>>>>>
>>>>>> On 2024-06-14 02:18 am, Martin Storsjö wrote:
>>>>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote:
>>>>>>>
>>>>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote:
>>>>>>>>>
>>>>>>>>> I'd otherwise want to push this, but I'm not entirely satisfied 
>>>>>>>>> with the option name quite yet. I'm pondering if we should call 
>>>>>>>>> it "hybrid_fragmented" - any opinions, Dennis or Timo?
>>>>>>>>
>>>>>>>> How about `resilient_mode` or `recoverable`?
>>>>>>>> I agree that the how is secondary.
>>>>>>>
>>>>>>> Those are good suggestions as well - but I think I prefer 
>>>>>>> "hybrid_fragmented" still.
>>>>>>>
>>>>>>> In theory, I guess one could implement resilient writing in a 
>>>>>>> number of different ways, whereas the hybrid 
>>>>>>> fragmented/non-fragmented only is one.
>>>>>>>
>>>>>>> So with a couple other voices agreeing with the name 
>>>>>>> "hybrid_fragmented", I'll post a new patch with the option in 
>>>>>>> that form - hopefully you don't object to it.
>>>>>>
>>>>>> The term hybrid is not applicable here. The fragmented state is 
>>>>>> transient during writing and contingent in the finished artifact 
>>>>>> depending on how the writing process concluded.
>>>>>> Hybrid implies both modes available e.g.. a hybrid vehicle can use 
>>>>>> both types of energy sources. The artifact here will be one _or_ 
>>>>>> the other.
>>>>>
>>>>> Sure, the file itself is either or, but the process of writing will 
>>>>> have utilized both. TBH, I don't see it as such a black-or-white 
>>>>> thing.
>>>>>
>>>>> What do the others who have chimed in on the thread think, compared 
>>>>> to calling it "recoverable" or "resilient_mode"?
>>>>
>>>> I don't have a super strong opinion on it, but out of the options 
>>>> provided, I'd prefer the hybrid_ one, since there's a good chance 
>>>> it'll become an established term now that OBS presents it quite 
>>>> publicly visible.
>>>
>>> The OBS dev intends to change the term:
>>>
>>> "Come up with a better name than "Hybrid MP4" that hopefully won't 
>>> confuse users"
>>> 
> https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024 
>>>
>>>
>>> Regards,
>>> Gyan
>>
>> Now that it's merged and in the hands of users I don't have any 
>> intention of changing the name any more.
>> We had some chats about about it, but nobody suggested anything that 
>> people agreed was better, so it stuck.
>>
>> While "resilient" certainly fits, it could equally apply to regular 
>> fragmented MP4 (e.g. vMix uses that terminology for fMP4 if I'm not 
>> mistaken).
>> The important attribute with this approach is that it's resilient 
>> *and* compatible, and I'm still not sure how to get that across in 
>> name alone.
>
> How about `failsafe`?

I don't see how that differs from "resilient", as a regular fragmented 
file also is failsafe (or resilient) in the same way - while the special 
thing here is that it's both fragmented and not.

// Martin
Gyan Doshi June 17, 2024, 12:41 p.m. UTC | #14
On 2024-06-17 04:08 pm, Martin Storsjö wrote:
> On Sat, 15 Jun 2024, Gyan Doshi wrote:
>
>> On 2024-06-15 03:54 am, Dennis Sädtler via ffmpeg-devel wrote:
>>> On 2024-06-14 13:23, Gyan Doshi wrote:
>>>>
>>>> On 2024-06-14 04:35 pm, Timo Rothenpieler wrote:
>>>>> On 14/06/2024 12:44, Martin Storsjö wrote:
>>>>>> On Fri, 14 Jun 2024, Gyan Doshi wrote:
>>>>>>
>>>>>>> On 2024-06-14 02:18 am, Martin Storsjö wrote:
>>>>>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote:
>>>>>>>>
>>>>>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote:
>>>>>>>>>>
>>>>>>>>>> I'd otherwise want to push this, but I'm not entirely 
>>>>>>>>>> satisfied with the option name quite yet. I'm pondering if we 
>>>>>>>>>> should call it "hybrid_fragmented" - any opinions, Dennis or 
>>>>>>>>>> Timo?
>>>>>>>>>
>>>>>>>>> How about `resilient_mode` or `recoverable`?
>>>>>>>>> I agree that the how is secondary.
>>>>>>>>
>>>>>>>> Those are good suggestions as well - but I think I prefer 
>>>>>>>> "hybrid_fragmented" still.
>>>>>>>>
>>>>>>>> In theory, I guess one could implement resilient writing in a 
>>>>>>>> number of different ways, whereas the hybrid 
>>>>>>>> fragmented/non-fragmented only is one.
>>>>>>>>
>>>>>>>> So with a couple other voices agreeing with the name 
>>>>>>>> "hybrid_fragmented", I'll post a new patch with the option in 
>>>>>>>> that form - hopefully you don't object to it.
>>>>>>>
>>>>>>> The term hybrid is not applicable here. The fragmented state is 
>>>>>>> transient during writing and contingent in the finished artifact 
>>>>>>> depending on how the writing process concluded.
>>>>>>> Hybrid implies both modes available e.g.. a hybrid vehicle can 
>>>>>>> use both types of energy sources. The artifact here will be one 
>>>>>>> _or_ the other.
>>>>>>
>>>>>> Sure, the file itself is either or, but the process of writing 
>>>>>> will have utilized both. TBH, I don't see it as such a 
>>>>>> black-or-white thing.
>>>>>>
>>>>>> What do the others who have chimed in on the thread think, 
>>>>>> compared to calling it "recoverable" or "resilient_mode"?
>>>>>
>>>>> I don't have a super strong opinion on it, but out of the options 
>>>>> provided, I'd prefer the hybrid_ one, since there's a good chance 
>>>>> it'll become an established term now that OBS presents it quite 
>>>>> publicly visible.
>>>>
>>>> The OBS dev intends to change the term:
>>>>
>>>> "Come up with a better name than "Hybrid MP4" that hopefully won't 
>>>> confuse users"
>>>>
>> https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024 
>>
>>>>
>>>>
>>>> Regards,
>>>> Gyan
>>>
>>> Now that it's merged and in the hands of users I don't have any 
>>> intention of changing the name any more.
>>> We had some chats about about it, but nobody suggested anything that 
>>> people agreed was better, so it stuck.
>>>
>>> While "resilient" certainly fits, it could equally apply to regular 
>>> fragmented MP4 (e.g. vMix uses that terminology for fMP4 if I'm not 
>>> mistaken).
>>> The important attribute with this approach is that it's resilient 
>>> *and* compatible, and I'm still not sure how to get that across in 
>>> name alone.
>>
>> How about `failsafe`?
>
> I don't see how that differs from "resilient", as a regular fragmented 
> file also is failsafe (or resilient) in the same way - while the 
> special thing here is that it's both fragmented and not.

The expert user already knows to save a fragmented file if they want 
resilience. This option saves them a remux step if the original writing 
ends gracefully.
For all other users,  the value proposition _is_ the resilience.  If the 
muxing ends normally, they just have a normal file. If it ends 
prematurely, they just want to be able to convert to a regular seekable 
MP4. The fact that it is saved in fragmented or any other mode is 
irrelevant - an academic detail at best.

Ultimately, as long as the doc is clear about what the use of this 
option is, and what to do next if the muxing does abort, it should not 
matter too much what the option is called. But just like the faststart 
flag name identifies the purpose instead of being called something like 
moov_in_front, hopefully so will the name here.

Regards,
Gyan
Martin Storsjö June 19, 2024, 12:34 p.m. UTC | #15
On Mon, 17 Jun 2024, Gyan Doshi via ffmpeg-devel wrote:

> On 2024-06-17 04:08 pm, Martin Storsjö wrote:
>> On Sat, 15 Jun 2024, Gyan Doshi wrote:
>>
>>> On 2024-06-15 03:54 am, Dennis Sädtler via ffmpeg-devel wrote:
>>>> On 2024-06-14 13:23, Gyan Doshi wrote:
>>>>>
>>>>> On 2024-06-14 04:35 pm, Timo Rothenpieler wrote:
>>>>>> On 14/06/2024 12:44, Martin Storsjö wrote:
>>>>>>> On Fri, 14 Jun 2024, Gyan Doshi wrote:
>>>>>>>
>>>>>>>> On 2024-06-14 02:18 am, Martin Storsjö wrote:
>>>>>>>>> On Thu, 13 Jun 2024, Gyan Doshi wrote:
>>>>>>>>>
>>>>>>>>>> On 2024-06-13 06:20 pm, Martin Storsjö wrote:
>>>>>>>>>>>
>>>>>>>>>>> I'd otherwise want to push this, but I'm not entirely 
>>>>>>>>>>> satisfied with the option name quite yet. I'm pondering if we 
>>>>>>>>>>> should call it "hybrid_fragmented" - any opinions, Dennis or 
>>>>>>>>>>> Timo?
>>>>>>>>>>
>>>>>>>>>> How about `resilient_mode` or `recoverable`?
>>>>>>>>>> I agree that the how is secondary.
>>>>>>>>>
>>>>>>>>> Those are good suggestions as well - but I think I prefer 
>>>>>>>>> "hybrid_fragmented" still.
>>>>>>>>>
>>>>>>>>> In theory, I guess one could implement resilient writing in a 
>>>>>>>>> number of different ways, whereas the hybrid 
>>>>>>>>> fragmented/non-fragmented only is one.
>>>>>>>>>
>>>>>>>>> So with a couple other voices agreeing with the name 
>>>>>>>>> "hybrid_fragmented", I'll post a new patch with the option in 
>>>>>>>>> that form - hopefully you don't object to it.
>>>>>>>>
>>>>>>>> The term hybrid is not applicable here. The fragmented state is 
>>>>>>>> transient during writing and contingent in the finished artifact 
>>>>>>>> depending on how the writing process concluded.
>>>>>>>> Hybrid implies both modes available e.g.. a hybrid vehicle can 
>>>>>>>> use both types of energy sources. The artifact here will be one 
>>>>>>>> _or_ the other.
>>>>>>>
>>>>>>> Sure, the file itself is either or, but the process of writing 
>>>>>>> will have utilized both. TBH, I don't see it as such a 
>>>>>>> black-or-white thing.
>>>>>>>
>>>>>>> What do the others who have chimed in on the thread think, 
>>>>>>> compared to calling it "recoverable" or "resilient_mode"?
>>>>>>
>>>>>> I don't have a super strong opinion on it, but out of the options 
>>>>>> provided, I'd prefer the hybrid_ one, since there's a good chance 
>>>>>> it'll become an established term now that OBS presents it quite 
>>>>>> publicly visible.
>>>>>
>>>>> The OBS dev intends to change the term:
>>>>>
>>>>> "Come up with a better name than "Hybrid MP4" that hopefully won't 
>>>>> confuse users"
>>>>>
>>> 
> https://github.com/obsproject/obs-studio/pull/10608#issuecomment-2095222024 
>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>> Gyan
>>>>
>>>> Now that it's merged and in the hands of users I don't have any 
>>>> intention of changing the name any more.
>>>> We had some chats about about it, but nobody suggested anything that 
>>>> people agreed was better, so it stuck.
>>>>
>>>> While "resilient" certainly fits, it could equally apply to regular 
>>>> fragmented MP4 (e.g. vMix uses that terminology for fMP4 if I'm not 
>>>> mistaken).
>>>> The important attribute with this approach is that it's resilient 
>>>> *and* compatible, and I'm still not sure how to get that across in 
>>>> name alone.
>>>
>>> How about `failsafe`?
>>
>> I don't see how that differs from "resilient", as a regular fragmented 
>> file also is failsafe (or resilient) in the same way - while the 
>> special thing here is that it's both fragmented and not.
>
> The expert user already knows to save a fragmented file if they want 
> resilience. This option saves them a remux step if the original writing 
> ends gracefully.
> For all other users,  the value proposition _is_ the resilience.  If the 
> muxing ends normally, they just have a normal file. If it ends 
> prematurely, they just want to be able to convert to a regular seekable 
> MP4. The fact that it is saved in fragmented or any other mode is 
> irrelevant - an academic detail at best.

Sure

> Ultimately, as long as the doc is clear about what the use of this 
> option is, and what to do next if the muxing does abort, it should not 
> matter too much what the option is called.

So, are you saying you'd accept the hybrid_fragmented name, as long as the 
docs explain this correctly?

> But just like the faststart flag name identifies the purpose instead of 
> being called something like moov_in_front, hopefully so will the name 
> here.

TBH, the "faststart" name isn't necessarily the best either - although 
it's clearly better than moov_in_front though. But here, the name 
"faststart" was already established for the concept of this kind of 
operation, from the "qt-faststart" tool before it was integrated into the 
muxer.

And here, we seem to begin to have some sort of similar establishing of 
"hybrid" as name for this kind of operation.

But if this really is inacceptable to you, I guess all others who support 
that name, will have to figure out which of the alternate names we prefer.

// Martin
Gyan Doshi June 19, 2024, 12:38 p.m. UTC | #16
On 2024-06-19 06:04 pm, Martin Storsjö wrote:
> On Mon, 17 Jun 2024, Gyan Doshi via ffmpeg-devel wrote:
>
>
>> Ultimately, as long as the doc is clear about what the use of this 
>> option is, and what to do next if the muxing does abort, it should 
>> not matter too much what the option is called.
>
> So, are you saying you'd accept the hybrid_fragmented name, as long as 
> the docs explain this correctly?

Yes,

Regards,
Gyan
diff mbox series

Patch

diff --git a/doc/muxers.texi b/doc/muxers.texi
index 6340c8e54d..e313b5e631 100644
--- a/doc/muxers.texi
+++ b/doc/muxers.texi
@@ -569,6 +569,13 @@  experimental, may be renamed or changed, do not use from scripts.
 
 @item write_gama
 write deprecated gama atom
+
+@item hide_fragments
+After writing a fragmented file, convert it to a regular, non-fragmented
+file at the end. This keeps the file readable while it is being
+written, and makes it recoverable if the process of writing the file
+gets aborted uncleanly, while still producing an easily seekable
+and widely compatible non-fragmented file in the end.
 @end table
 
 @item movie_timescale @var{scale}
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index d1517870fc..da16a9dae5 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -110,6 +110,7 @@  static const AVOption options[] = {
       { "use_metadata_tags", "Use mdta atom for metadata.", 0, AV_OPT_TYPE_CONST, {.i64 = FF_MOV_FLAG_USE_MDTA}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, .unit = "movflags" },
       { "write_colr", "Write colr atom even if the color info is unspecified (Experimental, may be renamed or changed, do not use from scripts)", 0, AV_OPT_TYPE_CONST, {.i64 = FF_MOV_FLAG_WRITE_COLR}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, .unit = "movflags" },
       { "write_gama", "Write deprecated gama atom", 0, AV_OPT_TYPE_CONST, {.i64 = FF_MOV_FLAG_WRITE_GAMA}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, .unit = "movflags" },
+      { "hide_fragments", "Write a fragmented file that is converted to non-fragmented at the end", 0, AV_OPT_TYPE_CONST, {.i64 = FF_MOV_FLAG_HIDE_FRAGMENTS}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, .unit = "movflags" },
     { "min_frag_duration", "Minimum fragment duration", offsetof(MOVMuxContext, min_fragment_duration), AV_OPT_TYPE_INT, {.i64 = 0}, 0, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM},
     { "mov_gamma", "gamma value for gama atom", offsetof(MOVMuxContext, gamma), AV_OPT_TYPE_FLOAT, {.dbl = 0.0 }, 0.0, 10, AV_OPT_FLAG_ENCODING_PARAM},
     { "movie_timescale", "set movie timescale", offsetof(MOVMuxContext, movie_timescale), AV_OPT_TYPE_INT, {.i64 = MOV_TIMESCALE}, 1, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM},
@@ -5993,10 +5994,30 @@  static int mov_write_squashed_packets(AVFormatContext *s)
     return 0;
 }
 
-static int mov_finish_fragment(MOVTrack *track)
+static int mov_finish_fragment(MOVMuxContext *mov, MOVTrack *track,
+                               int64_t ref_pos)
 {
+    int i;
     if (!track->entry)
         return 0;
+    if (mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS) {
+        for (i = 0; i < track->entry; i++)
+            track->cluster[i].pos += ref_pos + track->data_offset;
+        if (track->cluster_written == 0 && !(mov->flags & FF_MOV_FLAG_EMPTY_MOOV)) {
+            // First flush. If this was a case of not using empty moov, reset chunking.
+            for (i = 0; i < track->entry; i++) {
+                track->cluster[i].chunkNum = 0;
+                track->cluster[i].samples_in_chunk = track->cluster[i].entries;
+            }
+        }
+        if (av_reallocp_array(&track->cluster_written,
+                              track->entry_written + track->entry,
+                              sizeof(*track->cluster)))
+            return AVERROR(ENOMEM);
+        memcpy(&track->cluster_written[track->entry_written],
+               track->cluster, track->entry * sizeof(*track->cluster));
+        track->entry_written += track->entry;
+    }
     track->entry = 0;
     track->entries_flushed = 0;
     track->end_reliable = 0;
@@ -6007,7 +6028,7 @@  static int mov_flush_fragment(AVFormatContext *s, int force)
 {
     MOVMuxContext *mov = s->priv_data;
     int i, first_track = -1;
-    int64_t mdat_size = 0;
+    int64_t mdat_size = 0, mdat_start = 0;
     int ret;
     int has_video = 0, starts_with_key = 0, first_video_track = 1;
 
@@ -6113,7 +6134,7 @@  static int mov_flush_fragment(AVFormatContext *s, int force)
         mov->moov_written = 1;
         mov->mdat_size = 0;
         for (i = 0; i < mov->nb_tracks; i++)
-            mov_finish_fragment(&mov->tracks[i]);
+            mov_finish_fragment(mov, &mov->tracks[i], 0);
         avio_write_marker(s->pb, AV_NOPTS_VALUE, AVIO_DATA_MARKER_FLUSH_POINT);
         return 0;
     }
@@ -6182,9 +6203,10 @@  static int mov_flush_fragment(AVFormatContext *s, int force)
 
             avio_wb32(s->pb, mdat_size + 8);
             ffio_wfourcc(s->pb, "mdat");
+            mdat_start = avio_tell(s->pb);
         }
 
-        mov_finish_fragment(&mov->tracks[i]);
+        mov_finish_fragment(mov, &mov->tracks[i], mdat_start);
         if (!mov->frag_interleave) {
             if (!track->mdat_buf)
                 continue;
@@ -7169,6 +7191,7 @@  static void mov_free(AVFormatContext *s)
         else if (track->tag == MKTAG('t','m','c','d') && mov->nb_meta_tmcd)
             av_freep(&track->par);
         av_freep(&track->cluster);
+        av_freep(&track->cluster_written);
         av_freep(&track->frag_info);
         av_packet_free(&track->cover_image);
 
@@ -7363,6 +7386,9 @@  static int mov_init(AVFormatContext *s)
         mov->flags |= FF_MOV_FLAG_FRAGMENT;
 
     /* Set other implicit flags immediately */
+    if (mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS)
+        mov->flags |= FF_MOV_FLAG_FRAGMENT;
+
     if (mov->mode == MODE_ISM)
         mov->flags |= FF_MOV_FLAG_EMPTY_MOOV | FF_MOV_FLAG_SEPARATE_MOOF |
                       FF_MOV_FLAG_FRAGMENT | FF_MOV_FLAG_NEGATIVE_CTS_OFFSETS;
@@ -7887,6 +7913,11 @@  static int mov_write_header(AVFormatContext *s)
                             FF_MOV_FLAG_FRAG_EVERY_FRAME)) &&
             !mov->max_fragment_duration && !mov->max_fragment_size)
             mov->flags |= FF_MOV_FLAG_FRAG_KEYFRAME;
+        if (mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS) {
+            avio_wb32(pb, 8); // placeholder for extended size field (64 bit)
+            ffio_wfourcc(pb, mov->mode == MODE_MOV ? "wide" : "free");
+            mov->mdat_pos = avio_tell(pb);
+        }
     } else if (mov->mode != MODE_AVIF) {
         if (mov->flags & FF_MOV_FLAG_FASTSTART)
             mov->reserved_header_pos = avio_tell(pb);
@@ -8090,13 +8121,34 @@  static int mov_write_trailer(AVFormatContext *s)
         }
     }
 
-    if (!(mov->flags & FF_MOV_FLAG_FRAGMENT)) {
+    if (!(mov->flags & FF_MOV_FLAG_FRAGMENT) ||
+        mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS) {
+        if (mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS) {
+            mov_flush_fragment(s, 1);
+            mov->mdat_size = avio_tell(pb) - mov->mdat_pos - 8;
+            for (i = 0; i < mov->nb_tracks; i++) {
+                MOVTrack *track = &mov->tracks[i];
+                track->data_offset = 0;
+                av_free(track->cluster);
+                track->cluster = track->cluster_written;
+                track->entry   = track->entry_written;
+                track->cluster_written = NULL;
+                track->entry_written   = 0;
+                track->chunkCount = 0; // Force build_chunks to rebuild the list of chunks
+            }
+            // Clear the empty_moov flag, as we do want the moov to include
+            // all the samples at this point.
+            mov->flags &= ~FF_MOV_FLAG_EMPTY_MOOV;
+        }
+
         moov_pos = avio_tell(pb);
 
         /* Write size of mdat tag */
         if (mov->mdat_size + 8 <= UINT32_MAX) {
             avio_seek(pb, mov->mdat_pos, SEEK_SET);
             avio_wb32(pb, mov->mdat_size + 8);
+            if (mov->flags & FF_MOV_FLAG_HIDE_FRAGMENTS)
+                ffio_wfourcc(pb, "mdat"); // overwrite the original moov into a mdat
         } else {
             /* overwrite 'wide' placeholder atom */
             avio_seek(pb, mov->mdat_pos - 8, SEEK_SET);
diff --git a/libavformat/movenc.h b/libavformat/movenc.h
index 08d580594d..59daf8946b 100644
--- a/libavformat/movenc.h
+++ b/libavformat/movenc.h
@@ -85,7 +85,7 @@  typedef struct MOVFragmentInfo {
 
 typedef struct MOVTrack {
     int         mode;
-    int         entry;
+    int         entry, entry_written;
     unsigned    timescale;
     uint64_t    time;
     int64_t     track_duration;
@@ -114,6 +114,7 @@  typedef struct MOVTrack {
     int         vos_len;
     uint8_t     *vos_data;
     MOVIentry   *cluster;
+    MOVIentry   *cluster_written;
     unsigned    cluster_capacity;
     int         audio_vbr;
     int         height; ///< active picture (w/o VBI) height for D-10/IMX
@@ -282,6 +283,7 @@  typedef struct MOVMuxContext {
 #define FF_MOV_FLAG_SKIP_SIDX             (1 << 21)
 #define FF_MOV_FLAG_CMAF                  (1 << 22)
 #define FF_MOV_FLAG_PREFER_ICC            (1 << 23)
+#define FF_MOV_FLAG_HIDE_FRAGMENTS        (1 << 24)
 
 int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt);
 
diff --git a/libavformat/version.h b/libavformat/version.h
index 4687cd857c..af7d0a1024 100644
--- a/libavformat/version.h
+++ b/libavformat/version.h
@@ -31,8 +31,8 @@ 
 
 #include "version_major.h"
 
-#define LIBAVFORMAT_VERSION_MINOR   3
-#define LIBAVFORMAT_VERSION_MICRO 104
+#define LIBAVFORMAT_VERSION_MINOR   4
+#define LIBAVFORMAT_VERSION_MICRO 100
 
 #define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \
                                                LIBAVFORMAT_VERSION_MINOR, \
diff --git a/tests/fate/lavf-container.mak b/tests/fate/lavf-container.mak
index d84117c50f..035c394c09 100644
--- a/tests/fate/lavf-container.mak
+++ b/tests/fate/lavf-container.mak
@@ -5,7 +5,7 @@  FATE_LAVF_CONTAINER-$(call ENCDEC,  FLV,                   FLV)                +
 FATE_LAVF_CONTAINER-$(call ENCDEC,  RAWVIDEO,              FILMSTRIP)          += flm
 FATE_LAVF_CONTAINER-$(call ENCDEC2, MPEG2VIDEO, PCM_S16LE, GXF)                += gxf gxf_pal gxf_ntsc
 FATE_LAVF_CONTAINER-$(call ENCDEC2, MPEG4,      MP2,       MATROSKA)           += mkv mkv_attachment
-FATE_LAVF_CONTAINER-$(call ENCDEC2, MPEG4,      PCM_ALAW,  MOV)                += mov mov_rtphint ismv
+FATE_LAVF_CONTAINER-$(call ENCDEC2, MPEG4,      PCM_ALAW,  MOV)                += mov mov_rtphint mov_hide_frag ismv
 FATE_LAVF_CONTAINER-$(call ENCDEC,  MPEG4,                 MOV)                += mp4
 FATE_LAVF_CONTAINER-$(call ENCDEC2, MPEG1VIDEO, MP2,       MPEG1SYSTEM MPEGPS) += mpg
 FATE_LAVF_CONTAINER-$(call ENCDEC , FFV1,                  MXF)                += mxf_ffv1
@@ -51,6 +51,7 @@  fate-lavf-mkv: CMD = lavf_container "" "-c:a mp2 -c:v mpeg4 -ar 44100 -threads 1
 fate-lavf-mkv_attachment: CMD = lavf_container_attach "-c:a mp2 -c:v mpeg4 -threads 1 -f matroska"
 fate-lavf-mov: CMD = lavf_container_timecode "-movflags +faststart -c:a pcm_alaw -c:v mpeg4 -threads 1"
 fate-lavf-mov_rtphint: CMD = lavf_container "" "-movflags +rtphint -c:a pcm_alaw -c:v mpeg4 -threads 1 -f mov"
+fate-lavf-mov_hide_frag: CMD = lavf_container "" "-movflags +frag_keyframe+hide_fragments -c:a pcm_alaw -c:v mpeg4 -threads 1 -f mov"
 fate-lavf-mp4: CMD = lavf_container_timecode "-c:v mpeg4 -an -threads 1"
 fate-lavf-mpg: CMD = lavf_container_timecode "-ar 44100 -threads 1"
 fate-lavf-mxf: CMD = lavf_container_timecode "-af aresample=48000:tsf=s16p -bf 2 -threads 1"
diff --git a/tests/ref/lavf/mov_hide_frag b/tests/ref/lavf/mov_hide_frag
new file mode 100644
index 0000000000..af3af5b171
--- /dev/null
+++ b/tests/ref/lavf/mov_hide_frag
@@ -0,0 +1,3 @@ 
+4871796f41234350f1b050317d0288a3 *tests/data/lavf/lavf.mov_hide_frag
+358508 tests/data/lavf/lavf.mov_hide_frag
+tests/data/lavf/lavf.mov_hide_frag CRC=0xbb2b949b