[FFmpeg-devel] lavc/avcodec: Allow libavcodec to overwrite profile and level

Submitted by Carl Eugen Hoyos on Nov. 23, 2017, 3:01 p.m.

Details

Message ID CAB0OVGo8a4PKpYmexCx5ynt8LD66nPvR4Wr2ZPyDzD8Ezd5i=g@mail.gmail.com
State New
Headers show

Commit Message

Carl Eugen Hoyos Nov. 23, 2017, 3:01 p.m.
Hi!

The (external) encoders may overwrite level and profile because of
requested encoding properties, allowing libavcodec to (also) overwrite
them in the context makes sense (and is already done in some cases
afaict).

Please comment, Carl Eugen

Comments

Michael Niedermayer Nov. 23, 2017, 9:58 p.m.
On Thu, Nov 23, 2017 at 04:01:06PM +0100, Carl Eugen Hoyos wrote:
> Hi!
> 
> The (external) encoders may overwrite level and profile because of
> requested encoding properties, allowing libavcodec to (also) overwrite
> them in the context makes sense (and is already done in some cases
> afaict).
> 
> Please comment, Carl Eugen

If a user needs to generate a file with a specific profile/level
for example because its for broadcast, some specification or a hw
decoder.
How could he after this patch ensure that exactly the needed profile
is used?

[...]
Carl Eugen Hoyos Nov. 23, 2017, 11:47 p.m.
2017-11-23 22:58 GMT+01:00 Michael Niedermayer <michael@niedermayer.cc>:
> On Thu, Nov 23, 2017 at 04:01:06PM +0100, Carl Eugen Hoyos wrote:
>> Hi!
>>
>> The (external) encoders may overwrite level and profile because of
>> requested encoding properties, allowing libavcodec to (also) overwrite
>> them in the context makes sense (and is already done in some cases
>> afaict).
>>
>> Please comment, Carl Eugen
>
> If a user needs to generate a file with a specific profile/level
> for example because its for broadcast, some specification or a hw
> decoder.
> How could he after this patch ensure that exactly the needed profile
> is used?

Afair, x264 does change profile and / or level depending on properties
set by the user. Currently there is no way for the libavcodec user to
know that libx264 changed something.
With this change the user can know that he does not get the
requested values.

Or am I wrong and libx264 never overwrites requested values
for level and / or profile?

Carl Eugen
kjeyapal@akamai.com Nov. 24, 2017, 2:54 a.m.
On 11/24/17, 5:17 AM, "Carl Eugen Hoyos" <ceffmpeg@gmail.com> wrote:

>2017-11-23 22:58 GMT+01:00 Michael Niedermayer <michael@niedermayer.cc>:

>> On Thu, Nov 23, 2017 at 04:01:06PM +0100, Carl Eugen Hoyos wrote:

>>> Hi!

>>>

>>> The (external) encoders may overwrite level and profile because of

>>> requested encoding properties, allowing libavcodec to (also) overwrite

>>> them in the context makes sense (and is already done in some cases

>>> afaict).

>>>

>>> Please comment, Carl Eugen

>>

>> If a user needs to generate a file with a specific profile/level

>> for example because its for broadcast, some specification or a hw

>> decoder.

>> How could he after this patch ensure that exactly the needed profile

>> is used?

There is no way of ensuring the same even without this patch.

>Afair, x264 does change profile and / or level depending on properties

>set by the user. Currently there is no way for the libavcodec user to

>know that libx264 changed something.

>With this change the user can know that he does not get the

>requested values.

>

>Or am I wrong and libx264 never overwrites requested values

>for level and / or profile?

>


I just tested x264 by giving contradicting profiles and levels. 
Following are the results:
Case 1:
Input : 1080p30, 420p
Config : Set level wrongly to 1.1
Output : Throws some warnings, but level is still written wrongly as 1.1 is the SPS header.
Impact of overwrite : None. Both the user set value and header’s value are same.

Case 2:1080p30, 422p
Config : Set profile wrongly to baseline
Output : Throws errors, doesn’t encode
Impact of overwrite : None. 

Case 3:1080p30, 420p
Config : Set profile wrongly to high422
Output : No warnings or errors. Automatically switches to high profile. SPS header also has high profile.
Impact of overwrite : User will get to know the correct profile being encoded.

Regards,
Karthick
Mark Thompson Nov. 24, 2017, 11:13 a.m.
On 24/11/17 02:54, Jeyapal, Karthick wrote:
> 
> 
> On 11/24/17, 5:17 AM, "Carl Eugen Hoyos" <ceffmpeg@gmail.com> wrote:
> 
>> 2017-11-23 22:58 GMT+01:00 Michael Niedermayer <michael@niedermayer.cc>:
>>> On Thu, Nov 23, 2017 at 04:01:06PM +0100, Carl Eugen Hoyos wrote:
>>>> Hi!
>>>>
>>>> The (external) encoders may overwrite level and profile because of
>>>> requested encoding properties, allowing libavcodec to (also) overwrite
>>>> them in the context makes sense (and is already done in some cases
>>>> afaict).
>>>>
>>>> Please comment, Carl Eugen
>>>
>>> If a user needs to generate a file with a specific profile/level
>>> for example because its for broadcast, some specification or a hw
>>> decoder.
>>> How could he after this patch ensure that exactly the needed profile
>>> is used?
> There is no way of ensuring the same even without this patch.
> 
>> Afair, x264 does change profile and / or level depending on properties
>> set by the user. Currently there is no way for the libavcodec user to
>> know that libx264 changed something.
>> With this change the user can know that he does not get the
>> requested values.
>>
>> Or am I wrong and libx264 never overwrites requested values
>> for level and / or profile?
>>
> 
> I just tested x264 by giving contradicting profiles and levels. 
> Following are the results:
> Case 1:
> Input : 1080p30, 420p
> Config : Set level wrongly to 1.1
> Output : Throws some warnings, but level is still written wrongly as 1.1 is the SPS header.
> Impact of overwrite : None. Both the user set value and header’s value are same.
> 
> Case 2:1080p30, 422p
> Config : Set profile wrongly to baseline
> Output : Throws errors, doesn’t encode
> Impact of overwrite : None. 
> 
> Case 3:1080p30, 420p
> Config : Set profile wrongly to high422

That's not wrong.  High 4:2:2 profile admits bit_depth_*_minus8 == 0 and chroma_format_idc == 1.

> Output : No warnings or errors. Automatically switches to high profile. SPS header also has high profile.

That feels like ffmpeg and libx264 aren't quite agreeing about what the profile field should mean.  libx264 is using it as an extra set of constraints and then picking a profile conforming to everything specified later, whereas the usual ffmpeg use is to apply that specific setting.  The libx264-in-ffmpeg documentation does admittedly say "Set profile restrictions." kindof suggesting that use, but it is still at odds with most other encoders.

> Impact of overwrite : User will get to know the correct profile being encoded.

I don't like this overwrite, but I'm not sure what a better answer is.

- Mark
Michael Niedermayer Nov. 24, 2017, 4:42 p.m.
On Fri, Nov 24, 2017 at 12:47:16AM +0100, Carl Eugen Hoyos wrote:
> 2017-11-23 22:58 GMT+01:00 Michael Niedermayer <michael@niedermayer.cc>:
> > On Thu, Nov 23, 2017 at 04:01:06PM +0100, Carl Eugen Hoyos wrote:
> >> Hi!
> >>
> >> The (external) encoders may overwrite level and profile because of
> >> requested encoding properties, allowing libavcodec to (also) overwrite
> >> them in the context makes sense (and is already done in some cases
> >> afaict).
> >>
> >> Please comment, Carl Eugen
> >
> > If a user needs to generate a file with a specific profile/level
> > for example because its for broadcast, some specification or a hw
> > decoder.
> > How could he after this patch ensure that exactly the needed profile
> > is used?
> 
> Afair, x264 does change profile and / or level depending on properties
> set by the user. Currently there is no way for the libavcodec user to
> know that libx264 changed something.
> With this change the user can know that he does not get the
> requested values.
> 
> Or am I wrong and libx264 never overwrites requested values
> for level and / or profile?

IIUC (someone please correct me if iam wrong) libx264 can change
the profile/level to a compatible one.

The proposed API would allow any change.
That would add a requirement to the user apps to check if the actual
profile/level is compatible to the users requirements. And that would
make this require special treatment in many user apps, not just pass
user options to ffmpeg


[...]
Hendrik Leppkes Nov. 24, 2017, 4:50 p.m.
On Fri, Nov 24, 2017 at 5:42 PM, Michael Niedermayer
<michael@niedermayer.cc> wrote:
> On Fri, Nov 24, 2017 at 12:47:16AM +0100, Carl Eugen Hoyos wrote:
>> 2017-11-23 22:58 GMT+01:00 Michael Niedermayer <michael@niedermayer.cc>:
>> > On Thu, Nov 23, 2017 at 04:01:06PM +0100, Carl Eugen Hoyos wrote:
>> >> Hi!
>> >>
>> >> The (external) encoders may overwrite level and profile because of
>> >> requested encoding properties, allowing libavcodec to (also) overwrite
>> >> them in the context makes sense (and is already done in some cases
>> >> afaict).
>> >>
>> >> Please comment, Carl Eugen
>> >
>> > If a user needs to generate a file with a specific profile/level
>> > for example because its for broadcast, some specification or a hw
>> > decoder.
>> > How could he after this patch ensure that exactly the needed profile
>> > is used?
>>
>> Afair, x264 does change profile and / or level depending on properties
>> set by the user. Currently there is no way for the libavcodec user to
>> know that libx264 changed something.
>> With this change the user can know that he does not get the
>> requested values.
>>
>> Or am I wrong and libx264 never overwrites requested values
>> for level and / or profile?
>
> IIUC (someone please correct me if iam wrong) libx264 can change
> the profile/level to a compatible one.
>
> The proposed API would allow any change.
> That would add a requirement to the user apps to check if the actual
> profile/level is compatible to the users requirements. And that would
> make this require special treatment in many user apps, not just pass
> user options to ffmpeg
>

But this change doesn't actually change behavior. If you want to
propose a different wording to allow encoders to slightly adjust the
value in a compatible manner, we're all ears. :)

- Hendrik
Michael Niedermayer Nov. 24, 2017, 5:22 p.m.
On Fri, Nov 24, 2017 at 05:50:36PM +0100, Hendrik Leppkes wrote:
> On Fri, Nov 24, 2017 at 5:42 PM, Michael Niedermayer
> <michael@niedermayer.cc> wrote:
> > On Fri, Nov 24, 2017 at 12:47:16AM +0100, Carl Eugen Hoyos wrote:
> >> 2017-11-23 22:58 GMT+01:00 Michael Niedermayer <michael@niedermayer.cc>:
> >> > On Thu, Nov 23, 2017 at 04:01:06PM +0100, Carl Eugen Hoyos wrote:
> >> >> Hi!
> >> >>
> >> >> The (external) encoders may overwrite level and profile because of
> >> >> requested encoding properties, allowing libavcodec to (also) overwrite
> >> >> them in the context makes sense (and is already done in some cases
> >> >> afaict).
> >> >>
> >> >> Please comment, Carl Eugen
> >> >
> >> > If a user needs to generate a file with a specific profile/level
> >> > for example because its for broadcast, some specification or a hw
> >> > decoder.
> >> > How could he after this patch ensure that exactly the needed profile
> >> > is used?
> >>
> >> Afair, x264 does change profile and / or level depending on properties
> >> set by the user. Currently there is no way for the libavcodec user to
> >> know that libx264 changed something.
> >> With this change the user can know that he does not get the
> >> requested values.
> >>
> >> Or am I wrong and libx264 never overwrites requested values
> >> for level and / or profile?
> >
> > IIUC (someone please correct me if iam wrong) libx264 can change
> > the profile/level to a compatible one.
> >
> > The proposed API would allow any change.
> > That would add a requirement to the user apps to check if the actual
> > profile/level is compatible to the users requirements. And that would
> > make this require special treatment in many user apps, not just pass
> > user options to ffmpeg
> >
> 

> But this change doesn't actually change behavior. If you want to

No change to any (API) documentation changes implementation behavior.

It allows though implementations to change behavior and then also
requires the other side of the interface to support this.

aka badly worded API docs can lead to misunderstanding and pain


> propose a different wording to allow encoders to slightly adjust the
> value in a compatible manner, we're all ears. :)

Maybe something like this:

An encoder may change this to a compatible profile / level.
For example if profile.level 5.7 requires decoder support of lower
levels then an encoder may choose to use 5.6 instead and update these
fields.

[...]

Patch hide | download patch | download mbox

From c6a84f9d8b511a9f4db541f0271748ae5257a0ae Mon Sep 17 00:00:00 2001
From: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Date: Thu, 23 Nov 2017 15:58:33 +0100
Subject: [PATCH] lavc/avcodec: Allow lavc to overwrite
 AVCodecContext->profile and level.

This makes more sense and is already done in some cases.
---
 libavcodec/avcodec.h |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 4cd72b5..dd12353 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -2800,7 +2800,7 @@  typedef struct AVCodecContext {
 
     /**
      * profile
-     * - encoding: Set by user.
+     * - encoding: Set by user, may be overwritten by libavcodec.
      * - decoding: Set by libavcodec.
      */
      int profile;
@@ -2898,7 +2898,7 @@  typedef struct AVCodecContext {
 
     /**
      * level
-     * - encoding: Set by user.
+     * - encoding: Set by user, may be overwritten by libavcodec.
      * - decoding: Set by libavcodec.
      */
      int level;
-- 
1.7.10.4