diff mbox

[FFmpeg-devel] amfenc: Add support for pict_type field

Message ID 20190115220552.18116-1-michael.dirks@xaymar.com
State New
Headers show

Commit Message

Michael Fabian 'Xaymar' Dirks Jan. 15, 2019, 10:05 p.m. UTC
From: Michael Fabian 'Xaymar' Dirks <info@xaymar.com>

Adds support for the pict_type field in AVFrame to amf_h264 and amf_h265 simultaneously. This field is needed in cases where the application wishes to override the frame type with another one, such as forcefully inserting a key frame for chapter markers or similar.

Additionally this abuses AV_PICTURE_TYPE_S for marking Skip frames, a special type of frame in AVC, SVC and HEVC which is a flag for the decoder to repeat the last frame.

Signed-off-by: Michael Fabian 'Xaymar' Dirks <info@xaymar.com>
---
 Changelog           |  1 +
 libavcodec/amfenc.c | 46 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+)

Comments

Michael Fabian 'Xaymar' Dirks Jan. 15, 2019, 10:41 p.m. UTC | #1
On 15.01.2019 23:05, michael.dirks@xaymar.com wrote:
> From: Michael Fabian 'Xaymar' Dirks <info@xaymar.com>
>
> Adds support for the pict_type field in AVFrame to amf_h264 and amf_h265 simultaneously. This field is needed in cases where the application wishes to override the frame type with another one, such as forcefully inserting a key frame for chapter markers or similar.
>
> Additionally this abuses AV_PICTURE_TYPE_S for marking Skip frames, a special type of frame in AVC, SVC and HEVC which is a flag for the decoder to repeat the last frame.
>
> Signed-off-by: Michael Fabian 'Xaymar' Dirks <info@xaymar.com>
> ---
>   Changelog           |  1 +
>   libavcodec/amfenc.c | 46 +++++++++++++++++++++++++++++++++++++++++++++
>   2 files changed, 47 insertions(+)
>
> diff --git a/Changelog b/Changelog
> index 1cd53916cb..4fc48ba210 100644
> --- a/Changelog
> +++ b/Changelog
> @@ -15,6 +15,7 @@ version <next>:
>   - hymt decoder
>   - anlmdn filter
>   - maskfun filter
> +- AMF now supports AVFrame->pict_type override
>   
>   
>   version 4.1:
> diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c
> index 384d8efc92..3be9ff9ee2 100644
> --- a/libavcodec/amfenc.c
> +++ b/libavcodec/amfenc.c
> @@ -680,6 +680,52 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame)
>               break;
>           }
>   
> +        // Override Picture Type for Frame
> +        if (avctx->codec->id == AV_CODEC_ID_H264) {
> +            switch (frame->pict_type) {
> +            case AV_PICTURE_TYPE_I:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_I);
> +                break;
> +            case AV_PICTURE_TYPE_P:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_P);
> +                break;
> +            case AV_PICTURE_TYPE_B:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_B);
> +                break;
> +            case AV_PICTURE_TYPE_S:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_SKIP);
> +                break;
> +            default:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_NONE);
> +                break;
> +            }
> +            // Keyframe overrides previous assignment.
> +            if (frame->key_frame) {
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_IDR);
> +            }
> +        } else if (avctx->codec->id == AV_CODEC_ID_HEVC) {
> +            switch (frame->pict_type) {
> +            case AV_PICTURE_TYPE_I:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_I);
> +                break;
> +            case AV_PICTURE_TYPE_P:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_P);
> +                break;
> +            case AV_PICTURE_TYPE_B:
> +                av_log(ctx, AV_LOG_WARNING, "Ignoring B-Frame, unsupported by AMD AMF H.265 Encoder.");
> +                break;
> +            case AV_PICTURE_TYPE_S:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_SKIP);
> +                break;
> +            default:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_NONE);
> +                break;
> +            }
> +            // Keyframe overrides previous assignment.
> +            if (frame->key_frame) {
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_IDR);
> +            }
> +        }
>   
>           // submit surface
>           res = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)surface);
This patch has been tested with the small tool I wrote here: 
https://github.com/Xaymar/ffmpeg-test. It works in both H264 and H265 
and produces the correct sequence of IDR, I, P and Skip frames - 
including the visual corruption that AMD's Skip frames have.

Michael Fabian Dirks
Michael Fabian 'Xaymar' Dirks Feb. 1, 2019, 6:25 p.m. UTC | #2
On 15.01.2019 23:41, Michael Dirks wrote:
> On 15.01.2019 23:05, michael.dirks@xaymar.com wrote:
>> From: Michael Fabian 'Xaymar' Dirks <info@xaymar.com>
>>
>> Adds support for the pict_type field in AVFrame to amf_h264 and 
>> amf_h265 simultaneously. This field is needed in cases where the 
>> application wishes to override the frame type with another one, such 
>> as forcefully inserting a key frame for chapter markers or similar.
>>
>> Additionally this abuses AV_PICTURE_TYPE_S for marking Skip frames, a 
>> special type of frame in AVC, SVC and HEVC which is a flag for the 
>> decoder to repeat the last frame.
> This patch has been tested with the small tool I wrote here: 
> https://github.com/Xaymar/ffmpeg-test. It works in both H264 and H265 
> and produces the correct sequence of IDR, I, P and Skip frames - 
> including the visual corruption that AMD's Skip frames have.

As there currently seems to be no active maintainer for amfenc files, is 
there any chance of this getting merged into ffmpeg? Personally I can 
work with a patch for ffmpeg, but it makes building the project a bit 
more difficult (integrating make plus cross compiling tools is a bit much).

Sincerely
Michael Fabian 'Xaymar' Dirks
Carl Eugen Hoyos Feb. 1, 2019, 6:31 p.m. UTC | #3
2019-02-01 19:25 GMT+01:00, Michael Dirks <michael.dirks@xaymar.com>:
> On 15.01.2019 23:41, Michael Dirks wrote:
>> On 15.01.2019 23:05, michael.dirks@xaymar.com wrote:
>>> From: Michael Fabian 'Xaymar' Dirks <info@xaymar.com>
>>>
>>> Adds support for the pict_type field in AVFrame to amf_h264 and
>>> amf_h265 simultaneously. This field is needed in cases where the
>>> application wishes to override the frame type with another one, such
>>> as forcefully inserting a key frame for chapter markers or similar.
>>>
>>> Additionally this abuses AV_PICTURE_TYPE_S for marking Skip frames, a
>>> special type of frame in AVC, SVC and HEVC which is a flag for the
>>> decoder to repeat the last frame.
>> This patch has been tested with the small tool I wrote here:
>> https://github.com/Xaymar/ffmpeg-test. It works in both H264 and H265
>> and produces the correct sequence of IDR, I, P and Skip frames -
>> including the visual corruption that AMD's Skip frames have.

> As there currently seems to be no active maintainer for amfenc files

(I don't find this surprising given the history of the AMD patch.)

Could you take over mantainership?

Carl Eugen
Alexander Kravchenko Feb. 1, 2019, 6:39 p.m. UTC | #4
Hi guys,

I could take manteinership and one person from my team, hopefully it could help to move pending patches which are required to be applied and have been waiting for a long time

 

Thanks

Alexander

 

 

От: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> от имени Carl Eugen Hoyos <ceffmpeg@gmail.com>
Обратный адрес: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Дата: пятница, 1 февраля 2019 г., 21:31
Кому: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Тема: Re: [FFmpeg-devel] [PATCH] amfenc: Add support for pict_type field

 

2019-02-01 19:25 GMT+01:00, Michael Dirks <michael.dirks@xaymar.com>:

On 15.01.2019 23:41, Michael Dirks wrote:

On 15.01.2019 23:05, michael.dirks@xaymar.com wrote:

From: Michael Fabian 'Xaymar' Dirks <info@xaymar.com>

 

Adds support for the pict_type field in AVFrame to amf_h264 and

amf_h265 simultaneously. This field is needed in cases where the

application wishes to override the frame type with another one, such

as forcefully inserting a key frame for chapter markers or similar.

 

Additionally this abuses AV_PICTURE_TYPE_S for marking Skip frames, a

special type of frame in AVC, SVC and HEVC which is a flag for the

decoder to repeat the last frame.

This patch has been tested with the small tool I wrote here:

https://github.com/Xaymar/ffmpeg-test. It works in both H264 and H265

and produces the correct sequence of IDR, I, P and Skip frames -

including the visual corruption that AMD's Skip frames have.

 

As there currently seems to be no active maintainer for amfenc files

 

(I don't find this surprising given the history of the AMD patch.)

 

Could you take over mantainership?

 

Carl Eugen
Michael Fabian 'Xaymar' Dirks Feb. 1, 2019, 6:48 p.m. UTC | #5
On 01.02.2019 19:31, Carl Eugen Hoyos wrote:
> Could you take over mantainership?

Unfortunately I lack the necessary FFmpeg experience for this task. Even 
though I am the maintainer of the OBS Studio AMD Encoder integration, 
I'd be horribly lost in FFmpegs code when it comes to AMD/AMF.

Sincerely

Michael Fabian Dirks
Carl Eugen Hoyos Feb. 1, 2019, 6:51 p.m. UTC | #6
2019-02-01 19:39 GMT+01:00, Alexander Kravchenko <akravchenko188@gmail.com>:

> I could take manteinership and one person from my team, hopefully
> it could help to move pending patches which are required to be
> applied and have been waiting for a long time

A good start would be to review this patch an stop top-posting.

Carl Eugen
Alexander Kravchenko Feb. 1, 2019, 7:32 p.m. UTC | #7
Hello Carl

Ok, I’m going to test and coming back.

 

Thanks,

Alexander

 

От: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> от имени Carl Eugen Hoyos <ceffmpeg@gmail.com>
Обратный адрес: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Дата: пятница, 1 февраля 2019 г., 21:51
Кому: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Тема: Re: [FFmpeg-devel] [PATCH] amfenc: Add support for pict_type field

 

2019-02-01 19:39 GMT+01:00, Alexander Kravchenko <akravchenko188@gmail.com>:

 

I could take manteinership and one person from my team, hopefully

it could help to move pending patches which are required to be

applied and have been waiting for a long time

 

A good start would be to review this patch an stop top-posting.

 

Carl Eugen
Mark Thompson Feb. 4, 2019, 10:05 a.m. UTC | #8
On 15/01/2019 22:05, michael.dirks@xaymar.com wrote:
> From: Michael Fabian 'Xaymar' Dirks <info@xaymar.com>
> 
> Adds support for the pict_type field in AVFrame to amf_h264 and amf_h265 simultaneously. This field is needed in cases where the application wishes to override the frame type with another one, such as forcefully inserting a key frame for chapter markers or similar.
> 
> Additionally this abuses AV_PICTURE_TYPE_S for marking Skip frames, a special type of frame in AVC, SVC and HEVC which is a flag for the decoder to repeat the last frame.

Overloading the meaning of a flag like this is probably confusing.

Can you explain what this "skip frame" actually does in the encoder?  The concept does not exist in H.264 or H.265, as far as I know.

>  Changelog           |  1 +
>  libavcodec/amfenc.c | 46 +++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 47 insertions(+)
> 
> diff --git a/Changelog b/Changelog
> index 1cd53916cb..4fc48ba210 100644
> --- a/Changelog
> +++ b/Changelog
> @@ -15,6 +15,7 @@ version <next>:
>  - hymt decoder
>  - anlmdn filter
>  - maskfun filter
> +- AMF now supports AVFrame->pict_type override
>  
>  
>  version 4.1:
> diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c
> index 384d8efc92..3be9ff9ee2 100644
> --- a/libavcodec/amfenc.c
> +++ b/libavcodec/amfenc.c
> @@ -680,6 +680,52 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame)
>              break;
>          }
>  
> +        // Override Picture Type for Frame
> +        if (avctx->codec->id == AV_CODEC_ID_H264) {
> +            switch (frame->pict_type) {
> +            case AV_PICTURE_TYPE_I:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_I);

Consider whether you need to set IDR here rather than I, and maybe add a comment explaining the choice?  The normal use of this is to indicate that an IRAP should be generated, not just a single intra picture.  (Compare libx264, which has an additional flag controlling whether the forced picture is IDR or I, but I believe it is still always an IRAP there.)

> +                break;
> +            case AV_PICTURE_TYPE_P:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_P);
> +                break;
> +            case AV_PICTURE_TYPE_B:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_B);
> +                break;
> +            case AV_PICTURE_TYPE_S:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_SKIP);
> +                break;
> +            default:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_NONE);
> +                break;
> +            }
> +            // Keyframe overrides previous assignment.
> +            if (frame->key_frame) {

This flag shouldn't be read by encoders.  (I believe it is set by the decoder and never cleared, so with this test you will have surprising behaviour where the output stream gets key frames everywhere that the input stream had them, in addition to those dictated by its own GOP structure.)

> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_IDR);
> +            }
> +        } else if (avctx->codec->id == AV_CODEC_ID_HEVC) {
> +            switch (frame->pict_type) {
> +            case AV_PICTURE_TYPE_I:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_I);
> +                break;
> +            case AV_PICTURE_TYPE_P:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_P);
> +                break;
> +            case AV_PICTURE_TYPE_B:
> +                av_log(ctx, AV_LOG_WARNING, "Ignoring B-Frame, unsupported by AMD AMF H.265 Encoder.");
> +                break;
> +            case AV_PICTURE_TYPE_S:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_SKIP);
> +                break;
> +            default:
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_NONE);
> +                break;
> +            }
> +            // Keyframe overrides previous assignment.
> +            if (frame->key_frame) {
> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_IDR);
> +            }
> +        }
>  
>          // submit surface
>          res = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)surface);
> 

- Mark
Michael Fabian 'Xaymar' Dirks Feb. 4, 2019, 2:41 p.m. UTC | #9
On 04.02.2019 11:05, Mark Thompson wrote:
> Can you explain what this "skip frame" actually does in the encoder?  The concept does not exist in H.264 or H.265, as far as I know.

I believe this has to do with the pic_struct flag which has a value of 7 
for frame doubling and 8 for frame tripling, see page 345 (PDF page 367) 
Table D-1 Interpretation of pic_struct in Rec. ITU-T H.264 (04/2017). 
However I do not work for AMD (and their encoder is closed source and a 
piece of hardware), so I can't confirm that this is actually the 
behavior, nor do I have any tools that can read and show all H.264 
headers. An alternative would be that AMDs encoder is instead choosing 
to emit a frame that has maximum compression and references the previous 
frame for all data, thus causing a copy of it.

>> +            case AV_PICTURE_TYPE_I:
>> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_I);
> Consider whether you need to set IDR here rather than I, and maybe add a comment explaining the choice?  The normal use of this is to indicate that an IRAP should be generated, not just a single intra picture.  (Compare libx264, which has an additional flag controlling whether the forced picture is IDR or I, but I believe it is still always an IRAP there.)
>> +            // Keyframe overrides previous assignment.
>> +            if (frame->key_frame) {
> This flag shouldn't be read by encoders.  (I believe it is set by the decoder and never cleared, so with this test you will have surprising behaviour where the output stream gets key frames everywhere that the input stream had them, in addition to those dictated by its own GOP structure.)
I went by the documentation in the individual header files, which does 
not actually claim key_frame to be a decoder only field (libavutil/frame.h):

 > /**
 >     * 1 -> keyframe, 0-> not
 >     */
 >    int key_frame;

Therefore this patch uses the field exactly as it is documented in the 
frame.h file, and also treats AV_PICTURE_TYPE_I as a request for an 
Intra Picture instead of an IDR Picture.

Sincerely
Michael Fabian Dirks
Mark Thompson Feb. 10, 2019, 6:32 p.m. UTC | #10
On 04/02/2019 14:41, Michael Dirks wrote:
> On 04.02.2019 11:05, Mark Thompson wrote:
>> Can you explain what this "skip frame" actually does in the encoder?  The concept does not exist in H.264 or H.265, as far as I know.
> 
> I believe this has to do with the pic_struct flag which has a value of 7 for frame doubling and 8 for frame tripling, see page 345 (PDF page 367) Table D-1 Interpretation of pic_struct in Rec. ITU-T H.264 (04/2017). However I do not work for AMD (and their encoder is closed source and a piece of hardware), so I can't confirm that this is actually the behavior, nor do I have any tools that can read and show all H.264 headers. An alternative would be that AMDs encoder is instead choosing to emit a frame that has maximum compression and references the previous frame for all data, thus causing a copy of it.

You can see this with the trace_headers bitstream filter.  (Try "-c:v h264_amf -options... -bsf:v trace_headers -f null -", or it can be used with streamcopy to examine an existing file.)

>>> +            case AV_PICTURE_TYPE_I:
>>> +                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_I);
>> Consider whether you need to set IDR here rather than I, and maybe add a comment explaining the choice?  The normal use of this is to indicate that an IRAP should be generated, not just a single intra picture.  (Compare libx264, which has an additional flag controlling whether the forced picture is IDR or I, but I believe it is still always an IRAP there.)
>>> +            // Keyframe overrides previous assignment.
>>> +            if (frame->key_frame) {
>> This flag shouldn't be read by encoders.  (I believe it is set by the decoder and never cleared, so with this test you will have surprising behaviour where the output stream gets key frames everywhere that the input stream had them, in addition to those dictated by its own GOP structure.)
> I went by the documentation in the individual header files, which does not actually claim key_frame to be a decoder only field (libavutil/frame.h):
> 
>> /**
>>     * 1 -> keyframe, 0-> not
>>     */
>>    int key_frame;
> 
> Therefore this patch uses the field exactly as it is documented in the frame.h file, and also treats AV_PICTURE_TYPE_I as a request for an Intra Picture instead of an IDR Picture.

Well yes, but that's not actually going to work - try it with a set of JPEGs as input and you'll observe the problem.  (Note that no other encoder uses this field on input, though some do use it internally.)

- Mark
Thilo Borgmann via ffmpeg-devel April 12, 2019, 1:37 a.m. UTC | #11
On 10.02.2019 19:32, Mark Thompson wrote:
> On 04/02/2019 14:41, Michael Dirks wrote:
> You can see this with the trace_headers bitstream filter.  (Try "-c:v h264_amf -options... -bsf:v trace_headers -f null -", or it can be used with streamcopy to examine an existing file.)
>
> Well yes, but that's not actually going to work - try it with a set of JPEGs as input and you'll observe the problem.  (Note that no other encoder uses this field on input, though some do use it internally.)
>
> - Mark
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Hi Mark,

I should have a newer AMD card to test with in the near future, so I 
should be able to confirm the behavior across multiple generations of 
cards (GCN2, GCN3, GCN4, GCN4.1? (Vega)). As I currently only have the 
ability to test GCN2, GCN3 and Polaris (GCN4) I'm not fully sure how it 
behaves yet.

As for the key_frame change, you are correct. How do you actually insert 
a normal I frame without the extra overhead of a IDR frame in ffmpeg? 
The AMD AMF Encoder is not exactly efficient in speed, quality or power 
usage, and inserting an IDR frame does more "damage" to the AMF encoded 
stream than just inserting an I frame.
diff mbox

Patch

diff --git a/Changelog b/Changelog
index 1cd53916cb..4fc48ba210 100644
--- a/Changelog
+++ b/Changelog
@@ -15,6 +15,7 @@  version <next>:
 - hymt decoder
 - anlmdn filter
 - maskfun filter
+- AMF now supports AVFrame->pict_type override
 
 
 version 4.1:
diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c
index 384d8efc92..3be9ff9ee2 100644
--- a/libavcodec/amfenc.c
+++ b/libavcodec/amfenc.c
@@ -680,6 +680,52 @@  int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame)
             break;
         }
 
+        // Override Picture Type for Frame
+        if (avctx->codec->id == AV_CODEC_ID_H264) {
+            switch (frame->pict_type) {
+            case AV_PICTURE_TYPE_I:
+                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_I);
+                break;
+            case AV_PICTURE_TYPE_P:
+                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_P);
+                break;
+            case AV_PICTURE_TYPE_B:
+                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_B);
+                break;
+            case AV_PICTURE_TYPE_S:
+                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_SKIP);
+                break;
+            default:
+                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_NONE);
+                break;
+            }
+            // Keyframe overrides previous assignment.
+            if (frame->key_frame) {
+                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_IDR);
+            }
+        } else if (avctx->codec->id == AV_CODEC_ID_HEVC) {
+            switch (frame->pict_type) {
+            case AV_PICTURE_TYPE_I:
+                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_I);
+                break;
+            case AV_PICTURE_TYPE_P:
+                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_P);
+                break;
+            case AV_PICTURE_TYPE_B:
+                av_log(ctx, AV_LOG_WARNING, "Ignoring B-Frame, unsupported by AMD AMF H.265 Encoder.");
+                break;
+            case AV_PICTURE_TYPE_S:
+                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_SKIP);
+                break;
+            default:
+                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_NONE);
+                break;
+            }
+            // Keyframe overrides previous assignment.
+            if (frame->key_frame) {
+                AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_IDR);
+            }
+        }
 
         // submit surface
         res = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)surface);