| Message ID | 20231007162503.1057-1-jamrial@gmail.com |
|---|---|
| Headers | show |
| Series | Container level frame cropping parameters | expand |
Quoting James Almer (2023-10-07 18:24:56) > This is a simple set to add support to stream wide, container level cropping > parameters, as featured in formats like Matroska and ISOBMFF. It's needed for > things like AV1 streams generated by certain hardware encoders that produce > dimension aligned output, and unlike H26x, can't export cropping info within > the bitstream. > In this set i add the packet side data type, mux and demux support to Matroska, > and handling code to ffmpeg and ffplay. Missing is support for clap boxes in > ISOBMFF, and exporting the relevant side data from encoders like AMF. > > It's a rebased and updated version to the set i sent a month ago. Why is there no lavc-decoder handling? I would expect cropping side data submitted to lavc to be somehow propagated to decoded frames. Though there is a question of how to combine side data with the codec-level cropping.
On 10/10/2023 8:17 AM, Anton Khirnov wrote: > Quoting James Almer (2023-10-07 18:24:56) >> This is a simple set to add support to stream wide, container level cropping >> parameters, as featured in formats like Matroska and ISOBMFF. It's needed for >> things like AV1 streams generated by certain hardware encoders that produce >> dimension aligned output, and unlike H26x, can't export cropping info within >> the bitstream. >> In this set i add the packet side data type, mux and demux support to Matroska, >> and handling code to ffmpeg and ffplay. Missing is support for clap boxes in >> ISOBMFF, and exporting the relevant side data from encoders like AMF. >> >> It's a rebased and updated version to the set i sent a month ago. > > Why is there no lavc-decoder handling? I would expect cropping side data > submitted to lavc to be somehow propagated to decoded frames. > Though there is a question of how to combine side data with the > codec-level cropping. I thought about adding lavc support alongside the CLI handling, yeah. And it should be a matter of adding the values to the existing ones in frame->crop_* as exported by decoders, which av_frame_apply_cropping() will then handle. And the reason i add support to the CLI by including a filter instead of leaving it up to lavc is that av_frame_apply_cropping() may not do exact cropping if AV_CODEC_FLAG_UNALIGNED is not set.
Quoting James Almer (2023-10-10 13:44:19) > On 10/10/2023 8:17 AM, Anton Khirnov wrote: > > Quoting James Almer (2023-10-07 18:24:56) > >> This is a simple set to add support to stream wide, container level cropping > >> parameters, as featured in formats like Matroska and ISOBMFF. It's needed for > >> things like AV1 streams generated by certain hardware encoders that produce > >> dimension aligned output, and unlike H26x, can't export cropping info within > >> the bitstream. > >> In this set i add the packet side data type, mux and demux support to Matroska, > >> and handling code to ffmpeg and ffplay. Missing is support for clap boxes in > >> ISOBMFF, and exporting the relevant side data from encoders like AMF. > >> > >> It's a rebased and updated version to the set i sent a month ago. > > > > Why is there no lavc-decoder handling? I would expect cropping side data > > submitted to lavc to be somehow propagated to decoded frames. > > Though there is a question of how to combine side data with the > > codec-level cropping. > > I thought about adding lavc support alongside the CLI handling, yeah. > And it should be a matter of adding the values to the existing ones in > frame->crop_* as exported by decoders, which av_frame_apply_cropping() > will then handle. > > And the reason i add support to the CLI by including a filter instead of > leaving it up to lavc is that av_frame_apply_cropping() may not do exact > cropping if AV_CODEC_FLAG_UNALIGNED is not set. I intend to have the CLI always set AVCodecContext.apply_cropping=0 and have it handled by lavfi for this exact reason, just didn't get to actually doing it yet. Feel free and welcome to do it yourself.
On 10/10/2023 8:54 AM, Anton Khirnov wrote: > Quoting James Almer (2023-10-10 13:44:19) >> On 10/10/2023 8:17 AM, Anton Khirnov wrote: >>> Quoting James Almer (2023-10-07 18:24:56) >>>> This is a simple set to add support to stream wide, container level cropping >>>> parameters, as featured in formats like Matroska and ISOBMFF. It's needed for >>>> things like AV1 streams generated by certain hardware encoders that produce >>>> dimension aligned output, and unlike H26x, can't export cropping info within >>>> the bitstream. >>>> In this set i add the packet side data type, mux and demux support to Matroska, >>>> and handling code to ffmpeg and ffplay. Missing is support for clap boxes in >>>> ISOBMFF, and exporting the relevant side data from encoders like AMF. >>>> >>>> It's a rebased and updated version to the set i sent a month ago. >>> >>> Why is there no lavc-decoder handling? I would expect cropping side data >>> submitted to lavc to be somehow propagated to decoded frames. >>> Though there is a question of how to combine side data with the >>> codec-level cropping. >> >> I thought about adding lavc support alongside the CLI handling, yeah. >> And it should be a matter of adding the values to the existing ones in >> frame->crop_* as exported by decoders, which av_frame_apply_cropping() >> will then handle. >> >> And the reason i add support to the CLI by including a filter instead of >> leaving it up to lavc is that av_frame_apply_cropping() may not do exact >> cropping if AV_CODEC_FLAG_UNALIGNED is not set. > > I intend to have the CLI always set AVCodecContext.apply_cropping=0 and > have it handled by lavfi for this exact reason, just didn't get to > actually doing it yet. Feel free and welcome to do it yourself. The cropping filter doesn't look at the frame cropping fields, though. It in fact exports cropping into them if you pass it hw frames. It only looks at input arguments. So it doesn't seem like that can be done without changing the filter.
Quoting James Almer (2023-10-10 13:58:00) > On 10/10/2023 8:54 AM, Anton Khirnov wrote: > > Quoting James Almer (2023-10-10 13:44:19) > >> On 10/10/2023 8:17 AM, Anton Khirnov wrote: > >>> Quoting James Almer (2023-10-07 18:24:56) > >>>> This is a simple set to add support to stream wide, container level cropping > >>>> parameters, as featured in formats like Matroska and ISOBMFF. It's needed for > >>>> things like AV1 streams generated by certain hardware encoders that produce > >>>> dimension aligned output, and unlike H26x, can't export cropping info within > >>>> the bitstream. > >>>> In this set i add the packet side data type, mux and demux support to Matroska, > >>>> and handling code to ffmpeg and ffplay. Missing is support for clap boxes in > >>>> ISOBMFF, and exporting the relevant side data from encoders like AMF. > >>>> > >>>> It's a rebased and updated version to the set i sent a month ago. > >>> > >>> Why is there no lavc-decoder handling? I would expect cropping side data > >>> submitted to lavc to be somehow propagated to decoded frames. > >>> Though there is a question of how to combine side data with the > >>> codec-level cropping. > >> > >> I thought about adding lavc support alongside the CLI handling, yeah. > >> And it should be a matter of adding the values to the existing ones in > >> frame->crop_* as exported by decoders, which av_frame_apply_cropping() > >> will then handle. > >> > >> And the reason i add support to the CLI by including a filter instead of > >> leaving it up to lavc is that av_frame_apply_cropping() may not do exact > >> cropping if AV_CODEC_FLAG_UNALIGNED is not set. > > > > I intend to have the CLI always set AVCodecContext.apply_cropping=0 and > > have it handled by lavfi for this exact reason, just didn't get to > > actually doing it yet. Feel free and welcome to do it yourself. > > The cropping filter doesn't look at the frame cropping fields, though. > It in fact exports cropping into them if you pass it hw frames. It only > looks at input arguments. So it doesn't seem like that can be done > without changing the filter. I mean ffmpeg CLI would look at the frame cropping fields and insert the crop filter accordingly, as it does for other bits of metadata.
This is a simple set to add support to stream wide, container level cropping parameters, as featured in formats like Matroska and ISOBMFF. It's needed for things like AV1 streams generated by certain hardware encoders that produce dimension aligned output, and unlike H26x, can't export cropping info within the bitstream. In this set i add the packet side data type, mux and demux support to Matroska, and handling code to ffmpeg and ffplay. Missing is support for clap boxes in ISOBMFF, and exporting the relevant side data from encoders like AMF. It's a rebased and updated version to the set i sent a month ago. James Almer (7): avcodec/packet: add a decoded frame cropping side data type avformat/dump: print Frame Cropping side data info avformat/matroskadec: export cropping values avformat/avformat: add a flag to signal muxers that support storing cropping values avformat/matroskaenc: support writing cropping values fftools/ffmpeg: support applying container level cropping fftools/ffplay: support applying container level cropping fftools/ffmpeg.h | 3 ++ fftools/ffmpeg_demux.c | 6 ++++ fftools/ffmpeg_enc.c | 24 ++++++++-------- fftools/ffmpeg_filter.c | 23 +++++++++++++++ fftools/ffmpeg_opt.c | 3 ++ fftools/ffplay.c | 26 +++++++++++++++++ libavcodec/packet.h | 14 ++++++++++ libavformat/avformat.h | 1 + libavformat/dump.c | 21 ++++++++++++++ libavformat/matroskadec.c | 53 ++++++++++++++++++++++++++++------- libavformat/matroskaenc.c | 59 ++++++++++++++++++++++++++++++--------- libavformat/mux.c | 8 ++++++ 12 files changed, 207 insertions(+), 34 deletions(-)