[FFmpeg-devel] avcodec: libdav1d AV1 decoder wrapper

Submitted by James Almer on Sept. 28, 2018, 9:42 p.m.

Details

Message ID 20180928214202.6884-1-jamrial@gmail.com
State New
Headers show

Commit Message

James Almer Sept. 28, 2018, 9:42 p.m.
From: Ronald S. Bultje <rsbultje@gmail.com>

Originally written by Ronald S. Bultje, with fixes, optimizations and
improvements by James Almer.

Signed-off-by: James Almer <jamrial@gmail.com>
---
Missing Changelog entry and version bump.

 configure              |   4 +
 libavcodec/Makefile    |   1 +
 libavcodec/allcodecs.c |   1 +
 libavcodec/libdav1d.c  | 267 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 273 insertions(+)
 create mode 100644 libavcodec/libdav1d.c

Comments

Rostislav Pehlivanov Sept. 29, 2018, 9:57 a.m.
On Fri, 28 Sep 2018 at 22:51, James Almer <jamrial@gmail.com> wrote:

> From: Ronald S. Bultje <rsbultje@gmail.com>
>
> Originally written by Ronald S. Bultje, with fixes, optimizations and
> improvements by James Almer.
>
> Signed-off-by: James Almer <jamrial@gmail.com>
>

I'd much rather go with the original intent which was to merge the decoder
into lavc.
We already have libaomdec which is fast-ish but works as a stop-gap
solution until we do that.
Hendrik Leppkes Sept. 29, 2018, 11:05 a.m.
On Sat, Sep 29, 2018 at 12:05 PM Rostislav Pehlivanov
<atomnuker@gmail.com> wrote:
>
> On Fri, 28 Sep 2018 at 22:51, James Almer <jamrial@gmail.com> wrote:
>
> > From: Ronald S. Bultje <rsbultje@gmail.com>
> >
> > Originally written by Ronald S. Bultje, with fixes, optimizations and
> > improvements by James Almer.
> >
> > Signed-off-by: James Almer <jamrial@gmail.com>
> >
>
> I'd much rather go with the original intent which was to merge the decoder
> into lavc.
> We already have libaomdec which is fast-ish but works as a stop-gap
> solution until we do that.

I would definitely support that. Not having an internal decoder for
av1 would be a real pain.

- Hendrik
James Almer Sept. 29, 2018, 12:52 p.m.
On 9/29/2018 8:05 AM, Hendrik Leppkes wrote:
> On Sat, Sep 29, 2018 at 12:05 PM Rostislav Pehlivanov
> <atomnuker@gmail.com> wrote:
>>
>> On Fri, 28 Sep 2018 at 22:51, James Almer <jamrial@gmail.com> wrote:
>>
>>> From: Ronald S. Bultje <rsbultje@gmail.com>
>>>
>>> Originally written by Ronald S. Bultje, with fixes, optimizations and
>>> improvements by James Almer.
>>>
>>> Signed-off-by: James Almer <jamrial@gmail.com>
>>>
>>
>> I'd much rather go with the original intent which was to merge the decoder
>> into lavc.
>> We already have libaomdec which is fast-ish but works as a stop-gap
>> solution until we do that.
> 
> I would definitely support that. Not having an internal decoder for
> av1 would be a real pain.
> 
> - Hendrik

I want the same, but until that happens this wrapper wont hurt, much
like the one we added for libdcadec a few years ago before it was ported
into an internal one.

If we don't add this wrapper in the meantime, people will start asking
us about it.
Devin Heitmueller Sept. 29, 2018, 12:54 p.m.
On Sat, Sep 29, 2018 at 6:04 AM Rostislav Pehlivanov
<atomnuker@gmail.com> wrote:
> I'd much rather go with the original intent which was to merge the decoder
> into lavc.

Ronald can correct me if I'm wrong, but I suspect a key goal behind
the decoder was to have a standalone library which could be shared
across a variety of projects (both open and closed source).  Merging
it in directly will create a maintenance headache as the two source
trees diverge.  It also makes unit testing more difficult, since
Ronald/VideoLAN can write unit tests for the library (which will
presumably be consumed by a number of projects/products) and be
confident that those same unit tests apply to the version that is used
by ffmpeg.

I don't think having libx264/libx265 out of tree hasn't been a
nightmare for anyone.  I don't think this case would be any different.

Devin
James Almer Sept. 29, 2018, 1:04 p.m.
On 9/29/2018 9:54 AM, Devin Heitmueller wrote:
> On Sat, Sep 29, 2018 at 6:04 AM Rostislav Pehlivanov
> <atomnuker@gmail.com> wrote:
>> I'd much rather go with the original intent which was to merge the decoder
>> into lavc.
> 
> Ronald can correct me if I'm wrong, but I suspect a key goal behind
> the decoder was to have a standalone library which could be shared
> across a variety of projects (both open and closed source).  Merging
> it in directly will create a maintenance headache as the two source
> trees diverge.  It also makes unit testing more difficult, since
> Ronald/VideoLAN can write unit tests for the library (which will
> presumably be consumed by a number of projects/products) and be
> confident that those same unit tests apply to the version that is used
> by ffmpeg.
> 
> I don't think having libx264/libx265 out of tree hasn't been a
> nightmare for anyone.  I don't think this case would be any different.
> 
> Devin

There's a considerable difference between an encoder and a decoder from
a ffmpeg PoV. For starters, hwaccel hooks, and then threading is instead
handled by the library.
Paul B Mahol Sept. 29, 2018, 1:10 p.m.
On 9/29/18, Devin Heitmueller <dheitmueller@kernellabs.com> wrote:
> On Sat, Sep 29, 2018 at 6:04 AM Rostislav Pehlivanov
> <atomnuker@gmail.com> wrote:
>> I'd much rather go with the original intent which was to merge the decoder
>> into lavc.
>
> Ronald can correct me if I'm wrong, but I suspect a key goal behind
> the decoder was to have a standalone library which could be shared
> across a variety of projects (both open and closed source).  Merging
> it in directly will create a maintenance headache as the two source
> trees diverge.  It also makes unit testing more difficult, since
> Ronald/VideoLAN can write unit tests for the library (which will
> presumably be consumed by a number of projects/products) and be
> confident that those same unit tests apply to the version that is used
> by ffmpeg.
>
> I don't think having libx264/libx265 out of tree hasn't been a
> nightmare for anyone.  I don't think this case would be any different.
>

I simply disagree with you. Encoders are different thing.

We need native decoders.
Hendrik Leppkes Sept. 29, 2018, 1:14 p.m.
On Sat, Sep 29, 2018 at 3:02 PM Devin Heitmueller
<dheitmueller@kernellabs.com> wrote:
>
> On Sat, Sep 29, 2018 at 6:04 AM Rostislav Pehlivanov
> <atomnuker@gmail.com> wrote:
> > I'd much rather go with the original intent which was to merge the decoder
> > into lavc.
>
> Ronald can correct me if I'm wrong, but I suspect a key goal behind
> the decoder was to have a standalone library which could be shared
> across a variety of projects (both open and closed source).  Merging
> it in directly will create a maintenance headache as the two source
> trees diverge.  It also makes unit testing more difficult, since
> Ronald/VideoLAN can write unit tests for the library (which will
> presumably be consumed by a number of projects/products) and be
> confident that those same unit tests apply to the version that is used
> by ffmpeg.
>
> I don't think having libx264/libx265 out of tree hasn't been a
> nightmare for anyone.  I don't think this case would be any different.
>

Decoders and encoders are quite a different type of beast. For one
decoders are ultimately "finished" at some point, where the number of
changes drastically reduces over time, and decoding is the bread and
butter of avcodec, which would give immediate av1 support to hundreds
if not thousands of applications out there, without any further
developer intervention.

Additionally, I can already name an important key area where not
having an internal decoder will be a nightmare: hardware decoding.
Our hardware decoding framework heavily relies on having a decoder
that does all the header parsing, frame management, and whatnot, and
the hardware acceleration hooks just by-pass the actual slice decoding
then. If we don't have an internal decoder, we would have to either
build half decoder just to hook up hwaccel, or dav1d would have to
give us extremely detailed bitstream information and slice-level
bitstream hooks, which I don't think is very practical either.

A new mainstream codec without hardware acceleration support is not
going to make it, no matter how fast you make the decoder. I'm still a
bit annoyed that this wasn't even a consideration when it was decided
to move it into a separate library, but its not like they actually
asked for arguments.

- Hendrik
James Almer Sept. 29, 2018, 2:19 p.m.
On 9/29/2018 10:14 AM, Hendrik Leppkes wrote:
> On Sat, Sep 29, 2018 at 3:02 PM Devin Heitmueller
> <dheitmueller@kernellabs.com> wrote:
>>
>> On Sat, Sep 29, 2018 at 6:04 AM Rostislav Pehlivanov
>> <atomnuker@gmail.com> wrote:
>>> I'd much rather go with the original intent which was to merge the decoder
>>> into lavc.
>>
>> Ronald can correct me if I'm wrong, but I suspect a key goal behind
>> the decoder was to have a standalone library which could be shared
>> across a variety of projects (both open and closed source).  Merging
>> it in directly will create a maintenance headache as the two source
>> trees diverge.  It also makes unit testing more difficult, since
>> Ronald/VideoLAN can write unit tests for the library (which will
>> presumably be consumed by a number of projects/products) and be
>> confident that those same unit tests apply to the version that is used
>> by ffmpeg.
>>
>> I don't think having libx264/libx265 out of tree hasn't been a
>> nightmare for anyone.  I don't think this case would be any different.
>>
> 
> Decoders and encoders are quite a different type of beast. For one
> decoders are ultimately "finished" at some point, where the number of
> changes drastically reduces over time, and decoding is the bread and
> butter of avcodec, which would give immediate av1 support to hundreds
> if not thousands of applications out there, without any further
> developer intervention.

I'm guessing LGPL is a deterrent for some potential users VideoLAN
wanted for this decoder, hence the 2-Clause BSD license in libdav1d.

> 
> Additionally, I can already name an important key area where not
> having an internal decoder will be a nightmare: hardware decoding.
> Our hardware decoding framework heavily relies on having a decoder
> that does all the header parsing, frame management, and whatnot, and
> the hardware acceleration hooks just by-pass the actual slice decoding
> then. If we don't have an internal decoder, we would have to either
> build half decoder just to hook up hwaccel, or dav1d would have to
> give us extremely detailed bitstream information and slice-level
> bitstream hooks, which I don't think is very practical either.
> 
> A new mainstream codec without hardware acceleration support is not
> going to make it, no matter how fast you make the decoder.

Yeah, this is one of the main reasons to have it as an internal decoder.
But considering actual hardware capable of decoding AV1 will not come
out until like late 2019 at the earliest, there's time for libdav1d to
mature before porting it becomes a must.
At some point it will be "complete" and development pretty much halted,
much like vp9 currently is (not counting some missing assembly), so the
argument about extra maintenance of separate codebases will not apply
anymore.

And to be honest the extra maintenance argument is not too strong to
begin with, considering all the time lost having to NIH a lot of
standard features already available in libavutil, like memory allocator
wrappers and reference counted buffers, or muxer/demuxers to use with
their CLI, all already available in libavformat, because they can't link
to either of them.

> I'm still a
> bit annoyed that this wasn't even a consideration when it was decided
> to move it into a separate library, but its not like they actually
> asked for arguments.
> 
> - Hendrik
Jean-Baptiste Kempf Sept. 29, 2018, 2:36 p.m.
On Sat, 29 Sep 2018, at 15:14, Hendrik Leppkes wrote:
>  I'm still a bit annoyed that this wasn't even a consideration when it was

How do you know it was not considered?
Knowing that most developers are part of the FFmpeg/x264/VLC community, you can be sure it was considered.

> decided to move it into a separate library, but its not like they actually
> asked for arguments.

Arguments have been asked to numerous people, who just might have a different opinion that yours.

Also, it was not 'moved' to a separate library, but created elsewhere.
Paul B Mahol Sept. 29, 2018, 2:46 p.m.
On 9/29/18, Jean-Baptiste Kempf <jb@videolan.org> wrote:
> On Sat, 29 Sep 2018, at 15:14, Hendrik Leppkes wrote:
>>  I'm still a bit annoyed that this wasn't even a consideration when it was
>
> How do you know it was not considered?
> Knowing that most developers are part of the FFmpeg/x264/VLC community, you
> can be sure it was considered.
>
>> decided to move it into a separate library, but its not like they actually
>> asked for arguments.
>
> Arguments have been asked to numerous people, who just might have a
> different opinion that yours.
>
> Also, it was not 'moved' to a separate library, but created elsewhere.

Was I asked?
Hendrik Leppkes Sept. 29, 2018, 3:32 p.m.
On Sat, Sep 29, 2018 at 4:37 PM Jean-Baptiste Kempf <jb@videolan.org> wrote:
>
> On Sat, 29 Sep 2018, at 15:14, Hendrik Leppkes wrote:
> >  I'm still a bit annoyed that this wasn't even a consideration when it was
>
> How do you know it was not considered?

I took BBB by his word when he said so.

- Hendrik
Jean-Baptiste Kempf Sept. 29, 2018, 3:46 p.m.
On Sat, 29 Sep 2018, at 17:32, Hendrik Leppkes wrote:
> On Sat, Sep 29, 2018 at 4:37 PM Jean-Baptiste Kempf <jb@videolan.org> wrote:
> >
> > On Sat, 29 Sep 2018, at 15:14, Hendrik Leppkes wrote:
> > >  I'm still a bit annoyed that this wasn't even a consideration when it was
> >
> > How do you know it was not considered?
> 
> I took BBB by his word when he said so.

You really think that the person that wrote ffvp9 (and merged it inside libavcodec) and a large part of dav1d, and who is a large contributor to FFmpeg, did not consider whether it was better to start inside libavcodec or as an external library?

Come on...
Hendrik Leppkes Sept. 29, 2018, 4:16 p.m.
On Sat, Sep 29, 2018 at 5:46 PM Jean-Baptiste Kempf <jb@videolan.org> wrote:
>
> On Sat, 29 Sep 2018, at 17:32, Hendrik Leppkes wrote:
> > On Sat, Sep 29, 2018 at 4:37 PM Jean-Baptiste Kempf <jb@videolan.org> wrote:
> > >
> > > On Sat, 29 Sep 2018, at 15:14, Hendrik Leppkes wrote:
> > > >  I'm still a bit annoyed that this wasn't even a consideration when it was
> > >
> > > How do you know it was not considered?
> >
> > I took BBB by his word when he said so.
>
> You really think that the person that wrote ffvp9 (and merged it inside libavcodec) and a large part of dav1d, and who is a large contributor to FFmpeg, did not consider whether it was better to start inside libavcodec or as an external library?
>

Just to make sure, I was specifically talking about hardware
acceleration (which my entire post was more or less about), which
according to Ronald was not considered in the reasoning here.

- Hendrik
Rostislav Pehlivanov Sept. 29, 2018, 9:47 p.m.
On Sat, 29 Sep 2018 at 15:19, James Almer <jamrial@gmail.com> wrote:

> On 9/29/2018 10:14 AM, Hendrik Leppkes wrote:
> > On Sat, Sep 29, 2018 at 3:02 PM Devin Heitmueller
> > <dheitmueller@kernellabs.com> wrote:
> >>
> >> On Sat, Sep 29, 2018 at 6:04 AM Rostislav Pehlivanov
> >> <atomnuker@gmail.com> wrote:
> >>> I'd much rather go with the original intent which was to merge the
> decoder
> >>> into lavc.
> >>
> >> Ronald can correct me if I'm wrong, but I suspect a key goal behind
> >> the decoder was to have a standalone library which could be shared
> >> across a variety of projects (both open and closed source).  Merging
> >> it in directly will create a maintenance headache as the two source
> >> trees diverge.  It also makes unit testing more difficult, since
> >> Ronald/VideoLAN can write unit tests for the library (which will
> >> presumably be consumed by a number of projects/products) and be
> >> confident that those same unit tests apply to the version that is used
> >> by ffmpeg.
> >>
> >> I don't think having libx264/libx265 out of tree hasn't been a
> >> nightmare for anyone.  I don't think this case would be any different.
> >>
> >
> > Decoders and encoders are quite a different type of beast. For one
> > decoders are ultimately "finished" at some point, where the number of
> > changes drastically reduces over time, and decoding is the bread and
> > butter of avcodec, which would give immediate av1 support to hundreds
> > if not thousands of applications out there, without any further
> > developer intervention.
>
> I'm guessing LGPL is a deterrent for some potential users VideoLAN
> wanted for this decoder, hence the 2-Clause BSD license in libdav1d.
>

On the other hand the 2-Clause BSD is a deterrent for me to write some
assembly and just generally pour work on it.


> Additionally, I can already name an important key area where not
> > having an internal decoder will be a nightmare: hardware decoding.
> > Our hardware decoding framework heavily relies on having a decoder
> > that does all the header parsing, frame management, and whatnot, and
> > the hardware acceleration hooks just by-pass the actual slice decoding
> > then. If we don't have an internal decoder, we would have to either
> > build half decoder just to hook up hwaccel, or dav1d would have to
> > give us extremely detailed bitstream information and slice-level
> > bitstream hooks, which I don't think is very practical either.
> >
> > A new mainstream codec without hardware acceleration support is not
> > going to make it, no matter how fast you make the decoder.
>
> Yeah, this is one of the main reasons to have it as an internal decoder.
> But considering actual hardware capable of decoding AV1 will not come
> out until like late 2019 at the earliest, there's time for libdav1d to
> mature before porting it becomes a must.
>

Considering the code style and some parts of the internal API are quite
close I reckon the wrapper won't survive for more than a few months, and
will definitely be gone by January.
I'd agree to the wrapper if it didn't make it to 4.1, since it won't be in
4.2. No point in adding a non-default decoder that users may need to
specify (not sure how precedence works with both libaomdec and libdav1id).
Especially knowing how long cargo cult command lines and API usage lasts.


At some point it will be "complete" and development pretty much halted,
> much like vp9 currently is (not counting some missing assembly), so the
> argument about extra maintenance of separate codebases will not apply
> anymore.
>

Given the license difference, the preference of everyone involved with it
for LGPL (its only BSD and a separate library because money) I think
chances are people/companies will sooner abandon it to work on their fork
than use the base version as-is (or in case of companies contribute).
Jean-Baptiste Kempf Sept. 30, 2018, 7:07 a.m.
On Sat, 29 Sep 2018, at 23:47, Rostislav Pehlivanov wrote:
> Given the license difference, the preference of everyone involved with it

How can you speak for "everyone involved"? Don't speak for other, please.

> for LGPL (its only BSD and a separate library because money) I think

It's BSD because we want everyone to use, because we want the AV1 format to be successful.
And AV1 can only be successful if you have good decoders, until the hardware decoders arrive (2 years).
The same reasoning is why rav1e is BSD.

Having a BSD licensed library makes sense, like for libopus, libvorbis and so on, in these cases, where you want your format to be popular.
Even RMS agrees...

It's a separate library to avoid the VP9 issue where people refused to use ffvp9 because of the size of the library, and kept using libvpx, or their fork of libvpx.

> chances are people/companies will sooner abandon it to work on their fork
> than use the base version as-is (or in case of companies contribute).

If people fork without contributing, it's probably because it does not fit their use case.
It's not going to be because of the speed, because de facto, this community makes things fast.
It's a decoder, not something where you can have a strategic advantage (a contrario from encoders).

Also, you know, the world has moved on, from the 90s/2000s.
People understand now the cost of maintenance of software, and the cost of forking.
Microsoft contributes .Net as MIT, Google and Microsoft work together on numerous frameworks, most go, js, python projects are licensed under MIT/BSD, and they still have thousand of contributors, not people forking all the time.

The reason is because the money is on services, not just software.

Patch hide | download patch | download mbox

diff --git a/configure b/configure
index fdca47cf2c..8cd8fbd353 100755
--- a/configure
+++ b/configure
@@ -226,6 +226,7 @@  External library support:
   --enable-libcelt         enable CELT decoding via libcelt [no]
   --enable-libcdio         enable audio CD grabbing with libcdio [no]
   --enable-libcodec2       enable codec2 en/decoding using libcodec2 [no]
+  --enable-libdav1d        enable AV1 decoding via libdav1d [no]
   --enable-libdavs2        enable AVS2 decoding via libdavs2 [no]
   --enable-libdc1394       enable IIDC-1394 grabbing using libdc1394
                            and libraw1394 [no]
@@ -1712,6 +1713,7 @@  EXTERNAL_LIBRARY_LIST="
     libcaca
     libcelt
     libcodec2
+    libdav1d
     libdc1394
     libdrm
     libflite
@@ -3087,6 +3089,7 @@  libaom_av1_encoder_select="extract_extradata_bsf"
 libcelt_decoder_deps="libcelt"
 libcodec2_decoder_deps="libcodec2"
 libcodec2_encoder_deps="libcodec2"
+libdav1d_decoder_deps="libdav1d"
 libdavs2_decoder_deps="libdavs2"
 libfdk_aac_decoder_deps="libfdk_aac"
 libfdk_aac_encoder_deps="libfdk_aac"
@@ -6062,6 +6065,7 @@  enabled libcelt           && require libcelt celt/celt.h celt_decode -lcelt0 &&
                                die "ERROR: libcelt must be installed and version must be >= 0.11.0."; }
 enabled libcaca           && require_pkg_config libcaca caca caca.h caca_create_canvas
 enabled libcodec2         && require libcodec2 codec2/codec2.h codec2_create -lcodec2
+enabled libdav1d          && require_pkg_config libdav1d "dav1d >= 0.0.1" "dav1d/dav1d.h" dav1d_version
 enabled libdavs2          && require_pkg_config libdavs2 "davs2 >= 1.5.115" davs2.h davs2_decoder_open
 enabled libdc1394         && require_pkg_config libdc1394 libdc1394-2 dc1394/dc1394.h dc1394_new
 enabled libdrm            && require_pkg_config libdrm libdrm xf86drm.h drmGetVersion
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index b9cc20b5ef..6cfbfbcff6 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -954,6 +954,7 @@  OBJS-$(CONFIG_LIBAOM_AV1_ENCODER)         += libaomenc.o
 OBJS-$(CONFIG_LIBCELT_DECODER)            += libcelt_dec.o
 OBJS-$(CONFIG_LIBCODEC2_DECODER)          += libcodec2.o codec2utils.o
 OBJS-$(CONFIG_LIBCODEC2_ENCODER)          += libcodec2.o codec2utils.o
+OBJS-$(CONFIG_LIBDAV1D_DECODER)           += libdav1d.o
 OBJS-$(CONFIG_LIBDAVS2_DECODER)           += libdavs2.o
 OBJS-$(CONFIG_LIBFDK_AAC_DECODER)         += libfdk-aacdec.o
 OBJS-$(CONFIG_LIBFDK_AAC_ENCODER)         += libfdk-aacenc.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index c0b4d56d0d..d2628df620 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -674,6 +674,7 @@  extern AVCodec ff_libaom_av1_encoder;
 extern AVCodec ff_libcelt_decoder;
 extern AVCodec ff_libcodec2_encoder;
 extern AVCodec ff_libcodec2_decoder;
+extern AVCodec ff_libdav1d_decoder;
 extern AVCodec ff_libdavs2_decoder;
 extern AVCodec ff_libfdk_aac_encoder;
 extern AVCodec ff_libfdk_aac_decoder;
diff --git a/libavcodec/libdav1d.c b/libavcodec/libdav1d.c
new file mode 100644
index 0000000000..26ebf769e7
--- /dev/null
+++ b/libavcodec/libdav1d.c
@@ -0,0 +1,267 @@ 
+/*
+ * Copyright (c) 2018 Ronald S. Bultje <rsbultje gmail com>
+ * Copyright (c) 2018 James Almer <jamrial gmail com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include <dav1d/dav1d.h>
+
+#include "libavutil/avassert.h"
+#include "libavutil/fifo.h"
+#include "libavutil/opt.h"
+
+#include "avcodec.h"
+#include "decode.h"
+#include "internal.h"
+
+typedef struct Libdav1dContext {
+    AVClass *class;
+    Dav1dContext *c;
+
+    AVFifoBuffer *cache;
+    Dav1dData data;
+    int frame_threads, tile_threads;
+} Libdav1dContext;
+
+static av_cold int libdav1d_init(AVCodecContext *c)
+{
+    Libdav1dContext *dav1d = c->priv_data;
+    Dav1dSettings s;
+    int res;
+
+    dav1d_init();
+    dav1d_default_settings(&s);
+
+    av_log(c, AV_LOG_VERBOSE, "%s\n", dav1d_version());
+
+    s.n_tile_threads = dav1d->tile_threads;
+    s.n_frame_threads = dav1d->frame_threads;
+
+    dav1d->cache = av_fifo_alloc(8 * sizeof(AVPacket));
+    if (!dav1d->cache)
+        return AVERROR(ENOMEM);
+
+    res = dav1d_open(&dav1d->c, &s);
+    if (res < 0)
+        return AVERROR(ENOMEM);
+
+    c->has_b_frames = 1;
+
+    return 0;
+}
+
+static void clear_cache(AVCodecContext *c)
+{
+    Libdav1dContext *dav1d = c->priv_data;
+    AVPacket pkt;
+
+    while (av_fifo_size(dav1d->cache)) {
+        av_fifo_generic_read(dav1d->cache, &pkt, sizeof(pkt), NULL);
+        av_packet_unref(&pkt);
+    }
+
+    dav1d_data_unref(&dav1d->data);
+}
+
+static av_cold int libdav1d_close(AVCodecContext *c)
+{
+    Libdav1dContext *dav1d = c->priv_data;
+
+    clear_cache(c);
+    av_fifo_freep(&dav1d->cache);
+
+    dav1d_close(&dav1d->c);
+
+    return 0;
+}
+
+static void libdav1d_flush(AVCodecContext *c)
+{
+    Libdav1dContext *dav1d = c->priv_data;
+
+    clear_cache(c);
+
+    dav1d_flush(dav1d->c);
+}
+
+static void libdav1d_data_free_wrapper(uint8_t *data, void *opaque) {
+    AVBufferRef *buf = opaque;
+
+    av_buffer_unref(&buf);
+}
+
+static void libdav1d_frame_free_wrapper(void *opaque, uint8_t *data) {
+    Dav1dPicture p = { 0 };
+
+    p.ref = opaque;
+    p.data[0] = (void *) 0x1; // this has to be non-NULL
+    dav1d_picture_unref(&p);
+}
+
+static int libdav1d_receive_frame(AVCodecContext *c, AVFrame *frame)
+{
+    Libdav1dContext *dav1d = c->priv_data;
+    Dav1dData *data_ptr;
+    AVPacket pkt = { 0 };
+    Dav1dPicture p = { 0 };
+    int res;
+
+    if (!dav1d->data.sz) {
+        res = ff_decode_get_packet(c, &pkt);
+        if (res < 0 && res != AVERROR_EOF)
+            return res;
+
+        if (pkt.size) {
+            AVBufferRef *buf;
+
+            if (!av_fifo_space(dav1d->cache)) {
+                res = av_fifo_grow(dav1d->cache, 8 * sizeof(pkt));
+                if (res < 0)
+                    return res;
+            }
+
+            av_fifo_generic_write(dav1d->cache, &pkt, sizeof(pkt), NULL);
+
+            buf = av_buffer_ref(pkt.buf);
+            if (!buf)
+                return AVERROR(ENOMEM);
+
+            data_ptr = &dav1d->data;
+            res = dav1d_data_wrap(data_ptr, pkt.data, pkt.size, libdav1d_data_free_wrapper, buf);
+            if (res < 0) {
+                av_buffer_unref(&buf);
+                return res;
+            }
+        } else {
+            data_ptr = NULL;
+        }
+    } else
+        data_ptr = &dav1d->data;
+
+    res = dav1d_decode(dav1d->c, data_ptr, &p);
+    if (res < 0 && res == -EINVAL)
+        return AVERROR_INVALIDDATA;
+
+    if (!res) {
+        static const enum AVPixelFormat pix_fmt[][2] = {
+            [DAV1D_PIXEL_LAYOUT_I400] = { AV_PIX_FMT_GRAY8,   AV_PIX_FMT_GRAY10 },
+            [DAV1D_PIXEL_LAYOUT_I420] = { AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV420P10 },
+            [DAV1D_PIXEL_LAYOUT_I422] = { AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV422P10 },
+            [DAV1D_PIXEL_LAYOUT_I444] = { AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV444P10 },
+        };
+        // TODO: Update once 12bit support is added.
+        static const int profiles[] = {
+            [DAV1D_PIXEL_LAYOUT_I400] = FF_PROFILE_AV1_MAIN,
+            [DAV1D_PIXEL_LAYOUT_I420] = FF_PROFILE_AV1_MAIN,
+            [DAV1D_PIXEL_LAYOUT_I422] = FF_PROFILE_AV1_PROFESSIONAL,
+            [DAV1D_PIXEL_LAYOUT_I444] = FF_PROFILE_AV1_HIGH,
+        };
+        int err;
+
+        av_assert0(p.data[0] != NULL);
+
+        c->profile = profiles[p.p.layout];
+        frame->format = c->pix_fmt = pix_fmt[p.p.layout][p.p.bpc == 10];
+        frame->width = p.p.w;
+        frame->height = p.p.h;
+        if (c->width != p.p.w || c->height != p.p.h) {
+            err = ff_set_dimensions(c, p.p.w, p.p.h);
+            if (err < 0)
+                return err;
+        }
+
+        switch (p.p.chr) {
+        case DAV1D_CHR_VERTICAL:
+            frame->chroma_location = c->chroma_sample_location = AVCHROMA_LOC_LEFT;
+            break;
+        case DAV1D_CHR_COLOCATED:
+            frame->chroma_location = c->chroma_sample_location = AVCHROMA_LOC_TOPLEFT;
+            break;
+        }
+        frame->colorspace = c->colorspace = p.p.mtrx;
+        frame->color_primaries = c->color_primaries = p.p.pri;
+        frame->color_trc = c->color_trc = p.p.trc;
+        frame->color_range = c->color_range = p.p.fullrange ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG;
+
+        frame->buf[0] = av_buffer_create(NULL, 0,
+                                         libdav1d_frame_free_wrapper,
+                                         p.ref, AV_BUFFER_FLAG_READONLY);
+        if (!frame->buf[0])
+            return AVERROR(ENOMEM);
+
+        frame->data[0] = p.data[0];
+        frame->data[1] = p.data[1];
+        frame->data[2] = p.data[2];
+        frame->linesize[0] = p.stride[0];
+        frame->linesize[1] = p.stride[1];
+        frame->linesize[2] = p.stride[1];
+
+        av_fifo_generic_read(dav1d->cache, &pkt, sizeof(pkt), NULL);
+
+        // match timestamps and packet size
+        frame->pkt_pos = pkt.pos;
+        frame->pts = frame->best_effort_timestamp = pkt.pts;
+#if FF_API_PKT_PTS
+FF_DISABLE_DEPRECATION_WARNINGS
+        frame->pkt_pts = pkt.pts;
+FF_ENABLE_DEPRECATION_WARNINGS
+#endif
+        frame->pkt_dts = pkt.dts;
+        frame->pkt_size = pkt.size;
+        frame->pkt_duration = pkt.duration;
+        frame->key_frame = !!(pkt.flags & AV_PKT_FLAG_KEY);
+        frame->pict_type = frame->key_frame ? AV_PICTURE_TYPE_I : AV_PICTURE_TYPE_P;
+        av_packet_unref(&pkt);
+    }
+
+    return res == -EAGAIN && c->internal->draining ? AVERROR_EOF : res;
+}
+
+#define OFFSET(x) offsetof(Libdav1dContext, x)
+#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM
+static const AVOption libdav1d_options[] = {
+    { "framethreads", "Frame threads", OFFSET(frame_threads),
+        AV_OPT_TYPE_INT, { .i64 = 1 }, 1, 256, VD, NULL },
+    { "tilethreads", "Tile threads", OFFSET(tile_threads),
+        AV_OPT_TYPE_INT, { .i64 = 1 }, 1, 64, VD, NULL },
+    { NULL }
+};
+
+static const AVClass libdav1d_class = {
+    .class_name = "libdav1d decoder",
+    .item_name  = av_default_item_name,
+    .option     = libdav1d_options,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+AVCodec ff_libdav1d_decoder = {
+    .name           = "libdav1d",
+    .long_name      = NULL_IF_CONFIG_SMALL("dav1d AV1 decoder by VideoLAN"),
+    .type           = AVMEDIA_TYPE_VIDEO,
+    .id             = AV_CODEC_ID_AV1,
+    .priv_data_size = sizeof(Libdav1dContext),
+    .init           = libdav1d_init,
+    .close          = libdav1d_close,
+    .flush          = libdav1d_flush,
+    .receive_frame  = libdav1d_receive_frame,
+    .capabilities   = AV_CODEC_CAP_DELAY,
+    .caps_internal  = FF_CODEC_CAP_INIT_THREADSAFE | FF_CODEC_CAP_INIT_CLEANUP |
+                      FF_CODEC_CAP_SETS_PKT_DTS,
+    .priv_class     = &libdav1d_class,
+    .wrapper_name   = "libdav1d",
+};