diff mbox series

[FFmpeg-devel,5/8] lavu: add a JSON writer API (WIP)

Message ID 20230428095508.221826-5-george@nsup.org
State New
Headers show
Series [FFmpeg-devel,1/8] lavu: add macros to help making future-proof structures | expand

Checks

Context Check Description
andriy/configure_x86 warning Failed to apply patch

Commit Message

Nicolas George April 28, 2023, 9:55 a.m. UTC
Signed-off-by: Nicolas George <george@nsup.org>
---
 libavutil/Makefile |   1 +
 libavutil/json.c   | 368 +++++++++++++++++++++++++++++++++++
 libavutil/json.h   | 470 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 839 insertions(+)
 create mode 100644 libavutil/json.c
 create mode 100644 libavutil/json.h

Comments

Nicolas George April 29, 2023, 9:11 a.m. UTC | #1
Nicolas George (12023-04-28):
> Signed-off-by: Nicolas George <george@nsup.org>
> ---
>  libavutil/Makefile |   1 +
>  libavutil/json.c   | 368 +++++++++++++++++++++++++++++++++++
>  libavutil/json.h   | 470 +++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 839 insertions(+)
>  create mode 100644 libavutil/json.c
>  create mode 100644 libavutil/json.h

I forgot to write: I wrote this code not only because we have half-baked
JSON output in multiple places in the code, but also to show the
kind of API AVWriter makes possible.

Typical JSON APIs would have a function to write a string value or an
object key, requiring the caller to build the string beforehand. Of
course, this API can do that:

> +void av_json_add_string(AVJson *jc, const char *str);

including with a format string, which is less usual:

> +void av_json_add_string_printf(AVJson *jc, const char *fmt, ...) av_printf_format(2, 3);

But these are just wrappers for convenience over the real API:

> +AVWriter av_json_begin_string(AVJson *jc);

It starts a string, i.e. outputs a quote, and then we get a writer to
write into that string. It will be automatically escaped, no
intermediate buffer will be used, and all the functions of the writer
API are available, including all the av_something_write() to come to
serialize our various types.

Also note that this API as a whole can produce small JSON outputs
without any dynamic allocation, making it suitable for once-per-frame
calls, but is not limited to that.

Do we *need* that: of course not, we still put “len += snprintf()” and
“av_realloc()” and error checks all over the place.

Now, while people maybe look at the code, there are a few things I can
work on, and I wonder which one you would like to see first:

- Finishing this JSON API.

- A XML API: that would be useful for dashenc, movenc, ttmlenc, ffprobe
  and possibly others.

- Add serialization functions for our various types, like I did for
  av_disposition_write().

- Go forward with the others API enhancements that I promised and that
  depend on AVWriter.

Regards,
Anton Khirnov April 29, 2023, 9:41 a.m. UTC | #2
Quoting Nicolas George (2023-04-29 11:11:59)
> Nicolas George (12023-04-28):
> > Signed-off-by: Nicolas George <george@nsup.org>
> > ---
> >  libavutil/Makefile |   1 +
> >  libavutil/json.c   | 368 +++++++++++++++++++++++++++++++++++
> >  libavutil/json.h   | 470 +++++++++++++++++++++++++++++++++++++++++++++
> >  3 files changed, 839 insertions(+)
> >  create mode 100644 libavutil/json.c
> >  create mode 100644 libavutil/json.h
> 
> I forgot to write: I wrote this code not only because we have half-baked
> JSON output in multiple places in the code

As far as I can see, there are exactly two places in the codebase that
produce JSON: af_loudnorm and ffprobe.

af_loudnorm:
* can optionally produce final filter stats as JSON output with av_log()
* the relevant code has ~25 lines and is unlikely to be simplified by
  this
* IMO the filter should not be doing this at all and instead produce some
  sort of a struct and let the users process it as they wish

ffprobe:
* is not one of the libraries, but rather their caller
* we are not in business of providing random non-multimedia-related
  services to callers, unless they are useful in our libraries;
  if this code is only useful in ffprobe then it should live in fftools/
* it is not at all obvious that switching ffprobe to this code would
  be an improvement; a patch actually demonstrating this would be most
  useful
James Almer April 29, 2023, 2:06 p.m. UTC | #3
On 4/29/2023 6:41 AM, Anton Khirnov wrote:
> Quoting Nicolas George (2023-04-29 11:11:59)
>> Nicolas George (12023-04-28):
>>> Signed-off-by: Nicolas George <george@nsup.org>
>>> ---
>>>   libavutil/Makefile |   1 +
>>>   libavutil/json.c   | 368 +++++++++++++++++++++++++++++++++++
>>>   libavutil/json.h   | 470 +++++++++++++++++++++++++++++++++++++++++++++
>>>   3 files changed, 839 insertions(+)
>>>   create mode 100644 libavutil/json.c
>>>   create mode 100644 libavutil/json.h
>>
>> I forgot to write: I wrote this code not only because we have half-baked
>> JSON output in multiple places in the code
> 
> As far as I can see, there are exactly two places in the codebase that
> produce JSON: af_loudnorm and ffprobe.
> 
> af_loudnorm:
> * can optionally produce final filter stats as JSON output with av_log()
> * the relevant code has ~25 lines and is unlikely to be simplified by
>    this
> * IMO the filter should not be doing this at all and instead produce some
>    sort of a struct and let the users process it as they wish

It should be exporting frame metadata, like aphasemeter, cropdetect, etc.
av_log() output is not meant to be parseable. It's why the relevant 
output of vf_showinfo can be freely changed, whereas output from things 
like framecrc/framehash muxers is standardized.

> 
> ffprobe:
> * is not one of the libraries, but rather their caller
> * we are not in business of providing random non-multimedia-related
>    services to callers, unless they are useful in our libraries;
>    if this code is only useful in ffprobe then it should live in fftools/
> * it is not at all obvious that switching ffprobe to this code would
>    be an improvement; a patch actually demonstrating this would be most
>    useful
>
Derek Buitenhuis April 29, 2023, 3:06 p.m. UTC | #4
On 4/29/2023 10:41 AM, Anton Khirnov wrote:
> ffprobe:
> * is not one of the libraries, but rather their caller
> * we are not in business of providing random non-multimedia-related
>   services to callers, unless they are useful in our libraries;
>   if this code is only useful in ffprobe then it should live in fftools/
> * it is not at all obvious that switching ffprobe to this code would
>   be an improvement; a patch actually demonstrating this would be most
>   useful

+1

- Derek
Nicolas George April 29, 2023, 5:11 p.m. UTC | #5
Anton Khirnov (12023-04-29):
> As far as I can see, there are exactly two places in the codebase that
> produce JSON: af_loudnorm and ffprobe.

I think I remember finding a few other places, but it does not matter
much.

Users have been asking¹ for parse-friendly output from filters for years,
and JSON is often the format they prefer. Therefore, all the filters
that log useful information are candidates where a JSON writing API is
useful.

(See my answer to James for why “put it in frame metadata” is not a real
solution.)

> * IMO the filter should not be doing this at all and instead produce some
>   sort of a struct and let the users process it as they wish

And to let users process it as they wish, it needs to be formatted into
a parse-friendly syntax like JSON.

> ffprobe:
> * is not one of the libraries, but rather their caller
> * we are not in business of providing random non-multimedia-related
>   services to callers, unless they are useful in our libraries;
>   if this code is only useful in ffprobe then it should live in fftools/

This is your opinion, not the project policy.

My opinion is that anything that is useful for fftools and can be made
to have a properly-defined API has its place in libavutil.

Let us see which opinion has more support.

> * it is not at all obvious that switching ffprobe to this code would
>   be an improvement; a patch actually demonstrating this would be most
>   useful

I distinctly remember writing the changes to ffprobe and getting the
same output except for the indentation level, but I do not find the
code.


1: To be aware what would be useful to the project, reading the users
mailing-lists helps.
Nicolas George April 29, 2023, 5:17 p.m. UTC | #6
James Almer (12023-04-29):
> It should be exporting frame metadata

Not really. Exporting information to frame data is a hack, “this looks
pointy, we have a hammer, let's nail it”.

It only works if the data is very flat.

It does not work for data that comes at the end of the stream.

It still requires a way to get the data out in a parse-friendly syntax
when the program running is not ffprobe.

Having an API and user interface for filters to return structured data
is one of the reasons I want this API in libavutil.

Regards,
Anton Khirnov April 29, 2023, 6:27 p.m. UTC | #7
Quoting Nicolas George (2023-04-29 19:11:52)
> Anton Khirnov (2023-04-29):
> > As far as I can see, there are exactly two places in the codebase that
> > produce JSON: af_loudnorm and ffprobe.
> 
> I think I remember finding a few other places, but it does not matter
> much.
> 
> Users have been asking¹ for parse-friendly output from filters for years,
> and JSON is often the format they prefer. Therefore, all the filters
> that log useful information are candidates where a JSON writing API is
> useful.

libavfilter is a C library with a C API. Any structured output from
filters should be in the form of a C object, typically a struct. I do
not see why are you so in love with strings, they make for terrible
APIs.

We could conceivably use e.g. the AVOption mechanism to allow automated
serialization of such structs, but the actual mechanism should be left
to the callers.

> (See my answer to James for why “put it in frame metadata” is not a real
> solution.)

I believe frame metadata is a mistake and should not exist, so I am
certainly not suggesting to use it for anything.
Nicolas George April 29, 2023, 6:33 p.m. UTC | #8
Anton Khirnov (12023-04-29):
> libavfilter is a C library with a C API. Any structured output from
> filters should be in the form of a C object, typically a struct. I do
> not see why are you so in love with strings, they make for terrible
> APIs.

Yes, strings are a terrible API, but our project is not only a set of
libraries, it is also a set of command-line tools, and command-line
tools work with strings and nothing else.
Kieran Kunhya April 30, 2023, 12:29 a.m. UTC | #9
On Sat, 29 Apr 2023 at 05:07, Derek Buitenhuis <derek.buitenhuis@gmail.com>
wrote:

> On 4/29/2023 10:41 AM, Anton Khirnov wrote:
> > ffprobe:
> > * is not one of the libraries, but rather their caller
> > * we are not in business of providing random non-multimedia-related
> >   services to callers, unless they are useful in our libraries;
> >   if this code is only useful in ffprobe then it should live in fftools/
> > * it is not at all obvious that switching ffprobe to this code would
> >   be an improvement; a patch actually demonstrating this would be most
> >   useful
>
> +1
>
>
+2
Michael Niedermayer April 30, 2023, 3:06 p.m. UTC | #10
On Sat, Apr 29, 2023 at 08:27:32PM +0200, Anton Khirnov wrote:
> Quoting Nicolas George (2023-04-29 19:11:52)
> > Anton Khirnov (2023-04-29):
> > > As far as I can see, there are exactly two places in the codebase that
> > > produce JSON: af_loudnorm and ffprobe.
> > 
> > I think I remember finding a few other places, but it does not matter
> > much.
> > 
> > Users have been asking¹ for parse-friendly output from filters for years,
> > and JSON is often the format they prefer. Therefore, all the filters
> > that log useful information are candidates where a JSON writing API is
> > useful.
> 
> libavfilter is a C library with a C API. Any structured output from
> filters should be in the form of a C object, typically a struct. I do
> not see why are you so in love with strings, they make for terrible
> APIs.

There are many projects which use libavcodec, format, filter
Human users use these projects

If the standarization is at a C struct level only then the human interface
for each application can be different.
Thats fine if the 2 cases use fundamentally different interfaces like a
GUI draging, droping and connecting components vs some command line
interface.
But if 2 applications both use command line / string based interfaces
it would be nice to the human user if she could use/learn the same syntax
and transfer a working example / script from one to the other.

So i think strings do matter for C libs because of that.

thx

[...]
Kieran Kunhya April 30, 2023, 9:51 p.m. UTC | #11
> There are many projects which use libavcodec, format, filter
> Human users use these projects
>
> If the standarization is at a C struct level only then the human interface
> for each application can be different.
> Thats fine if the 2 cases use fundamentally different interfaces like a
> GUI draging, droping and connecting components vs some command line
> interface.
> But if 2 applications both use command line / string based interfaces
> it would be nice to the human user if she could use/learn the same syntax
> and transfer a working example / script from one to the other.
>
> So i think strings do matter for C libs because of that.
>
> thx
>

The interface and the exchange format are completely different things.

Kieran
Vittorio Giovara May 1, 2023, 6:20 a.m. UTC | #12
On Sat, Apr 29, 2023 at 8:29 PM Kieran Kunhya <kierank@obe.tv> wrote:

> On Sat, 29 Apr 2023 at 05:07, Derek Buitenhuis <derek.buitenhuis@gmail.com
> >
> wrote:
>
> > On 4/29/2023 10:41 AM, Anton Khirnov wrote:
> > > ffprobe:
> > > * is not one of the libraries, but rather their caller
> > > * we are not in business of providing random non-multimedia-related
> > >   services to callers, unless they are useful in our libraries;
> > >   if this code is only useful in ffprobe then it should live in
> fftools/
> > > * it is not at all obvious that switching ffprobe to this code would
> > >   be an improvement; a patch actually demonstrating this would be most
> > >   useful
> >
> > +1
> >
> >
> +2
>

+3
Leo Izen May 1, 2023, 6:57 a.m. UTC | #13
On 4/29/23 14:33, Nicolas George wrote:
> Anton Khirnov (12023-04-29):
>> libavfilter is a C library with a C API. Any structured output from
>> filters should be in the form of a C object, typically a struct. I do
>> not see why are you so in love with strings, they make for terrible
>> APIs.
> 
> Yes, strings are a terrible API, but our project is not only a set of
> libraries, it is also a set of command-line tools, and command-line
> tools work with strings and nothing else.
> 

This is a good argument for putting the code in fftools/ and not 
libavutil, fwiw.

- Leo Izen (Traneptora / thebombzen)
Nicolas George May 1, 2023, 9:46 a.m. UTC | #14
Michael Niedermayer (12023-04-30):
> There are many projects which use libavcodec, format, filter
> Human users use these projects
> 
> If the standarization is at a C struct level only then the human interface
> for each application can be different.
> Thats fine if the 2 cases use fundamentally different interfaces like a
> GUI draging, droping and connecting components vs some command line
> interface.
> But if 2 applications both use command line / string based interfaces
> it would be nice to the human user if she could use/learn the same syntax
> and transfer a working example / script from one to the other.
> 
> So i think strings do matter for C libs because of that.

Thank you for stating it that way.

I think I can make it even a little stronger:

The API of the avlibraries is so rich that applications cannot
realistically cover all of them, and this is why we have the options
system: so that applications can expose all the knobs and controls of
avlibs without having to maintain code for every one of them.

But the options system has severe limitations, including the occasional
need for half-a-dozen backslashes or more for escaping and the inability
to define AV_OPT_TYPE_SOMETHING if SOMETHING is defined in another
library than lavu or nor generic enough.

Overcoming the limitations of the options system is a project I have had
for a long time, and it connects to the project of embedding the
documentation into the libraries (which has received some support).

http://ffmpeg.org/pipermail/ffmpeg-devel/2015-December/184525.html
(the technical details in my mind have evolved a little, but not much)
http://ffmpeg.org/pipermail/ffmpeg-devel/2020-August/268389.html

And for that, we absolutely need an efficient strings API (this is now
supported by a majority of developers, thankfully) and standardized
serialization functions. In the libraries, not the avtools.

To say it in a more concise way:

The avlibraries must not only perform work for applications, they also
must help applications communicate with users about that work, and that
is done with text.

Regards,
Nicolas George May 1, 2023, 9:51 a.m. UTC | #15
Leo Izen (12023-05-01):
> > Yes, strings are a terrible API, but our project is not only a set of
> > libraries, it is also a set of command-line tools, and command-line
> > tools work with strings and nothing else.
> This is a good argument for putting the code in fftools/ and not libavutil,
> fwiw.

This is not wrong. But I realize now my argument was widely incomplete.
GUI applications not of our own still need text to communicate with
users about things that do not have their specific widget, for example.

The avlibraries must not only perform work for applications, they also
must help applications communicate with users about that work, and that
is done with text.

See there for a more complete wording:
http://ffmpeg.org/pipermail/ffmpeg-devel/2023-May/309077.html

Regards,
Jean-Baptiste Kempf May 1, 2023, 10:18 a.m. UTC | #16
On Mon, 1 May 2023, at 08:57, Leo Izen wrote:
> On 4/29/23 14:33, Nicolas George wrote:
>> Anton Khirnov (12023-04-29):
>>> libavfilter is a C library with a C API. Any structured output from
>>> filters should be in the form of a C object, typically a struct. I do
>>> not see why are you so in love with strings, they make for terrible
>>> APIs.
>> 
>> Yes, strings are a terrible API, but our project is not only a set of
>> libraries, it is also a set of command-line tools, and command-line
>> tools work with strings and nothing else.
>> 
>
> This is a good argument for putting the code in fftools/ and not 
> libavutil, fwiw.

This is also my understanding.
diff mbox series

Patch

diff --git a/libavutil/Makefile b/libavutil/Makefile
index 4526ec80ca..a8a9700778 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -140,6 +140,7 @@  OBJS = adler32.o                                                        \
        imgutils.o                                                       \
        integer.o                                                        \
        intmath.o                                                        \
+       json.o                                                           \
        lfg.o                                                            \
        lls.o                                                            \
        log.o                                                            \
diff --git a/libavutil/json.c b/libavutil/json.c
new file mode 100644
index 0000000000..7ee68aaa4a
--- /dev/null
+++ b/libavutil/json.c
@@ -0,0 +1,368 @@ 
+/*
+ * Copyright (c) 2021 Nicolas George
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "avassert.h"
+#include "common.h"
+#include "json.h"
+#include "opt.h"
+
+#define FIELDOK(st, f) ((char *)(&(st)->f + 1) <= (char *)(st) + (st)->self_size)
+
+#define json_assert_abi(jwr) av_assert1(FIELDOK(jwr, padding))
+#define json_escape_writer_assert_abi(jwr) av_assert1(FIELDOK(jwr, padding))
+
+AVClass *av_json_get_class(void)
+{
+    return NULL;
+}
+
+int av_json_alloc(AVJson **jc, unsigned max_depth)
+{
+    return AVERROR_BUG;
+}
+
+void av_json_free(AVJson **jc)
+{
+}
+
+AVJson *av_json_preinit(AVJson *jc, AVJsonEscapeWriter *jwr)
+{
+    json_assert_abi(jc);
+    json_escape_writer_assert_abi(jwr);
+    jc->av_class = av_json_get_class();
+    jc->escape_writer = jwr;
+    av_opt_set_defaults(jc);
+    return jc;
+}
+
+void av_json_init(AVJson *jc, AVWriter wr, unsigned flags, AVDictionary *options)
+{
+    jc->out = wr;
+    jc->flags = flags;
+    jc->in_string = 0;
+    jc->in_object = 0;
+    jc->first_element = 1;
+    jc->depth = 0;
+    jc->stack = NULL;
+}
+
+static inline int check_value_must_be_string(AVJson *jc, int string)
+{
+    return !jc->object_key || string;
+}
+
+static inline int check_stack_clean(AVJsonStack *stack)
+{
+    return !stack->prev;
+}
+
+static inline int check_begin_end_balanced(AVJson *jc, int obj)
+{
+    return jc->in_object == obj && jc->stack;
+}
+
+static inline int flag_pretty_print(AVJson *jc)
+{
+    return !!(jc->flags & AV_JSON_FLAG_PRETTY_PRINT);
+}
+
+static void end_value(AVJson *jc)
+{
+    if (!jc->stack && flag_pretty_print(jc))
+        av_writer_print(jc->out, "\n");
+}
+
+static void auto_end_string(AVJson *jc)
+{
+    if (jc->in_string) {
+        av_writer_print(jc->out, "\"");
+        jc->in_string = 0;
+        end_value(jc);
+    }
+}
+
+static void auto_add_separator(AVJson *jc)
+{
+    int indent = flag_pretty_print(jc);
+
+    auto_end_string(jc);
+    if (!jc->first_element) {
+        if (jc->object_key) {
+            av_writer_print(jc->out, flag_pretty_print(jc) ? " : " : ":");
+            indent = jc->object_key = 0;
+        } else {
+            av_writer_print(jc->out, ",");
+            jc->object_key = jc->in_object;
+        }
+    }
+    if (indent) {
+        if (jc->stack)
+            av_writer_print(jc->out, "\n");
+        av_writer_add_chars(jc->out, ' ', 3 * jc->depth);
+    }
+    jc->first_element = 0;
+}
+
+static void begin_value(AVJson *jc, int string)
+{
+    auto_add_separator(jc);
+    av_assert1(check_value_must_be_string(jc, string));
+}
+
+static void begin_compound(AVJson *jc, AVJsonStack *stack, unsigned obj)
+{
+    av_assert1(FIELDOK(stack, in_object));
+    av_assert1(check_stack_clean(stack));
+    stack->prev = jc->stack;
+    stack->in_object = jc->in_object;
+    jc->stack = stack;
+    jc->first_element = 1;
+    jc->in_object = jc->object_key = obj;
+    jc->depth++;
+}
+
+static void end_compound(AVJson *jc, unsigned obj)
+{
+    AVJsonStack *stack = jc->stack;
+
+    av_assert1(check_begin_end_balanced(jc, obj));
+    auto_end_string(jc);
+    jc->depth--;
+    if (!jc->first_element && flag_pretty_print(jc)) {
+        av_writer_print(jc->out, "\n");
+        av_writer_add_chars(jc->out, ' ', 3 * jc->depth);
+    }
+    jc->in_object = stack->in_object;
+    jc->object_key = 0;
+    jc->first_element = 0;
+    jc->stack = stack->prev;
+    stack->prev = NULL;
+}
+
+void av_json_begin_object_with_stack(AVJson *jc, AVJsonStack *stack)
+{
+    begin_value(jc, 0);
+    begin_compound(jc, stack, 1);
+    av_writer_print(jc->out, "{");
+}
+
+void av_json_end_object(AVJson *jc)
+{
+    end_compound(jc, 1);
+    av_writer_print(jc->out, "}");
+    end_value(jc);
+}
+
+void av_json_begin_array_with_stack(AVJson *jc, AVJsonStack *stack)
+{
+    begin_value(jc, 0);
+    begin_compound(jc, stack, 0);
+    av_writer_print(jc->out, "[");
+}
+
+void av_json_end_array(AVJson *jc)
+{
+    end_compound(jc, 0);
+    av_writer_print(jc->out, "]");
+    end_value(jc);
+}
+
+AVWriter av_json_begin_string(AVJson *jc)
+{
+    begin_value(jc, 1);
+    jc->in_string = 1;
+    av_writer_print(jc->out, "\"");
+    av_json_escape_writer_init(jc->escape_writer, jc->out, jc->flags);
+    return av_json_escape_writer_wrap(jc->escape_writer);
+}
+
+void av_json_end_string(AVJson *jc)
+{
+    av_assert1(jc->in_string);
+    auto_end_string(jc);
+}
+
+void av_json_add_string(AVJson *jc, const char *str)
+{
+    AVWriter wr = av_json_begin_string(jc);
+    av_writer_print(wr, str);
+}
+
+void av_json_add_string_vprintf(AVJson *jc, const char *fmt, va_list va)
+{
+    AVWriter wr = av_json_begin_string(jc);
+    av_writer_vprintf(wr, fmt, va);
+    av_json_end_string(jc);
+}
+
+void av_json_add_string_printf(AVJson *jc, const char *fmt, ...)
+{
+    va_list va;
+    va_start(va, fmt);
+    av_json_add_string_vprintf(jc, fmt, va);
+    va_end(va);
+}
+
+void av_json_add_int(AVJson *jc, intmax_t val)
+{
+    begin_value(jc, 0);
+    av_writer_printf(jc->out, "%jd", val);
+    end_value(jc);
+}
+
+void av_json_add_double(AVJson *jc, double val)
+{
+    begin_value(jc, 0);
+    av_writer_printf(jc->out, "%.15g", val);
+    end_value(jc);
+}
+
+void av_json_add_bool(AVJson *jc, int val)
+{
+    begin_value(jc, 0);
+    av_writer_print(jc->out, val ? "true" : "false");
+    end_value(jc);
+}
+
+void av_json_add_null(AVJson *jc)
+{
+    begin_value(jc, 0);
+    av_writer_print(jc->out, "null");
+    end_value(jc);
+}
+
+void av_json_add_raw_vprintf(AVJson *jc, const char *fmt, va_list va)
+{
+    begin_value(jc, 1);
+    av_writer_vprintf(jc->out, fmt, va);
+    end_value(jc);
+}
+
+void av_json_add_raw_printf(AVJson *jc, const char *fmt, ...)
+{
+    va_list va;
+    va_start(va, fmt);
+    av_json_add_raw_vprintf(jc, fmt, va);
+    va_end(va);
+}
+
+/***************************************************************************
+ * AVJsonEscapeWriter - escpae JSON strings
+ ***************************************************************************/
+
+static void json_escape_writer_write(AVWriter wr, const char *data, size_t size)
+{
+    AVJsonEscapeWriter *jwr = wr.obj;
+    const uint8_t *end = data + size;
+    const uint8_t *cur = data;
+    const uint8_t *written = data;
+    const uint8_t *raw = data;
+    unsigned char buf[13];
+    const char *escape;
+    int32_t c;
+    int ret;
+
+    av_assert1(av_json_escape_writer_check(wr));
+    json_escape_writer_assert_abi(jwr);
+    if (jwr->error)
+        return;
+    while (cur < end) {
+        raw = cur;
+        ret = av_utf8_decode(&c, &cur, end, 0);
+        if (ret < 0) {
+            if ((jwr->flags & AV_JSON_FLAG_BAD_ENCODING_EXPLODE)) {
+                av_log(NULL, AV_LOG_PANIC, "Bad UTF-8 in JSON\n");
+                abort();
+            }
+            if ((jwr->flags & AV_JSON_FLAG_BAD_ENCODING_REPLACE)) {
+                c = 0xFFFD;
+            } else {
+                jwr->error = ret;
+                return;
+            }
+        }
+        av_assert1(c >= 0 && c <= 0x10FFFF);
+        if ((unsigned)(c - ' ') < 127 - ' ' && c != '"' && c != '\\')
+            continue;
+        if ((unsigned)(c - 0x0080) <= (0xFFFF - 0x0080)) /* TODO flag */
+            continue;
+        if (c == '"') {
+            escape = "\\\"";
+        } else if (c == '\\') {
+            escape = "\\\\";
+        } else if (c == '\n') {
+            escape = "\\n";
+        } else if (c == '\r') {
+            escape = "\\r";
+        } else if (c == '\t') {
+            escape = "\\t";
+        } else if (c == '\f') {
+            escape = "\\f";
+        } else if (c == '\b') {
+            escape = "\\b";
+        } else {
+            if (c < 0x10000)
+                snprintf(buf, sizeof(buf), "\\u%04X", c);
+            else /* JSON sucks: UTF-16 */
+                snprintf(buf, sizeof(buf), "\\u%04X\\u%04X",
+                         0xD800 + (((c - 0x10000) >> 10) & 0x3FF),
+                         0xDC00 + (((c - 0x10000) >>  0) & 0x3FF));
+            escape = buf;
+        }
+        if (raw > written)
+            av_writer_write(jwr->owr, written, raw - written);
+        av_writer_print(jwr->owr, escape);
+        written = raw = cur;
+    }
+    if (cur > written)
+        av_writer_write(jwr->owr, written, cur - written);
+}
+
+static int json_escape_writer_get_error(AVWriter wr, int self_only)
+{
+    AVJsonEscapeWriter *jwr = wr.obj;
+
+    av_assert1(av_json_escape_writer_check(wr));
+    return jwr->error ? jwr->error :
+           self_only ? 0 : av_writer_get_error(jwr->owr, 0);
+}
+
+AV_WRITER_DEFINE_METHODS(/*public*/, AVJsonEscapeWriter, av_json_escape_writer) {
+    .self_size        = sizeof(AVWriterMethods),
+    .name             = "AVJsonEscapeWriter",
+    .write            = json_escape_writer_write,
+    .get_error        = json_escape_writer_get_error,
+};
+
+AVJsonEscapeWriter *av_json_escape_writer_init(AVJsonEscapeWriter *jwr, AVWriter owr, unsigned flags)
+{
+    json_escape_writer_assert_abi(jwr);
+    jwr->owr = owr;
+    jwr->flags = flags;
+    return jwr;
+}
+
+AVWriter av_json_escape_writer_wrap(AVJsonEscapeWriter *jwr)
+{
+    AVWriter r = { av_json_escape_writer_get_methods(), jwr };
+    json_escape_writer_assert_abi(jwr);
+    return r;
+}
+
diff --git a/libavutil/json.h b/libavutil/json.h
new file mode 100644
index 0000000000..6cdcdc1348
--- /dev/null
+++ b/libavutil/json.h
@@ -0,0 +1,470 @@ 
+/*
+ * Copyright (c) 2023 The FFmpeg project
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVUTIL_JSON_H
+#define AVUTIL_JSON_H
+
+#include <stddef.h>
+#include <stdint.h>
+#include "dict.h"
+#include "extendable.h"
+#include "log.h"
+#include "writer.h"
+
+/**
+ * @defgroup av_json AVJson
+ *
+ * API to serialize data to JSON.
+ *
+ * API summary and quick HOWTO:
+ *
+ * AVWriter out = av_dynbuf_writer();
+ * AVJson *jc = AV_JSON_DEFINE();
+ * av_json_init(jc, out, AV_JSON_FLAG_PRETTY_PRINT, NULL);
+ * av_json_begin_object(jc);
+ * av_json_add_string("user");
+ * av_json_add_string(user_name);
+ * av_json_add_string("score");
+ * av_json_add_int("%d", score);
+ * av_json_end_object(jc);
+ * av_json_finish(jc);
+ * ret = av_writer_get_error(out, 0);
+ * if (ret < 0) ...
+ * data = av_dynbuf_writer_get_data(wr, &size);
+ *
+ * @{
+ */
+
+typedef struct AVJson AVJson;
+typedef struct AVJsonStack AVJsonStack;
+typedef struct AVJsonEscapeWriter AVJsonEscapeWriter;
+typedef struct AVJsonValue AVJsonValue;
+typedef enum AVJsonValueType AVJsonValueType;
+
+/**
+ * Produce ASCII output by escaping non-ASCII characters
+ */
+#define AV_JSON_FLAG_ASCII                      0x0001
+
+/**
+ * Abort if the input is not valid UTF-8.
+ */
+#define AV_JSON_FLAG_BAD_ENCODING_EXPLODE       0x0002
+
+/**
+ * If the input is not valid UTF-8, replace the offending partis with a
+ * replacement character.
+ */
+#define AV_JSON_FLAG_BAD_ENCODING_REPLACE       0x0004
+
+/**
+ * Pretty-print the output with spaces and indentation.
+ * XXX
+ */
+#define AV_JSON_FLAG_PRETTY_PRINT               0x0008
+
+/**
+ * Consider strings to be pseudo-binary instead of UTF-8.
+ *
+ * Warning: without AV_JSON_FLAG_ASCII, it will probably produce
+ * non-standard JSON.
+ * XXX
+ */
+#define AV_JSON_FLAG_PSEUDO_BINARY              0x0010
+
+/**
+ * Define a JSON context by allocating it as compound literals
+ * (hidden local variables).
+ * The context will be valid in the current scope and no further.
+ */
+#define AV_JSON_DEFINE(max_depth) \
+    av_json_preinit(&FF_NEW_SZ(AVJson), &FF_NEW_SZ(AVJsonEscapeWriter))
+
+/**
+ * Get the AVClass for a JSON context.
+ */
+AVClass *av_json_get_class(void);
+
+/**
+ * Pre-initialize a JSON context with its escape writer.
+ * @return  jc itself
+ */
+AVJson *av_json_preinit(AVJson *jc, AVJsonEscapeWriter *jwr);
+
+/**
+ * Allocate a new JSON context.
+ * Only use this if AV_JSON_DEFINE() is not suitable.
+ * @return  0 or an AVERROR code, including AVERROR(ENOMEM).
+ */
+int av_json_alloc(AVJson **jc, unsigned max_depth);
+
+/**
+ * Free a JSON context allocated with av_json_alloc().
+ */
+void av_json_free(AVJson **jc);
+
+/**
+ * Initialize a JSON context with output, flags an options.
+ * This function can be called several times on the same context to reuse
+ * it.
+ */
+void av_json_init(AVJson *jc, AVWriter wr, unsigned flags, AVDictionary *options);
+
+/**
+ * Begin an object, i.e. a { key : value... } dictionary.
+ * After this, every other value must be a string.
+ * The behavior is undefined if a key is not a string.
+ * The stack object must be allocated and inited, and valid until the
+ * corresponding av_json_end_object().
+ * See av_json_begin_object() for a more convenient version.
+ */
+void av_json_begin_object_with_stack(AVJson *jc, AVJsonStack *stack);
+
+/**
+ * End an object.
+ */
+void av_json_end_object(AVJson *jc);
+
+/**
+ * Begin an array, i.e. a [ value, value... ] list.
+ * The stack object must be allocated and inited, and valid until the
+ * corresponding av_json_end_object().
+ * See av_json_begin_array() for a more convenient version.
+ */
+void av_json_begin_array_with_stack(AVJson *jc, AVJsonStack *stack);
+
+/**
+ * End an array.
+ */
+void av_json_end_array(AVJson *jc);
+
+/**
+ * Begin a string and a return an AVWriter to write its contents.
+ */
+AVWriter av_json_begin_string(AVJson *jc);
+
+/**
+ * End a string. Optional.
+ */
+void av_json_end_string(AVJson *jc);
+
+/**
+ * Add a string all at once.
+ */
+void av_json_add_string(AVJson *jc, const char *str);
+
+/**
+ * Add a string all at once from a format string and a va_list.
+ */
+void av_json_add_string_vprintf(AVJson *jc, const char *fmt, va_list va);
+
+/**
+ * Add a sring all at once from a format string and arguments.
+ */
+void av_json_add_string_printf(AVJson *jc, const char *fmt, ...) av_printf_format(2, 3);
+
+/**
+ * Add an integer number.
+ */
+void av_json_add_int(AVJson *jc, intmax_t val);
+
+/**
+ * Add a floating-point number.
+ */
+void av_json_add_double(AVJson *jc, double val);
+
+/**
+ * Add a boolean value (true/false).
+ */
+void av_json_add_bool(AVJson *jc, int val);
+
+/**
+ * Add a null value.
+ */
+void av_json_add_null(AVJson *jc);
+
+/**
+ * Add an arbitrary value from a format string and a va_list.
+ * Useful for adding a floating point number and controlling the format.
+ * Warning: the validity of the output cannot be guaranteed.
+ */
+void av_json_add_raw_vprintf(AVJson *jc, const char *fmt, va_list va);
+
+/**
+ * Add an arbitrary value from a format string and arguments.
+ * Useful for adding a floating point number and controlling the format.
+ * Warning: the validity of the output cannot be guaranteed.
+ */
+void av_json_add_raw_printf(AVJson *jc, const char *fmt, ...) av_printf_format(2, 3);
+
+/**
+ * Define and init a stack element as a compound literal
+ * (hidden local variable).
+ * Using this directly is not recommended.
+ */
+#define AV_JSON_STACK() (&FF_NEW_SZ(AVJsonStack))
+
+/**
+ * Allocate a stack element.
+ * Only use this if av_json_begin_object() / av_json_begin_array() cannot be
+ * used.
+ * @return  0 or an AVERROR code, including AVERROR(ENOMEM).
+ */
+int av_json_stack_alloc(AVJsonStack **stack);
+
+/**
+ * Free a stack element allocated with av_json_stack_alloc().
+ */
+void av_json_stack_free(AVJsonStack **stack);
+
+/**
+ * Begin an object with a stack element as a compound literal
+ * (hidden local variable).
+ * The corresponding av_json_end_object() must be in the same scope.
+ * After this, every other value must be a string.
+ * The behavior is undefined if a key is not a string.
+ */
+#define av_json_begin_object(jc) av_json_begin_object_with_stack((jc), AV_JSON_STACK())
+
+/**
+ * Begin an array with a stack element as a compound literal
+ * (hidden local variable).
+ * The corresponding av_json_end_array() must be in the same scope.
+ */
+#define av_json_begin_array(jc) av_json_begin_array_with_stack((jc), AV_JSON_STACK())
+
+/**
+ * An AVWriter object to escape JSON strings.
+ *
+ * Can be allocated on the stack.
+ *
+ * Should be inited with one of the utility functions.
+ */
+struct AVJsonEscapeWriter {
+    size_t self_size; /**< Size of the structure itself */
+    AVWriter owr; /**< AVWriter to send the output */
+    unsigned flags; /**< Escaping flags, see AV_JSON_FLAG_* */
+    int error; /**< Error status */
+    unsigned replacement_char; /**< Replacement character for bad UTF-8 */
+    FFReservedPadding padding[2];
+};
+
+/**
+ * Get the methods for a JSON escape writer.
+ * Probably not useful to use directly.
+ */
+const AVWriterMethods *av_json_escape_writer_get_methods(void);
+
+/**
+ * Check if a writer is a JSON escape writer.
+ */
+int av_json_escape_writer_check(AVWriter wr);
+
+/**
+ * Initialize an AVJsonEscapeWriter to an already-allocated memory buffer.
+ *
+ * @return  jwr itself
+ * jwr->self_size must be set.
+ */
+AVJsonEscapeWriter *av_json_escape_writer_init(AVJsonEscapeWriter *jwr, AVWriter owr, unsigned flags);
+
+/**
+ * Create an AVWriter from an AVJsonEscapeWriter structure.
+ */
+AVWriter av_json_escape_writer_wrap(AVJsonEscapeWriter *jwr);
+
+/**
+ * Create an AVWriter to escape JSON strings.
+ *
+ * Note: as it relies on a compound statement, the AVJsonEscapeWriter object has
+ * a scope limited to the block where this macro is called.
+ */
+#define av_json_escape_writer(owr, flags) \
+    av_json_escape_writer_wrap(av_json_escape_writer_init(&FF_NEW_SZ(AVJsonEscapeWriter), (owr), (flags)))
+
+/**
+ * JSON encoding context.
+ *
+ * This structure must be allocated with AV_JSON_DEFINE() or equivalent
+ * code that will set av_class, self_size and escape_writer.
+ *
+ * It then must be initialized using av_json_init().
+ */
+struct AVJson {
+
+    /**
+     * Class; must be initialized to av_json_get_class().
+    */
+    const AVClass *av_class;
+
+    /**
+     * Size of the structure, must be initialized at allocation.
+     */
+    size_t self_size;
+
+    /**
+     * Encoding flags, see AV_JSON_FLAG_*.
+     */
+    unsigned flags;
+
+    /**
+     * Indentation shift.
+     */
+    unsigned indentation;
+
+    /**
+     * Output writer.
+     */
+    AVWriter out;
+
+    /****************************************************************
+     * The fields below this limit are private.
+     ****************************************************************/
+
+    /**
+     * Stack of states (object/array)
+     */
+    AVJsonStack *stack;
+
+    /**
+     * Pre-allocated writer for escaping strings.
+     *
+     * Must be allocated before init.
+     */
+    AVJsonEscapeWriter *escape_writer;
+
+    /**
+     * Depth of nested structures, for indentation.
+     */
+    unsigned depth;
+
+    /**
+     * True if a string is being constructed.
+     */
+    unsigned in_string : 1;
+
+    /**
+     * True if we currently are directly in an object.
+     */
+    unsigned in_object : 1;
+
+    /**
+     * True if we are about to write the first element of a structure.
+     */
+    unsigned first_element : 1;
+
+    /**
+     * True if we are about to write the key in an object.
+     */
+    unsigned object_key : 1;
+
+    FFReservedPadding padding[8];
+};
+
+/**
+ * Stack element for the JSON context.
+ */
+struct AVJsonStack {
+    size_t self_size;
+    AVJsonStack *prev;
+    unsigned short in_object;
+    FFReservedPadding padding;
+};
+
+/**
+ * Type of a JSON value.
+ *
+ * Note that JSON does not distinguish int and float values.
+ */
+enum AVJsonValueType {
+    AV_JSON_TYPE_NULL,
+    AV_JSON_TYPE_BOOL,
+    AV_JSON_TYPE_STRING,
+    AV_JSON_TYPE_INT,
+    AV_JSON_TYPE_DOUBLE,
+    AV_JSON_TYPE_OBJECT,
+    AV_JSON_TYPE_ARRAY,
+};
+
+/**
+ * Typed atomic JSON values.
+ * Objects and arrays cannot be represented.
+ * This structure is meant to be passed by value.
+ */
+struct AVJsonValue {
+    AVJsonValueType type;
+    union AVJsonValueValue {
+        intmax_t i;
+        double d;
+        const char *s;
+    } val;
+};
+
+/**
+ * Build an AVJsonValue for null.
+ */
+static inline AVJsonValue av_json_value_null(void)
+{
+    AVJsonValue ret = { .type = AV_JSON_TYPE_NULL };
+    return ret;
+}
+
+/**
+ * Build an AVJsonValue for a boolean.
+ */
+static inline AVJsonValue av_json_value_bool(int val)
+{
+    AVJsonValue ret = { .type = AV_JSON_TYPE_BOOL, .val.i = val };
+    return ret;
+}
+
+/**
+ * Build an AVJsonValue for a string.
+ * The pointer must stay valid while the value is used.
+ */
+static inline AVJsonValue av_json_value_string(const char *val)
+{
+    AVJsonValue ret = { .type = AV_JSON_TYPE_STRING, .val.s = val };
+    return ret;
+}
+
+/**
+ * Build an AVJsonValue for an integer.
+ */
+static inline AVJsonValue av_json_value_int(intmax_t val)
+{
+    AVJsonValue ret = { .type = AV_JSON_TYPE_INT, .val.i = val };
+    return ret;
+}
+
+/**
+ * Build an AVJsonValue for an integer.
+ */
+static inline AVJsonValue av_json_value_double(double val)
+{
+    AVJsonValue ret = { .type = AV_JSON_TYPE_INT, .val.d = val };
+    return ret;
+}
+
+/**
+ * @}
+ */
+
+#endif /* AVUTIL_JSON_H */