[FFmpeg-devel,v4,1/4] doc: Explain what "context" means

Derived from detailed explanations kindly provided by Stefano Sabatini:
https://ffmpeg.org/pipermail/ffmpeg-devel/2024-April/325903.html
---
 doc/context.md | 394 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 394 insertions(+)
 create mode 100644 doc/context.md

Sorry for the slow reply.

On date Wednesday 2024-05-15 16:54:19 +0100, Andrew Sayers wrote:
> Derived from detailed explanations kindly provided by Stefano Sabatini:
> https://ffmpeg.org/pipermail/ffmpeg-devel/2024-April/325903.html
> ---
>  doc/context.md | 394 +++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 394 insertions(+)
>  create mode 100644 doc/context.md
> 
> diff --git a/doc/context.md b/doc/context.md
> new file mode 100644
> index 0000000000..fb85b3f366
> --- /dev/null
> +++ b/doc/context.md
> @@ -0,0 +1,394 @@
> +# Introduction to contexts
> +
> +“%Context”

Is this style of quoting needed? Especially I'd avoid special markup
to simplify unredendered text reading (which is the point of markdown
afterall).

> is a name for a widely-used programming idiom.

> +This document explains the general idiom and the conventions FFmpeg has built around it.
> +
> +This document uses object-oriented analogies to help readers familiar with
> +[object-oriented programming](https://en.wikipedia.org/wiki/Object-oriented_programming)
> +learn about contexts.  But contexts can also be used outside of OOP,
> +and even in situations where OOP isn't helpful.  So these analogies
> +should only be used as a first step towards understanding contexts.
> +
> +## “Context” as a way to think about code
> +
> +A context is any data structure that is passed to several functions
> +(or several instances of the same function) that all operate on the same entity.
> +For example, [object-oriented programming](https://en.wikipedia.org/wiki/Object-oriented_programming)
> +languages usually provide member functions with a `this` or `self` value:
> +

> +```c
> +class my_cxx_class {
> +  void my_member_function() {
> +    // the implicit object parameter provides context for the member function:
> +    std::cout << this;
> +  }
> +};
> +```

I'm not convinced this is really useful: if you know C++ this is
redundant, if you don't this is confusing and don't add much information.

> +
> +Contexts are a fundamental building block of OOP, but can also be used in procedural code.

I'd drop this line, and drop the anchor on OOP at the same time since
it's adding no much information.

> +For example, most callback functions can be understood to use contexts:

> +
> +```c
> +struct MyStruct {
> +  int counter;
> +};
> +
> +void my_callback( void *my_var_ ) {
> +  // my_var provides context for the callback function:
> +  struct MyStruct *my_var = (struct MyStruct *)my_var_;
> +  printf("Called %d time(s)", ++my_var->counter);
> +}
> +
> +void init() {
> +  struct MyStruct my_var;
> +  my_var.counter = 0;
> +  register_callback( my_callback, &my_var );

style: fun(my_callback, ...) (so spaces around parentheses) here and
below

> +}
> +```
> +
> +In the broadest sense, &ldquo;context&rdquo; is just a way to think about code.
> +You can even use it to think about code written by people who have never
> +heard the term, or who would disagree with you about what it means.
> +
> +## &ldquo;Context&rdquo; as a tool of communication
> +
> +&ldquo;%Context&ldquo; can just be a word to understand code in your own head,
> +but it can also be a term you use to explain your interfaces.
> +Here is a version of the callback example that makes the context explicit:
> +
> +```c
> +struct CallbackContext {
> +  int counter;
> +};
> +
> +void my_callback( void *ctx_ ) {
> +  // ctx provides context for the callback function:
> +  struct CallbackContext *ctx = (struct CallbackContext *)ctx_;
> +  printf("Called %d time(s)", ++ctx->counter);
> +}
> +
> +void init() {
> +  struct CallbackContext ctx;
> +  ctx.counter = 0;
> +  register_callback( my_callback, &ctx );
> +}
> +```
> +
> +The difference here is subtle, but important.  If a piece of code
> +*appears compatible with contexts*, then you are *allowed to think
> +that way*, but if a piece of code *explicitly states it uses
> +contexts*, then you are *required to follow that approach*.
> +

> +For example, imagine someone modified `MyStruct` in the earlier example
> +to count several unrelated events across the whole program.  That would mean
> +it contained information about multiple entities, so was not a context.
> +But nobody ever *said* it was a context, so that isn't necessarily wrong.
> +However, proposing the same change to the `CallbackContext` in the later example
> +would violate a guarantee, and should be pointed out in a code review.
> +

I'm not very convinced by the callback example. The use of contexts in
the FFmpeg API is very much simpler, it is used to keep track of
configuration and state (that is they track the "object" where to
operate on), so the callback example here is a bit misleading.

Callbacks are used in the internals to implement different elements
(codecs, protocols, filters, etc...) implementing a common API, but in
this case the relation with "contexts" is less straightforward.

> +@warning Guaranteeing to use contexts does not mean guaranteeing to use
> +object-oriented programming.  For example, FFmpeg creates its contexts
> +procedurally instead of with constructors.

I'm afraid this is more confusing than helpful, since the FFmpeg API
is not OOP. I'd drop this sentence.

> +
> +## Contexts in the real world
> +
> +To understand how contexts are used in the real world, it might be
> +useful to compare [curl's MD5 hash context](https://github.com/curl/curl/blob/bbeeccdea8507ff50efca70a0b33d28aef720267/lib/curl_md5.h#L48)
> +with @ref AVMD5 "FFmpeg's equivalent context".
> +

> +The [MD5 algorithm](https://en.wikipedia.org/wiki/MD5) produces
> +a fixed-length digest from arbitrary-length data.  It does this by calculating
> +the digest for a prefix of the data, then loading the next part and adding it
> +to the previous digest, and so on.  Projects that use MD5 generally use some
> +kind of context, so comparing them can reveal differences between projects.
> +
> +```c
> +// Curl's MD5 context looks like this:
> +struct MD5_context {
> +  const struct MD5_params *md5_hash;    /* Hash function definition */
> +  void                  *md5_hashctx;   /* Hash function context */
> +};
> +
> +// FFmpeg's MD5 context looks like this:
> +typedef struct AVMD5 {
> +    uint64_t len;
> +    uint8_t  block[64];
> +    uint32_t ABCD[4];
> +} AVMD5;
> +```
> +
> +Curl's struct name ends with `_context`, guaranteeing contexts are the correct
> +interpretation.  FFmpeg's struct does not explicitly say it's a context, but
> +@ref libavutil/md5.c "its functions do" so we can reasonably assume
> +it's the intended interpretation.
> +
> +Curl's struct uses `void *md5_hashctx` to avoid guaranteeing
> +implementation details in the public interface, whereas FFmpeg makes
> +everything accessible.  This kind of data hiding is an advanced context-oriented
> +convention, and is discussed below.  Using it in this case has strengths and
> +weaknesses.  On one hand, it means changing the layout in a future version
> +of curl won't break downstream programs that used that data.  On the other hand,
> +the MD5 algorithm has been stable for 30 years, so it's arguably more important
> +to let people dig in when debugging their own code.
> +
> +Curl's struct is declared as `struct <type> { ... }`, whereas FFmpeg uses
> +`typedef struct <type> { ... } <type>`.  These conventions are used with both
> +context and non-context structs, so don't say anything about contexts as such.
> +Specifically, FFmpeg's convention is a workaround for an issue with C grammar:
> +
> +```c
> +void my_function( ... ) {
> +  int                my_var;        // good
> +  MD5_context        my_curl_ctx;   // error: C needs you to explicitly say "struct"
> +  struct MD5_context my_curl_ctx;   // good: added "struct"
> +  AVMD5              my_ffmpeg_ctx; // good: typedef's avoid the need for "struct"
> +}
> +```
> +
> +Both MD5 implementations are long-tested, widely-used examples of contexts
> +in the real world.  They show how contexts can solve the same problem
> +in different ways.

I'm concerned that this is adding more information than really
needed. Especially comparing with internals of curl means that now the
docs needs to be kept in synch also with the curl's API, meaning that
it will be outdated very soon. I'd rather drop the curl comparison
altogether.

> +
> +## FFmpeg's advanced context-oriented conventions
> +
> +Projects that make heavy use of contexts tend to develop conventions
> +to make them more useful.  This section discusses conventions used in FFmpeg,
> +some of which are used in other projects, others are unique to this project.
> +
> +### Naming: &ldquo;Context&rdquo; and &ldquo;ctx&rdquo;
> +
> +```c
> +// Context struct names usually end with `Context`:
> +struct AVSomeContext {
> +  ...
> +};
> +
> +// Functions are usually named after their context,
> +// context parameters usually come first and are often called `ctx`:
> +void av_some_function( AVSomeContext *ctx, ... );
> +```
> +
> +If an FFmpeg struct is intended for use as a context, its name usually
> +makes that clear.  Exceptions to this rule include AVMD5 (discussed above),
> +which is only identified as a context by the functions that call it.
> +
> +If a function is associated with a context, its name usually
> +begins with some variant of the context name (e.g. av_md5_alloc()
> +or avcodec_alloc_context3()).  Exceptions to this rule include
> +@ref avformat.h "AVFormatContext's functions", many of which
> +begin with just `av_`.
> +
> +If a function has a context parameter, it usually comes first and its name
> +often contains `ctx`.  Exceptions include av_bsf_alloc(), which puts the
> +context argument second to emphasise it's an out variable.
> +
> +### Data hiding: private contexts
> +
> +```c
> +// Context structs often hide private context:
> +struct AVSomeContext {
> +  void *priv_data; // sometimes just called "internal"
> +};
> +```
> +
> +Contexts usually present a public interface, so changing a context's members
> +forces everyone that uses the library to at least recompile their program,
> +if not rewrite it to remain compatible.  Hiding information in a private context
> +ensures it can be modified without affecting downstream software.
> +
> +Object-oriented programmers may be tempted to compare private contexts to
> +*private class members*.  That's often accurate, but for example it can also
> +be used like a *virtual function table* - a list of functions that are
> +guaranteed to exist, but may be implemented differently for different
> +sub-classes.  When thinking about private contexts, remember that FFmpeg
> +isn't *large enough* to need some common OOP techniques, even though it's
> +solving a problem that's *complex enough* to benefit from some rarer techniques.
> +
> +### Manage lifetime: allocate, initialize and free
> +
> +```c
> +void my_function( ... ) {
> +
> +    // Context structs are allocated then initialized with associated functions:
> +
> +    AVSomeContext *ctx = av_some_context_alloc( ... );
> +
> +    // ... configure ctx ...
> +
> +    av_some_context_init( ctx, ... );
> +
> +    // ... use ctx ...
> +
> +    // Context structs are freed with associated functions:
> +
> +    av_some_context_free( ctx );
> +
> +}
> +```
> +
> +FFmpeg contexts go through the following stages of life:
> +
> +1. allocation (often a function that ends with `_alloc`)
> +   * a range of memory is allocated for use by the structure
> +   * memory is allocated on boundaries that improve caching
> +   * memory is reset to zeroes, some internal structures may be initialized
> +2. configuration (implemented by setting values directly on the object)
> +   * no function for this - calling code populates the structure directly
> +   * memory is populated with useful values
> +   * simple contexts can skip this stage
> +3. initialization (often a function that ends with `_init`)
> +   * setup actions are performed based on the configuration (e.g. opening files)
> +5. normal usage
> +   * most functions are called in this stage
> +   * documentation implies some members are now read-only (or not used at all)
> +   * some contexts allow re-initialization
> +6. closing (often a function that ends with `_close()`)
> +   * teardown actions are performed (e.g. closing files)
> +7. deallocation (often a function that ends with `_free()`)
> +   * memory is returned to the pool of available memory
> +
> +This can mislead object-oriented programmers, who expect something more like:
> +
> +1. allocation (usually a `new` keyword)
> +   * a range of memory is allocated for use by the structure
> +   * memory *may* be reset (e.g. for security reasons)
> +2. initialization (usually a constructor)
> +   * memory is populated with useful values
> +   * related setup actions are performed based on arguments (e.g. opening files)
> +3. normal usage
> +   * most functions are called in this stage
> +   * compiler enforces that some members are read-only (or private)
> +   * no going back to the previous stage
> +4. finalization (usually a destructor)
> +   * teardown actions are performed (e.g. closing files)
> +5. deallocation (usually a `delete` keyword)
> +   * memory is returned to the pool of available memory
> +
> +FFmpeg's allocation stage is broadly similar to OOP, but can do some higher-level
> +operations.  For example, AVOptions-enabled structs (discussed below) contain an
> +AVClass member that is set during allocation.
> +
> +FFmpeg's "configuration" and "initialization" stages combine to resemble OOP's
> +"initialization" stage.  This can mislead object-oriented developers,
> +who are used to doing both at once.  This means FFmpeg contexts don't have
> +a direct equivalent of OOP constructors, as they would be doing
> +two jobs in one function.
> +
> +FFmpeg's three-stage creation process is useful for complicated structures.
> +For example, AVCodecContext contains many members that *can* be set before
> +initialization, but in practice most programs set few if any of them.
> +Implementing this with a constructor would involve a function with a list
> +of arguments that was extremely long and changed whenever the struct was
> +updated.  For contexts that don't need the extra flexibility, FFmpeg usually
> +provides a combined allocator and initializer function.  For historical reasons,
> +suffixes like `_alloc`, `_init`, `_alloc_context` and even `_open` can indicate
> +the function does any combination of allocation and initialization.
> +
> +FFmpeg's "closing" stage is broadly similar to OOP's "finalization" stage,
> +but some contexts allow re-initialization after finalization.  For example,
> +SwrContext lets you call swr_close() then swr_init() to reuse a context.
> +
> +FFmpeg's "deallocation" stage is broadly similar to OOP, but can perform some
> +higher-level functions (similar to the allocation stage).
> +
> +Very few contexts need the flexibility of separate "closing" and
> +"deallocation" stages, so these are usually combined into a single function.
> +Closing functions usually end with "_close", while deallocation
> +functions usually end with "_free".
> +
> +### Reflection: AVOptions-enabled structs
> +
> +Object-oriented programming puts more focus on data hiding than FFmpeg needs,
> +but it also puts less focus on
> +[reflection](https://en.wikipedia.org/wiki/Reflection_(computer_programming)).
> +
> +To understand FFmpeg's reflection requirements, run `ffmpeg -h full` on the
> +command-line, then ask yourself how you would implement all those options
> +with the C standard [`getopt` function](https://en.wikipedia.org/wiki/Getopt).
> +You can also ask the same question for any other programming languages you know.
> +[Python's argparse module](https://docs.python.org/3/library/argparse.html)
> +is a good example - its approach works well with far more complex programs
> +than `getopt`, but would you like to maintain an argparse implementation
> +with 15,000 options and growing?
> +
> +Most solutions assume you can just put all options in a single block,
> +which is unworkable at FFmpeg's scale.  Instead, we split configuration
> +across many *AVOptions-enabled structs*, which use the @ref avoptions
> +"AVOptions API" to reflect information about their user-configurable members,
> +including members in private contexts.
> +
> +An *AVOptions-enabled struct* is a struct that contains an AVClass element as
> +its first member, and uses that element to provide access to instances of
> +AVOption, each of which provides information about a single option.
> +The AVClass can also include more @ref AVClass "AVClasses" for private contexts,
> +making it possible to set options through the API that aren't
> +accessible directly.
> +
> +AVOptions-accessible members of a context should be accessed through the
> +AVOptions API whenever possible, even if they're not hidden away in a private
> +context.  That ensures values are validated as they're set, and means you won't
> +have to do as much work if a future version of FFmpeg changes the layout.
> +
> +AVClass was created very early in FFmpeg's history, long before AVOptions.
> +Its name suggests some kind of relationship to an OOP
> +base [class](https://en.wikipedia.org/wiki/Class_(computer_programming)),
> +but the name has become less accurate as FFmpeg evolved, to the point where
> +AVClass and AVOption are largely synonymous in modern usage.  The difference
> +might still matter if you need to support old versions of FFmpeg,
> +where you might find *AVClass context structures* (contain an AVClass element
> +as their first member) that are not *AVOptions-enabled* (don't use that element
> +to provide access to instances of AVOption).
> +
> +Object-oriented programmers may be tempted to compare @ref avoptions "AVOptions"
> +to OOP getters and setters.  There is some overlap in functionality, but OOP
> +getters and setters are usually specific to a single member and don't provide
> +metadata about the member; whereas AVOptions has a single API that covers
> +every option, and provides help text etc. as well.
> +
> +Object-oriented programmers may be tempted to compare AVOptions-accessible
> +members of a public context to protected members of a class.  Both provide
> +global access through an API, and unrestricted access for trusted friends.
> +But this is just a happy accident, not a guarantee.

This part looks fine, although there is too much OOP jargon for my
taste: this would make reading for programmers not familiar with OOP
harder than needed since she will miss many references.

> +
> +## Final example: context for a codec
> +
> +AVCodecContext is an AVOptions-enabled struct that contains information
> +about encoding or decoding one stream of data (e.g. the video in a movie).
> +It's a good example of many of the issues above.
> +
> +The name "AVCodecContext" tells us this is a context.  Many of
> +@ref libavcodec/avcodec.h "its functions" start with an `avctx` parameter,
> +indicating this object provides context for that function.
> +
> +AVCodecContext::internal contains the private context.  For example,
> +codec-specific information might be stored here.
> +
> +AVCodecContext is allocated with avcodec_alloc_context3(), initialized with
> +avcodec_open2(), and freed with avcodec_free_context().  Most of its members
> +are configured with the @ref avoptions "AVOptions API", but for example you
> +can set AVCodecContext::opaque or AVCodecContext::draw_horiz_band() if your
> +program happens to need them.
> +
> +AVCodecContext provides an abstract interface to many different *codecs*.
> +Options supported by many codecs (e.g. "bitrate") are kept in AVCodecContext
> +and reflected as AVOptions.  Options that are specific to one codec are
> +stored in the internal context, and reflected from there.
> +
> +To support a specific codec, AVCodecContext's private context is set to
> +an encoder-specific data type.  For example, the video codec
> +[H.264](https://en.wikipedia.org/wiki/Advanced_Video_Coding) is supported via
> +[the x264 library](https://www.videolan.org/developers/x264.html), and
> +implemented in X264Context.

> Although included in the documentation, X264Context is not part of the public API.

Why included in the doc? That is a private struct and therefore should
not be included in the doxy.

> +[...] Whereas there are strict rules about
> +changing AVCodecContext, a version of FFmpeg could modify X264Context or
> +replace it with another type altogether.  An adverse legal ruling or security
> +problem could even force us to switch to a completely different library
> +without a major version bump.
> +
> +The design of AVCodecContext provides several important guarantees:
> +
> +- lets you use the same interface for any codec
> +- supports common encoder options like "bitrate" without duplicating code
> +- supports encoder-specific options like "profile" without bulking out the public interface
> +- reflects both types of options to users, with help text and detection of missing options
> +- hides implementation details (e.g. its encoding buffer)
> -- 
> 2.43.0
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

On Thu, May 23, 2024 at 09:00:39PM +0100, Andrew Sayers wrote:
> NOTE: this patchset depends on [1], and should not be applied before that.
> 
> I think it's important to guide readers between parts of FFmpeg, because
> learning how the pieces of the puzzle fit together is a big part of the
> newbie experience.  So this patchset replaces the "@ref Context for foo"
> statements in public structs with "@see" blocks, giving us a hook we can
> hang more links on in future.
> 
> That said, there's a rule against internal links from private structs,
> so I've removed the @ref's from them.  By the way, is this rule written
> somewhere?  If not, where would be a good place to write it?
> And either way, it would be good to link to this as part of [2].
> 
> Previous patches had to change the language for many structs, but "@see" blocks
> avoid the need to include those changes in this patchset.  Rather than waste
> that work, I've temporarily moved those changes to the final patch in this set.
> My feelings about that last patch aren't strong, but I guess I'll propose them
> in a separate thread unless anyone wants them here or chucked altogether.
> 
> 
> I've rewritten AVOptions and AVClass based on feedback.  The new version
> reflects a hypothetical that's been going round my head all week...
> 
> Imagine you wanted to write a system that nudged people to try new codecs.
> It might say e.g. "you seem to be using H.264, would you like to try H.265?"
> Implementing that would probably involve a struct like:
> 
> struct AVOldNew {
>   AVClass* old;
>   AVClass* new;
> };

AVClass would describe the internal decoder structures. This would not be
correct at all in this example.
Thats like handing a man 2 CAD documents about 2 engines of 2 cars

If you wanted to suggest to get a tesla instead of a ford. One would have to
describe the 2 cars and their differences
thats 2 AVCodecDescriptor maybe

thx

[...]

On Fri, May 24, 2024 at 03:50:52AM +0200, Michael Niedermayer wrote:
> On Thu, May 23, 2024 at 09:00:39PM +0100, Andrew Sayers wrote:
<snip>
> > Imagine you wanted to write a system that nudged people to try new codecs.
> > It might say e.g. "you seem to be using H.264, would you like to try H.265?"
> > Implementing that would probably involve a struct like:
> > 
> > struct AVOldNew {
> >   AVClass* old;
> >   AVClass* new;
> > };
> 
> AVClass would describe the internal decoder structures. This would not be
> correct at all in this example.
> Thats like handing a man 2 CAD documents about 2 engines of 2 cars
> 
> If you wanted to suggest to get a tesla instead of a ford. One would have to
> describe the 2 cars and their differences
> thats 2 AVCodecDescriptor maybe

Hmm, yes fair point.  A better example might be a simple linked list:

struct AVClassList {
    AVClass* cur;
    AVClassList* next;
};

Again, that clearly is a struct that begins with AVClass*, but clearly isn't an
AVClass context structure.

I realise it's a bit of an academic distinction, but IMHO these hypotheticals
suggest it's more accurate to define the term "AVClass context structure"
in terms of usage rather than layout.

On date Wednesday 2024-05-22 17:07:51 +0100, Andrew Sayers wrote:
[...]
> > > diff --git a/doc/context.md b/doc/context.md
> > > new file mode 100644
> > > index 0000000000..fb85b3f366
> > > --- /dev/null
> > > +++ b/doc/context.md
> > > @@ -0,0 +1,394 @@
> > > +# Introduction to contexts
> > > +
> > > +“%Context”
> > 
> > Is this style of quoting needed? Especially I'd avoid special markup
> > to simplify unredendered text reading (which is the point of markdown
> > afterall).
> 
> Short answer: I'll change it in the next patch and see what happens.
> 
> Long answer: HTML quotes are ugly for everyone, UTF-8 is great until someone
> turns up complaining we broke their Latin-1 workflow.  I've always preferred
> ASCII-only representations for that reason, but happy to try the other way
> and see if anyone still cares.

I think it's safe to assume UTF-8 as the default nowadays, and every
other encoding broken in same way. Latin-1 is assumed probably only by
a minor group of FFmpeg users.
 
> > 
> > > is a name for a widely-used programming idiom.
> > 
> > > +This document explains the general idiom and the conventions FFmpeg has built around it.
> > > +
> > > +This document uses object-oriented analogies to help readers familiar with
> > > +[object-oriented programming](https://en.wikipedia.org/wiki/Object-oriented_programming)
> > > +learn about contexts.  But contexts can also be used outside of OOP,
> > > +and even in situations where OOP isn't helpful.  So these analogies
> > > +should only be used as a first step towards understanding contexts.
> > > +
> > > +## “Context” as a way to think about code
> > > +
> > > +A context is any data structure that is passed to several functions
> > > +(or several instances of the same function) that all operate on the same entity.
> > > +For example, [object-oriented programming](https://en.wikipedia.org/wiki/Object-oriented_programming)
> > > +languages usually provide member functions with a `this` or `self` value:
> > > +
> > 
> > > +```c
> > > +class my_cxx_class {
> > > +  void my_member_function() {
> > > +    // the implicit object parameter provides context for the member function:
> > > +    std::cout << this;
> > > +  }
> > > +};
> > > +```
> > 
> > I'm not convinced this is really useful: if you know C++ this is
> > redundant, if you don't this is confusing and don't add much information.
> 
> The example is there to break up a wall of text (syntax-highlighted in the
> rendered output), and to let the reader know that this is going to be one of
> those documents that alternates between text and code, so they're ready for the
> more substantive examples later on.  I take the point about C++ though -
> would this Python example be more readable?
> 
>     class MyClass:
>         def my_func(self):
>             # If a Python function is part of a class,
>             # its first parameter must be an instance of that class
> 

But again, this is redundnat if you know Python, otherwise is not very
helpful if you don't. In this case probably it's good just to keep the
mention to self/this, without a specific example.

> > 
> > > +
> > > +Contexts are a fundamental building block of OOP, but can also be used in procedural code.
> > 
> > I'd drop this line, and drop the anchor on OOP at the same time since
> > it's adding no much information.
> 
> Fundamentally, this document addresses two audiences:
> 
> 1. people coming from a non-OOP background, who want to learn contexts
>    from first principles, and at best see OOP stuff as background information
> 
> 2. people coming from an OOP background.  There's no polite way to say this -
>    their incentive is to write FFmpeg off as a failed attempt at OOP, so they
>    don't have to learn a new way of working that's just different enough to
>    make them feel dumb
> 
> I think a good way to evaluate the document might be to read it through twice,
> stopping after each paragraph to ask two unfair questions...
> 
> 1. what has this told me about FFmpeg itself, as opposed to some other thing
>    you wish I cared about?
> 
> 2. couldn't you have just done this the standard OOP way?
> 
> The earlier paragraph acknowledged that contexts resemble OOP (telling the OOP
> audience we get it), then this paragraph adds "but they're not the same"
> (telling the OOP audience we disagree).  To be more useful to non-OOP folk,
> how about:
> 

>     Contexts can be a fundamental building block of OOP, but can also be used in
>     procedural projects like FFmpeg.

What perplexes me is that "context" is not part of the standard OOP
jargon, so this is probably adding more to the confusion. I think the
target here is to clarify what context is meant for in FFmpeg, and
this can be simply defined without any reference to OOP:

|A context is a structure used to contain data (allocated memory,
|configuration, and/or state) for operations working on the same
|entity.

As such a context is just a glorified struct referecenced by a set of
functions operating on them, with the convention (common in most C
APIs) that this is the first argument of these functions.

Then we have AVClass+log+options as a trick to define common
functionality for the most complex "contexts".

This is also why I'm concerned with the idea of explainig this with a
strong focus on OOP: the effect is that the reader not familiar with
OOP is convinced that she needs to be familiar with OOP to understand
it (which is not the case). That's also why I'm in favor of having a
comparison with OOP in a dedicated section: this might help the
OOP-minded reader, and might be skipped by the other ones.

[...]
> > I'm not very convinced by the callback example. The use of contexts in
> > the FFmpeg API is very much simpler, it is used to keep track of
> > configuration and state (that is they track the "object" where to
> > operate on), so the callback example here is a bit misleading.
> > 
> > Callbacks are used in the internals to implement different elements
> > (codecs, protocols, filters, etc...) implementing a common API, but in
> > this case the relation with "contexts" is less straightforward.
> 
> I go back and forth on this one, but your point made me think about it
> in a new way...
> 

> AVIOContext::read_packet is a callback function, and a reader who has just
> learnt about contexts would naturally assume we intend its first argument
> to be interpreted as a context.  Given that new readers are likely to learn
> avio_alloc_context() around the same time as reading this document,
> it's important we give them the tools to understand that function.
> 
> How about changing the topmost callback example to read data from a FILE*
> (without mentioning AVIOContext), then emphasising how you can think of it
> as a context despite not following FFmpeg's rules, then finally mentioning
> how you could pass the callback to avio_alloc_context() if you wanted?

Let's discuss this in the context of the new patch.

> > > +@warning Guaranteeing to use contexts does not mean guaranteeing to use
> > > +object-oriented programming.  For example, FFmpeg creates its contexts
> > > +procedurally instead of with constructors.
> > 
> > I'm afraid this is more confusing than helpful, since the FFmpeg API
> > is not OOP. I'd drop this sentence.
> 
> My concern is that if an OOP reader asks "couldn't you have just done this
> the standard OOP way?", they will be tempted to answer "oh, so you *used to*
> fail at OOP but nowadays you promise to do it right", and not bother reading
> any further.  So there needs to be something eye-catching here, but yes this
> paragraph needs to be more useful to non-OOP readers.

If you write:
> FFmpeg creates its contexts procedurally instead of with constructors.

then I don't know what this means, since there are no explicit
"contructors" in plain C as they are not part of the language
(although you can mimic constructors using special conventions, but
this is something built on top of the language).

>[...]
> This will probably need to be rewritten based on the callback discussion,
> so I'll think about ways to change this at the same time.
> > > +Curl's struct is declared as `struct <type> { ... }`, whereas FFmpeg uses
> > > +`typedef struct <type> { ... } <type>`.  These conventions are used with both
> > > +context and non-context structs, so don't say anything about contexts as such.
> > > +Specifically, FFmpeg's convention is a workaround for an issue with C grammar:
> > > +
> > > +```c
> > > +void my_function( ... ) {
> > > +  int                my_var;        // good
> > > +  MD5_context        my_curl_ctx;   // error: C needs you to explicitly say "struct"
> > > +  struct MD5_context my_curl_ctx;   // good: added "struct"
> > > +  AVMD5              my_ffmpeg_ctx; // good: typedef's avoid the need for "struct"
> > > +}
> > > +```
> > > +
> > > +Both MD5 implementations are long-tested, widely-used examples of contexts
> > > +in the real world.  They show how contexts can solve the same problem
> > > +in different ways.
> > 
> > I'm concerned that this is adding more information than really
> > needed. Especially comparing with internals of curl means that now the
> > docs needs to be kept in synch also with the curl's API, meaning that
> > it will be outdated very soon. I'd rather drop the curl comparison
> > altogether.
> 
> The value of this section depends on the reader...
> 
> This tells a non-OOP reader that FFmpeg didn't invent contexts, and reasonable
> people can disagree about what they mean.  So when we get into the ambiguous
> cases later on, they have a better idea about which things are just how
> contexts work, and which are specifically how FFmpeg uses them.
> 
> This tells an OOP reader that it's not "OOP standard, FFmpeg non-standard",
> it's that FFmpeg is using a C standard that's not used in OOP languages.
> 
> Curl's `MD5_context` was last modified in 2020 (interestingly, to get rid of
> the `typedef struct` trick).  It's a slow-moving target, but you're right it's
> not a static one.
> 
> I'd argue the above means this should be *somewhere* in the document, but not
> necessarily here in the middle.  I'll see if it works better as a paragraph
> here and an appendix or something at the bottom.

Yes, moving to a less prominent section might help, since we want to
avoid the long-wall-of-text effect with information not impacting the
basic understanding.

[...]
> > > +To support a specific codec, AVCodecContext's private context is set to
> > > +an encoder-specific data type.  For example, the video codec
> > > +[H.264](https://en.wikipedia.org/wiki/Advanced_Video_Coding) is supported via
> > > +[the x264 library](https://www.videolan.org/developers/x264.html), and
> > > +implemented in X264Context.
> > 
> > > Although included in the documentation, X264Context is not part of the public API.
> > 
> > Why included in the doc? That is a private struct and therefore should
> > not be included in the doxy.
> 

> Doxygen doesn't provide an obvious mechanism to include only the public API.
> Changing that would be at best a big job, and it isn't obvious where to even
> start until/unless e.g. [1] gets merged in.  It seems like a better plan
> to put the warning in and take it out if and when the site gets updated.

I think it's fine to keep as is, this is somehow a doxygen glitch, but
at the same time might help with internal development (in this case
you want to know the internal structures).

It feels like we've got through most of the mid-level "how FFmpeg works" stuff,
and now we're left with language choices (e.g "options" vs. "introspection")
and philosophical discussions (e.g. the relationship between contexts and OOP).
It's probably best to philosophise first, then come back to language.

This message has been sent as a reply to one specific message, but is actually
springboarding off messages from two sub-threads.  Hopefully that will keep
the big questions contained in one place.

On Sat, May 25, 2024 at 11:49:48AM +0200, Stefano Sabatini wrote:
> What perplexes me is that "context" is not part of the standard OOP
> jargon, so this is probably adding more to the confusion.

This actually speaks to a more fundamental issue about how we learn.
To be clear, everything I'm about to describe applies every human that ever
lived, but starting with this message makes it easier to explain
(for reasons that will hopefully become obvious).

When you ask why "context" is not part of OOP jargon, one could equally ask
why "object" isn't part of FFmpeg jargon.  The document hints at some arguments:
their lifetime stages are different, their rules are enforced at the
language vs. community level, OOP encourages homogenous interfaces while FFmpeg
embraces unique interfaces that precisely suit each use case, and so on.
But the honest answer is much simpler - humans are lazy, and we want the things
we learn today to resemble the things we learnt yesterday.

Put another way - if we had infinite time every day, we could probably write an
object-oriented interface to FFmpeg.  But our time is sadly finite so we stick
with the thing that's proven to work.  Similarly, if our readers had infinite
free time every day, they could probably learn a completely new approach to
programming.  But their time is finite, so they stick to what they know.

That means people reading this document aren't just passively soaking up
information, they're looking for shortcuts that fit their assumptions.
And as anyone that's ever seen a political discussion can tell you,
humans are *really good* at finding shortcuts that fit their assumptions.
For example, when an OOP developer sees the words "alloc" and "init",
they will assume these map precisely to OOP allocators and initializers.  One
reason for the long section about context lifetimes is because it needs to
meet them where they are, then walk them step-by-step to a better place.

Aside: if FFmpeg had a blog, I could turn this discussion into a great post
called something like "reflections on object- vs. context-oriented development".
But the project's voice is more objective than that, so this document is limited
to discussing the subset of issues that relate specifically to the FFmpeg API.


On Sat, May 25, 2024 at 01:00:14PM +0200, Stefano Sabatini wrote:
> > +Some functions fit awkwardly within FFmpeg's context idiom.  For example,
> > +av_ambient_viewing_environment_create_side_data() creates an
> > +AVAmbientViewingEnvironment context, then adds it to the side-data of an
> > +AVFrame context.
> 
> To go back to this unfitting example, can you state what would be
> fitting in this case?

"Awkwardly" probably isn't the right word to use, but that's a language choice
we can come back to.

The problem with FFmpeg's interface isn't that any one part is illogical,
it's that different parts of the interface follow incompatible logic.

It's hard to give specific examples, because any given learner's journey looks
like a random walk through the API, and you can always say "well nobody else
would have that problem".  But if everyone has a different problem, that means
everyone has *a* problem, even though there's no localised code fix.

For sake of argument, let's imagine a user who was a world-leading expert in
Microsoft QBasic in the eighties, then fell into a forty-year coma and woke up
in front of the FFmpeg documentation.  In other words, a highly adept
programmer with zero knowledge of programming conventions more recent than
"a function is a special type of subroutine for returning a value".
Their journey might look like...

1. there's this thing called "context", and some functions "have" contexts
2. sws_init_context() says "Initialize the swscaler context sws_context",
   and `sws_context` is a `SwsContext *`, so I think it has a SwsContext context
3. sws_alloc_context() says "Allocate an empty SwsContext",
   and it returns a `SwsContext *`, so I think it has the same context
   as sws_init_context()
4. avio_alloc_context() and avio_open2() are both variations on this theme,
   so I should look for creative ways to interpret things as "having" contexts
5. av_alloc_format_context() puts the type in the middle of the function name,
   so I should only treat prefixes as a weak signal
6. av_ambient_viewing_environment_create_side_data() allocates like an alloc,
   so I think the return value is the context; but it also operates on AVFrame
   in a way that affects related functions, so I think the arg is the context.
   Its prefix is too a weak a signal to be a tiebreaker, so I'll just guess
   one of them at random and wait until something goes wrong

In the above case, the interface rewarded the developer for looking harder and
harder for ways to call something a context, to the point where saying "neither"
becomes inconceivable (or at best an admission of defeat).

There's no way to avoid that sort of inconsistency in a project like FFmpeg,
and explaining the logic behind each choice would involve an order of magnitude
more documentation.  So the only practical choice is to present a sort of
conceptual buffet - "here are several ways to think about the problem, choose
whatever suits your tastes".

On date Sunday 2024-05-26 13:06:52 +0100, Andrew Sayers wrote:
> It feels like we've got through most of the mid-level "how FFmpeg works" stuff,
> and now we're left with language choices (e.g "options" vs. "introspection")
> and philosophical discussions (e.g. the relationship between contexts and OOP).
> It's probably best to philosophise first, then come back to language.
> 
> This message has been sent as a reply to one specific message, but is actually
> springboarding off messages from two sub-threads.  Hopefully that will keep
> the big questions contained in one place.
> 
> On Sat, May 25, 2024 at 11:49:48AM +0200, Stefano Sabatini wrote:
> > What perplexes me is that "context" is not part of the standard OOP
> > jargon, so this is probably adding more to the confusion.
> 

> This actually speaks to a more fundamental issue about how we learn.
> To be clear, everything I'm about to describe applies every human that ever
> lived, but starting with this message makes it easier to explain
> (for reasons that will hopefully become obvious).
> 

> When you ask why "context" is not part of OOP jargon, one could equally ask
> why "object" isn't part of FFmpeg jargon.  The document hints at some arguments:
> their lifetime stages are different, their rules are enforced at the
> language vs. community level, OOP encourages homogenous interfaces while FFmpeg
> embraces unique interfaces that precisely suit each use case, and so on.
> But the honest answer is much simpler - humans are lazy, and we want the things
> we learn today to resemble the things we learnt yesterday.
> 
> Put another way - if we had infinite time every day, we could probably write an
> object-oriented interface to FFmpeg.  But our time is sadly finite so we stick
> with the thing that's proven to work.  Similarly, if our readers had infinite
> free time every day, they could probably learn a completely new approach to
> programming.  But their time is finite, so they stick to what they know.
> 
> That means people reading this document aren't just passively soaking up
> information, they're looking for shortcuts that fit their assumptions.
> And as anyone that's ever seen a political discussion can tell you,
> humans are *really good* at finding shortcuts that fit their assumptions.
> For example, when an OOP developer sees the words "alloc" and "init",
> they will assume these map precisely to OOP allocators and initializers.  One
> reason for the long section about context lifetimes is because it needs to
> meet them where they are, then walk them step-by-step to a better place.

I think we start with different assumptions: you assume that most of
the readers are familiar with OOP jargon, and that they will leverage
the OOP jargon to understand the FFmpeg API. I think this is
misleading: historically what defined FFmpeg is its minimalistic
approach, in this case we don't want the user, familiar with C, to be
familiar with OOP in order to use and understand the FFmpeg API, as
this is simply not necessary as FFmpeg is plain C language.

In fact, if this might help with some users, this will probably
confuse all the others. Even more, if we adopt the FFmpeg jargon to
OOP (using "context") we are adding more confusion, as now the OOP
reader will have to adopt the FFmpeg jargon applied to OOP.

My advice is to move all the content to OOP to a dedicated section, or
to make the references to OOP jargon as less as possible, to not get
in the way with the non-OOP reader.

> Aside: if FFmpeg had a blog, I could turn this discussion into a great post
> called something like "reflections on object- vs. context-oriented development".
> But the project's voice is more objective than that, so this document is limited
> to discussing the subset of issues that relate specifically to the FFmpeg API.

One option would be the wiki:
http://trac.ffmpeg.org/

but probably a personal blog entry would be better, given that this
would not be reference material but more as an article (since the API
and its own philosophy is a moving target).

> 
> On Sat, May 25, 2024 at 01:00:14PM +0200, Stefano Sabatini wrote:
> > > +Some functions fit awkwardly within FFmpeg's context idiom.  For example,
> > > +av_ambient_viewing_environment_create_side_data() creates an
> > > +AVAmbientViewingEnvironment context, then adds it to the side-data of an
> > > +AVFrame context.
> > 
> > To go back to this unfitting example, can you state what would be
> > fitting in this case?
> 
> "Awkwardly" probably isn't the right word to use, but that's a language choice
> we can come back to.
> 
> The problem with FFmpeg's interface isn't that any one part is illogical,
> it's that different parts of the interface follow incompatible logic.
> 
> It's hard to give specific examples, because any given learner's journey looks
> like a random walk through the API, and you can always say "well nobody else
> would have that problem".  But if everyone has a different problem, that means
> everyone has *a* problem, even though there's no localised code fix.

Fine, but can you propose the signature of the functions that you
would consider optimal in this case?

One option I suggested is to move the function to frame.h, but this
would have other implications (that's why keeping as is looks to me
like a reasonable tradeoff - you can consider it as a sort of static
class method if you like).

> 
> For sake of argument, let's imagine a user who was a world-leading expert in
> Microsoft QBasic in the eighties, then fell into a forty-year coma and woke up
> in front of the FFmpeg documentation.  In other words, a highly adept
> programmer with zero knowledge of programming conventions more recent than
> "a function is a special type of subroutine for returning a value".
> Their journey might look like...
> 
> 1. there's this thing called "context", and some functions "have" contexts
> 2. sws_init_context() says "Initialize the swscaler context sws_context",
>    and `sws_context` is a `SwsContext *`, so I think it has a SwsContext context
> 3. sws_alloc_context() says "Allocate an empty SwsContext",
>    and it returns a `SwsContext *`, so I think it has the same context
>    as sws_init_context()
> 4. avio_alloc_context() and avio_open2() are both variations on this theme,
>    so I should look for creative ways to interpret things as "having" contexts
> 5. av_alloc_format_context() puts the type in the middle of the function name,
>    so I should only treat prefixes as a weak signal

> 6. av_ambient_viewing_environment_create_side_data() allocates like an alloc,
>    so I think the return value is the context; but it also operates on AVFrame
>    in a way that affects related functions, so I think the arg is the context.
>    Its prefix is too a weak a signal to be a tiebreaker, so I'll just guess
>    one of them at random and wait until something goes wrong

I see, but the "exception" should be apparent from the doc.

Also you might think at it like:
AVAmbientViewingEnvironment.create_side_data(AVFrame *frame)

that is as a static class method creating the object starting from a
frame component.

[...]

Anyway resuming, I'm mostly fine with your latest patch, but for the
nits and the specific points I raised. Also I'd tend to loose the
references to OOP jargon for the reasons I explained.

On Tue, May 28, 2024 at 07:24:55PM +0200, Stefano Sabatini wrote:
> 
> I think we start with different assumptions: you assume that most of
> the readers are familiar with OOP jargon, and that they will leverage
> the OOP jargon to understand the FFmpeg API. I think this is

Not exactly.  I'm saying the document has two equally valid use cases:

1. non-OOP developers, who need us to add new information to their brains
2. OOP developers, who need us to replace existing information in their brains

So it's not a matter of "leveraging OOP" jargon so much as "calling out
OOP assumptions".  For example, I originally assumed AVClass was to FFmpeg
as the Object struct[1] is to GObject, and struggled to take anything onboard
until that been explicitly debunked.  You may even be able to see that
opinion gradually eroding in each rewrite of the context document.

But actually, two groups is probably over-simplifying.  Another group would be
people coming from other C projects, who have incompatible ideas about contexts
that would never occur to us.  Or Fortran developers, who (according to the
first link on Google[2]) can be expected to understand modules but not objects.

> misleading: historically what defined FFmpeg is its minimalistic
> approach, in this case we don't want the user, familiar with C, to be
> familiar with OOP in order to use and understand the FFmpeg API, as
> this is simply not necessary as FFmpeg is plain C language.
> 
> In fact, if this might help with some users, this will probably
> confuse all the others. Even more, if we adopt the FFmpeg jargon to
> OOP (using "context") we are adding more confusion, as now the OOP
> reader will have to adopt the FFmpeg jargon applied to OOP.
> 
> My advice is to move all the content to OOP to a dedicated section, or
> to make the references to OOP jargon as less as possible, to not get
> in the way with the non-OOP reader.

Earlier versions of the document probably drew too strong an analogy with OOP,
but I suspect the problem may be simpler in the latest draft.  The majority of
OOP references explain how things are different to OOP - it might be possible
to make these more readable, but they can't be moved or removed, because
replacing information involves calling out the old information first.  Only the
first few OOP references draw the analogy in the positive, aiming to show OOP
devs how this is similar but different.

Given the OOP vs. non-OOP distinction is too simplistic, the solution might be
to start the document with a collection of code samples - a generic C function,
a Python object, a JavaScript event listener, and an FFmpeg sample.  That would
show how "context" is a name for a broad concept you've probably used before,
without singling out OOP.  It would also reinforce the idea of the document as
a buffet of concepts - don't like the OOP example?  Never mind, here are some
procedural ones for you to enjoy.

> 
> > Aside: if FFmpeg had a blog, I could turn this discussion into a great post
> > called something like "reflections on object- vs. context-oriented development".
> > But the project's voice is more objective than that, so this document is limited
> > to discussing the subset of issues that relate specifically to the FFmpeg API.
> 
> One option would be the wiki:
> http://trac.ffmpeg.org/
> 
> but probably a personal blog entry would be better, given that this
> would not be reference material but more as an article (since the API
> and its own philosophy is a moving target).
> 
> > 
> > On Sat, May 25, 2024 at 01:00:14PM +0200, Stefano Sabatini wrote:
> > > > +Some functions fit awkwardly within FFmpeg's context idiom.  For example,
> > > > +av_ambient_viewing_environment_create_side_data() creates an
> > > > +AVAmbientViewingEnvironment context, then adds it to the side-data of an
> > > > +AVFrame context.
> > > 
> > > To go back to this unfitting example, can you state what would be
> > > fitting in this case?
> > 
> > "Awkwardly" probably isn't the right word to use, but that's a language choice
> > we can come back to.
> > 
> > The problem with FFmpeg's interface isn't that any one part is illogical,
> > it's that different parts of the interface follow incompatible logic.
> > 
> > It's hard to give specific examples, because any given learner's journey looks
> > like a random walk through the API, and you can always say "well nobody else
> > would have that problem".  But if everyone has a different problem, that means
> > everyone has *a* problem, even though there's no localised code fix.
> 
> Fine, but can you propose the signature of the functions that you
> would consider optimal in this case?

I think we largely agree on this, but since you ask -

The code solution would be to pick a rule and rewrite the entire codebase to
fit that rule.  I don't have a strong opinion about what the rule is, but one
example might be "the first context parameter to a function must be that
function's context", which I guess would mean renaming this to something like
"av_frame_create_ambient_viewing_environment_side_data()".  That would be a
huge amount of work to implement, and even more work to maintain, so yes it's
fine to just document it and move on.

> One option I suggested is to move the function to frame.h, but this
> would have other implications (that's why keeping as is looks to me
> like a reasonable tradeoff - you can consider it as a sort of static
> class method if you like).

That's fine for this example, but interesting in other ways.

I'm increasingly reading the API in terms of what signals different parts of it
emit.  For example, if a function ends with "_alloc", that signals it's probably
an allocator, but av_max_alloc() doesn't allocate `max`es, so that signal's
information value is less than 1.  I think you're saying the location of a
function is an important signal - what sort of things does it tell you?

[...]

[1] https://docs.gtk.org/gobject/class.Object.html
[2] https://fortranwiki.org/fortran/show/Object-oriented+programming

Posting this separately, as these are practical "how does FFmpeg work" issues
vaguely inspired by recent discussions.


*How do namespaces work in FFmpeg?*

We've talked a bit about function namespaces recently.  One reason I've
suggested they're a weak signal is because they aren't really addressed in the
documentation.  How about adding something like this to the context doc:

    Most FFmpeg functions are grouped into namespaces, usually indicated by
    prefixes (e.g. `av_format_*`) but sometimes indicated by infixes
    (e.g. av_alloc_format_context()).  Namespaces group functions by *topic*,
    which doesn't always correlate with *context*.  For example, `av_find_*`
    functions search for information across various contexts.


*Should external API devs treat Sw{r,s}Context as AVClass context structures?*

This is probably an uninteresting edge case, but just to be sure...

The website says Sw{r,s}Context start with AVClass[1], they have _get_class
functions, are shown in `ffmpeg -h full`, and I haven't found anything that says
to treat them differently to e.g. AVCodecContext.  So I'm pretty sure these
are AVClass context structures, at least as far as internal devs are concerned.

But their definitions are only in the private interface, so the layout is just
an implementation detail that can change without even a major version bump.
AVFrame used to have a _get_class function despite never having an actual
AVClass member, so that's not a signal to external API devs.  And `URLContext`
appears in `ffmpeg -h full` despite having being made private long ago,
so that's not much of a signal either.

My guess is that the above should be addressed with a patch like:

+/**
+ * @brief Context for SWResampler
+ *
+ * @note The public ABI only guarantees this is an AVOptions-enabled struct.
+ * Its size and other members are not a part of the public ABI.
+ *
+ * @see
+ * - @ref Context
+ */
 struct SwrContext {

Let me know if the above is on the right track.  If so, I'll queue up a patch
for after the context document is done.


*Are AVOptions just command-line options?*

I have trouble with statements like "AVOptions is a framework for options",
both because it's circular and because the term "option" is too broad to be
meaningful.

I've previously pushed the word "reflection" on the assumption that options
can be used anywhere variables are used.  For example, imagine a decoder that
could be reconfigured on-the-fly to reduce CPU usage at the cost of displaying
blurier images.  That can't be part of the public interface because it's
codec-specific, but I could imagine updating some kind of "output_quality"
AVOption as a user slides a slider up and down.

But the CLI tools are largely non-interactive, so have I just misunderstood?

How about saying "AVOptions is a framework for command-line options"?

[1] https://ffmpeg.org/doxygen/trunk/structSwrContext.html

On Wed, May 29, 2024 at 12:50 PM Andrew Sayers <ffmpeg-devel@pileofstuff.org>
wrote:

> Posting this separately, as these are practical "how does FFmpeg work"
> issues
> vaguely inspired by recent discussions.
>
>
> *How do namespaces work in FFmpeg?*
>
> We've talked a bit about function namespaces recently.  One reason I've
> suggested they're a weak signal is because they aren't really addressed in
> the
> documentation.  How about adding something like this to the context doc:
>
>     Most FFmpeg functions are grouped into namespaces, usually indicated by
>     prefixes (e.g. `av_format_*`) but sometimes indicated by infixes
>     (e.g. av_alloc_format_context()).  Namespaces group functions by
> *topic*,
>     which doesn't always correlate with *context*.  For example,
> `av_find_*`
>     functions search for information across various contexts.
>
>
> *Should external API devs treat Sw{r,s}Context as AVClass context
> structures?*
>
> This is probably an uninteresting edge case, but just to be sure...
>
> The website says Sw{r,s}Context start with AVClass[1], they have _get_class
> functions, are shown in `ffmpeg -h full`, and I haven't found anything
> that says
> to treat them differently to e.g. AVCodecContext.  So I'm pretty sure these
> are AVClass context structures, at least as far as internal devs are
> concerned.
>
> But their definitions are only in the private interface, so the layout is
> just
> an implementation detail that can change without even a major version bump.
> AVFrame used to have a _get_class function despite never having an actual
> AVClass member, so that's not a signal to external API devs.  And
> `URLContext`
> appears in `ffmpeg -h full` despite having being made private long ago,
> so that's not much of a signal either.
>
> My guess is that the above should be addressed with a patch like:
>
> +/**
> + * @brief Context for SWResampler
> + *
> + * @note The public ABI only guarantees this is an AVOptions-enabled
> struct.
> + * Its size and other members are not a part of the public ABI.
> + *
> + * @see
> + * - @ref Context
> + */
>  struct SwrContext {
>
> Let me know if the above is on the right track.  If so, I'll queue up a
> patch
> for after the context document is done.
>
>
> *Are AVOptions just command-line options?*
>
> I have trouble with statements like "AVOptions is a framework for options",
> both because it's circular and because the term "option" is too broad to be
> meaningful.
>
> I've previously pushed the word "reflection" on the assumption that options
> can be used anywhere variables are used.  For example, imagine a decoder
> that
> could be reconfigured on-the-fly to reduce CPU usage at the cost of
> displaying
> blurier images.  That can't be part of the public interface because it's
> codec-specific, but I could imagine updating some kind of "output_quality"
> AVOption as a user slides a slider up and down.
>
> But the CLI tools are largely non-interactive, so have I just
> misunderstood?
>
> How about saying "AVOptions is a framework for command-line options"?
>

ffmpeg is cli tool

libavfilter is library

AVOptions is certainly and primarily not framework for command-line options.


>
> [1] https://ffmpeg.org/doxygen/trunk/structSwrContext.html
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>

On Wed, May 29, 2024 at 01:06:30PM +0200, Paul B Mahol wrote:
> On Wed, May 29, 2024 at 12:50 PM Andrew Sayers <ffmpeg-devel@pileofstuff.org>
> wrote:
> 
[...]
> > *Are AVOptions just command-line options?*
> >
> > I have trouble with statements like "AVOptions is a framework for options",
> > both because it's circular and because the term "option" is too broad to be
> > meaningful.
> >
> > I've previously pushed the word "reflection" on the assumption that options
> > can be used anywhere variables are used.  For example, imagine a decoder
> > that
> > could be reconfigured on-the-fly to reduce CPU usage at the cost of
> > displaying
> > blurier images.  That can't be part of the public interface because it's
> > codec-specific, but I could imagine updating some kind of "output_quality"
> > AVOption as a user slides a slider up and down.
> >
> > But the CLI tools are largely non-interactive, so have I just
> > misunderstood?
> >
> > How about saying "AVOptions is a framework for command-line options"?
> >
> 
> ffmpeg is cli tool
> 
> libavfilter is library
> 
> AVOptions is certainly and primarily not framework for command-line options.

"Command-line option" might be the wrong word, but I've just checked
write_number() in libavutil/opt.c, and it seems to do non-atomic updates
without locking anything.  That suggests I was indeed wrong to think it could
be used on-the-fly - maybe "initial configuration options" would be a better
term?  Possibly with a warning somewhere about how AVOptions is not reentrant?

On date Wednesday 2024-05-29 11:50:40 +0100, Andrew Sayers wrote:
> Posting this separately, as these are practical "how does FFmpeg work" issues
> vaguely inspired by recent discussions.
> 
> 
> *How do namespaces work in FFmpeg?*
> 
> We've talked a bit about function namespaces recently.  One reason I've
> suggested they're a weak signal is because they aren't really addressed in the
> documentation.  How about adding something like this to the context doc:
> 
>     Most FFmpeg functions are grouped into namespaces, usually indicated by
>     prefixes (e.g. `av_format_*`) but sometimes indicated by infixes
>     (e.g. av_alloc_format_context()).  Namespaces group functions by *topic*,
>     which doesn't always correlate with *context*.  For example, `av_find_*`
>     functions search for information across various contexts.
> 
> 
> *Should external API devs treat Sw{r,s}Context as AVClass context structures?*
> 
> This is probably an uninteresting edge case, but just to be sure...
> 

> The website says Sw{r,s}Context start with AVClass[1], they have _get_class
> functions, are shown in `ffmpeg -h full`, and I haven't found anything that says
> to treat them differently to e.g. AVCodecContext.  So I'm pretty sure these
> are AVClass context structures, at least as far as internal devs are concerned.

It impacts the user as this changes the API. I guess when AVClass was
removed from AVFrame that was done with a major bump to advertise an
API break.

> But their definitions are only in the private interface, so the layout is just
> an implementation detail that can change without even a major version bump.
> AVFrame used to have a _get_class function despite never having an actual
> AVClass member, so that's not a signal to external API devs.  And `URLContext`
> appears in `ffmpeg -h full` despite having being made private long ago,
> so that's not much of a signal either.
> 
> My guess is that the above should be addressed with a patch like:
> 
> +/**
> + * @brief Context for SWResampler
> + *

> + * @note The public ABI only guarantees this is an AVOptions-enabled struct.
> + * Its size and other members are not a part of the public ABI.

This looks redundant to me, this is true for all the opaque structus,
and we don't want to repeat this again and again (this is a feature of
the C language, not of the API itself).

> + *
> + * @see
> + * - @ref Context
> + */
>  struct SwrContext {
> 
> Let me know if the above is on the right track.  If so, I'll queue up a patch
> for after the context document is done.

Better to treat this as a separate patch anyway.
 
> *Are AVOptions just command-line options?*

No, in fact this is not about comman-line options at all.

> 
> I have trouble with statements like "AVOptions is a framework for options",
> both because it's circular and because the term "option" is too broad to be
> meaningful.

AVOptions (or better the av_opt API) is a system to set
fields/properties/options on an enabled struct. This is a high level
API since it allows multiple options setting, validation, setting
from a dictionary etc., and enables to abstract the struct field name
(since the option names might be different from the field name).

> 
> I've previously pushed the word "reflection" on the assumption that options
> can be used anywhere variables are used.  For example, imagine a decoder that
> could be reconfigured on-the-fly to reduce CPU usage at the cost of displaying
> blurier images.  That can't be part of the public interface because it's
> codec-specific, but I could imagine updating some kind of "output_quality"
> AVOption as a user slides a slider up and down.
> 
> But the CLI tools are largely non-interactive, so have I just misunderstood?
> 
> How about saying "AVOptions is a framework for command-line options"?

It is not about CLI options, although you can use introspection to get
the supported options and build an interface to set them.