From patchwork Tue Jun 4 14:47:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Sayers X-Patchwork-Id: 49550 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9183:0:b0:460:55fa:d5ed with SMTP id s3csp2743492vqg; Tue, 4 Jun 2024 07:49:51 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUi32wh4VQDSnk132xpvyCkOh/N71qABLI/nosu6A69N0TcIg+HZW7GEJa/coiBCHzwnm4SGnuLtmiAZA+rYuCzlineMWN8lJkBzQ== X-Google-Smtp-Source: AGHT+IEfgN5LU3IFzRV+TMGVF7iZZ6iUAZ8qboZ9MQbNrM/Qhhf644oCnaiU+MrNLiIYHlEM4qr2 X-Received: by 2002:a17:906:703:b0:a55:8f2a:950d with SMTP id a640c23a62f3a-a681fc5c73amr803480366b.16.1717512590812; Tue, 04 Jun 2024 07:49:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1717512590; cv=none; d=google.com; s=arc-20160816; b=XPduenzFEN6x6V84tkaTfeY9cF4lfUbml0oUGU3kwdCWSW+9iKpUM9UN2j/39zfvlO j4onF+uXCFIU7nfx4zbWYQZU8GhB2ofwhdtn1ZJP90qEBRAsELudeaotQdVu3AAvUDSk yBVTi7h2KGYQvzooRZI7JmXfAhpbbxE7ilnU1JohNAy9H4PrQDSyiJ0nkW1J4HHqSv9r xmqGe+KJE8MjBBmtmqSs2xVaaL9dnOoXnz0cjJSUN03BjaJ9+oWfeHiEI/4M6IO9bm+L WofySAe/R34Qval8cFXfuYVUme0dGzpB8lVqpg8vd4L2VMomvs1nCJm03Og0Lldr2cGZ Mc0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=lJw9bqT9coUCEc0KnMAVwZZAu/N0kzDHVqvUjIjR5mU=; fh=73ExZnkQ8FYbu/qeQNmI0dtHCfShNh8/NmZJs1umltM=; b=ZnFudDVHHJKde7n/X05D/OKbFkSRvskoWN4UAuS9r9ZWAoQ8kT50kE4QDC5+6oQNkk Ul5Udj6/5fa3snWkoAaIX3R7+XH1AoCCXadvr98S3o3lCAqn77Jn5m2nWgknsN/6i6Gx Lt5pKDfwVvbA2CSHbYKuLvotAGl3+U1K1Y7iMn9kwl27d+et2oPN8YrdLEvxDpv3yAjR PkFFk4YGU1WeHhFzh+DLgsHwtAvMGa0Kdo7LzqWEh1Wk3dXngVsVyUt/9BXqbuKpr5Ux CT92rK70NTV229bD0kdiehIH2Ojdp/cxUP/haINx+tEtxrTMM9hYrkrsz9QWdof0O+PK /gIg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a68f828255esi249948266b.910.2024.06.04.07.49.50; Tue, 04 Jun 2024 07:49:50 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9E9DF68D70F; Tue, 4 Jun 2024 17:49:38 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from alt2.a-painless.mh.aa.net.uk (alt2.a-painless.mh.aa.net.uk [81.187.30.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 91C1C68D6BA for ; Tue, 4 Jun 2024 17:49:31 +0300 (EEST) Received: from 0.b.4.b.7.4.0.8.c.4.a.5.d.8.b.2.0.5.8.0.9.1.8.0.0.b.8.0.1.0.0.2.ip6.arpa ([2001:8b0:819:850:2b8d:5a4c:8047:b4b0] helo=andrews-2024-laptop.lan) by painless-a.thn.aa.net.uk with esmtp (Exim 4.96) (envelope-from ) id 1sEVTe-007B97-1v; Tue, 04 Jun 2024 15:49:31 +0100 From: Andrew Sayers To: ffmpeg-devel@ffmpeg.org Date: Tue, 4 Jun 2024 15:47:21 +0100 Message-ID: <20240604144919.213799-2-ffmpeg-devel@pileofstuff.org> X-Mailer: git-send-email 2.45.1 In-Reply-To: <20240604144919.213799-1-ffmpeg-devel@pileofstuff.org> References: <20240418150614.3952107-1-ffmpeg-devel@pileofstuff.org> <20240604144919.213799-1-ffmpeg-devel@pileofstuff.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v6 1/4] doc: Explain what "context" means X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andrew Sayers Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: gipqVRlD6QH0 Derived from explanations kindly provided by Stefano Sabatini and others: https://ffmpeg.org/pipermail/ffmpeg-devel/2024-April/325903.html --- doc/context.md | 430 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 430 insertions(+) create mode 100644 doc/context.md diff --git a/doc/context.md b/doc/context.md new file mode 100644 index 0000000000..bd8cb58696 --- /dev/null +++ b/doc/context.md @@ -0,0 +1,430 @@ +@page Context Introduction to contexts + +@tableofcontents + +FFmpeg uses the term “context” to refer to an idiom +you have probably used before: + +```c +// C structs often share context between functions: + +FILE *my_file; // my_file stores information about a filehandle + +printf(my_file, "hello "); // my_file provides context to this function, +printf(my_file, "world!"); // and also to this function +``` + +```python +# Python classes provide context for the methods they contain: + +class MyClass: + def print(self,message): + if self.prev_message != message: + self.prev_message = message + print(message) +``` + + +```c +// Many JavaScript callbacks accept an optional context argument: + +const my_object = {}; + +my_array.forEach(function_1, my_object); +my_array.forEach(function_2, my_object); +``` + +Be careful comparing FFmpeg contexts to things you're already familiar with - +FFmpeg may sometimes happen to reuse words you recognise, but mean something +completely different. For example, the AVClass struct has nothing to do with +[object-oriented classes](https://en.wikipedia.org/wiki/Class_(computer_programming)). + +If you've used contexts in other C projects, you may want to read +@ref Context_comparison before the rest of the document. + +@section Context_general “Context” as a general concept + +@par +A context is any data structure used by several functions +(or several instances of the same function) that all operate on the same entity. + +In the broadest sense, “context” is just a way to think about code. +You can even use it to think about code written by people who have never +heard the term, or who would disagree with you about what it means. +Consider the following snippet: + +```c +struct DualWriter { + int fd1, fd2; +}; + +ssize_t write_to_two_files( + struct DualWriter *my_writer, + uint8_t *buf, + int buf_size +) { + + ssize_t bytes_written_1 = write(my_writer->fd1, buf, buf_size); + ssize_t bytes_written_2 = write(my_writer->fd2, buf, buf_size); + + if ( bytes_written_1 != bytes_written_2 ) { + // ... handle this edge case ... + } + + return bytes_written_1; + +} + +int main() { + + struct DualWriter my_writer; + my_writer.fd1 = open("file1", 0644, "wb"); + my_writer.fd2 = open("file2", 0644, "wb"); + + write_to_two_files(&my_writer, "hello ", sizeof("hello ")); + write_to_two_files(&my_writer, "world!", sizeof("world!")); + + close( my_writer.fd1 ); + close( my_writer.fd2 ); + +} +``` + +The term “context” doesn't appear anywhere in the snippet. But `DualWriter` +is passed to several instances of `write_to_two_files()` that operate on +the same entity, so it fits the definition of a context. + +When reading code that isn't explicitly described in terms of contexts, +remember that your interpretation may differ from other people's. +For example, FFmpeg's avio_alloc_context() accepts a set of callback functions +and an `opaque` argument - even though this function guarantees to *return* +a context, it does not require `opaque` to *provide* context for the callback +functions. So you could choose to pass a struct like `DualWriter` as the +`opaque` argument, or you could pass callbacks that use `stdin` and `stdout` +and just pass a `NULL` argument for `opaque`. + +When reading code that *is* explicitly described in terms of contexts, +remember that the term's meaning is guaranteed by *the project's community*, +not *the language it's written in*. That means guarantees may be more flexible +and change more over time. For example, programming languages that use +[encapsulation](https://en.wikipedia.org/wiki/Encapsulation_(computer_programming)) +will simply refuse to compile code that violates its rules about access, +while communities can put up with special cases if they improve code quality. + +The next section will discuss what specific conventions FFmpeg developers mean +when they describe parts of their code as using “contexts”. + +@section Context_ffmpeg FFmpeg contexts + +This section discusses specific context-related conventions used in FFmpeg. +Some of these are used in other projects, others are unique to this project. + +@subsection Context_indicating Indicating context: “Context”, “ctx” etc. + +```c +// Context struct names usually end with `Context`: +struct AVSomeContext { + ... +}; + +// Functions are usually named after their context, +// context parameters usually come first and are often called `ctx`: +void av_some_function(AVSomeContext *ctx, ...); +``` + +FFmpeg struct names usually signal whether they are contexts (e.g. AVBSFContext +or AVCodecContext). Exceptions to this rule include AVMD5, which is only +identified as a context by @ref libavutil/md5.c "the functions that call it". + +Function names usually signal the context they're associated with (e.g. +av_md5_alloc() or avcodec_alloc_context3()). Exceptions to this rule include +@ref avformat.h "AVFormatContext's functions", many of which begin with +just `av_`. + +Functions usually signal their context parameter by putting it first and +naming it some variant of `ctx`. Exceptions include av_bsf_alloc(), which puts +its context argument second to emphasise it's an out variable. + +Some functions fit awkwardly within FFmpeg's context idiom, so they send mixed +signals. For example, av_ambient_viewing_environment_create_side_data() creates +an AVAmbientViewingEnvironment context, then adds it to the side-data of an +AVFrame context. So its name hints at one context, its parameter hints at +another, and its documentation is silent on the issue. You might prefer to +think of such functions as not having a context, or as “receiving” one context +and “producing” another. + +@subsection Context_data_hiding Data hiding: private contexts + +```c +// Context structs often hide private context: +struct AVSomeContext { + void *priv_data; // sometimes just called "internal" +}; +``` + +Contexts present a public interface, so changing a context's members forces +everyone that uses the library to at least recompile their program, +if not rewrite it to remain compatible. Many contexts reduce this problem +by including a private context with a type that is not exposed in the public +interface. Hiding information this way ensures it can be modified without +affecting downstream software. + +Private contexts often store variables users aren't supposed to see +(similar to an [OOP](https://en.wikipedia.org/wiki/Object-oriented_programming) +private block), but can be used for more than just access control. They can +also store information shared between some but not all instances of a context +(e.g. codec-specific functionality), and @ref Context_avoptions +"AVOptions-enabled structs" can provide user configuration options through +the @ref avoptions "AVOptions API". + +@subsection Context_lifetime Manage lifetime: creation, use, destruction + +```c +void my_function(...) { + + // Context structs are allocated then initialized with associated functions: + + AVSomeContext *ctx = av_some_context_alloc(...); + + // ... configure ctx ... + + av_some_context_init(ctx, ...); + + // ... use ctx ... + + // Context structs are closed then freed with associated functions: + + av_some_context_close(ctx); + av_some_context_free(ctx); + +} +``` + +FFmpeg contexts go through the following stages of life: + +1. allocation (often a function that ends with `_alloc`) + * a range of memory is allocated for use by the structure + * memory is allocated on boundaries that improve caching + * memory is reset to zeroes, some internal structures may be initialized +2. configuration (implemented by setting values directly on the context) + * no function for this - calling code populates the structure directly + * memory is populated with useful values + * simple contexts can skip this stage +3. initialization (often a function that ends with `_init`) + * setup actions are performed based on the configuration (e.g. opening files) +5. normal usage + * most functions are called in this stage + * documentation implies some members are now read-only (or not used at all) + * some contexts allow re-initialization +6. closing (often a function that ends with `_close()`) + * teardown actions are performed (e.g. closing files) +7. deallocation (often a function that ends with `_free()`) + * memory is returned to the pool of available memory + +This can mislead object-oriented programmers, who expect something more like: + +1. allocation (usually a `new` keyword) + * a range of memory is allocated for use by the structure + * memory *may* be reset (e.g. for security reasons) +2. initialization (usually a constructor) + * memory is populated with useful values + * related setup actions are performed based on arguments (e.g. opening files) +3. normal usage + * most functions are called in this stage + * compiler enforces that some members are read-only (or private) + * no going back to the previous stage +4. finalization (usually a destructor) + * teardown actions are performed (e.g. closing files) +5. deallocation (usually a `delete` keyword) + * memory is returned to the pool of available memory + +The remainder of this section discusses FFmpeg's differences from OOP, to help +object-oriented programmers avoid misconceptions. You can safely skip this +section if you aren't familiar with the OOP lifetime described above. + +FFmpeg's allocation stage is broadly similar to the OOP stage of the same name. +Both set aside some memory for use by a new entity, but FFmpeg's stage can also +do some higher-level operations. For example, @ref Context_avoptions +"AVOptions-enabled structs" set their AVClass member during allocation. + +FFmpeg's configuration stage involves setting any variables you want before +you start using the context. Complicated FFmpeg structures like AVCodecContext +tend to have many members you *could* set, but in practice most programs set +few if any of them. The freeform configuration stage works better than bundling +these into the initialization stage, which would lead to functions with +impractically many parameters, and would mean each new option was an +incompatible change to the API. One way to understand the problem is to read +@ref Context_avoptions "the AVOptions section below" and think how a constructor +would handle those options. + +FFmpeg's initialization stage involves calling a function that sets the context +up based on your configuration. + +FFmpeg's first three stages do the same job as OOP's first two stages. +This can mislead object-oriented developers, who expect to do less work in the +allocation stage, and more work in the initialization stage. To simplify this, +most FFmpeg contexts provide a combined allocator and initializer function. +For historical reasons, suffixes like `_alloc`, `_init`, `_alloc_context` and +even `_open` can indicate the function does any combination of allocation and +initialization. + +FFmpeg's "closing" stage is broadly similar to OOP's "finalization" stage, +but some contexts allow re-initialization after finalization. For example, +SwrContext lets you call swr_close() then swr_init() to reuse a context. +Be aware that some FFmpeg functions happen to use the word "finalize" in a way +that has nothing to do with the OOP stage (e.g. av_bsf_list_finalize()). + +FFmpeg's "deallocation" stage is broadly similar to OOP, but can perform some +higher-level functions (similar to the allocation stage). + +Closing functions usually end with "_close", while deallocation +functions usually end with "_free". Very few contexts need the flexibility of +separate "closing" and "deallocation" stages, so many "_free" functions +implicitly close the context first. + +@subsection Context_avoptions Configuration options: AVOptions-enabled structs + +The @ref avoptions "AVOptions API" is a framework to configure user-facing +options, e.g. on the command-line or in GUI configuration forms. + +To understand FFmpeg's configuration requirements, run `ffmpeg -h full` on the +command-line, then ask yourself how you would implement all those options +with the C standard [`getopt` function](https://en.wikipedia.org/wiki/Getopt). +You can also ask the same question for other approaches - for example, how would +you maintain a GUI with 15,000+ configuration options? + +Most solutions assume you can just put all options in a single code block, +which is unworkable at FFmpeg's scale. Instead, we split configuration +across many *AVOptions-enabled structs*, which use the @ref avoptions +"AVOptions API" to inspect and configure options, including in private contexts. + +AVOptions-accessible members of a context should be accessed through the +@ref avoptions "AVOptions API" whenever possible, even if they're not hidden +in a private context. That ensures values are validated as they're set, and +means you won't have to do as much work if a future version of FFmpeg changes +the allowed values. + +Although not strictly required, it is best to only modify options during +the configuration stage. Initialized structs may be accessed by internal +FFmpeg threads, and modifying them can cause weird intermittent bugs. + +@subsection Context_logging Logging: AVClass context structures + +FFmpeg's @ref lavu_log "logging facility" needs to be simple to use, +but flexible enough to let people debug problems. And much like options, +it needs to work the same across a wide variety of unrelated structs. + +FFmpeg structs that support the logging framework are called *@ref AVClass +context structures*. The name @ref AVClass was chosen early in FFmpeg's +development, but in practice it only came to store information about +logging, and about options. + +@section Context_further Further information about contexts + +So far, this document has provided a theoretical guide to FFmpeg contexts. +This final section provides some alternative approaches to the topic, +which may help round out your understanding. + +@subsection Context_example Learning by example: context for a codec + +It can help to learn contexts by doing a deep dive into a specific struct. +This section will discuss AVCodecContext - an AVOptions-enabled struct +that contains information about encoding or decoding one stream of data +(e.g. the video in a movie). + +The name "AVCodecContext" tells us this is a context. Many of +@ref libavcodec/avcodec.h "its functions" start with an `avctx` parameter, +indicating this parameter provides context for that function. + +AVCodecContext::internal contains the private context. For example, +codec-specific information might be stored here. + +AVCodecContext is allocated with avcodec_alloc_context3(), initialized with +avcodec_open2(), and freed with avcodec_free_context(). Most of its members +are configured with the @ref avoptions "AVOptions API", but for example you +can set AVCodecContext::draw_horiz_band() if your program happens to need it. + +AVCodecContext provides an abstract interface to many different *codecs*. +Options supported by many codecs (e.g. "bitrate") are kept in AVCodecContext +and exposed with AVOptions. Options that are specific to one codec are +stored in the private context, and also exposed with AVOptions. + +AVCodecContext::av_class contains logging metadata to ensure all codec-related +error messages look the same, plus implementation details about options. + +To support a specific codec, AVCodecContext's private context is set to +an encoder-specific data type. For example, the video codec +[H.264](https://en.wikipedia.org/wiki/Advanced_Video_Coding) is supported via +[the x264 library](https://www.videolan.org/developers/x264.html), and +implemented in X264Context. Although included in the documentation, X264Context +is not part of the public API. That means FFmpeg's @ref ffmpeg_versioning +"strict rules about changing public structs" aren't as important here, so a +version of FFmpeg could modify X264Context or replace it with another type +altogether. An adverse legal ruling or security problem could even force us to +switch to a completely different library without a major version bump. + +The design of AVCodecContext provides several important guarantees: + +- lets you use the same interface for any codec +- supports common encoder options like "bitrate" without duplicating code +- supports encoder-specific options like "profile" without bulking out the public interface +- exposes both types of options to users, with help text and detection of missing options +- provides uniform logging output +- hides implementation details (e.g. its encoding buffer) + +@subsection Context_comparison Learning by comparison: FFmpeg vs. Curl contexts + +It can help to learn contexts by comparing how different projects tackle +similar problems. This section will compare @ref AVMD5 "FFmpeg's MD5 context" +with [curl 8.8.0's equivalent](https://github.com/curl/curl/blob/curl-8_8_0/lib/md5.c#L48). + +The [MD5 algorithm](https://en.wikipedia.org/wiki/MD5) produces +a fixed-length digest from arbitrary-length data. It does this by calculating +the digest for a prefix of the data, then loading the next part and adding it +to the previous digest, and so on. + +```c +// FFmpeg's MD5 context looks like this: +typedef struct AVMD5 { + uint64_t len; + uint8_t block[64]; + uint32_t ABCD[4]; +} AVMD5; + +// Curl 8.8.0's MD5 context looks like this: +struct MD5_context { + const struct MD5_params *md5_hash; /* Hash function definition */ + void *md5_hashctx; /* Hash function context */ +}; +``` + +Curl's struct name ends with `_context`, guaranteeing contexts are the correct +interpretation. FFmpeg's struct does not explicitly say it's a context, but +@ref libavutil/md5.c "its functions do" so we can reasonably assume +it's the intended interpretation. + +Curl's struct uses `void *md5_hashctx` to avoid guaranteeing +implementation details in the public interface, whereas FFmpeg makes +everything accessible. This disagreement about data hiding is a good example +of how contexts can be used differently. Hiding the data means changing the +layout in a future version of curl won't break downstream programs that used +that data. But the MD5 algorithm has been stable for 30 years, and making the +data public makes it easier for people to follow a bug in their own code. + +Curl's struct is declared as `struct { ... }`, whereas FFmpeg uses +`typedef struct { ... } `. These conventions are used with both +context and non-context structs, so don't say anything about contexts as such. +Specifically, FFmpeg's convention is a workaround for an issue with C grammar: + +```c +void my_function(...) { + int my_var; // good + MD5_context my_curl_ctx; // error: C needs you to explicitly say "struct" + struct MD5_context my_curl_ctx; // good: added "struct" + AVMD5 my_ffmpeg_ctx; // good: typedef's avoid the need for "struct" +} +``` + +Both MD5 implementations are long-tested, widely-used examples of contexts +in the real world. They show how contexts can solve the same problem +in different ways.