From patchwork Wed Jun 29 19:52:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 36532 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8b27:b0:88:1bbf:7fd2 with SMTP id l39csp522256pzh; Wed, 29 Jun 2022 12:53:17 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tsZ3T5BE9LYK69S5kNR+GiVd5v5Hs/uGFqmOtU3OKo33SIxnOwXK1xiwirpGzJtWlO8BiT X-Received: by 2002:a17:907:2d86:b0:711:da8b:b579 with SMTP id gt6-20020a1709072d8600b00711da8bb579mr5140648ejc.67.1656532397056; Wed, 29 Jun 2022 12:53:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1656532397; cv=none; d=google.com; s=arc-20160816; b=R3zeaZtMNWVQLsfF1k3YwX+RheqMI3h+Z7HYxPGvDIYhFa9LXP7kJ+N92kEjh9PaoV RnAfl6HjUb2PkHbcYOTQTE0YXm3zTZGbFUasKABBAqEbGtCCFHCwNuIYkDCW9mk51KMe +LimjdbC7inhWo95cQ3UPVQIstvAbdz/zulgt9PUeseP6Om77dJnj2bOThIxaAygZ41P wMYgRpKCbu5UKZi3Qs44UaCfZxmsnWMmwVfs9AgzQq2SgQ9yhp3BDhIzgtjJGELavjhr PSA10m6Al4nLd64iTu9jJNnEaKdCGQUjjkvKi+IXIiUP48uUtoweAYNmbevmQpP9Iw13 RjyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=US3IyGMCzc4H0kY96uEIe4cmKSAgIyDY3KxMpqNqJXk=; b=VbenCC4ys+4S2BWsb/ZVa89tVUFI9ANDIPU1sFNuuCNyxMV3JG1esDbGqQV7/aUEWh xOnA4Zpja58B6MjwFWPpoKJRaNcL7q4ADaBCB7BWyjpSSyonBpQyCWCQ3Khqezic7CZI ZMMLg0vvnAmKOvHXl+SjD7BaqmTuO7QdZ3XMQJ2oUhnaX1skYZipQPbiJPfeajt1yMxa vXl0ARWk9CHpa2H1yGTnUUEzjOy2t9bA2u7F+X4kTfTKjwhTCZcw+zqjxSwRCo4YpeFd 5wJDJHkiGkDW1UpgB9l9zQ/nXVfmX6+hRAvRivN+J2gR4a+Z5GvviUjeomBvsKFPtJ6l izMA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=sXcgPcbF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ec2-20020a170906b6c200b00711c9e20c40si20079209ejb.243.2022.06.29.12.53.16; Wed, 29 Jun 2022 12:53:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=sXcgPcbF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3FB9E68037B; Wed, 29 Jun 2022 22:53:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [136.243.74.85]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3341768B411 for ; Wed, 29 Jun 2022 22:53:05 +0300 (EEST) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id EF3B025E7D6; Wed, 29 Jun 2022 21:53:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1656532383; bh=0mAlRuW104qLvFLrmWt/cS9aNI6buJSiOouwoJWscTo=; h=From:To:Cc:Subject:Date; b=sXcgPcbFwBt/eRZGKfwl1bM5R+wnPOummdLJnU58lceMmmgBH+81+ohMUDe8p3Tt9 HzKngLEQgInrKmeJMFhRmKZiMXxtxibejNiGJhE2uakqGTutJUaC40OWhEJtrx5Eeh 4ruS2dZWbzdNYpvnudILyO2f7O+2roEdfky/fl9CBnvJ2Hvi0NMnmQvNIiE3+Is/6k ChA9DRrSENtCD5ZHTD1BEZIalsj1OsD+hUVIEwCY3h1y/OW1ayw4UBueCgDQAQQsID LHK5mTvvvE/2NQm8y0tBxbUQktPmNlxsQb8qhcWYbaXl3WnsnQmI/Ws2tG6Q0YX8wW s2aO2b7Z0h3Dg== From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Wed, 29 Jun 2022 21:52:50 +0200 Message-Id: <20220629195250.29341-1-timo@rothenpieler.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [RFC] avdevice/lavfi: output wrapped AVFrames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: o1Xjb3ijO8UN The lavfi avdevice as it is right now can't output "special frames" like hardware frames. This patch makes the lavfi avdevice output wrapped AVFrames instead of copying the actual data, greatly simplifying it in the process. --- I am not at all sure if this has some unexpected consequences. It works just fine with ffmpeg.c, but it might be an ABI break for potential library consumers? libavdevice/lavfi.c | 108 +++++++++++++------------------------------- 1 file changed, 31 insertions(+), 77 deletions(-) diff --git a/libavdevice/lavfi.c b/libavdevice/lavfi.c index db5d0b94de..6bfdb5033e 100644 --- a/libavdevice/lavfi.c +++ b/libavdevice/lavfi.c @@ -54,32 +54,10 @@ typedef struct { int *sink_eof; int *stream_sink_map; int *sink_stream_subcc_map; - AVFrame *decoded_frame; int nb_sinks; AVPacket subcc_packet; } LavfiContext; -static int *create_all_formats(int n) -{ - int i, j, *fmts, count = 0; - - for (i = 0; i < n; i++) { - const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(i); - if (!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) - count++; - } - - if (!(fmts = av_malloc_array(count + 1, sizeof(*fmts)))) - return NULL; - for (j = 0, i = 0; i < n; i++) { - const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(i); - if (!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) - fmts[j++] = i; - } - fmts[j] = AV_PIX_FMT_NONE; - return fmts; -} - av_cold static int lavfi_read_close(AVFormatContext *avctx) { LavfiContext *lavfi = avctx->priv_data; @@ -90,7 +68,6 @@ av_cold static int lavfi_read_close(AVFormatContext *avctx) av_freep(&lavfi->sink_stream_subcc_map); av_freep(&lavfi->sinks); avfilter_graph_free(&lavfi->graph); - av_frame_free(&lavfi->decoded_frame); return 0; } @@ -125,15 +102,11 @@ av_cold static int lavfi_read_header(AVFormatContext *avctx) LavfiContext *lavfi = avctx->priv_data; AVFilterInOut *input_links = NULL, *output_links = NULL, *inout; const AVFilter *buffersink, *abuffersink; - int *pix_fmts = create_all_formats(AV_PIX_FMT_NB); enum AVMediaType type; int ret = 0, i, n; #define FAIL(ERR) { ret = ERR; goto end; } - if (!pix_fmts) - FAIL(AVERROR(ENOMEM)); - buffersink = avfilter_get_by_name("buffersink"); abuffersink = avfilter_get_by_name("abuffersink"); @@ -264,22 +237,12 @@ av_cold static int lavfi_read_header(AVFormatContext *avctx) ret = avfilter_graph_create_filter(&sink, buffersink, inout->name, NULL, NULL, lavfi->graph); - if (ret >= 0) - ret = av_opt_set_int_list(sink, "pix_fmts", pix_fmts, AV_PIX_FMT_NONE, AV_OPT_SEARCH_CHILDREN); if (ret < 0) goto end; } else if (type == AVMEDIA_TYPE_AUDIO) { - static const enum AVSampleFormat sample_fmts[] = { - AV_SAMPLE_FMT_U8, AV_SAMPLE_FMT_S16, AV_SAMPLE_FMT_S32, - AV_SAMPLE_FMT_FLT, AV_SAMPLE_FMT_DBL, - }; - ret = avfilter_graph_create_filter(&sink, abuffersink, inout->name, NULL, NULL, lavfi->graph); - if (ret >= 0) - ret = av_opt_set_bin(sink, "sample_fmts", (const uint8_t*)sample_fmts, - sizeof(sample_fmts), AV_OPT_SEARCH_CHILDREN); if (ret < 0) goto end; ret = av_opt_set_int(sink, "all_channel_counts", 1, @@ -320,39 +283,25 @@ av_cold static int lavfi_read_header(AVFormatContext *avctx) AVCodecParameters *const par = st->codecpar; avpriv_set_pts_info(st, 64, time_base.num, time_base.den); par->codec_type = av_buffersink_get_type(sink); + par->codec_id = AV_CODEC_ID_WRAPPED_AVFRAME; + par->format = av_buffersink_get_format(sink); if (par->codec_type == AVMEDIA_TYPE_VIDEO) { - int64_t probesize; - par->codec_id = AV_CODEC_ID_RAWVIDEO; - par->format = av_buffersink_get_format(sink); par->width = av_buffersink_get_w(sink); par->height = av_buffersink_get_h(sink); - probesize = par->width * par->height * 30 * - av_get_padded_bits_per_pixel(av_pix_fmt_desc_get(par->format)); - avctx->probesize = FFMAX(avctx->probesize, probesize); - st ->sample_aspect_ratio = + st ->sample_aspect_ratio = par->sample_aspect_ratio = av_buffersink_get_sample_aspect_ratio(sink); } else if (par->codec_type == AVMEDIA_TYPE_AUDIO) { par->sample_rate = av_buffersink_get_sample_rate(sink); ret = av_buffersink_get_ch_layout(sink, &par->ch_layout); if (ret < 0) goto end; - par->format = av_buffersink_get_format(sink); - par->codec_id = av_get_pcm_codec(par->format, -1); - if (par->codec_id == AV_CODEC_ID_NONE) - av_log(avctx, AV_LOG_ERROR, - "Could not find PCM codec for sample format %s.\n", - av_get_sample_fmt_name(par->format)); } } if ((ret = create_subcc_streams(avctx)) < 0) goto end; - if (!(lavfi->decoded_frame = av_frame_alloc())) - FAIL(AVERROR(ENOMEM)); - end: - av_free(pix_fmts); avfilter_inout_free(&input_links); avfilter_inout_free(&output_links); return ret; @@ -378,22 +327,31 @@ static int create_subcc_packet(AVFormatContext *avctx, AVFrame *frame, return 0; } +static void lavfi_free_frame(void *opaque, uint8_t *data) +{ + AVFrame *frame = (AVFrame*)data; + av_frame_free(&frame); +} + static int lavfi_read_packet(AVFormatContext *avctx, AVPacket *pkt) { LavfiContext *lavfi = avctx->priv_data; double min_pts = DBL_MAX; int stream_idx, min_pts_sink_idx = 0; - AVFrame *frame = lavfi->decoded_frame; + AVFrame *frame; AVDictionary *frame_metadata; int ret, i; int size = 0; - AVStream *st; if (lavfi->subcc_packet.size) { av_packet_move_ref(pkt, &lavfi->subcc_packet); return pkt->size; } + frame = av_frame_alloc(); + if (!frame) + return AVERROR(ENOMEM); + /* iterate through all the graph sinks. Select the sink with the * minimum PTS */ for (i = 0; i < lavfi->nb_sinks; i++) { @@ -411,7 +369,7 @@ static int lavfi_read_packet(AVFormatContext *avctx, AVPacket *pkt) lavfi->sink_eof[i] = 1; continue; } else if (ret < 0) - return ret; + goto fail; d = av_rescale_q_rnd(frame->pts, tb, AV_TIME_BASE_Q, AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX); ff_dlog(avctx, "sink_idx:%d time:%f\n", i, d); av_frame_unref(frame); @@ -421,29 +379,15 @@ static int lavfi_read_packet(AVFormatContext *avctx, AVPacket *pkt) min_pts_sink_idx = i; } } - if (min_pts == DBL_MAX) - return AVERROR_EOF; + if (min_pts == DBL_MAX) { + ret = AVERROR_EOF; + goto fail; + } ff_dlog(avctx, "min_pts_sink_idx:%i\n", min_pts_sink_idx); av_buffersink_get_frame_flags(lavfi->sinks[min_pts_sink_idx], frame, 0); stream_idx = lavfi->sink_stream_map[min_pts_sink_idx]; - st = avctx->streams[stream_idx]; - - if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) { - size = av_image_get_buffer_size(frame->format, frame->width, frame->height, 1); - if ((ret = av_new_packet(pkt, size)) < 0) - goto fail; - - av_image_copy_to_buffer(pkt->data, size, (const uint8_t **)frame->data, frame->linesize, - frame->format, frame->width, frame->height, 1); - } else if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { - size = frame->nb_samples * av_get_bytes_per_sample(frame->format) * - frame->ch_layout.nb_channels; - if ((ret = av_new_packet(pkt, size)) < 0) - goto fail; - memcpy(pkt->data, frame->data[0], size); - } frame_metadata = frame->metadata; if (frame_metadata) { @@ -465,13 +409,23 @@ static int lavfi_read_packet(AVFormatContext *avctx, AVPacket *pkt) goto fail; } + pkt->buf = av_buffer_create((uint8_t*)frame, sizeof(*frame), + &lavfi_free_frame, NULL, 0); + if (!pkt->buf) { + ret = AVERROR(ENOMEM); + goto fail; + } + + pkt->data = pkt->buf->data; + pkt->size = pkt->buf->size; + pkt->flags |= AV_PKT_FLAG_TRUSTED; + pkt->stream_index = stream_idx; pkt->pts = frame->pts; pkt->pos = frame->pkt_pos; - av_frame_unref(frame); return size; fail: - av_frame_unref(frame); + av_frame_free(&frame); return ret; }