From patchwork Fri Oct 21 08:30:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 38863 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp640689pzb; Fri, 21 Oct 2022 01:30:58 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5S6vzk6wohg2XQs2YShXPtows0h293tw9NxQhcmm3o2+KSXxw/hREQKpW3dZhwnKznK9/n X-Received: by 2002:a17:907:a06d:b0:78d:ceae:ba5a with SMTP id ia13-20020a170907a06d00b0078dceaeba5amr14707218ejc.586.1666341057677; Fri, 21 Oct 2022 01:30:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666341057; cv=none; d=google.com; s=arc-20160816; b=a9fmujVhOH6a1mO9yX4HR5OJig6Q9QdGUiXVsRKnDeu887ZDagVCffFxc/oYcFFb5g 8vkTnZCM9Zjxt8+cPJfLIsrZbDX2f6u0TqpLdH9sw80U4tSrV8iuvzKqcFC6zsNEIvE7 3MSSwjwrQMInSUaAjgu/X2P0+BNTZXfP6yMWWTY7VaXxXQ3ZxGXun1GYadofTROLizs5 XwOhXCPLxwP6I1KT6zDp9SRgcoPpW0wlfBld+dWqqXGDbYxl6DCUSIt1odWP/GZc6vVg DCxxzW8X3gMlw3f5emskiwLRY9KXFtnaLNnKUDaQyqx+MSSaUdeVCEf62ToUcVOLMbFf ML5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=zONCxa0NCGW5jeRYoXFgqTSHIByBi7iqdv19kebMPaE=; b=XaVOLg3OSRhHCRems08jPBezrjW8dze4+NmlyrefwUwoCeKcC/yM1wN6HX3FGm7VDz 9DxT90NKvAbmGt8t/F7YVBZMwu7P2cIZ3c3rmIDiPokPGXZKp3Qer58vpbjJsYGq+kFV SQKs3UbtyIMnpt9mLx1HIwH3mrel2sx8eonFK77I/FpQyTXkGOo6beSpSyT0QeeS2cSq f7EMjx0jQYTpEdPTUhjV0ksljiU3NiuxVEICuyVVguCMMgvXmI65Bb/sF9ZIvMf3+s/k 6PhGX+H+JXPDOIGGjZPeAe6LLkgUcypwEzW6EGo+KgzHX+aSuqwFn8cYovncL1iHR3iK 6n3w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h9-20020a170906854900b0078d10478051si15779500ejy.265.2022.10.21.01.30.56; Fri, 21 Oct 2022 01:30:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 29C7868BE80; Fri, 21 Oct 2022 11:30:53 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9113A68BDEB for ; Fri, 21 Oct 2022 11:30:46 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 142C02404F5 for ; Fri, 21 Oct 2022 10:30:46 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id 0zg2QVDmgK7K for ; Fri, 21 Oct 2022 10:30:45 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id F2B072400F4 for ; Fri, 21 Oct 2022 10:30:44 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 162B93A056E for ; Fri, 21 Oct 2022 10:30:39 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Oct 2022 10:30:36 +0200 Message-Id: <20221021083036.17264-1-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 In-Reply-To: <9eb483c2-4802-f013-0fcf-b2b073c9afe3@gmail.com> References: <9eb483c2-4802-f013-0fcf-b2b073c9afe3@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] tools: add an AV_CODEC_CAP_ENCODER_RECON_FRAME test tool X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: R2He/i1sIga9 --- Makefile | 2 + tools/Makefile | 3 +- tools/enc_recon_frame_test.c | 391 +++++++++++++++++++++++++++++++++++ 3 files changed, 395 insertions(+), 1 deletion(-) create mode 100644 tools/enc_recon_frame_test.c diff --git a/Makefile b/Makefile index 1fb742f390..bf1b69f96b 100644 --- a/Makefile +++ b/Makefile @@ -67,6 +67,8 @@ tools/target_io_dem_fuzzer$(EXESUF): tools/target_io_dem_fuzzer.o $(FF_DEP_LIBS) tools/enum_options$(EXESUF): ELIBS = $(FF_EXTRALIBS) tools/enum_options$(EXESUF): $(FF_DEP_LIBS) +tools/enc_recon_frame_test$(EXESUF): $(FF_DEP_LIBS) +tools/enc_recon_frame_test$(EXESUF): ELIBS = $(FF_EXTRALIBS) tools/scale_slice_test$(EXESUF): $(FF_DEP_LIBS) tools/scale_slice_test$(EXESUF): ELIBS = $(FF_EXTRALIBS) tools/sofa2wavs$(EXESUF): ELIBS = $(FF_EXTRALIBS) diff --git a/tools/Makefile b/tools/Makefile index 4afa23342d..dee6a41668 100644 --- a/tools/Makefile +++ b/tools/Makefile @@ -1,4 +1,4 @@ -TOOLS = enum_options qt-faststart scale_slice_test trasher uncoded_frame +TOOLS = enc_recon_frame_test enum_options qt-faststart scale_slice_test trasher uncoded_frame TOOLS-$(CONFIG_LIBMYSOFA) += sofa2wavs TOOLS-$(CONFIG_ZLIB) += cws2fws @@ -17,6 +17,7 @@ tools/target_dem_fuzzer.o: tools/target_dem_fuzzer.c tools/target_io_dem_fuzzer.o: tools/target_dem_fuzzer.c $(COMPILE_C) -DIO_FLAT=0 +tools/enc_recon_frame_test$(EXESUF): tools/decode_simple.o tools/venc_data_dump$(EXESUF): tools/decode_simple.o tools/scale_slice_test$(EXESUF): tools/decode_simple.o diff --git a/tools/enc_recon_frame_test.c b/tools/enc_recon_frame_test.c new file mode 100644 index 0000000000..8450bad4d1 --- /dev/null +++ b/tools/enc_recon_frame_test.c @@ -0,0 +1,391 @@ +/* + * copyright (c) 2022 Anton Khirnov + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* A test for AV_CODEC_FLAG_RECON_FRAME + * TODO: dump reconstructed frames to disk */ + +#include +#include +#include + +#include "decode_simple.h" + +#include "libavutil/adler32.h" +#include "libavutil/common.h" +#include "libavutil/error.h" +#include "libavutil/frame.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" + +#include "libavformat/avformat.h" + +#include "libavcodec/avcodec.h" +#include "libavcodec/codec.h" + +#include "libswscale/swscale.h" + +typedef struct FrameChecksum { + int64_t ts; + uint32_t checksum[4]; +} FrameChecksum; + +typedef struct PrivData { + AVCodecContext *enc; + AVCodecContext *dec; + + int64_t pts_in; + + AVPacket *pkt; + AVFrame *frame, *frame_recon; + + struct SwsContext *scaler; + + FrameChecksum *checksums_decoded; + size_t nb_checksums_decoded; + FrameChecksum *checksums_recon; + size_t nb_checksums_recon; +} PrivData; + +static int frame_hash(FrameChecksum **pc, size_t *nb_c, int64_t ts, + const AVFrame *frame) +{ + FrameChecksum *c; + int shift_h, shift_v; + + c = av_realloc_array(*pc, *nb_c + 1, sizeof(*c)); + if (!c) + return AVERROR(ENOMEM); + *pc = c; + (*nb_c)++; + + c += *nb_c - 1; + memset(c, 0, sizeof(*c)); + + av_pix_fmt_get_chroma_sub_sample(frame->format, &shift_h, &shift_v); + + c->ts = ts; + for (int p = 0; frame->data[p]; p++) { + const uint8_t *data = frame->data[p]; + int linesize = av_image_get_linesize(frame->format, frame->width, p); + uint32_t checksum = 0; + + for (int j = 0; j < frame->height >> shift_v; j++) { + checksum = av_adler32_update(checksum, data, linesize); + data += frame->linesize[p]; + } + + c->checksum[p] = checksum; + } + + return 0; +} + +static int recon_frame_process(PrivData *pd, const AVPacket *pkt) +{ + AVFrame *f = pd->frame_recon; + int ret; + + ret = avcodec_receive_frame(pd->enc, f); + if (ret < 0) { + fprintf(stderr, "Error retrieving a reconstructed frame\n"); + return ret; + } + + // the encoder's internal format (in which the reconsturcted frames are + // exported) may be different from the user-facing pixel format + if (f->format != pd->enc->pix_fmt) { + if (!pd->scaler) { + pd->scaler = sws_getContext(f->width, f->height, f->format, + f->width, f->height, pd->enc->pix_fmt, + SWS_BITEXACT, NULL, NULL, NULL); + if (!pd->scaler) + return AVERROR(ENOMEM); + } + + ret = sws_scale_frame(pd->scaler, pd->frame, f); + if (ret < 0) { + fprintf(stderr, "Error converting pixel formats\n"); + return ret; + } + + av_frame_unref(f); + f = pd->frame; + } + + ret = frame_hash(&pd->checksums_recon, &pd->nb_checksums_recon, + pkt->pts, f); + av_frame_unref(f); + + return 0; +} + +static int process_frame(DecodeContext *dc, AVFrame *frame) +{ + PrivData *pd = dc->opaque; + int ret; + + if (!avcodec_is_open(pd->enc)) { + if (!frame) { + fprintf(stderr, "No input frames were decoded\n"); + return AVERROR_INVALIDDATA; + } + + pd->enc->width = frame->width; + pd->enc->height = frame->height; + pd->enc->pix_fmt = frame->format; + pd->enc->thread_count = dc->decoder->thread_count; + pd->enc->thread_type = dc->decoder->thread_type; + + // real timestamps do not matter for this test, so we just + // pretend the input is 25fps CFR to avoid any timestamp issues + pd->enc->time_base = (AVRational){ 1, 25 }; + + ret = avcodec_open2(pd->enc, NULL, NULL); + if (ret < 0) { + fprintf(stderr, "Error opening the encoder\n"); + return ret; + } + } + + if (frame) { + frame->pts = pd->pts_in++; + + // avoid forcing coded frame type + frame->pict_type = AV_PICTURE_TYPE_NONE; + } + + ret = avcodec_send_frame(pd->enc, frame); + if (ret < 0) { + fprintf(stderr, "Error submitting a frame for encoding\n"); + return ret; + } + + while (1) { + AVPacket *pkt = pd->pkt; + + ret = avcodec_receive_packet(pd->enc, pkt); + if (ret == AVERROR(EAGAIN)) + break; + else if (ret == AVERROR_EOF) + pkt = NULL; + else if (ret < 0) { + fprintf(stderr, "Error receiving a frame from the encoder\n"); + return ret; + } + + if (pkt) { + ret = recon_frame_process(pd, pkt); + if (ret < 0) + return ret; + } + + if (!avcodec_is_open(pd->dec)) { + if (!pkt) { + fprintf(stderr, "No packets were received from the encoder\n"); + return AVERROR(EINVAL); + } + + pd->dec->width = pd->enc->width; + pd->dec->height = pd->enc->height; + pd->dec->pix_fmt = pd->enc->pix_fmt; + pd->dec->thread_count = dc->decoder->thread_count; + pd->dec->thread_type = dc->decoder->thread_type; + if (pd->enc->extradata_size) { + pd->dec->extradata = av_memdup(pd->enc->extradata, + pd->enc->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE); + if (!pd->dec->extradata) + return AVERROR(ENOMEM); + } + + ret = avcodec_open2(pd->dec, NULL, NULL); + if (ret < 0) { + fprintf(stderr, "Error opening the decoder\n"); + return ret; + } + } + + ret = avcodec_send_packet(pd->dec, pkt); + if (ret < 0) { + fprintf(stderr, "Error sending a packet to decoder\n"); + return ret; + } + + while (1) { + ret = avcodec_receive_frame(pd->dec, pd->frame); + if (ret == AVERROR(EAGAIN)) + break; + else if (ret == AVERROR_EOF) + return 0; + else if (ret < 0) { + fprintf(stderr, "Error receving a frame from decoder\n"); + return ret; + } + + ret = frame_hash(&pd->checksums_decoded, &pd->nb_checksums_decoded, + pd->frame->pts, pd->frame); + av_frame_unref(pd->frame); + if (ret < 0) + return ret; + } + + } + + return 0; +} + +static int frame_checksum_compare(const void *a, const void *b) +{ + const FrameChecksum *ca = a; + const FrameChecksum *cb = b; + if (ca->ts == cb->ts) + return 0; + return FFSIGN(ca->ts - cb->ts); +} + +int main(int argc, char **argv) +{ + PrivData pd; + DecodeContext dc; + + const char *filename, *enc_name, *enc_opts, *thread_type = NULL, *nb_threads = NULL; + const AVCodec *enc, *dec; + int ret = 0, max_frames = 0; + + if (argc < 4) { + fprintf(stderr, + "Usage: %s " + "[ [ ]\n", + argv[0]); + return 0; + } + + filename = argv[1]; + enc_name = argv[2]; + enc_opts = argv[3]; + if (argc >= 5) + max_frames = strtol(argv[4], NULL, 0); + if (argc >= 6) + nb_threads = argv[5]; + if (argc >= 7) + thread_type = argv[6]; + + memset(&dc, 0, sizeof(dc)); + memset(&pd, 0, sizeof(pd)); + + enc = avcodec_find_encoder_by_name(enc_name); + if (!enc) { + fprintf(stderr, "No such encoder: %s\n", enc_name); + return 1; + } + if (!(enc->capabilities & AV_CODEC_CAP_ENCODER_RECON_FRAME)) { + fprintf(stderr, "Encoder '%s' cannot ouput reconstructed frames\n", + enc->name); + return 1; + } + + dec = avcodec_find_decoder(enc->id); + if (!dec) { + fprintf(stderr, "No decoder for: %s\n", avcodec_get_name(enc->id)); + return 1; + } + + pd.enc = avcodec_alloc_context3(enc); + if (!pd.enc) { + fprintf(stderr, "Error allocating encoder\n"); + return 1; + } + + ret = av_set_options_string(pd.enc, enc_opts, "=", ","); + if (ret < 0) { + fprintf(stderr, "Error setting encoder options\n"); + goto fail; + } + pd.enc->flags |= AV_CODEC_FLAG_RECON_FRAME | AV_CODEC_FLAG_BITEXACT; + + pd.dec = avcodec_alloc_context3(dec); + if (!pd.dec) { + fprintf(stderr, "Error allocating decoder\n"); + goto fail; + } + + pd.dec->flags |= AV_CODEC_FLAG_BITEXACT; + pd.dec->err_recognition |= AV_EF_CRCCHECK; + + pd.frame = av_frame_alloc(); + pd.frame_recon = av_frame_alloc(); + pd.pkt = av_packet_alloc(); + if (!pd.frame ||!pd.frame_recon || !pd.pkt) { + ret = 1; + goto fail; + } + + ret = ds_open(&dc, filename, 0); + if (ret < 0) { + fprintf(stderr, "Error opening the file\n"); + goto fail; + } + + dc.process_frame = process_frame; + dc.opaque = &pd; + dc.max_frames = max_frames; + + ret = av_dict_set(&dc.decoder_opts, "threads", nb_threads, 0); + ret |= av_dict_set(&dc.decoder_opts, "thread_type", thread_type, 0); + + ret = ds_run(&dc); + if (ret < 0) + goto fail; + + if (pd.nb_checksums_decoded != pd.nb_checksums_recon) { + fprintf(stderr, "Mismatching frame counts: recon=%zu decoded=%zu\n", + pd.nb_checksums_recon, pd.nb_checksums_decoded); + ret = 1; + goto fail; + } + + // reconstructed frames are in coded order, sort them by pts into presentation order + qsort(pd.checksums_recon, pd.nb_checksums_recon, sizeof(*pd.checksums_recon), + frame_checksum_compare); + + for (size_t i = 0; i < pd.nb_checksums_decoded; i++) { + const FrameChecksum *d = &pd.checksums_decoded[i]; + const FrameChecksum *r = &pd.checksums_recon[i]; + + for (int p = 0; p < FF_ARRAY_ELEMS(d->checksum); p++) + if (d->checksum[p] != r->checksum[p]) { + fprintf(stderr, "Checksum mismatch in frame ts=%"PRId64", plane %d\n", + d->ts, p); + ret = 1; + goto fail; + } + } + fprintf(stderr, "All %zu encoded frames match\n", pd.nb_checksums_decoded); + +fail: + avcodec_free_context(&pd.enc); + avcodec_free_context(&pd.dec); + av_freep(&pd.checksums_decoded); + av_freep(&pd.checksums_recon); + av_frame_free(&pd.frame); + av_frame_free(&pd.frame_recon); + av_packet_free(&pd.pkt); + ds_free(&dc); + return !!ret; +}