From patchwork Tue May 30 17:28:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Jones X-Patchwork-Id: 3765 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.10.2 with SMTP id 2csp1991875vsk; Tue, 30 May 2017 10:28:27 -0700 (PDT) X-Received: by 10.28.0.13 with SMTP id 13mr2490565wma.19.1496165306932; Tue, 30 May 2017 10:28:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1496165306; cv=none; d=google.com; s=arc-20160816; b=bRFzrgC12CJIHJYaA7CwFWUHvJ6M52ssStcaqevd0Hb/1YHjnuVVjE/XlLYyoTyDuy tsWiLjaaR1nviCqNy2WUQfob+3RSS3s6E+2YD4TIUWEHe07fIpsDBSOm06PINQTBBtnh MeET+nbffHdZbdZr/DtMRgCH5FDxg+dksXw6bJlGQtN0TiQdMBEguMk+v1oefRAXWEMO 345F0fZvYIHRGdsMmFE0fgROLW8Z6UD3zWABsn16ThfWuT04yLj/CQlpRy1MV/n3l88N CqrRM0IDNwPGWqy5/0+nd8annLBONU0m7keVmDZNzfAngoRTE4W40bKmeM2qQjCcEAYh WaVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject:user-agent :mime-version:message-id:to:from:date:dkim-signature:delivered-to :arc-authentication-results; bh=WK1PzaIt30vK+G2xmqozH+A2HWkJA4rDmeNbwiI37oI=; b=eYMTVKAXibA0AonviRM7yPJyXxLrSCFOIMYsS8t5dFUZv6DilMl6chILpVUWf1TpKs XOhwHEXuqz51df/oQucI6up1Nv0z4p6j/cfcnrMmgGP/UbQ29o7IegBr8pCHTDijJQDB kDrx7FEQiGzGO/a0kpqIe7Ag7Kb2+/TMpqqGJ6TeUL+tVkDzpjnwf7QnrzRbNOgj/3iY 6T7FD8xLkXJ+tyXnt4fl0D5tYim/eEpAeaCBaK8y2vBCyhrG/kYwifJoVHBP1/g3gIeO Xop9cuEsRub5daBDS8P2L/IKsIPBbbra4U5tdrhZBpXzKkmLumMGrY/0Y+A2T+LWM3VG PAdA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r99si4107567wrb.259.2017.05.30.10.28.26; Tue, 30 May 2017 10:28:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D14A96883CA; Tue, 30 May 2017 20:28:18 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf0-f181.google.com (mail-pf0-f181.google.com [209.85.192.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DF15A6882CF for ; Tue, 30 May 2017 20:28:16 +0300 (EEST) Received: by mail-pf0-f181.google.com with SMTP id e193so78387383pfh.0 for ; Tue, 30 May 2017 10:28:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:subject:message-id:mime-version:content-disposition :user-agent; bh=VFVTrpcBpGkCCHYKm/aCTRHmOUWyUiXximAac4RSxDg=; b=SEhsr4AxO2Df/WGJ1wayg+zmcJfFiq2dVrxoQPoLkF1dKAo0xLcxIRU+mR5OChYzTJ W/KZnzVvrrOmMqioBSfNVpLh1tyTwqRW3V0Y9ePgPc5UzKS4+0BhT9MFSjy7tX8poGKt B/gkbuRDAnxq3avLIW3vzVH+Tqj0/xlWYXpu8gl27b8VrgblO1bk54yKR6jGr52iqPcx CMrUxAHH95l5BCxZQJVnDrqX1rxMhBMwMM0NpryXt30M6y+otn4GlekboKweh7xPK0iK 4WQ0knq7/pIIxREWAN4iJVOp02k9LLAAaAtqBcvUfjdbyHbESZRcyRSUzH9rT9gUhPdG dxpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-disposition:user-agent; bh=VFVTrpcBpGkCCHYKm/aCTRHmOUWyUiXximAac4RSxDg=; b=esbmlIP+wAD4UcPzKEGjzDIU9aH100AFNdk5XYRF3hCMdE9i9nNKrIlao4Hw9VO34O AKaRZzgqiJjL3Is1orRxjgKnU5LGHNFbgWieBE7HzGwkxrvt2uM5ncQTfKX6cALhUv1O clwsgkPyYKSKjc4TnaEjXeaQWwZBpmwK7rQ3uaHT1NTU8CeaJHycAQ3+nrkKD3F34daY DH4W//NkZWTlQi7ibx8Lx//DgJ/dreIJljnpccIrlCFeGfgLwGB046ab5/36UXP3yQJC Hlicvzkwj6wtyr+OxRJpUupfwitBMyPROHJTmkWevAcgunMgYO+cyuEzkkT38ADOLXV9 2GfQ== X-Gm-Message-State: AODbwcDnysz4wL/Rd3q3o9uu46ZgdHC8BR3ZBBpf4WVVtIp0PAOyGLSX LIY6zpIeHFun6K5Q X-Received: by 10.98.58.83 with SMTP id h80mr25277952pfa.27.1496165299557; Tue, 30 May 2017 10:28:19 -0700 (PDT) Received: from tdjones879 (host-184-167-177-46.csp-wy.client.bresnan.net. [184.167.177.46]) by smtp.gmail.com with ESMTPSA id 6sm24221414pfo.132.2017.05.30.10.28.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 May 2017 10:28:18 -0700 (PDT) Date: Tue, 30 May 2017 11:28:16 -0600 From: Tyler Jones To: ffmpeg-devel@ffmpeg.org Message-ID: <20170530172816.GA20222@tdjones879> MIME-Version: 1.0 User-Agent: Mutt/1.5.24 (2015-08-30) Subject: [FFmpeg-devel] [PATCH 2/2] avcodec/vorbisenc: Use a bufqueue in encoding with smaller lengths X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Switching the vorbis encoder to use a buffer queue for input frames allows saving lookahead samples more easily and safely for psychoacoustic systems, requiring less pointer arithmetic in the case of transient windows. --- libavcodec/vorbisenc.c | 120 +++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 106 insertions(+), 14 deletions(-) diff --git a/libavcodec/vorbisenc.c b/libavcodec/vorbisenc.c index fdce864..ea2b7f5 100644 --- a/libavcodec/vorbisenc.c +++ b/libavcodec/vorbisenc.c @@ -1024,20 +1024,117 @@ static int apply_window_and_mdct(vorbis_enc_context *venc, return 1; } +/* Used for padding the last encoded packet */ +static AVFrame *spawn_empty_frame(AVCodecContext *avctx, int channels) +{ + AVFrame *f = av_frame_alloc(); + + if (!f) + return NULL; + + f->format = avctx->sample_fmt; + f->nb_samples = avctx->frame_size; + f->channel_layout = avctx->channel_layout; + + if (av_frame_get_buffer(f, 4)) { + av_frame_free(&f); + return NULL; + } + + for (int ch = 0; ch < channels; ch++) { + size_t bps = av_get_bytes_per_sample(f->format); + memset(f->extended_data[ch], 0, bps * f->nb_samples); + } + return f; +} + +static float **alloc_audio_arrays(int channels, int frame_size) +{ + float **audio = av_mallocz_array(channels, sizeof(float *)); + if (!audio) + return NULL; + + for (int ch = 0; ch < channels; ch++) { + audio[ch] = av_mallocz_array(frame_size, sizeof(float)); + if (!audio[ch]) { + // alloc has failed, free everything allocated thus far + for (ch--; ch >= 0; ch--) + av_free(audio[ch]); + av_free(audio); + return NULL; + } + } + + return audio; +} + +/* Concatenate audio frames into an appropriately sized array of samples */ +static void move_audio(vorbis_enc_context *venc, float **audio, int *samples, int sf_size) +{ + AVFrame *cur = NULL; + int frame_size = 1 << (venc->log2_blocksize[1] - 1); + int subframes = frame_size / sf_size; + + for (int sf = 0; sf < subframes; sf++) { + cur = ff_bufqueue_get(&venc->bufqueue); + *samples += cur->nb_samples; + + for (int ch = 0; ch < venc->channels; ch++) { + const float *input = (float *) cur->extended_data[ch]; + const size_t len = cur->nb_samples * sizeof(float); + memcpy(&audio[ch][sf*sf_size], input, len); + } + av_frame_free(&cur); + } +} + static int vorbis_encode_frame(AVCodecContext *avctx, AVPacket *avpkt, const AVFrame *frame, int *got_packet_ptr) { vorbis_enc_context *venc = avctx->priv_data; - float **audio = frame ? (float **)frame->extended_data : NULL; - int samples = frame ? frame->nb_samples : 0; + float **audio = NULL; + int i, ret, need_more; + int samples = 0, frame_size = 1 << (venc->log2_blocksize[1] - 1); vorbis_enc_mode *mode; vorbis_enc_mapping *mapping; PutBitContext pb; - int i, ret; + + if (frame) { + if ((ret = ff_af_queue_add(&venc->afq, frame)) < 0) + return ret; + ff_bufqueue_add(avctx, &venc->bufqueue, av_frame_clone(frame)); + } else + if (!venc->afq.remaining_samples) + return 0; + + need_more = venc->bufqueue.available * avctx->frame_size < frame_size; + need_more = frame && need_more; + if (need_more) + return 0; + + audio = alloc_audio_arrays(venc->channels, frame_size); + if (!audio) + return AVERROR(ENOMEM); + + /* Pad the bufqueue with empty frames for encoding the last packet. */ + if (!frame) { + if (venc->bufqueue.available * avctx->frame_size < frame_size) { + int frames_needed = (frame_size/avctx->frame_size) - venc->bufqueue.available; + + for (int i = 0; i < frames_needed; i++) { + AVFrame *empty = spawn_empty_frame(avctx, venc->channels); + if (!empty) + return AVERROR(ENOMEM); + + ff_bufqueue_add(avctx, &venc->bufqueue, empty); + } + } + } + + move_audio(venc, audio, &samples, avctx->frame_size); if (!apply_window_and_mdct(venc, audio, samples)) return 0; - samples = 1 << (venc->log2_blocksize[0] - 1); if ((ret = ff_alloc_packet2(avctx, avpkt, 8192, 0)) < 0) return ret; @@ -1096,16 +1193,11 @@ static int vorbis_encode_frame(AVCodecContext *avctx, AVPacket *avpkt, flush_put_bits(&pb); avpkt->size = put_bits_count(&pb) >> 3; - avpkt->duration = ff_samples_to_time_base(avctx, avctx->frame_size); - if (frame) { - if (frame->pts != AV_NOPTS_VALUE) - avpkt->pts = ff_samples_to_time_base(avctx, frame->pts); - } else { - avpkt->pts = venc->next_pts; - } - if (avpkt->pts != AV_NOPTS_VALUE) - venc->next_pts = avpkt->pts + avpkt->duration; + for (int ch = 0; ch < venc->channels; ch++) + av_free(audio[ch]); + av_free(audio); + ff_af_queue_remove(&venc->afq, frame_size, &avpkt->pts, &avpkt->duration); *got_packet_ptr = 1; return 0; } @@ -1196,7 +1288,7 @@ static av_cold int vorbis_encode_init(AVCodecContext *avctx) goto error; avctx->extradata_size = ret; - avctx->frame_size = 1 << (venc->log2_blocksize[0] - 1); + avctx->frame_size = 64; ff_af_queue_init(avctx, &venc->afq);