From patchwork Tue Oct 18 16:33:13 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rostislav Pehlivanov X-Patchwork-Id: 1055 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.133 with SMTP id o127csp759051vsd; Tue, 18 Oct 2016 09:33:25 -0700 (PDT) X-Received: by 10.194.62.205 with SMTP id a13mr837800wjs.89.1476808405883; Tue, 18 Oct 2016 09:33:25 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id do12si38184540wjb.261.2016.10.18.09.33.24; Tue, 18 Oct 2016 09:33:25 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A60C9689AF2; Tue, 18 Oct 2016 19:33:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf0-f68.google.com (mail-lf0-f68.google.com [209.85.215.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A8377689751 for ; Tue, 18 Oct 2016 19:33:12 +0300 (EEST) Received: by mail-lf0-f68.google.com with SMTP id x79so33422267lff.2 for ; Tue, 18 Oct 2016 09:33:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=N+Rpjdc+5KTHX9M4LjfItXbThTJOb1TevPlcpH/yWQE=; b=dsIlISmLN+RJFRQ498gW0kT2TVJAcH0Tx6tPBgyTt0rNp/ssN35u90YFuxObOTaTf8 6QpNWaYaPol2Uka0da0jzm5lBpzcldPtrxXOorFpL/uwihH6YMZ+i9tW+6GWxdeAa081 KAO3KtiMX+od9OObKNTMWtugsSjbPWoWB0HmjwMH+CFJa69luGaldyNM1JbLJzkKxE/1 70YkJziBcS4W4GZuF8tAv1ixx+MyUZTDMAr8blZuD7DAeMojSNdvqlCDrwevFXfenQyK qtVHLV7e9qcVkYHFLSKI/mqcsZNwl5GXyb3TbNYpcEufuVOrxjFub9aolwWom2J/HT4j Da0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=N+Rpjdc+5KTHX9M4LjfItXbThTJOb1TevPlcpH/yWQE=; b=RP37SmS0bjd3E0Vh1hKgdUu0p5HZapibe+osKOJNNQvzMfL1K3YWKT8u5B3irFWWPn teXzEVO/A+brUELmFiDJbrMKJO66XymjegxwW025714ADrfjgbYkB3MZO38LywXiOMED 672a8cx3GoEiqtGESaOIXgpsU9yUB13HB7gzHgQkyqZzuTESp1CCbEo/F/M4uh/ulG5K b0LG42R/Yo+CLps20sibPyYFVNQhQIiJ4IFZhObYJSLzNmP+afKaNjEkh7NKQrZj96R2 ojLmsHKQCU6/bUfg/0okC1hJSC5z/4LpEBZlY8v674VvT0PdGCiDjysld7dTNhWhGxYj HpSQ== X-Gm-Message-State: AA6/9Rn32+yIYCjo6n6SIZ1pi3y1k24OagIyXiWe3A0ujAneAel8u+qIKyd4PGM0h7srLbeLtTgYiaSoLU2b3g== X-Received: by 10.25.44.194 with SMTP id s185mr77524lfs.124.1476808394445; Tue, 18 Oct 2016 09:33:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.114.26.98 with HTTP; Tue, 18 Oct 2016 09:33:13 -0700 (PDT) In-Reply-To: References: <20161017212448.13208-1-atomnuker@gmail.com> <20161017224316.GK4602@nb4> <20161018135122.GT4602@nb4> From: Rostislav Pehlivanov Date: Tue, 18 Oct 2016 17:33:13 +0100 Message-ID: To: FFmpeg development discussions and patches X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: Re: [FFmpeg-devel] [PATCH v3] aacenc: add SIMD optimizations for abs_pow34 and quantization X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On 18 October 2016 at 16:32, James Almer wrote: > On 10/18/2016 12:07 PM, Rostislav Pehlivanov wrote: > > diff --git a/libavcodec/aacenc.c b/libavcodec/aacenc.c > > index ee3cbf8..622f0ba 100644 > > --- a/libavcodec/aacenc.c > > +++ b/libavcodec/aacenc.c > > @@ -1033,6 +1033,12 @@ static av_cold int aac_encode_init(AVCodecContext > *avctx) > > ff_lpc_init(&s->lpc, 2*avctx->frame_size, TNS_MAX_ORDER, > FF_LPC_TYPE_LEVINSON); > > s->random_state = 0x1f2e3d4c; > > > > + s->abs_pow34 = &abs_pow34_v; > > + s->quant_bands = &quantize_bands; > > No need for & in these. > > > + > > + if (ARCH_X86) > > + ff_aac_dsp_init_x86(s); > > + > > if (HAVE_MIPSDSP) > > ff_aac_coder_init_mips(s); > > [...] > > > diff --git a/libavcodec/x86/aacencdsp.asm b/libavcodec/x86/aacencdsp.asm > > new file mode 100644 > > index 0000000..dd7b022 > > --- /dev/null > > +++ b/libavcodec/x86/aacencdsp.asm > > @@ -0,0 +1,88 @@ > > +;********************************************************** > ******************** > > +;* SIMD optimized AAC encoder DSP functions > > +;* > > +;* Copyright (C) 2016 Rostislav Pehlivanov > > +;* > > +;* This file is part of FFmpeg. > > +;* > > +;* FFmpeg is free software; you can redistribute it and/or > > +;* modify it under the terms of the GNU Lesser General Public > > +;* License as published by the Free Software Foundation; either > > +;* version 2.1 of the License, or (at your option) any later version. > > +;* > > +;* FFmpeg is distributed in the hope that it will be useful, > > +;* but WITHOUT ANY WARRANTY; without even the implied warranty of > > +;* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > +;* Lesser General Public License for more details. > > +;* > > +;* You should have received a copy of the GNU Lesser General Public > > +;* License along with FFmpeg; if not, write to the Free Software > > +;* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA > > +;********************************************************** > ******************** > > + > > +%include "libavutil/x86/x86util.asm" > > + > > +SECTION_RODATA > > + > > +float_abs_mask: times 4 dd 0x7fffffff > > + > > +SECTION .text > > + > > +;******************************************************************* > > +;void ff_abs_pow34(float *out, const float *in, const int size); > > +;******************************************************************* > > +INIT_XMM sse > > +cglobal abs_pow34, 3, 3, 3, out, in, size > > + mova m2, [float_abs_mask] > > + shl sizeq, 2 > > + add inq, sizeq > > + add outq, sizeq > > + neg sizeq > > +.loop: > > + movaps m0, [inq+sizeq] > > + andps m0, m2 > > Remove the movaps and do > > andps m0, m2, [inq+sizeq] > > Instead. Sorry i didn't notice this last time. > > > + sqrtps m1, m0 > > + mulps m0, m1 > > + sqrtps m0, m0 > > + mova [outq+sizeq], m0 > > + add sizeq, mmsize > > + jl .loop > > + RET > > + > > +;******************************************************************* > > +;void ff_aac_quantize_bands(int *out, const float *in, const float > *scaled, > > +; int size, int is_signed, int maxval, const > float Q34, > > +; const float rounding) > > +;******************************************************************* > > +INIT_XMM sse2 > > +cglobal aac_quantize_bands, 5, 5, 6, out, in, scaled, size, is_signed, > maxval, Q34, rounding > > +%if UNIX64 == 0 > > + movss m0, Q34m > > + movss m1, roundingm > > + cvtsi2ss m3, maxvald > > +%else > > + cvtsi2ss m3, dword maxvalm > > +%endif > > The other way around. Unix64 is the one that has maxval on a reg regardless > of how you init the function, whereas win64 and any x86_32 target have it > on stack. > > > + shufps m0, m0, 0 > > + shufps m1, m1, 0 > > + shufps m3, m3, 0 > > + shl is_signedd, 31 > > + movd m4, is_signedd > > + shufps m4, m4, 0 > > + shl sized, 2 > > + add inq, sizeq > > + add outq, sizeq > > + add scaledq, sizeq > > + neg sizeq > > +.loop: > > + mulps m2, m0, [scaledq+sizeq] > > + addps m2, m1 > > + minps m2, m3 > > > + movaps m5, [inq+sizeq] > > + andps m5, m4 > > Same as in abs_pow34, remove movaps and do > > andps m5, m4, [inq+sizeq] > > > + orps m2, m5 > > + cvttps2dq m2, m2 > > + mova [outq+sizeq], m2 > > + add sizeq, mmsize > > + jl .loop > > + RET > > diff --git a/libavcodec/x86/aacencdsp_init.c b/libavcodec/x86/aacencdsp_ > init.c > > new file mode 100644 > > index 0000000..aefaa15 > > --- /dev/null > > +++ b/libavcodec/x86/aacencdsp_init.c > > @@ -0,0 +1,43 @@ > > +/* > > + * AAC encoder assembly optimizations > > + * Copyright (C) 2016 Rostislav Pehlivanov > > + * > > + * This file is part of FFmpeg. > > + * > > + * FFmpeg is free software; you can redistribute it and/or > > + * modify it under the terms of the GNU Lesser General Public > > + * License as published by the Free Software Foundation; either > > + * version 2.1 of the License, or (at your option) any later version. > > + * > > + * FFmpeg is distributed in the hope that it will be useful, > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + * Lesser General Public License for more details. > > + * > > + * You should have received a copy of the GNU Lesser General Public > > + * License along with FFmpeg; if not, write to the Free Software > > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA > > + */ > > + > > +#include "config.h" > > + > > +#include "libavutil/float_dsp.h" > > +#include "libavutil/x86/cpu.h" > > +#include "libavcodec/aacenc.h" > > + > > +void ff_abs_pow34_sse(float *out, const float *in, const int size); > > + > > +void ff_aac_quantize_bands_sse2(int *out, const float *in, const float > *scaled, > > + int size, int is_signed, int maxval, > const float Q34, > > + const float rounding); > > + > > +av_cold void ff_aac_dsp_init_x86(AACEncContext *s) > > +{ > > + int cpu_flags = av_get_cpu_flags(); > > + > > + if (EXTERNAL_SSE(cpu_flags)) > > + s->abs_pow34 = &ff_abs_pow34_sse; > > + > > + if (EXTERNAL_SSE2(cpu_flags)) > > + s->quant_bands = &ff_aac_quantize_bands_sse2; > > Same with these. > > Passes fate on mingw-w64 (x86_64) after fixing the maxval mixup above. > Crashes otherwise. > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > Hopefully I didn't mess anything up and the attached patch is the last revision of it. From eb95cf5aeacdc7d191c5b7cf8df40a1e5306b733 Mon Sep 17 00:00:00 2001 From: Rostislav Pehlivanov Date: Sat, 8 Oct 2016 15:59:14 +0100 Subject: [PATCH] aacenc: add SIMD optimizations for abs_pow34 and quantization Performance improvements: quant_bands: with: 681 decicycles in quant_bands, 8388453 runs, 155 skips without: 1190 decicycles in quant_bands, 8388386 runs, 222 skips Around 42% for the function Twoloop coder: abs_pow34: with/without: 7.82s/8.17s Around 4% for the entire encoder Both: with/without: 7.15s/8.17s Around 12% for the entire encoder Fast coder: abs_pow34: with/without: 3.40s/3.77s Around 10% for the entire encoder Both: with/without: 3.02s/3.77s Around 20% faster for the entire encoder Signed-off-by: Rostislav Pehlivanov --- libavcodec/aaccoder.c | 27 +++++++------ libavcodec/aaccoder_trellis.h | 2 +- libavcodec/aaccoder_twoloop.h | 2 +- libavcodec/aacenc.c | 6 +++ libavcodec/aacenc.h | 6 +++ libavcodec/aacenc_is.c | 6 +-- libavcodec/aacenc_ltp.c | 4 +- libavcodec/aacenc_pred.c | 6 +-- libavcodec/aacenc_quantization.h | 4 +- libavcodec/aacenc_utils.h | 2 +- libavcodec/x86/Makefile | 2 + libavcodec/x86/aacencdsp.asm | 86 ++++++++++++++++++++++++++++++++++++++++ libavcodec/x86/aacencdsp_init.c | 43 ++++++++++++++++++++ 13 files changed, 170 insertions(+), 26 deletions(-) create mode 100644 libavcodec/x86/aacencdsp.asm create mode 100644 libavcodec/x86/aacencdsp_init.c diff --git a/libavcodec/aaccoder.c b/libavcodec/aaccoder.c index 35787e8..9f3b4ed 100644 --- a/libavcodec/aaccoder.c +++ b/libavcodec/aaccoder.c @@ -88,7 +88,7 @@ static void encode_window_bands_info(AACEncContext *s, SingleChannelElement *sce float next_minrd = INFINITY; int next_mincb = 0; - abs_pow34_v(s->scoefs, sce->coeffs, 1024); + s->abs_pow34(s->scoefs, sce->coeffs, 1024); start = win*128; for (cb = 0; cb < CB_TOT_ALL; cb++) { path[0][cb].cost = 0.0f; @@ -299,7 +299,7 @@ static void search_for_quantizers_anmr(AVCodecContext *avctx, AACEncContext *s, } } idx = 1; - abs_pow34_v(s->scoefs, sce->coeffs, 1024); + s->abs_pow34(s->scoefs, sce->coeffs, 1024); for (w = 0; w < sce->ics.num_windows; w += sce->ics.group_len[w]) { start = w*128; for (g = 0; g < sce->ics.num_swb; g++) { @@ -446,7 +446,7 @@ static void search_for_quantizers_fast(AVCodecContext *avctx, AACEncContext *s, if (!allz) return; - abs_pow34_v(s->scoefs, sce->coeffs, 1024); + s->abs_pow34(s->scoefs, sce->coeffs, 1024); ff_quantize_band_cost_cache_init(s); for (w = 0; w < sce->ics.num_windows; w += sce->ics.group_len[w]) { @@ -652,8 +652,8 @@ static void search_for_pns(AACEncContext *s, AVCodecContext *avctx, SingleChanne s->fdsp->vector_fmul_scalar(PNS, PNS, scale, sce->ics.swb_sizes[g]); pns_senergy = s->fdsp->scalarproduct_float(PNS, PNS, sce->ics.swb_sizes[g]); pns_energy += pns_senergy; - abs_pow34_v(NOR34, &sce->coeffs[start_c], sce->ics.swb_sizes[g]); - abs_pow34_v(PNS34, PNS, sce->ics.swb_sizes[g]); + s->abs_pow34(NOR34, &sce->coeffs[start_c], sce->ics.swb_sizes[g]); + s->abs_pow34(PNS34, PNS, sce->ics.swb_sizes[g]); dist1 += quantize_band_cost(s, &sce->coeffs[start_c], NOR34, sce->ics.swb_sizes[g], @@ -757,8 +757,9 @@ static void search_for_ms(AACEncContext *s, ChannelElement *cpe) { int start = 0, i, w, w2, g, sid_sf_boost, prev_mid, prev_side; uint8_t nextband0[128], nextband1[128]; - float M[128], S[128]; - float *L34 = s->scoefs, *R34 = s->scoefs + 128, *M34 = s->scoefs + 128*2, *S34 = s->scoefs + 128*3; + float *M = s->scoefs + 128*0, *S = s->scoefs + 128*1; + float *L34 = s->scoefs + 128*2, *R34 = s->scoefs + 128*3; + float *M34 = s->scoefs + 128*4, *S34 = s->scoefs + 128*5; const float lambda = s->lambda; const float mslambda = FFMIN(1.0f, lambda / 120.f); SingleChannelElement *sce0 = &cpe->ch[0]; @@ -789,8 +790,8 @@ static void search_for_ms(AACEncContext *s, ChannelElement *cpe) S[i] = M[i] - sce1->coeffs[start+(w+w2)*128+i]; } - abs_pow34_v(M34, M, sce0->ics.swb_sizes[g]); - abs_pow34_v(S34, S, sce0->ics.swb_sizes[g]); + s->abs_pow34(M34, M, sce0->ics.swb_sizes[g]); + s->abs_pow34(S34, S, sce0->ics.swb_sizes[g]); for (i = 0; i < sce0->ics.swb_sizes[g]; i++ ) { Mmax = FFMAX(Mmax, M34[i]); Smax = FFMAX(Smax, S34[i]); @@ -833,10 +834,10 @@ static void search_for_ms(AACEncContext *s, ChannelElement *cpe) - sce1->coeffs[start+(w+w2)*128+i]; } - abs_pow34_v(L34, sce0->coeffs+start+(w+w2)*128, sce0->ics.swb_sizes[g]); - abs_pow34_v(R34, sce1->coeffs+start+(w+w2)*128, sce0->ics.swb_sizes[g]); - abs_pow34_v(M34, M, sce0->ics.swb_sizes[g]); - abs_pow34_v(S34, S, sce0->ics.swb_sizes[g]); + s->abs_pow34(L34, sce0->coeffs+start+(w+w2)*128, sce0->ics.swb_sizes[g]); + s->abs_pow34(R34, sce1->coeffs+start+(w+w2)*128, sce0->ics.swb_sizes[g]); + s->abs_pow34(M34, M, sce0->ics.swb_sizes[g]); + s->abs_pow34(S34, S, sce0->ics.swb_sizes[g]); dist1 += quantize_band_cost(s, &sce0->coeffs[start + (w+w2)*128], L34, sce0->ics.swb_sizes[g], diff --git a/libavcodec/aaccoder_trellis.h b/libavcodec/aaccoder_trellis.h index 0230052..940ebf0 100644 --- a/libavcodec/aaccoder_trellis.h +++ b/libavcodec/aaccoder_trellis.h @@ -70,7 +70,7 @@ static void codebook_trellis_rate(AACEncContext *s, SingleChannelElement *sce, float next_minbits = INFINITY; int next_mincb = 0; - abs_pow34_v(s->scoefs, sce->coeffs, 1024); + s->abs_pow34(s->scoefs, sce->coeffs, 1024); start = win*128; for (cb = 0; cb < CB_TOT_ALL; cb++) { path[0][cb].cost = run_bits+4; diff --git a/libavcodec/aaccoder_twoloop.h b/libavcodec/aaccoder_twoloop.h index 42aea52..fb9849e 100644 --- a/libavcodec/aaccoder_twoloop.h +++ b/libavcodec/aaccoder_twoloop.h @@ -291,7 +291,7 @@ static void search_for_quantizers_twoloop(AVCodecContext *avctx, if (!allz) return; - abs_pow34_v(s->scoefs, sce->coeffs, 1024); + s->abs_pow34(s->scoefs, sce->coeffs, 1024); ff_quantize_band_cost_cache_init(s); for (i = 0; i < sizeof(minsf) / sizeof(minsf[0]); ++i) diff --git a/libavcodec/aacenc.c b/libavcodec/aacenc.c index ee3cbf8..363ed05 100644 --- a/libavcodec/aacenc.c +++ b/libavcodec/aacenc.c @@ -1033,6 +1033,12 @@ static av_cold int aac_encode_init(AVCodecContext *avctx) ff_lpc_init(&s->lpc, 2*avctx->frame_size, TNS_MAX_ORDER, FF_LPC_TYPE_LEVINSON); s->random_state = 0x1f2e3d4c; + s->abs_pow34 = abs_pow34_v; + s->quant_bands = quantize_bands; + + if (ARCH_X86) + ff_aac_dsp_init_x86(s); + if (HAVE_MIPSDSP) ff_aac_coder_init_mips(s); diff --git a/libavcodec/aacenc.h b/libavcodec/aacenc.h index 1ace00d..38a9734 100644 --- a/libavcodec/aacenc.h +++ b/libavcodec/aacenc.h @@ -127,11 +127,17 @@ typedef struct AACEncContext { uint16_t quantize_band_cost_cache_generation; AACQuantizeBandCostCacheEntry quantize_band_cost_cache[256][128]; ///< memoization area for quantize_band_cost + void (*abs_pow34)(float *out, const float *in, const int size); + void (*quant_bands)(int *out, const float *in, const float *scaled, + int size, int is_signed, int maxval, const float Q34, + const float rounding); + struct { float *samples; } buffer; } AACEncContext; +void ff_aac_dsp_init_x86(AACEncContext *s); void ff_aac_coder_init_mips(AACEncContext *c); void ff_quantize_band_cost_cache_init(struct AACEncContext *s); diff --git a/libavcodec/aacenc_is.c b/libavcodec/aacenc_is.c index 473897b..2f5b7eb 100644 --- a/libavcodec/aacenc_is.c +++ b/libavcodec/aacenc_is.c @@ -59,9 +59,9 @@ struct AACISError ff_aac_is_encoding_err(AACEncContext *s, ChannelElement *cpe, float minthr = FFMIN(band0->threshold, band1->threshold); for (i = 0; i < sce0->ics.swb_sizes[g]; i++) IS[i] = (L[start+(w+w2)*128+i] + phase*R[start+(w+w2)*128+i])*sqrt(ener0/ener01); - abs_pow34_v(L34, &L[start+(w+w2)*128], sce0->ics.swb_sizes[g]); - abs_pow34_v(R34, &R[start+(w+w2)*128], sce0->ics.swb_sizes[g]); - abs_pow34_v(I34, IS, sce0->ics.swb_sizes[g]); + s->abs_pow34(L34, &L[start+(w+w2)*128], sce0->ics.swb_sizes[g]); + s->abs_pow34(R34, &R[start+(w+w2)*128], sce0->ics.swb_sizes[g]); + s->abs_pow34(I34, IS, sce0->ics.swb_sizes[g]); maxval = find_max_val(1, sce0->ics.swb_sizes[g], I34); is_band_type = find_min_book(maxval, is_sf_idx); dist1 += quantize_band_cost(s, &L[start + (w+w2)*128], L34, diff --git a/libavcodec/aacenc_ltp.c b/libavcodec/aacenc_ltp.c index b9d43b4..1bec85b 100644 --- a/libavcodec/aacenc_ltp.c +++ b/libavcodec/aacenc_ltp.c @@ -190,8 +190,8 @@ void ff_aac_search_for_ltp(AACEncContext *s, SingleChannelElement *sce, FFPsyBand *band = &s->psy.ch[s->cur_channel].psy_bands[(w+w2)*16+g]; for (i = 0; i < sce->ics.swb_sizes[g]; i++) PCD[i] = sce->coeffs[start+(w+w2)*128+i] - sce->lcoeffs[start+(w+w2)*128+i]; - abs_pow34_v(C34, &sce->coeffs[start+(w+w2)*128], sce->ics.swb_sizes[g]); - abs_pow34_v(PCD34, PCD, sce->ics.swb_sizes[g]); + s->abs_pow34(C34, &sce->coeffs[start+(w+w2)*128], sce->ics.swb_sizes[g]); + s->abs_pow34(PCD34, PCD, sce->ics.swb_sizes[g]); dist1 += quantize_band_cost(s, &sce->coeffs[start+(w+w2)*128], C34, sce->ics.swb_sizes[g], sce->sf_idx[(w+w2)*16+g], sce->band_type[(w+w2)*16+g], s->lambda/band->threshold, INFINITY, &bits_tmp1, NULL, 0); diff --git a/libavcodec/aacenc_pred.c b/libavcodec/aacenc_pred.c index e77a3de..d111192 100644 --- a/libavcodec/aacenc_pred.c +++ b/libavcodec/aacenc_pred.c @@ -270,7 +270,7 @@ void ff_aac_search_for_pred(AACEncContext *s, SingleChannelElement *sce) continue; /* Normal coefficients */ - abs_pow34_v(O34, &sce->coeffs[start_coef], num_coeffs); + s->abs_pow34(O34, &sce->coeffs[start_coef], num_coeffs); dist1 = quantize_and_encode_band_cost(s, NULL, &sce->coeffs[start_coef], NULL, O34, num_coeffs, sce->sf_idx[sfb], cb_n, s->lambda / band->threshold, INFINITY, &cost1, NULL, 0); @@ -279,7 +279,7 @@ void ff_aac_search_for_pred(AACEncContext *s, SingleChannelElement *sce) /* Encoded coefficients - needed for #bits, band type and quant. error */ for (i = 0; i < num_coeffs; i++) SENT[i] = sce->coeffs[start_coef + i] - sce->prcoeffs[start_coef + i]; - abs_pow34_v(S34, SENT, num_coeffs); + s->abs_pow34(S34, SENT, num_coeffs); if (cb_n < RESERVED_BT) cb_p = av_clip(find_min_book(find_max_val(1, num_coeffs, S34), sce->sf_idx[sfb]), cb_min, cb_max); else @@ -291,7 +291,7 @@ void ff_aac_search_for_pred(AACEncContext *s, SingleChannelElement *sce) /* Reconstructed coefficients - needed for distortion measurements */ for (i = 0; i < num_coeffs; i++) sce->prcoeffs[start_coef + i] += QERR[i] != 0.0f ? (sce->prcoeffs[start_coef + i] - QERR[i]) : 0.0f; - abs_pow34_v(P34, &sce->prcoeffs[start_coef], num_coeffs); + s->abs_pow34(P34, &sce->prcoeffs[start_coef], num_coeffs); if (cb_n < RESERVED_BT) cb_p = av_clip(find_min_book(find_max_val(1, num_coeffs, P34), sce->sf_idx[sfb]), cb_min, cb_max); else diff --git a/libavcodec/aacenc_quantization.h b/libavcodec/aacenc_quantization.h index 4250407..fc5a46b 100644 --- a/libavcodec/aacenc_quantization.h +++ b/libavcodec/aacenc_quantization.h @@ -74,10 +74,10 @@ static av_always_inline float quantize_and_encode_band_cost_template( return cost * lambda; } if (!scaled) { - abs_pow34_v(s->scoefs, in, size); + s->abs_pow34(s->scoefs, in, size); scaled = s->scoefs; } - quantize_bands(s->qcoefs, in, scaled, size, Q34, !BT_UNSIGNED, aac_cb_maxval[cb], ROUNDING); + s->quant_bands(s->qcoefs, in, scaled, size, !BT_UNSIGNED, aac_cb_maxval[cb], Q34, ROUNDING); if (BT_UNSIGNED) { off = 0; } else { diff --git a/libavcodec/aacenc_utils.h b/libavcodec/aacenc_utils.h index ff9188a..bef4c10 100644 --- a/libavcodec/aacenc_utils.h +++ b/libavcodec/aacenc_utils.h @@ -63,7 +63,7 @@ static inline int quant(float coef, const float Q, const float rounding) } static inline void quantize_bands(int *out, const float *in, const float *scaled, - int size, float Q34, int is_signed, int maxval, + int size, int is_signed, int maxval, const float Q34, const float rounding) { int i; diff --git a/libavcodec/x86/Makefile b/libavcodec/x86/Makefile index 522b6c2..1db1137 100644 --- a/libavcodec/x86/Makefile +++ b/libavcodec/x86/Makefile @@ -42,6 +42,7 @@ OBJS-$(CONFIG_XMM_CLOBBER_TEST) += x86/w64xmmtest.o # decoders/encoders OBJS-$(CONFIG_AAC_DECODER) += x86/aacpsdsp_init.o \ x86/sbrdsp_init.o +OBJS-$(CONFIG_AAC_ENCODER) += x86/aacencdsp_init.o OBJS-$(CONFIG_ADPCM_G722_DECODER) += x86/g722dsp_init.o OBJS-$(CONFIG_ADPCM_G722_ENCODER) += x86/g722dsp_init.o OBJS-$(CONFIG_ALAC_DECODER) += x86/alacdsp_init.o @@ -132,6 +133,7 @@ YASM-OBJS-$(CONFIG_VP8DSP) += x86/vp8dsp.o \ # decoders/encoders YASM-OBJS-$(CONFIG_AAC_DECODER) += x86/aacpsdsp.o \ x86/sbrdsp.o +YASM-OBJS-$(CONFIG_AAC_ENCODER) += x86/aacencdsp.o YASM-OBJS-$(CONFIG_ADPCM_G722_DECODER) += x86/g722dsp.o YASM-OBJS-$(CONFIG_ADPCM_G722_ENCODER) += x86/g722dsp.o YASM-OBJS-$(CONFIG_ALAC_DECODER) += x86/alacdsp.o diff --git a/libavcodec/x86/aacencdsp.asm b/libavcodec/x86/aacencdsp.asm new file mode 100644 index 0000000..97af571 --- /dev/null +++ b/libavcodec/x86/aacencdsp.asm @@ -0,0 +1,86 @@ +;****************************************************************************** +;* SIMD optimized AAC encoder DSP functions +;* +;* Copyright (C) 2016 Rostislav Pehlivanov +;* +;* This file is part of FFmpeg. +;* +;* FFmpeg is free software; you can redistribute it and/or +;* modify it under the terms of the GNU Lesser General Public +;* License as published by the Free Software Foundation; either +;* version 2.1 of the License, or (at your option) any later version. +;* +;* FFmpeg is distributed in the hope that it will be useful, +;* but WITHOUT ANY WARRANTY; without even the implied warranty of +;* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +;* Lesser General Public License for more details. +;* +;* You should have received a copy of the GNU Lesser General Public +;* License along with FFmpeg; if not, write to the Free Software +;* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +;****************************************************************************** + +%include "libavutil/x86/x86util.asm" + +SECTION_RODATA + +float_abs_mask: times 4 dd 0x7fffffff + +SECTION .text + +;******************************************************************* +;void ff_abs_pow34(float *out, const float *in, const int size); +;******************************************************************* +INIT_XMM sse +cglobal abs_pow34, 3, 3, 3, out, in, size + mova m2, [float_abs_mask] + shl sizeq, 2 + add inq, sizeq + add outq, sizeq + neg sizeq +.loop: + andps m0, m2, [inq+sizeq] + sqrtps m1, m0 + mulps m0, m1 + sqrtps m0, m0 + mova [outq+sizeq], m0 + add sizeq, mmsize + jl .loop + RET + +;******************************************************************* +;void ff_aac_quantize_bands(int *out, const float *in, const float *scaled, +; int size, int is_signed, int maxval, const float Q34, +; const float rounding) +;******************************************************************* +INIT_XMM sse2 +cglobal aac_quantize_bands, 5, 5, 6, out, in, scaled, size, is_signed, maxval, Q34, rounding +%if UNIX64 == 0 + movss m0, Q34m + movss m1, roundingm + cvtsi2ss m3, dword maxvalm +%else + cvtsi2ss m3, maxvald +%endif + shufps m0, m0, 0 + shufps m1, m1, 0 + shufps m3, m3, 0 + shl is_signedd, 31 + movd m4, is_signedd + shufps m4, m4, 0 + shl sized, 2 + add inq, sizeq + add outq, sizeq + add scaledq, sizeq + neg sizeq +.loop: + mulps m2, m0, [scaledq+sizeq] + addps m2, m1 + minps m2, m3 + andps m5, m4, [inq+sizeq] + orps m2, m5 + cvttps2dq m2, m2 + mova [outq+sizeq], m2 + add sizeq, mmsize + jl .loop + RET diff --git a/libavcodec/x86/aacencdsp_init.c b/libavcodec/x86/aacencdsp_init.c new file mode 100644 index 0000000..d761c3c --- /dev/null +++ b/libavcodec/x86/aacencdsp_init.c @@ -0,0 +1,43 @@ +/* + * AAC encoder assembly optimizations + * Copyright (C) 2016 Rostislav Pehlivanov + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config.h" + +#include "libavutil/float_dsp.h" +#include "libavutil/x86/cpu.h" +#include "libavcodec/aacenc.h" + +void ff_abs_pow34_sse(float *out, const float *in, const int size); + +void ff_aac_quantize_bands_sse2(int *out, const float *in, const float *scaled, + int size, int is_signed, int maxval, const float Q34, + const float rounding); + +av_cold void ff_aac_dsp_init_x86(AACEncContext *s) +{ + int cpu_flags = av_get_cpu_flags(); + + if (EXTERNAL_SSE(cpu_flags)) + s->abs_pow34 = ff_abs_pow34_sse; + + if (EXTERNAL_SSE2(cpu_flags)) + s->quant_bands = ff_aac_quantize_bands_sse2; +} -- 2.9.3