From patchwork Sun Apr 16 13:19:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ruyi Ji X-Patchwork-Id: 3429 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.3.129 with SMTP id 123csp920137vsd; Sun, 16 Apr 2017 06:19:56 -0700 (PDT) X-Received: by 10.28.173.66 with SMTP id w63mr3703220wme.76.1492348796184; Sun, 16 Apr 2017 06:19:56 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 15si11858214wrz.309.2017.04.16.06.19.54; Sun, 16 Apr 2017 06:19:56 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8DF7E689742; Sun, 16 Apr 2017 16:19:44 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg0-f67.google.com (mail-pg0-f67.google.com [74.125.83.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2219B688384 for ; Sun, 16 Apr 2017 16:19:37 +0300 (EEST) Received: by mail-pg0-f67.google.com with SMTP id g2so22824979pge.2 for ; Sun, 16 Apr 2017 06:19:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=sn5gBDWsDxIYe07C1p3xcJrvBDC1S44juLUWN3ULFFk=; b=QUIMDjRhCKAJYqzp8+zXQaJmHnfMOA7TEnPCt2p1p6Z5p8Y4JQFsJZkLIRUXvgIUaX QuMMKZDgszLhR6TpPLC0eyjF48B7IgWBT4RQ4S2PCyXHFa0+XJUfv+dHy9E0LmEnohp1 3it+78w/RCpb12ZaWXLnI7L1gjXDRJilB02slrjAHYJBC0yfaGkuJsGOoCH6F//T3tqI J8GRnU0Pt3XvdCK/WbmhlmvTruqc3nZVbmP/HVG9iPxA5qEsp1FKGz+Jq68dXZ7bkOub n04pmG3P8X+HRPu/alaT5//OXS4iD7zuruutw5S9MNbVVy5+NBQ/dFa4fAxbBHAyWylw EJ5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=sn5gBDWsDxIYe07C1p3xcJrvBDC1S44juLUWN3ULFFk=; b=TXjoW1J/4uaoLa4fYl8uIyDpb1ojR8ml2emU93J9c3srMY4xzUIte6o1Hm9PvbB1c0 y1/4qeWeqhW7Mhv5uy3uNkRHSGjCvJIMAMIz9JAE9sAj9Zu9OrZTlmOoYobuDi03rCW2 i99SN5/TPuXcfcbEKghzPV7R3katfbU8UiucZmtk4BWZ2ERtnTy6IxTgewrwMw3+rt6d nye1u+5Kz6I2WPoAbdnGn13Riq9FiNq5TCHVR79gGmNODVPFyaj7rw8CqBKoDYcYLTmp a/lsvUXUYVtz2uuv0Wc9OKb6/fB4TIKLCMgv6/9GQ7Cmou5+gpgzOSWnZaVTSkuN6Z18 3Ekw== X-Gm-Message-State: AN3rC/6aTy909TveMfbrH5Hhf955fC29kRE7z4Gf5JGMXPrfNPCy8k5y dzrgF9abOHA+xQ== X-Received: by 10.84.210.10 with SMTP id z10mr6525984plh.173.1492348782733; Sun, 16 Apr 2017 06:19:42 -0700 (PDT) Received: from jry-ThinkPad-L440.lan ([124.16.139.139]) by smtp.gmail.com with ESMTPSA id l22sm9732023pfi.2.2017.04.16.06.19.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 16 Apr 2017 06:19:42 -0700 (PDT) From: Ruyi Ji To: ffmpeg-devel@ffmpeg.org Date: Sun, 16 Apr 2017 09:19:33 -0400 Message-Id: <1492348773-7880-1-git-send-email-jiruyi1@gmail.com> X-Mailer: git-send-email 1.9.1 Subject: [FFmpeg-devel] [PATCH] this is the gsos qualification task which use psychoacoustic system to detect transients in vorbis encoder. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Ruyi Ji MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Ruyi Ji --- libavcodec/psymodel.c | 1 + libavcodec/vorbis_enc_data.h | 111 +++++++++++++++++++++++++++++++++++++++++++ libavcodec/vorbisenc.c | 60 +++++++++++++++++++++++ 3 files changed, 172 insertions(+) diff --git a/libavcodec/psymodel.c b/libavcodec/psymodel.c index 2b5f111..2e11c48 100644 --- a/libavcodec/psymodel.c +++ b/libavcodec/psymodel.c @@ -62,6 +62,7 @@ av_cold int ff_psy_init(FFPsyContext *ctx, AVCodecContext *avctx, int num_lens, switch (ctx->avctx->codec_id) { case AV_CODEC_ID_AAC: + case AV_CODEC_ID_VORBIS: ctx->model = &ff_aac_psy_model; break; } diff --git a/libavcodec/vorbis_enc_data.h b/libavcodec/vorbis_enc_data.h index a51aaec..5102d30 100644 --- a/libavcodec/vorbis_enc_data.h +++ b/libavcodec/vorbis_enc_data.h @@ -501,4 +501,115 @@ static const struct { { 3, 2, 3, { -1, 12, 13, 14 } }, }; + +static const uint8_t swb_size_128_96[] = { + 4, 4, 4, 4, 4, 4, 8, 8, 8, 16, 28, 36 +}; + +static const uint8_t swb_size_128_64[] = { + 4, 4, 4, 4, 4, 4, 8, 8, 8, 16, 28, 36 +}; + +static const uint8_t swb_size_128_48[] = { + 4, 4, 4, 4, 4, 8, 8, 8, 12, 12, 12, 16, 16, 16 +}; + +static const uint8_t swb_size_128_24[] = { + 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 12, 12, 16, 16, 20 +}; + +static const uint8_t swb_size_128_16[] = { + 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 12, 12, 16, 20, 20 +}; + +static const uint8_t swb_size_128_8[] = { + 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 12, 16, 20, 20 +}; + +static const uint8_t swb_size_1024_96[] = { + 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 8, + 12, 12, 12, 12, 12, 16, 16, 24, 28, 36, 44, + 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64 +}; + +static const uint8_t swb_size_1024_64[] = { + 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, + 12, 12, 12, 16, 16, 16, 20, 24, 24, 28, 36, + 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40 +}; + +static const uint8_t swb_size_1024_48[] = { + 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 8, 8, 8, + 12, 12, 12, 12, 16, 16, 20, 20, 24, 24, 28, 28, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 96 +}; + +static const uint8_t swb_size_1024_32[] = { + 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 8, 8, 8, + 12, 12, 12, 12, 16, 16, 20, 20, 24, 24, 28, 28, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32 +}; + +static const uint8_t swb_size_1024_24[] = { + 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, + 12, 12, 12, 12, 16, 16, 16, 20, 20, 24, 24, 28, 28, + 32, 36, 36, 40, 44, 48, 52, 52, 64, 64, 64, 64, 64 +}; + +static const uint8_t swb_size_1024_16[] = { + 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, + 12, 12, 12, 12, 12, 12, 12, 12, 12, 16, 16, 16, 16, 20, 20, 20, 24, 24, 28, 28, + 32, 36, 40, 40, 44, 48, 52, 56, 60, 64, 64, 64 +}; + +static const uint8_t swb_size_1024_8[] = { + 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, + 16, 16, 16, 16, 16, 16, 16, 20, 20, 20, 20, 24, 24, 24, 28, 28, + 32, 36, 36, 40, 44, 48, 52, 56, 60, 64, 80 +}; + +const uint8_t *ff_vorbis_swb_size_128[] = { + swb_size_128_96, swb_size_128_96, swb_size_128_64, + swb_size_128_48, swb_size_128_48, swb_size_128_48, + swb_size_128_24, swb_size_128_24, swb_size_128_16, + swb_size_128_16, swb_size_128_16, swb_size_128_8, + swb_size_128_8 +}; + +const uint8_t *ff_vorbis_swb_size_1024[] = { + swb_size_1024_96, swb_size_1024_96, swb_size_1024_64, + swb_size_1024_48, swb_size_1024_48, swb_size_1024_32, + swb_size_1024_24, swb_size_1024_24, swb_size_1024_16, + swb_size_1024_16, swb_size_1024_16, swb_size_1024_8, + swb_size_1024_8 +}; + +const int ff_vorbis_swb_size_128_len = FF_ARRAY_ELEMS(ff_vorbis_swb_size_128); +const int ff_vorbis_swb_size_1024_len = FF_ARRAY_ELEMS(ff_vorbis_swb_size_1024); + +/* duplicated from avpriv_mpeg4audio_sample_rates to avoid shared build + * failures */ +static const int mpeg4audio_sample_rates[16] = { + 96000, 88200, 64000, 48000, 44100, 32000, + 24000, 22050, 16000, 12000, 11025, 8000, 7350 +}; + +enum WindowSequence { + ONLY_LONG_SEQUENCE, + LONG_START_SEQUENCE, + EIGHT_SHORT_SEQUENCE, + LONG_STOP_SEQUENCE, +}; + +const uint8_t ff_vorbis_num_swb_1024[] = { + 41, 41, 47, 49, 49, 51, 47, 47, 43, 43, 43, 40, 40 +}; + +const uint8_t ff_vorbis_num_swb_128[] = { + 12, 12, 12, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15 +}; + + + #endif /* AVCODEC_VORBIS_ENC_DATA_H */ diff --git a/libavcodec/vorbisenc.c b/libavcodec/vorbisenc.c index 2974ca2..9f1adf8 100644 --- a/libavcodec/vorbisenc.c +++ b/libavcodec/vorbisenc.c @@ -32,6 +32,7 @@ #include "mathops.h" #include "vorbis.h" #include "vorbis_enc_data.h" +#include "psymodel.h" #define BITSTREAM_WRITER_LE #include "put_bits.h" @@ -126,6 +127,10 @@ typedef struct vorbis_enc_context { vorbis_enc_mode *modes; int64_t next_pts; + //stuff concerned with psymodel + FFPsyContext psy; + struct FFPsyPreprocessContext * psypp; + enum WindowSequence window_sequence[MAX_CHANNELS]; } vorbis_enc_context; #define MAX_CHANNELS 2 @@ -1029,6 +1034,35 @@ static int vorbis_encode_frame(AVCodecContext *avctx, AVPacket *avpkt, PutBitContext pb; int i, ret; + float *samples2, *la, *overlap; + int start_ch, ch, chans, cur_channel; + FFPsyWindowInfo windows[MAX_CHANNELS]; + + if (!avctx->frame_number) + return 0; + + if (venc->psypp) + ff_psy_preprocess(venc->psypp, audio, venc->channels); + + start_ch = 0; + cur_channel = 0; + for (i = 0; i < venc->channels - 1; i++) { + FFPsyWindowInfo *wi = windows + start_ch; + chans = 2; + for (ch = 0; ch < chans; ch ++) { + cur_channel = start_ch + ch; + overlap = &audio[cur_channel][0]; + samples2 = overlap + 1024; + la = samples2 + (448 + 64) + if (!frame) + la = NULL; + wi[ch] = venc->psy.model->window(&venc->psy, samples2, la, cur_channel, venc->window_sequence[0]); + venc->window_sequence[1] = venc->window_sequence[0]; + venc->window_sequence[0] = wi[ch].window_type[0]; + } + start_ch += chans; + } + if (!apply_window_and_mdct(venc, audio, samples)) return 0; samples = 1 << (venc->log2_blocksize[0] - 1); @@ -1159,6 +1193,11 @@ static av_cold int vorbis_encode_close(AVCodecContext *avctx) ff_mdct_end(&venc->mdct[0]); ff_mdct_end(&venc->mdct[1]); + ff_psy_end(venc->psy); + + if (venc->psypp) + ff_psy_preprocess_end(venc->psypp); + av_freep(&avctx->extradata); return 0 ; @@ -1169,6 +1208,11 @@ static av_cold int vorbis_encode_init(AVCodecContext *avctx) vorbis_enc_context *venc = avctx->priv_data; int ret; + const uint8_t *sizes[MAX_CHANNELS]; + uint8_t grouping[MAX_CHANNELS]; + int lengths[MAX_CHANNELS]; + int samplerate_index; + if (avctx->channels != 2) { av_log(avctx, AV_LOG_ERROR, "Current FFmpeg Vorbis encoder only supports 2 channels.\n"); return -1; @@ -1190,6 +1234,22 @@ static av_cold int vorbis_encode_init(AVCodecContext *avctx) avctx->frame_size = 1 << (venc->log2_blocksize[0] - 1); + for (samplerate_index = 0; samplerate_index < venc->channels - 1; samplerate_index ++) + if (avctx->sample_rate == mpeg4audio_sample_rates[samplerate_index]) + break; + + if (samplerate_index == 16 || samplerate_index >= ff_vorbis_swb_size_1024_len || samplerate_index >= ff_vorbis_swb_size_128_len) + av_log(avctx, AV_LOG_ERROR, "Unsupported sample rate %d\n", avctx->sample_rate); + + sizes[0] = ff_vorbis_swb_size_1024[samplerate_index]; + sizes[1] = ff_vorbis_swb_size_128[samplerate_index]; + lengths[0] = ff_vorbis_num_swb_1024[samplerate_index]; + lengths[1] = ff_vorbis_num_swb_128[samplerate_index]; + + if ((ret = ff_psy_init(&venc->psy, avctx, 2, sizes, lengths, 1, grouping)) < 0) + goto error; + venc->psypp = ff_psy_preprocess_init(avctx); + return 0; error: vorbis_encode_close(avctx);