From patchwork Tue Oct 9 23:15:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: bananaman255@gmail.com X-Patchwork-Id: 10625 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:ab0:73d2:0:0:0:0:0 with SMTP id m18csp116978uaq; Tue, 9 Oct 2018 16:15:58 -0700 (PDT) X-Google-Smtp-Source: ACcGV6344OEvufPOocPbRPb4mXG5i5GD0tCxVY1/8ETLGIt1s6YivA1tSXA3KQ+JMi0uqoCvn8M+ X-Received: by 2002:a1c:1b91:: with SMTP id b139-v6mr3461090wmb.147.1539126958459; Tue, 09 Oct 2018 16:15:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539126958; cv=none; d=google.com; s=arc-20160816; b=vE6zTnFPTdSH/RfaN3depJORmIMRzsn1irPMoc+cTZrXXi9uLlCpGQo7uDX32FweO9 NL3UTJNVuL+1MQ/vKU9AISaB+w/UXwzE4LFO73KiLhz4Oi0KC6SyjSkmJRN7lG7xyCaK 9SppbnaQ4V/AStJRdelOOuiKT6uhbj8BZJmpoVm9pYXG+wg35+zVhtETuYpMu6EV1Re4 VpOooKzpltpxjSlX1KG7TFfbLCXrbFMCDNDKiNNB4yJDvfA47GhYy1v1QE8dwszVas2U IkSVW2Wk0cG1IUUq87NJOKNfySVuIOBAiJ0lSj1ILYVVW18l+Og28LBQ67NOmUMNoMhD /6NQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to; bh=BbQKcvMJ4Laq5Dk3PvarMXo1j6CyKZtz1Ot2WXTQKhE=; b=xT5O+2RnoWshlDAuIUwkWoz3g7BxXoaX5NmX6OWPefDD4eABDl80oKvUfJ3BnKY5GV QH4iqM0YCgLNt3rYRCs9WsXOUTdvaiYbcojSznwrb5jtRH734h/z89EXZTmnMSr5VTz4 oqQrTOpycbw4rU0ttddUs/iyGpcHqr531fSS/Od2b0k32yk3TJOv0pQzxA+t+3XAtzO4 CxfO7iNkrMoTE/G/qnAv+KL/Q6Q7nz38VmQ5HaocYFnY0Iv6xENNFyidFX/YsRsOa+h7 xyO0BRqEXnblhfiQueTTJpuP7uxwGZpVb0SeqRNUqUMtdcH/GOZbVJmpgvacuAm53RPV MHCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=QHy82JVB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t143-v6si11758298wmd.37.2018.10.09.16.15.58; Tue, 09 Oct 2018 16:15:58 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=QHy82JVB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8C53B68A118; Wed, 10 Oct 2018 02:15:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 73F88689FC4 for ; Wed, 10 Oct 2018 02:15:28 +0300 (EEST) Received: by mail-wr1-f65.google.com with SMTP id n1-v6so3573847wrt.10 for ; Tue, 09 Oct 2018 16:15:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=H16SrWjOn+T/nT4nKHPWv73ckpXQ+OQLqYxSO7W8vLw=; b=QHy82JVBm58+j5Yu0ei9a+yBwLMLVP1wtM8e0PxbS5HJYEmubYkkPh2NheVeI+l752 Wp3lFgD4cJuC9R3VT5QhF96RIVcOOqGoquWIxswk9oTicu94bebU75ZeculVcfHPsISo GvmQ4V4cT936LSxuEwUmQWWM1mjxC3DYscmlmTDO/X0xgjOgfg0fM+OfCOXuIrMoBTjD Y0svdhXe5eCQJqUv2hFa1bMlUWQpYmx0tGDUASEZ3vd/jQ0Nx4BC5bfnQ5Jcbh5PdbQT 1P4fvyLxnB5VIMWaZ4lfQ+G7LG4/Rxfict4XEWSCt8kF0hh0pFwULclOzaBL1pOC7Olw wuVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=H16SrWjOn+T/nT4nKHPWv73ckpXQ+OQLqYxSO7W8vLw=; b=cpCItPXml53sULSJ6Rhl5GmU+QxJR8BwqE2z1S0/wBIMXLaAwwJWcuMu6dKfz9Jier uO6ojYCgeEQ15YACdy1OfxZXx1MrT9+xg5ypeXb6cAq89k55lPuLeJ3fhYDutzehToCc /kVPyvz/fdy72z8kKJMBMhLtka21YC/+UchJvIj9gzN7mJnKsM8rqaylMqKmJm9vJjF2 Ivm4zRbREVFX4GRZWnXBvPzL2KS+8JSSxCLsZ6+qxTIQLCa5LlnIa1DiaqLgen0dhTyS IxtKqABlvn7y/eHWo6w32uAPsYy/cYk2NzTsRa3Ko5KRLNxMGxEBD/VHqhN8JYg1LuiH 2Yuw== X-Gm-Message-State: ABuFfoiVeoYy5Hx+/SkkO0IMbYOZ+0Vrfj04u0n1anwpkA7Ejjny8kdG /tRXsCT6dyF4SlMLC1zshZ9Dg/6M X-Received: by 2002:a05:6000:1201:: with SMTP id e1mr23350294wrx.157.1539126949973; Tue, 09 Oct 2018 16:15:49 -0700 (PDT) Received: from localhost.localdomain (117.98.132.37.dynamic.jazztel.es. [37.132.98.117]) by smtp.googlemail.com with ESMTPSA id b1-v6sm13513411wrt.43.2018.10.09.16.15.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 09 Oct 2018 16:15:48 -0700 (PDT) From: bananaman255@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 10 Oct 2018 01:15:02 +0200 Message-Id: <20181009231502.5292-1-bananaman255@gmail.com> X-Mailer: git-send-email 2.11.0.windows.3 Subject: [FFmpeg-devel] [PATCH] avcodec/wmaprodec: improve WMAPRO/XMA gapless output X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: bnnm MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: bnnm Improves trac issue #6722. Fixes truncated WMAPRO/XMA output due to delay samples and partially applies bitstream gapless info. This can be tested with files encoding a small nb_samples (like 400), which couldn't output anything due to missing delay at the end. Applying end_skip and partial delay samples would require some extra changes, so files are slightly bigger than what should be. Signed-off-by: bnnm --- libavcodec/wmaprodec.c | 100 ++++++++++++++++++++++++++++++++++++------------- tests/fate/wma.mak | 1 + 2 files changed, 74 insertions(+), 27 deletions(-) diff --git a/libavcodec/wmaprodec.c b/libavcodec/wmaprodec.c index 9439bfa771..e18e2e438f 100644 --- a/libavcodec/wmaprodec.c +++ b/libavcodec/wmaprodec.c @@ -216,9 +216,9 @@ typedef struct WMAProDecodeCtx { GetBitContext gb; ///< bitstream reader context int buf_bit_size; ///< buffer size in bits uint8_t drc_gain; ///< gain for the DRC tool - int8_t skip_frame; ///< skip output step int8_t parsed_all_subframes; ///< all subframes decoded? uint8_t skip_packets; ///< packets to skip to find next packet in a stream (XMA1/2) + int8_t eof_done; ///< set when EOF reached and extra subframe is written (XMA1/2) /* subframe/block decode state */ int16_t subframe_len; ///< current subframe length @@ -379,12 +379,6 @@ static av_cold int decode_init(WMAProDecodeCtx *s, AVCodecContext *avctx, int nu return AVERROR_PATCHWELCOME; } - /** frame info */ - if (avctx->codec_id != AV_CODEC_ID_WMAPRO) - s->skip_frame = 0; - else - s->skip_frame = 1; /* skip first frame */ - s->packet_loss = 1; s->len_prefix = (s->decode_flags & 0x40); @@ -1450,21 +1444,38 @@ static int decode_frame(WMAProDecodeCtx *s, AVFrame *frame, int *got_frame_ptr) ff_dlog(s->avctx, "drc_gain %i\n", s->drc_gain); } - /** no idea what these are for, might be the number of samples - that need to be skipped at the beginning or end of a stream */ + /** read encoder delay/padding (gapless) info */ if (get_bits1(gb)) { - int av_unused skip; + int start_skip, end_skip; + - /** usually true for the first frame */ + /** usually true for the first frame and equal to 1 frame */ if (get_bits1(gb)) { - skip = get_bits(gb, av_log2(s->samples_per_frame * 2)); - ff_dlog(s->avctx, "start skip: %i\n", skip); + start_skip = get_bits(gb, av_log2(s->samples_per_frame * 2)); + start_skip = FFMIN(start_skip, s->samples_per_frame); + + /* must add for XMA to respect starting skip_samples */ + s->avctx->internal->skip_samples += start_skip; } - /** sometimes true for the last frame */ + /** usually true for the last frame and less than 1 frame */ if (get_bits1(gb)) { - skip = get_bits(gb, av_log2(s->samples_per_frame * 2)); - ff_dlog(s->avctx, "end skip: %i\n", skip); + end_skip = get_bits(gb, av_log2(s->samples_per_frame * 2)); + end_skip = FFMIN(end_skip, s->samples_per_frame); + + //TODO fix end_skip (also needs changes in XMA multistream's handling) + /* for XMA/WMAPRO end_skip may span last frame + final delay samples, + * so we'd need some counter to (possibly) skip some in this frame and + * rest after last frame (buf_size 0), so final samples would be: + * samples_per_frame + delay_samples - start_skip - end_skip + * (files with 1 packet are possible so both skips are considered). + * + * WMAPRO 5.1 in XWMA does't do this and audio is padded instead, + * and min packets is 2 (assumes only 1/2ch need it). */ + + if (s->avctx->codec_id == AV_CODEC_ID_WMAPRO && s->nb_channels > 2) { + frame->nb_samples = s->samples_per_frame - end_skip; + } } } @@ -1500,13 +1511,9 @@ static int decode_frame(WMAProDecodeCtx *s, AVFrame *frame, int *got_frame_ptr) s->samples_per_frame * sizeof(*s->channel[i].out) >> 1); } - if (s->skip_frame) { - s->skip_frame = 0; - *got_frame_ptr = 0; - av_frame_unref(frame); - } else { - *got_frame_ptr = 1; - } + /** frame may be partially discarded with gapless info in the bitstream */ + *got_frame_ptr = 1; + if (s->len_prefix) { if (len != (get_bits_count(gb) - s->frame_offset) + 2) { @@ -1609,7 +1616,35 @@ static int decode_packet(AVCodecContext *avctx, WMAProDecodeCtx *s, *got_frame_ptr = 0; - if (s->packet_done || s->packet_loss) { + if (!buf_size) { + AVFrame *frame = data; + int i; + + /** Must output remaining samples after stream end. WMAPRO 5.1 created + * by XWMA encoder don't though; assuming only 1/2ch streams need it. */ + s->packet_done = 0; + if (s->eof_done || s->nb_channels > 2) + return 0; + + /** clean output buffer and copy last IMCDT samples */ + for (i = 0; i < s->nb_channels; i++) { + memset(frame->extended_data[i], 0, + s->samples_per_frame * sizeof(*s->channel[i].out)); + + memcpy(frame->extended_data[i], s->channel[i].out, + s->samples_per_frame * sizeof(*s->channel[i].out) >> 1); + } + + /* TODO: change nb_samples to delay_samples. XMA always uses 128 + * (with 512 samples per frame) and WMAPRO may use 768 (2048 s.p.f.). + * XMA needs changes in multi-stream handling though. */ + + s->eof_done = 1; + s->packet_done = 1; + *got_frame_ptr = 1; + return 0; + } + else if (s->packet_done || s->packet_loss) { s->packet_done = 0; /** sanity check for the buffer length */ @@ -1896,6 +1931,12 @@ static av_cold int xma_decode_init(AVCodecContext *avctx) start_channels += s->xma[i].nb_channels; } + /** XMA seems to require this to create gapless files (WMAPRO encoder + * pads files with silence, while XMA can encode any nb_samples). + * This value seems to counter the extra subframe output at EOF + * (or perhaps, XMA decoder should output 64 samples earlier?) */ + avctx->internal->skip_samples = 128/2; + return ret; } @@ -1922,6 +1963,7 @@ static void flush(WMAProDecodeCtx *s) sizeof(*s->channel[i].out)); s->packet_loss = 1; s->skip_packets = 0; + s->eof_done = 0; } @@ -1934,6 +1976,8 @@ static void wmapro_flush(AVCodecContext *avctx) WMAProDecodeCtx *s = avctx->priv_data; flush(s); + + avctx->internal->skip_samples = 0; } static void xma_flush(AVCodecContext *avctx) @@ -1946,6 +1990,8 @@ static void xma_flush(AVCodecContext *avctx) memset(s->offset, 0, sizeof(s->offset)); s->current_stream = 0; + + avctx->internal->skip_samples = 128/2; } @@ -1961,7 +2007,7 @@ AVCodec ff_wmapro_decoder = { .init = wmapro_decode_init, .close = wmapro_decode_end, .decode = wmapro_decode_packet, - .capabilities = AV_CODEC_CAP_SUBFRAMES | AV_CODEC_CAP_DR1, + .capabilities = AV_CODEC_CAP_SUBFRAMES | AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY, .flush = wmapro_flush, .sample_fmts = (const enum AVSampleFormat[]) { AV_SAMPLE_FMT_FLTP, AV_SAMPLE_FMT_NONE }, @@ -1976,7 +2022,7 @@ AVCodec ff_xma1_decoder = { .init = xma_decode_init, .close = xma_decode_end, .decode = xma_decode_packet, - .capabilities = AV_CODEC_CAP_SUBFRAMES | AV_CODEC_CAP_DR1, + .capabilities = AV_CODEC_CAP_SUBFRAMES | AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY, .sample_fmts = (const enum AVSampleFormat[]) { AV_SAMPLE_FMT_FLTP, AV_SAMPLE_FMT_NONE }, }; @@ -1991,7 +2037,7 @@ AVCodec ff_xma2_decoder = { .close = xma_decode_end, .decode = xma_decode_packet, .flush = xma_flush, - .capabilities = AV_CODEC_CAP_SUBFRAMES | AV_CODEC_CAP_DR1, + .capabilities = AV_CODEC_CAP_SUBFRAMES | AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY, .sample_fmts = (const enum AVSampleFormat[]) { AV_SAMPLE_FMT_FLTP, AV_SAMPLE_FMT_NONE }, }; diff --git a/tests/fate/wma.mak b/tests/fate/wma.mak index 12a8fa989a..9c7ce6fd7d 100644 --- a/tests/fate/wma.mak +++ b/tests/fate/wma.mak @@ -11,6 +11,7 @@ fate-wmapro-5.1: SIZE_TOLERANCE = 24576 FATE_WMAPRO-$(call DEMDEC, MOV, WMAPRO) += fate-wmapro-ism fate-wmapro-ism: CMD = pcm -i $(TARGET_SAMPLES)/isom/vc1-wmapro.ism -vn fate-wmapro-ism: REF = $(SAMPLES)/isom/vc1-wmapro.pcm +fate-wmapro-ism: SIZE_TOLERANCE = 12976 $(FATE_WMAPRO-yes): CMP = oneoff