From patchwork Wed Oct 3 21:55:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: bananaman255@gmail.com X-Patchwork-Id: 10608 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:ab0:73d2:0:0:0:0:0 with SMTP id m18csp121229uaq; Wed, 3 Oct 2018 14:56:09 -0700 (PDT) X-Google-Smtp-Source: ACcGV62mE1Q7oLP+p+NTEwOp44a1hRhqzZfySGq8owAhTMKks6isb8vnL/8yA7zZty3+KVluQI3S X-Received: by 2002:a1c:dac9:: with SMTP id r192-v6mr2619194wmg.141.1538603768934; Wed, 03 Oct 2018 14:56:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538603768; cv=none; d=google.com; s=arc-20160816; b=rd5lwxiJUw8VkfzAkd1vIh580URQX+tDO8BHxND45uiokDv3FlsPXUD2gDvRleocB4 OBao5+m9ulVwVjBVE3kjUcDP30awrrwwlVbMDUBnqhBWFMYiQuOG7RCGxlCar2PfiQA4 Ge9QxTdCp9yP0f0ymUJabgC5DB2E+JzdJLcXqtaDfZReVXj6EAMVZB01oPxt3AofHcOt 5VHA7/YPRxV5gZ+SuEv/zjKeYm6SvHJQTVw57KrUI4cX5kOVbzQHZItutLEPIzVjvC3i uaFW5wQ+U8vSiVS7nedPSyzU1roZWEC0/70Bm8eAJdPbISABPf0A0OL/RGsY4KjU5FOy KOGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to; bh=Fg2QjSGo2VDF4FgB9w+b62IMRfleFZMTC4PSplP3bCw=; b=Q552zVH11c1SLfuDY2cPhwLprElM/FSZDMzbm55BsVYKTcoIQNnMQYCNxYU2C34LN/ 7Nt9xF+5vOxyww1hFEF2PT6WO70eoSPIGcFQ0q6LzPefVVJ15wxVbhUVOgHrUjSvBPGC /+K+OjLxH49nlcGfv7l2JckrruOoW9B2GgcRL6+uimpVwfmdVo/rBNzZS7jsZ8X8g7Vx 8EHD2jZPf4VgUHLc6skqF9ajblS6j0VCgEnofEyuFrL1sd6CdSM1aFfMa4aSFpKPHwpi r8cDffPxFRbxgZlbRZuqEwZ3HLoFr5fj5dLb3frYzKVSgErkCTVVyTqcwkyeVKwoOVqT boBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=mGw66xxs; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x23-v6si2181378wrd.120.2018.10.03.14.56.08; Wed, 03 Oct 2018 14:56:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=mGw66xxs; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0B6CF68A410; Thu, 4 Oct 2018 00:55:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B3EDA68A410 for ; Thu, 4 Oct 2018 00:55:40 +0300 (EEST) Received: by mail-wr1-f68.google.com with SMTP id y16so7750339wrw.3 for ; Wed, 03 Oct 2018 14:56:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=ivLBXCn+LzF80xbdtQglCeKQkTkVcmd/iU1WYdjGc+c=; b=mGw66xxsj53L0EwmWyCS7dLVbl0R9Pi+UNev1s9xAJnHOH8HKC88ojL4ricwBMH7Og LGi1QIRJR9k0krAgqqtor+GLkFw4SQWDDcWu4rgvI388GIfy5pIP5gicEXMnTAfGA13G 5cIoQEk+zmjrfLej3QDYHpYTjaJ2bO7iwIwBEv3hpai+z8QISKtINOub/cmo3Jjs5da0 0+liPvQen1Ev2eBvn46tXEFih0SU0xCqbf8ZOFvTOozu5Umd2m415qMQPMXH1StquFHa wYU1bMMHOn2Kpms0xEZ9LT/jRUTpAO5pckuDa/zbftc28i61uGY2LbXy3iwpRAEq17bD 9Xvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=ivLBXCn+LzF80xbdtQglCeKQkTkVcmd/iU1WYdjGc+c=; b=a63+oefg5ofv7uZqITxUDeZofckQ6pa6O3jzrYPygbZDj4WrCklLBa+qrYfbEGAPgG osl0SINFQILae7pHgcv7OGOt1kfCAebyvL1vq9drqOkNKtflwAEzCZaQEpqN0GsI+SLj f2HRYoYeoQ+Y0XBvvIXcxKqNEmClYK3QmzpAyRuym5dXsnv5LAvolz1q4yM1lHp74dYi tTMSw4pEdXKZMtUs0ZKMHDrK9X3o5GvcjZK6tp2h5mkRLrOeiV1i0B38NLorakr6/FEV uI/hZHJAQdvqB+2rveZmlq0Q5h6nD1gwJhwk/6wHP9DytOQBVizrB0Ziydnq+0/fMKfh kf2A== X-Gm-Message-State: ABuFfog0hjUEydKHBncwyfNc21tzS+X1UHr0jSHyqE7H6QGSbCpVbzyl HWt5A4mMiwEWwv6CMUdMgfnbFp3V X-Received: by 2002:adf:8382:: with SMTP id 2-v6mr2624201wre.13.1538603759860; Wed, 03 Oct 2018 14:55:59 -0700 (PDT) Received: from localhost.localdomain (117.98.132.37.dynamic.jazztel.es. [37.132.98.117]) by smtp.googlemail.com with ESMTPSA id 195-v6sm4701823wmx.21.2018.10.03.14.55.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 03 Oct 2018 14:55:58 -0700 (PDT) From: bananaman255@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 3 Oct 2018 23:55:32 +0200 Message-Id: <20181003215532.3672-1-bananaman255@gmail.com> X-Mailer: git-send-email 2.11.0.windows.3 Subject: [FFmpeg-devel] [PATCH] avcodec/wmaprodec: improve WMAPRO/XMA gapless output X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: bnnm MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: bnnm Improves trac issue #6722. Fixes truncated XMA output (was missing 128 samples) and applies bitstream gapless info (partially for XMA, fully for WMAPRO). Applying XMA end_skip would require some extra changes in the XMA multi-stream handling, so end samples are slightly bigger than what should be. Compared to MS's decoders, WMAPRO in XWMA are correct, while in ASF (.wma) don't seem to read the last frame, so output is around 128~512 samples smaller (this happens even with gapless disabled and affects other ASF codecs). Signed-off-by: bnnm --- libavcodec/wmaprodec.c | 92 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 66 insertions(+), 26 deletions(-) diff --git a/libavcodec/wmaprodec.c b/libavcodec/wmaprodec.c index 9439bfa771..e9cc5d4ed9 100644 --- a/libavcodec/wmaprodec.c +++ b/libavcodec/wmaprodec.c @@ -216,9 +216,9 @@ typedef struct WMAProDecodeCtx { GetBitContext gb; ///< bitstream reader context int buf_bit_size; ///< buffer size in bits uint8_t drc_gain; ///< gain for the DRC tool - int8_t skip_frame; ///< skip output step int8_t parsed_all_subframes; ///< all subframes decoded? uint8_t skip_packets; ///< packets to skip to find next packet in a stream (XMA1/2) + int8_t eof_done; ///< set when EOF reached and extra subframe is written (XMA1/2) /* subframe/block decode state */ int16_t subframe_len; ///< current subframe length @@ -379,12 +379,6 @@ static av_cold int decode_init(WMAProDecodeCtx *s, AVCodecContext *avctx, int nu return AVERROR_PATCHWELCOME; } - /** frame info */ - if (avctx->codec_id != AV_CODEC_ID_WMAPRO) - s->skip_frame = 0; - else - s->skip_frame = 1; /* skip first frame */ - s->packet_loss = 1; s->len_prefix = (s->decode_flags & 0x40); @@ -1450,21 +1444,34 @@ static int decode_frame(WMAProDecodeCtx *s, AVFrame *frame, int *got_frame_ptr) ff_dlog(s->avctx, "drc_gain %i\n", s->drc_gain); } - /** no idea what these are for, might be the number of samples - that need to be skipped at the beginning or end of a stream */ + /** read encoder delay/padding (gapless) info */ if (get_bits1(gb)) { - int av_unused skip; + int start_skip, end_skip; + - /** usually true for the first frame */ + /** usually true for the first frame and equal to 1 frame */ if (get_bits1(gb)) { - skip = get_bits(gb, av_log2(s->samples_per_frame * 2)); - ff_dlog(s->avctx, "start skip: %i\n", skip); + start_skip = get_bits(gb, av_log2(s->samples_per_frame * 2)); + start_skip = FFMIN(start_skip, s->samples_per_frame); + + /* must add for XMA to respect starting skip_samples */ + s->avctx->internal->skip_samples += start_skip; } - /** sometimes true for the last frame */ + /** usually true for the last frame and less than 1 frame */ if (get_bits1(gb)) { - skip = get_bits(gb, av_log2(s->samples_per_frame * 2)); - ff_dlog(s->avctx, "end skip: %i\n", skip); + end_skip = get_bits(gb, av_log2(s->samples_per_frame * 2)); + end_skip = FFMIN(end_skip, s->samples_per_frame); + + if (s->avctx->codec_id == AV_CODEC_ID_WMAPRO) { + frame->nb_samples = s->samples_per_frame - end_skip; + } + + //TODO XMA end skip (needs changes in multistream's handling) + /* for XMA this skip seems to consider last frame + extra subframe, + * and (unlike WMAPRO?) files may have only 1 packet. Last packet + * nb_samples would be: 512 + 128 - start_skip - end_skip + * (theoretically skips happen anywhere, test more_frames too) */ } } @@ -1500,13 +1507,9 @@ static int decode_frame(WMAProDecodeCtx *s, AVFrame *frame, int *got_frame_ptr) s->samples_per_frame * sizeof(*s->channel[i].out) >> 1); } - if (s->skip_frame) { - s->skip_frame = 0; - *got_frame_ptr = 0; - av_frame_unref(frame); - } else { - *got_frame_ptr = 1; - } + /** frame may be partially discarded with gapless info in the bitstream */ + *got_frame_ptr = 1; + if (s->len_prefix) { if (len != (get_bits_count(gb) - s->frame_offset) + 2) { @@ -1609,7 +1612,33 @@ static int decode_packet(AVCodecContext *avctx, WMAProDecodeCtx *s, *got_frame_ptr = 0; - if (s->packet_done || s->packet_loss) { + if (!buf_size) { + AVFrame *frame = data; + int i; + + /** XMA must output one extra subframe after stream end with + * remaining samples (WMAPRO encoder adds padding instead). */ + s->packet_done = 0; + if (s->eof_done || avctx->codec_id == AV_CODEC_ID_WMAPRO) + return 0; + + /** clean output buffer and copy last IMCDT samples */ + for (i = 0; i < s->nb_channels; i++) { + memset(frame->extended_data[i], 0, + s->samples_per_frame * sizeof(*s->channel[i].out)); + + memcpy(frame->extended_data[i], s->channel[i].out, + s->samples_per_frame * sizeof(*s->channel[i].out) >> 1); + } + /* TODO: should output 128 samples only, but needs changes in + * multi-stream handling, so currently writes full 512 samples. */ + + s->eof_done = 1; + s->packet_done = 1; + *got_frame_ptr = 1; + return 0; + } + else if (s->packet_done || s->packet_loss) { s->packet_done = 0; /** sanity check for the buffer length */ @@ -1896,6 +1925,12 @@ static av_cold int xma_decode_init(AVCodecContext *avctx) start_channels += s->xma[i].nb_channels; } + /** XMA seems to require this to create gapless files (WMAPRO encoder + * pads files with silence, while XMA can encode any nb_samples). + * This value seems to counter the extra subframe output at EOF + * (or perhaps, XMA decoder should output 64 samples earlier?) */ + avctx->internal->skip_samples = 128/2; + return ret; } @@ -1922,6 +1957,7 @@ static void flush(WMAProDecodeCtx *s) sizeof(*s->channel[i].out)); s->packet_loss = 1; s->skip_packets = 0; + s->eof_done = 0; } @@ -1934,6 +1970,8 @@ static void wmapro_flush(AVCodecContext *avctx) WMAProDecodeCtx *s = avctx->priv_data; flush(s); + + avctx->internal->skip_samples = 0; } static void xma_flush(AVCodecContext *avctx) @@ -1946,6 +1984,8 @@ static void xma_flush(AVCodecContext *avctx) memset(s->offset, 0, sizeof(s->offset)); s->current_stream = 0; + + avctx->internal->skip_samples = 128/2; } @@ -1976,7 +2016,7 @@ AVCodec ff_xma1_decoder = { .init = xma_decode_init, .close = xma_decode_end, .decode = xma_decode_packet, - .capabilities = AV_CODEC_CAP_SUBFRAMES | AV_CODEC_CAP_DR1, + .capabilities = AV_CODEC_CAP_SUBFRAMES | AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY, .sample_fmts = (const enum AVSampleFormat[]) { AV_SAMPLE_FMT_FLTP, AV_SAMPLE_FMT_NONE }, }; @@ -1991,7 +2031,7 @@ AVCodec ff_xma2_decoder = { .close = xma_decode_end, .decode = xma_decode_packet, .flush = xma_flush, - .capabilities = AV_CODEC_CAP_SUBFRAMES | AV_CODEC_CAP_DR1, + .capabilities = AV_CODEC_CAP_SUBFRAMES | AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY, .sample_fmts = (const enum AVSampleFormat[]) { AV_SAMPLE_FMT_FLTP, AV_SAMPLE_FMT_NONE }, };