From patchwork Fri Oct 13 16:59:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wm4 X-Patchwork-Id: 5550 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.90 with SMTP id m26csp903841jah; Fri, 13 Oct 2017 09:59:56 -0700 (PDT) X-Received: by 10.223.148.162 with SMTP id 31mr2176883wrr.149.1507913996530; Fri, 13 Oct 2017 09:59:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1507913996; cv=none; d=google.com; s=arc-20160816; b=PWFMJyd72wSvBpBfS7TpzJUpzB6kZm2375f2W5kD2vBT8UyA5IIW5+emxfPPYxWD3w hR6hPr1VJIfqKtywCMmF2rTkgkm53+z8BFAuVWqusODExF2cnVlEesEzlyocIIj9nVWe 9z0uhd+xmAfQQ3Vn+hs1NDgow2BFc1Jt5e1E3qR1E/ifCuSqQce8UbRMFX1cBtiOtk3R HFwzVRLjHEOWUjsYoNthFQ9fjMIJAPP3pb3M0khsMQBLXrtCNSmn4R/tFr76jwFv7saq rUhGPyfiG0rP7FCw3amZGaucV3Z3EpDLa5YGAjCdqZ+dIRjkf1uLbm0MKpsYvpW5Zl9Z sXcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=MvCvOLXLX7ZyaAyOKGT+xzQ0WM45o+sev0t6oGN+acI=; b=HHx7bySh1CJPGrfvKKF2gWfl6IUTk+3Ldp5y1yrTfCLhewwsdC1HedH2bfxBV2G6ej tH61Sl5Tnuup//991RWth7zLWspwxakK5v2eUOeSna/2FdkxVR8VpY39VJwpyQpuc29s EPwfJaHSZNELrRHm7B0BIJoHCf17PkxL+jwTi5TsFzJMMQZlGuwLywYt+1D4KbnQ2Ben jWwhp0MtCYEd08HzHqGVoUdLHjYPf98jrhuP2l5FESDvIwDXC/00oZTS6vugT0QNX1x8 naO48XU13PxWvn+qtvdLH/R6mub40DbXYwB1XVw9sD1Jwh5VGpFB+KjfOrvszrtG/ltz 4SyA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com header.s=20161025 header.b=Dtqr7E1O; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g2si1242055wmc.39.2017.10.13.09.59.55; Fri, 13 Oct 2017 09:59:56 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com header.s=20161025 header.b=Dtqr7E1O; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5ACF968A104; Fri, 13 Oct 2017 19:59:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0256168A0E7 for ; Fri, 13 Oct 2017 19:59:20 +0300 (EEST) Received: by mail-wm0-f67.google.com with SMTP id q132so22997862wmd.2 for ; Fri, 13 Oct 2017 09:59:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=iCzguK+nguvw9+B54QTw255o0wDvuQ5tH3no0LZ444g=; b=Dtqr7E1OQ8y7vRXXq819mV3AW9C9xIVbnlycviai7Bxk6NK8Aq3Ek9CRuiPMrxg4Wi avZ2j+tTIISkzcuqbr/o0T0onPKng9WLW1egKXbXYdQntKDq2V8NN7PdOVMqu5q7CXVr QgQXM49uhIWIsdinf0HAcxlWwltsiuw2uIkuK7jE9aKGRdc776HcTtz3dVPKRVvotb7m TSXj7T2J9KyeXdYX8wYnbGX6gWPg5O9mHX+IX9++6hMkhnheph1sXaPXkWjAtbo0DCHD E9lQcOvQRZ3eCjFQMn3eslXb2gC95XZXH+B8Et+N4jJqd49SDu80RUvwU9pBekdwaIJL 1lzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=iCzguK+nguvw9+B54QTw255o0wDvuQ5tH3no0LZ444g=; b=f2fsQO8w5aW2fc7wAUchOwXIIz+qRNpglf5dn7sCnflwW4kOknrsjpFD3MhqEuhss1 penpcNIQ8lQ1GLePYT44vTN5AIiA0mbuQtb7phJJc2gnFIUoN7M1WwvultSyuJln01HS CV1h+Fa+w3OQpexB9QPZT8TC1C83tYZOn709q+IQMtlbePKNikbaoQ1CABJuvlGJo848 vrmrfqWN5xPzKBA14PJ0lto9TChVBpVLMPRlJQwnwFw6biPf1rtqg5wnhpYGBBnzao7P TOEEVM5HWPQXCPQ0wyjml2zDxwCjIq0YwcKMqv12w2mqpdG0goliPse4H4ttdXAwN9Cb F6/Q== X-Gm-Message-State: AMCzsaWy6M++CRGyO8U7cnAgfyoS+jkDSExqgU4kcR8OgTSX/1QKcmqs Dq7nQdL5d5l2OkYw233AARw7Cg== X-Google-Smtp-Source: AOwi7QBOjOHSIiiqjIniIqNmONNt4yrm9QqbRiIIRflrIk55DC9HhQ/N5zWMhBnqbvwaPWyiCiWAKg== X-Received: by 10.80.193.26 with SMTP id l26mr2894721edf.97.1507913962712; Fri, 13 Oct 2017 09:59:22 -0700 (PDT) Received: from debian.speedport.ip (p2003006CCD5CDB3359F6019A8DC244E7.dip0.t-ipconnect.de. [2003:6c:cd5c:db33:59f6:19a:8dc2:44e7]) by smtp.googlemail.com with ESMTPSA id a1sm1199243edb.37.2017.10.13.09.59.21 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 13 Oct 2017 09:59:21 -0700 (PDT) From: wm4 To: ffmpeg-devel@ffmpeg.org Date: Fri, 13 Oct 2017 18:59:17 +0200 Message-Id: <20171013165919.7642-4-nfxjfg@googlemail.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20171013165919.7642-1-nfxjfg@googlemail.com> References: <20171013165919.7642-1-nfxjfg@googlemail.com> Subject: [FFmpeg-devel] [PATCH 3/5] decode: add a method for attaching lavc-internal data to frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Anton Khirnov Use the AVFrame.opaque_ref field. The original user's opaque_ref is wrapped in the lavc struct and then unwrapped before the frame is returned to the caller. This new struct will be useful in the following commits. Merges Libav commit 359a8a3e2d1194b52b6c386f94fd0929567dfb67. This adds proper handling of draw_horiz_band over Libav. Also, the wrapped_avframe decoder is adjusted to return the wrapped frame's opaque_ref field to the user (which may or may not be what the user expects). --- libavcodec/decode.c | 96 +++++++++++++++++++++++++++++++++++++++++++- libavcodec/decode.h | 19 +++++++++ libavcodec/h264dec.c | 5 ++- libavcodec/huffyuvdec.c | 3 +- libavcodec/mpegutils.c | 4 +- libavcodec/vp3.c | 3 +- libavcodec/wrapped_avframe.c | 7 ++++ 7 files changed, 129 insertions(+), 8 deletions(-) diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 437b848248..9395cfc43b 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -640,6 +640,26 @@ static int decode_receive_frame_internal(AVCodecContext *avctx, AVFrame *frame) if (ret == AVERROR_EOF) avci->draining_done = 1; + /* unwrap the per-frame decode data and restore the original opaque_ref*/ + if (!ret) { + /* the only case where decode data is not set should be decoders + * that do not call ff_get_buffer() */ + av_assert0((frame->opaque_ref && frame->opaque_ref->size == sizeof(FrameDecodeData)) || + !(avctx->codec->capabilities & AV_CODEC_CAP_DR1)); + + if (frame->opaque_ref) { + FrameDecodeData *fdd; + AVBufferRef *user_opaque_ref; + + fdd = (FrameDecodeData*)frame->opaque_ref->data; + + user_opaque_ref = fdd->user_opaque_ref; + fdd->user_opaque_ref = NULL; + av_buffer_unref(&frame->opaque_ref); + frame->opaque_ref = user_opaque_ref; + } + } + return ret; } @@ -1612,6 +1632,37 @@ static void validate_avframe_allocation(AVCodecContext *avctx, AVFrame *frame) } } +static void decode_data_free(void *opaque, uint8_t *data) +{ + FrameDecodeData *fdd = (FrameDecodeData*)data; + + av_buffer_unref(&fdd->user_opaque_ref); + + av_freep(&fdd); +} + +int ff_attach_decode_data(AVFrame *frame) +{ + AVBufferRef *fdd_buf; + FrameDecodeData *fdd; + + fdd = av_mallocz(sizeof(*fdd)); + if (!fdd) + return AVERROR(ENOMEM); + + fdd_buf = av_buffer_create((uint8_t*)fdd, sizeof(*fdd), decode_data_free, + NULL, AV_BUFFER_FLAG_READONLY); + if (!fdd_buf) { + av_freep(&fdd); + return AVERROR(ENOMEM); + } + + fdd->user_opaque_ref = frame->opaque_ref; + frame->opaque_ref = fdd_buf; + + return 0; +} + static int get_buffer_internal(AVCodecContext *avctx, AVFrame *frame, int flags) { const AVHWAccel *hwaccel = avctx->hwaccel; @@ -1648,8 +1699,14 @@ static int get_buffer_internal(AVCodecContext *avctx, AVFrame *frame, int flags) avctx->sw_pix_fmt = avctx->pix_fmt; ret = avctx->get_buffer2(avctx, frame, flags); - if (ret >= 0) - validate_avframe_allocation(avctx, frame); + if (ret < 0) + goto end; + + validate_avframe_allocation(avctx, frame); + + ret = ff_attach_decode_data(frame); + if (ret < 0) + goto end; end: if (avctx->codec_type == AVMEDIA_TYPE_VIDEO && !override_dimensions && @@ -1757,3 +1814,38 @@ void ff_decode_bsfs_uninit(AVCodecContext *avctx) av_freep(&s->bsfs); s->nb_bsfs = 0; } + +void ff_call_draw_horiz_band(struct AVCodecContext *s, + const AVFrame *src, int offset[AV_NUM_DATA_POINTERS], + int y, int type, int height) +{ + AVFrame *user_frame; + + if (!s->draw_horiz_band) + return; + + user_frame = av_frame_clone(src); + if (!user_frame) { + av_log(s, AV_LOG_ERROR, "draw_horiz_band() failed\n"); + goto done; + } + + av_buffer_unref(&user_frame->opaque_ref); + + if (src->opaque_ref) { + FrameDecodeData *fdd = (FrameDecodeData*)src->opaque_ref->data; + + if (fdd->user_opaque_ref) { + user_frame->opaque_ref = av_buffer_ref(fdd->user_opaque_ref); + if (!user_frame->opaque_ref) { + av_log(s, AV_LOG_ERROR, "draw_horiz_band() failed\n"); + goto done; + } + } + } + + s->draw_horiz_band(s, user_frame, offset, y, type, height); + +done: + av_frame_unref(user_frame); +} diff --git a/libavcodec/decode.h b/libavcodec/decode.h index c9630228dc..9326f1d952 100644 --- a/libavcodec/decode.h +++ b/libavcodec/decode.h @@ -21,8 +21,21 @@ #ifndef AVCODEC_DECODE_H #define AVCODEC_DECODE_H +#include "libavutil/buffer.h" + #include "avcodec.h" +/** + * This struct stores per-frame lavc-internal data and is attached to it via + * opaque_ref. + */ +typedef struct FrameDecodeData { + /** + * The original user-set opaque_ref. + */ + AVBufferRef *user_opaque_ref; +} FrameDecodeData; + /** * Called by decoders to get the next packet for decoding. * @@ -36,4 +49,10 @@ int ff_decode_get_packet(AVCodecContext *avctx, AVPacket *pkt); void ff_decode_bsfs_uninit(AVCodecContext *avctx); +void ff_call_draw_horiz_band(struct AVCodecContext *s, + const AVFrame *src, int offset[AV_NUM_DATA_POINTERS], + int y, int type, int height); + +int ff_attach_decode_data(AVFrame *frame); + #endif /* AVCODEC_DECODE_H */ diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index f29c3f9048..318899674f 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -39,6 +39,7 @@ #include "cabac_functions.h" #include "error_resilience.h" #include "avcodec.h" +#include "decode.h" #include "h264.h" #include "h264dec.h" #include "h2645_parse.h" @@ -129,8 +130,8 @@ void ff_h264_draw_horiz_band(const H264Context *h, H264SliceContext *sl, emms_c(); - avctx->draw_horiz_band(avctx, src, offset, - y, h->picture_structure, height); + ff_call_draw_horiz_band(avctx, src, offset, + y, h->picture_structure, height); } } diff --git a/libavcodec/huffyuvdec.c b/libavcodec/huffyuvdec.c index 979c4b9d5c..13ddad45f0 100644 --- a/libavcodec/huffyuvdec.c +++ b/libavcodec/huffyuvdec.c @@ -33,6 +33,7 @@ #define UNCHECKED_BITSTREAM_READER 1 #include "avcodec.h" +#include "decode.h" #include "get_bits.h" #include "huffyuv.h" #include "huffyuvdsp.h" @@ -874,7 +875,7 @@ static void draw_slice(HYuvContext *s, AVFrame *frame, int y) offset[i] = 0; emms_c(); - s->avctx->draw_horiz_band(s->avctx, frame, offset, y, 3, h); + ff_call_draw_horiz_band(s->avctx, frame, offset, y, 3, h); s->last_slice_end = y + h; } diff --git a/libavcodec/mpegutils.c b/libavcodec/mpegutils.c index 62cc36aa6e..8221c3e0cc 100644 --- a/libavcodec/mpegutils.c +++ b/libavcodec/mpegutils.c @@ -25,6 +25,7 @@ #include "libavutil/pixdesc.h" #include "avcodec.h" +#include "decode.h" #include "mpegutils.h" void ff_draw_horiz_band(AVCodecContext *avctx, @@ -74,7 +75,6 @@ void ff_draw_horiz_band(AVCodecContext *avctx, emms_c(); - avctx->draw_horiz_band(avctx, src, offset, - y, picture_structure, h); + ff_call_draw_horiz_band(avctx, src, offset, y, picture_structure, h); } } diff --git a/libavcodec/vp3.c b/libavcodec/vp3.c index f167acf4ee..7e4333c016 100644 --- a/libavcodec/vp3.c +++ b/libavcodec/vp3.c @@ -36,6 +36,7 @@ #include "libavutil/imgutils.h" #include "avcodec.h" +#include "decode.h" #include "get_bits.h" #include "hpeldsp.h" #include "internal.h" @@ -1457,7 +1458,7 @@ static void vp3_draw_horiz_band(Vp3DecodeContext *s, int y) offset[i] = 0; emms_c(); - s->avctx->draw_horiz_band(s->avctx, s->current_frame.f, offset, y, 3, h); + ff_call_draw_horiz_band(s->avctx, s->current_frame.f, offset, y, 3, h); } /** diff --git a/libavcodec/wrapped_avframe.c b/libavcodec/wrapped_avframe.c index 5f88a668b9..85ff32d13a 100644 --- a/libavcodec/wrapped_avframe.c +++ b/libavcodec/wrapped_avframe.c @@ -25,6 +25,7 @@ */ #include "avcodec.h" +#include "decode.h" #include "internal.h" #include "libavutil/internal.h" @@ -98,6 +99,12 @@ static int wrapped_avframe_decode(AVCodecContext *avctx, void *data, av_frame_move_ref(out, in); + err = ff_attach_decode_data(out); + if (err < 0) { + av_frame_unref(out); + return err; + } + *got_frame = 1; return 0; }