From patchwork Mon Jul 24 19:08:49 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jorge Ramirez-Ortiz X-Patchwork-Id: 4445 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.1.76 with SMTP id 73csp4583032vsb; Mon, 24 Jul 2017 12:09:49 -0700 (PDT) X-Received: by 10.28.208.78 with SMTP id h75mr5255634wmg.119.1500923389432; Mon, 24 Jul 2017 12:09:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1500923389; cv=none; d=google.com; s=arc-20160816; b=Zk+jgh9KXlSLm3Ki2Btc/GlEYHn9frHl+e5KkZwyA539z6hRgAF4r1bVnuOqOmpW3b KBLFoUUSrqSeEQ/pp40/M1ZTM6lG7ghzSQ/xS0M5A6pdlSdgb97YcDjzjwtVY22IyKJg hKL4Yp2fzsySgS/69dDQyR4RM8tKbUeOQk4sr27pHccvz58RPUJ7HdNORypYmGrcmFZC 6bun2Uk2/1Igm1JZ+MCxHdmHa0eqmTyXz9etBaM1V5HgK0x0ZSDhTO3aWDPvTv3/crDz onK2fr5mdR9nslZxhZ4z2yJfs8s/z07O5ij79jk0XOTwztsCpw5cLpR3QtbXnA0wFFc3 zdqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=VDxD/6JeV3cpBdnV1MCAXVNlsrU1UPcZmozrPppktTI=; b=IR4ieeZo2jHjDd3YFDfJF+R6AuWIpGf9pFQMxms+dT1GNMSS2c9Uy9m4LWx0ifUmP/ 3x3z477uRiYii0alcc1NMq8V/kKd3M3WqvQSHQjXOHf8j61AyCyKpqHKQ3RwL+fR9iMc 7DdvvFK6jsnfoFmwGiHEMjTybpdkflS1fiotEBp2L7UKXuZFiATYRzsuSWaTuKpPE8io +mk5YutD6VIZdFF8MElK/+J+BvJt5mdVz1gTBX6v89FkZrAGS0HUjykcbmxJ/+PWV1p7 JEi7EFD9qq+42lSRWHxFLkdEXnDoTnWmiVCyqZtgdKFvoCJwsTUURSz3+0sQiMRmoHuU 3nJg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.b=KKrQbD07; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 71si13096116wrl.88.2017.07.24.12.09.48; Mon, 24 Jul 2017 12:09:49 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.b=KKrQbD07; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 275FA689AF3; Mon, 24 Jul 2017 22:09:05 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f181.google.com (mail-wr0-f181.google.com [209.85.128.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AD690689AF3 for ; Mon, 24 Jul 2017 22:08:58 +0300 (EEST) Received: by mail-wr0-f181.google.com with SMTP id k71so56480642wrc.2 for ; Mon, 24 Jul 2017 12:09:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=E1j82YYz/1eTr6Pf48RB26PC2ErAv9YTI4IKUfr3+HQ=; b=KKrQbD076zJqtONPXXkgodqq6Ji3ZPCumDYHRilb6YcchLZSN5s2OqBX2glk50tSy0 bxcMlPmkwJI39qris6qhWlYscaInA1vqMlUxKjoKZC2DkxPyvalLdBnIUHET2111k2Yv B9n3ZVqm6g55EuA8xweV22ueT0V8nQT1Ce/2E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=E1j82YYz/1eTr6Pf48RB26PC2ErAv9YTI4IKUfr3+HQ=; b=mdytlAYwG1ym+Er3ES8en0XiqNA/LIFzCWZ2tqzA2RX0puYoSA4Xv46pwGrzOkHHoa t6etlcC3DfnSV0P/VUA1MpNyvuLggpQ+AszSIHh1SPFP2vt98fw/w0l9WMzTpg1aObVC 3wLV7UJH6ofm5P5GAzF8SIqf+ycFj25ivE64LMzZaVd3GxlamGN7BTCjSy70Q/OKwui5 zj+ktuTV7Sz6myvs6vKmK7/e74tvBwE5+K5ambWxw47hDALsZQzeeNuZ2EOBnYmgZ2x+ PYX1rMyiLFl2U6STfPEWwSf/d1CPotDbTbz/akM4mSs8lmkmcggp6WZpMA1giHbfUtws dgvQ== X-Gm-Message-State: AIVw113gG9JgLnZ9aiLmVy+u3IKwkM5DLkwhj8VwAYbuWnOZgDjfK1Bw mYRFk91mkNi15PPC X-Received: by 10.223.158.133 with SMTP id a5mr16386039wrf.188.1500923346697; Mon, 24 Jul 2017 12:09:06 -0700 (PDT) Received: from igloo.80.58.61.254 (170.red-81-33-164.dynamicip.rima-tde.net. [81.33.164.170]) by smtp.gmail.com with ESMTPSA id a14sm8793353wma.42.2017.07.24.12.09.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 24 Jul 2017 12:09:04 -0700 (PDT) From: Jorge Ramirez-Ortiz To: jorge.ramirez-ortiz@linaro.org, aballier@gentoo.org, ffmpeg-devel@ffmpeg.org Date: Mon, 24 Jul 2017 21:08:49 +0200 Message-Id: <1500923329-2510-5-git-send-email-jorge.ramirez-ortiz@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1500923329-2510-1-git-send-email-jorge.ramirez-ortiz@linaro.org> References: <1500923329-2510-1-git-send-email-jorge.ramirez-ortiz@linaro.org> Subject: [FFmpeg-devel] [PATCH 4/4] libavcodec/dev: v4l2: add support for v4l2 mem2mem codecs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: nicolas.dechesne@linaro.org, stanimir.varbanov@linaro.org MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Alexis Ballier This patchset enhances Alexis Ballier's original patch and validates it using Qualcomm's Venus hardware (driver recently landed upstream [1]). This has been tested on Qualcomm's DragonBoard 410c and 820c ffplay tested video decoders: - h264, - vp8 - mpeg4 Some of the changes introduced: - v4l2: some cleanup of the code. - v4l2: some cleaup before upstreaming. - v4l2: follow the new decode api. - v4l2: fix display size for NV12 output pool. - v4l2: handle EOS. - v4l2: fix vp8 and mpeg4 decoding. - v4l2: generate EOF on dequeue errors. - v4l2: h264_mp4toannexb filtering. [1] https://lwn.net/Articles/697956/ Reviewed-by: Jorge Ramirez Reviewed-by: Alexis Ballier Tested-by: Jorge Ramirez --- Changelog | 3 +- configure | 17 +- libavcodec/Makefile | 15 +- libavcodec/allcodecs.c | 7 + libavcodec/v4l2-buffers.c | 633 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/v4l2-buffers.h | 247 ++++++++++++++++ libavcodec/v4l2-common.c | 48 ++-- libavcodec/v4l2-common.h | 4 +- libavcodec/v4l2_m2m.c | 358 ++++++++++++++++++++++++ libavcodec/v4l2_m2m.h | 69 +++++ libavcodec/v4l2_m2m_avcodec.h | 32 +++ libavcodec/v4l2_m2m_dec.c | 244 ++++++++++++++++ libavcodec/v4l2_m2m_enc.c | 251 +++++++++++++++++ 13 files changed, 1897 insertions(+), 31 deletions(-) create mode 100644 libavcodec/v4l2-buffers.c create mode 100644 libavcodec/v4l2-buffers.h create mode 100644 libavcodec/v4l2_m2m.c create mode 100644 libavcodec/v4l2_m2m.h create mode 100644 libavcodec/v4l2_m2m_avcodec.h create mode 100644 libavcodec/v4l2_m2m_dec.c create mode 100644 libavcodec/v4l2_m2m_enc.c diff --git a/Changelog b/Changelog index 187ae79..d31de07 100644 --- a/Changelog +++ b/Changelog @@ -29,6 +29,7 @@ version : - limiter video filter - libvmaf video filter - Dolby E decoder and SMPTE 337M demuxer +- V4L2 mem2mem HW accelerated codecs support version 3.3: - CrystalHD decoder moved to new decode API @@ -65,7 +66,6 @@ version 3.3: - Intel QSV-accelerated VP8 video decoding - VAAPI-accelerated deinterlacing - version 3.2: - libopenmpt demuxer - tee protocol @@ -105,7 +105,6 @@ version 3.2: - Changed mapping of rtp MIME type G726 to codec g726le. - spec compliant VAAPI/DXVA2 VC-1 decoding of slices in frame-coded images - version 3.1: - DXVA2-accelerated HEVC Main10 decoding - fieldhint filter diff --git a/configure b/configure index 0ebc022..9762f32 100755 --- a/configure +++ b/configure @@ -10,7 +10,6 @@ # Prevent locale nonsense from breaking basic text processing. LC_ALL=C export LC_ALL - # make sure we are running under a compatible shell # try to make this part work with most shells @@ -149,6 +148,7 @@ Component options: --disable-pixelutils disable pixel utils in libavutil Individual component options: + --disable-v4l2_m2m disable V4L2 mem2mem code [autodetect] --disable-everything disable all components listed below --disable-encoder=NAME disable encoder NAME --enable-encoder=NAME enable encoder NAME @@ -1432,6 +1432,7 @@ AVCODEC_COMPONENTS=" AVDEVICE_COMPONENTS=" indevs + v4l2_m2m outdevs " AVFILTER_COMPONENTS=" @@ -2269,10 +2270,12 @@ map 'eval ${v}_inline_deps=inline_asm' $ARCH_EXT_LIST_ARM loongson2_deps="mips" loongson3_deps="mips" v4l2_deps_any="linux_videodev2_h sys_videoio_h" +v4l2_m2m_select="v4l2" mipsfpu_deps="mips" mipsdsp_deps="mips" mipsdspr2_deps="mips" mips32r2_deps="mips" +vc1_v4l2m2m_decoder_deps="v4l2_m2m" mips32r5_deps="mips" mips32r6_deps="mips" mips64r2_deps="mips" @@ -2283,6 +2286,8 @@ mmi_deps="mips" altivec_deps="ppc" dcbzl_deps="ppc" ldbrx_deps="ppc" +vp8_v4l2m2m_decoder_deps="v4l2_m2m" +vp8_v4l2m2m_encoder_deps="v4l2_m2m" ppc4xx_deps="ppc" vsx_deps="altivec" power8_deps="vsx" @@ -2436,11 +2441,15 @@ h261_decoder_select="mpegvideo" h261_encoder_select="aandcttables mpegvideoenc" h263_decoder_select="h263_parser h263dsp mpegvideo qpeldsp" h263_encoder_select="aandcttables h263dsp mpegvideoenc" +h263_v4l2m2m_decoder_deps="v4l2_m2m" +h263_v4l2m2m_encoder_deps="v4l2_m2m" h263i_decoder_select="h263_decoder" h263p_decoder_select="h263_decoder" h263p_encoder_select="h263_encoder" h264_decoder_select="cabac golomb h264chroma h264dsp h264parse h264pred h264qpel videodsp" h264_decoder_suggest="error_resilience" +h264_v4l2m2m_decoder_deps="v4l2_m2m" +h264_v4l2m2m_encoder_deps="v4l2_m2m" hap_decoder_select="snappy texturedsp" hap_encoder_deps="libsnappy" hap_encoder_select="texturedspenc" @@ -2481,6 +2490,7 @@ mpc7_decoder_select="bswapdsp mpegaudiodsp" mpc8_decoder_select="mpegaudiodsp" mpeg_xvmc_decoder_deps="X11_extensions_XvMClib_h" mpeg_xvmc_decoder_select="mpeg2video_decoder" +mpeg1_v4l2m2m_decoder_deps="v4l2_m2m" mpegvideo_decoder_select="mpegvideo" mpeg1video_decoder_select="mpegvideo" mpeg1video_encoder_select="aandcttables mpegvideoenc h263dsp" @@ -2488,6 +2498,8 @@ mpeg2video_decoder_select="mpegvideo" mpeg2video_encoder_select="aandcttables mpegvideoenc h263dsp" mpeg4_decoder_select="h263_decoder mpeg4video_parser" mpeg4_encoder_select="h263_encoder" +mpeg4_v4l2m2m_decoder_deps="v4l2_m2m" +mpeg4_v4l2m2m_encoder_deps="v4l2_m2m" msa1_decoder_select="mss34dsp" mscc_decoder_select="zlib" msmpeg4v1_decoder_select="h263_decoder" @@ -3590,7 +3602,7 @@ done enable_weak audiotoolbox # Enable hwaccels by default. -enable_weak d3d11va dxva2 vaapi vda vdpau videotoolbox_hwaccel xvmc +enable_weak d3d11va dxva2 vaapi v4l2_m2m vda vdpau videotoolbox_hwaccel xvmc enable_weak xlib enable_weak cuda cuvid nvenc vda_framework videotoolbox videotoolbox_encoder @@ -6059,6 +6071,7 @@ check_header linux/fb.h check_header linux/videodev.h check_header linux/videodev2.h check_code cc linux/videodev2.h "struct v4l2_frmsizeenum vfse; vfse.discrete.width = 0;" && enable_safe struct_v4l2_frmivalenum_discrete +check_code cc linux/videodev2.h "int i = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_VIDEO_M2M;" || disable v4l2_m2m check_header sys/videoio.h check_code cc sys/videoio.h "struct v4l2_frmsizeenum vfse; vfse.discrete.width = 0;" && enable_safe struct_v4l2_frmivalenum_discrete diff --git a/libavcodec/Makefile b/libavcodec/Makefile index a594b23..12be85b 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -6,6 +6,7 @@ HEADERS = avcodec.h \ avfft.h \ d3d11va.h \ dirac.h \ + v4l2.h \ dv_profile.h \ dxva2.h \ jni.h \ @@ -101,7 +102,8 @@ OBJS-$(CONFIG_LZF) += lzf.o OBJS-$(CONFIG_MDCT) += mdct_fixed.o mdct_float.o mdct_fixed_32.o OBJS-$(CONFIG_ME_CMP) += me_cmp.o OBJS-$(CONFIG_MEDIACODEC) += mediacodecdec_common.o mediacodec_surface.o mediacodec_wrapper.o mediacodec_sw_buffer.o -OBJS-$(CONFIG_V4L2) += v4l2-common.o +OBJS-$(CONFIG_V4L2) += v4l2-common.o v4l2-buffers.o +OBJS-$(CONFIG_V4L2_M2M) += v4l2_m2m.o OBJS-$(CONFIG_MPEG_ER) += mpeg_er.o OBJS-$(CONFIG_MPEGAUDIO) += mpegaudio.o OBJS-$(CONFIG_MPEGAUDIODSP) += mpegaudiodsp.o \ @@ -320,6 +322,8 @@ OBJS-$(CONFIG_H261_ENCODER) += h261enc.o h261data.o h261.o OBJS-$(CONFIG_H263_DECODER) += h263dec.o h263.o ituh263dec.o \ mpeg4video.o mpeg4videodec.o flvdec.o\ intelh263dec.o h263data.o +OBJS-$(CONFIG_H263_V4L2M2M_DECODER) += v4l2_m2m_dec.o +OBJS-$(CONFIG_H263_V4L2M2M_ENCODER) += v4l2_m2m_enc.o OBJS-$(CONFIG_H263_ENCODER) += mpeg4videoenc.o mpeg4video.o \ h263.o ituh263enc.o flvenc.o h263data.o OBJS-$(CONFIG_H264_DECODER) += h264dec.o h264_cabac.o h264_cavlc.o \ @@ -327,6 +331,8 @@ OBJS-$(CONFIG_H264_DECODER) += h264dec.o h264_cabac.o h264_cavlc.o \ h264_mb.o h264_picture.o \ h264_refs.o h264_sei.o \ h264_slice.o h264data.o +OBJS-$(CONFIG_H264_V4L2M2M_DECODER) += v4l2_m2m_dec.o +OBJS-$(CONFIG_H264_V4L2M2M_ENCODER) += v4l2_m2m_enc.o OBJS-$(CONFIG_H264_CUVID_DECODER) += cuvid.o OBJS-$(CONFIG_H264_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_H264_MMAL_DECODER) += mmaldec.o @@ -419,11 +425,13 @@ OBJS-$(CONFIG_MP3ON4FLOAT_DECODER) += mpegaudiodec_float.o mpeg4audio.o OBJS-$(CONFIG_MPC7_DECODER) += mpc7.o mpc.o OBJS-$(CONFIG_MPC8_DECODER) += mpc8.o mpc.o OBJS-$(CONFIG_MPEGVIDEO_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o +OBJS-$(CONFIG_MPEG1_V4L2M2M_DECODER) += v4l2_m2m_dec.o OBJS-$(CONFIG_MPEG1VIDEO_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o OBJS-$(CONFIG_MPEG1VIDEO_ENCODER) += mpeg12enc.o mpeg12.o OBJS-$(CONFIG_MPEG2_MMAL_DECODER) += mmaldec.o OBJS-$(CONFIG_MPEG2_QSV_DECODER) += qsvdec_other.o OBJS-$(CONFIG_MPEG2_QSV_ENCODER) += qsvenc_mpeg2.o +OBJS-$(CONFIG_MPEG2_V4L2M2M_DECODER) += v4l2_m2m_dec.o OBJS-$(CONFIG_MPEG2VIDEO_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o OBJS-$(CONFIG_MPEG2VIDEO_ENCODER) += mpeg12enc.o mpeg12.o OBJS-$(CONFIG_MPEG2_MEDIACODEC_DECODER) += mediacodecdec.o @@ -431,6 +439,8 @@ OBJS-$(CONFIG_MPEG2_VAAPI_ENCODER) += vaapi_encode_mpeg2.o OBJS-$(CONFIG_MPEG4_DECODER) += xvididct.o OBJS-$(CONFIG_MPEG4_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_MPEG4_OMX_ENCODER) += omx.o +OBJS-$(CONFIG_MPEG4_V4L2M2M_DECODER) += v4l2_m2m_dec.o +OBJS-$(CONFIG_MPEG4_V4L2M2M_ENCODER) += v4l2_m2m_enc.o OBJS-$(CONFIG_MPL2_DECODER) += mpl2dec.o ass.o OBJS-$(CONFIG_MSA1_DECODER) += mss3.o OBJS-$(CONFIG_MSCC_DECODER) += mscc.o @@ -503,6 +513,7 @@ OBJS-$(CONFIG_RALF_DECODER) += ralf.o OBJS-$(CONFIG_RAWVIDEO_DECODER) += rawdec.o OBJS-$(CONFIG_RAWVIDEO_ENCODER) += rawenc.o OBJS-$(CONFIG_REALTEXT_DECODER) += realtextdec.o ass.o +OBJS-$(CONFIG_VC1_V4L2M2M_DECODER) += v4l2_m2m_dec.o OBJS-$(CONFIG_RL2_DECODER) += rl2.o OBJS-$(CONFIG_ROQ_DECODER) += roqvideodec.o roqvideo.o OBJS-$(CONFIG_ROQ_ENCODER) += roqvideoenc.o roqvideo.o elbg.o @@ -518,6 +529,8 @@ OBJS-$(CONFIG_RV30_DECODER) += rv30.o rv34.o rv30dsp.o OBJS-$(CONFIG_RV40_DECODER) += rv40.o rv34.o rv40dsp.o OBJS-$(CONFIG_SAMI_DECODER) += samidec.o ass.o htmlsubtitles.o OBJS-$(CONFIG_S302M_DECODER) += s302m.o +OBJS-$(CONFIG_VP8_V4L2M2M_DECODER) += v4l2_m2m_dec.o +OBJS-$(CONFIG_VP8_V4L2M2M_ENCODER) += v4l2_m2m_enc.o OBJS-$(CONFIG_S302M_ENCODER) += s302menc.o OBJS-$(CONFIG_SANM_DECODER) += sanm.o OBJS-$(CONFIG_SCPR_DECODER) += scpr.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 4712592..12af5bd 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -207,8 +207,10 @@ static void register_all(void) REGISTER_ENCDEC (H263, h263); REGISTER_DECODER(H263I, h263i); REGISTER_ENCDEC (H263P, h263p); + REGISTER_ENCDEC (H263_V4L2M2M, h263_v4l2m2m); REGISTER_DECODER(H264, h264); REGISTER_DECODER(H264_CRYSTALHD, h264_crystalhd); + REGISTER_ENCDEC (H264_V4L2M2M, h264_v4l2m2m); REGISTER_DECODER(H264_MEDIACODEC, h264_mediacodec); REGISTER_DECODER(H264_MMAL, h264_mmal); REGISTER_DECODER(H264_QSV, h264_qsv); @@ -253,6 +255,7 @@ static void register_all(void) REGISTER_ENCDEC (MPEG2VIDEO, mpeg2video); REGISTER_ENCDEC (MPEG4, mpeg4); REGISTER_DECODER(MPEG4_CRYSTALHD, mpeg4_crystalhd); + REGISTER_ENCDEC (MPEG4_V4L2M2M, mpeg4_v4l2m2m); REGISTER_DECODER(MPEG4_MMAL, mpeg4_mmal); #if FF_API_VDPAU REGISTER_DECODER(MPEG4_VDPAU, mpeg4_vdpau); @@ -262,8 +265,10 @@ static void register_all(void) REGISTER_DECODER(MPEG_VDPAU, mpeg_vdpau); REGISTER_DECODER(MPEG1_VDPAU, mpeg1_vdpau); #endif + REGISTER_DECODER(MPEG1_V4L2M2M, mpeg1_v4l2m2m); REGISTER_DECODER(MPEG2_MMAL, mpeg2_mmal); REGISTER_DECODER(MPEG2_CRYSTALHD, mpeg2_crystalhd); + REGISTER_DECODER(MPEG2_V4L2M2M, mpeg2_v4l2m2m); REGISTER_DECODER(MPEG2_QSV, mpeg2_qsv); REGISTER_DECODER(MPEG2_MEDIACODEC, mpeg2_mediacodec); REGISTER_DECODER(MSA1, msa1); @@ -361,6 +366,7 @@ static void register_all(void) REGISTER_DECODER(VC1IMAGE, vc1image); REGISTER_DECODER(VC1_MMAL, vc1_mmal); REGISTER_DECODER(VC1_QSV, vc1_qsv); + REGISTER_DECODER(VC1_V4L2M2M, vc1_v4l2m2m); REGISTER_ENCODER(VC2, vc2); REGISTER_DECODER(VCR1, vcr1); REGISTER_DECODER(VMDVIDEO, vmdvideo); @@ -372,6 +378,7 @@ static void register_all(void) REGISTER_DECODER(VP6F, vp6f); REGISTER_DECODER(VP7, vp7); REGISTER_DECODER(VP8, vp8); + REGISTER_ENCDEC (VP8_V4L2M2M, vp8_v4l2m2m); REGISTER_DECODER(VP9, vp9); REGISTER_DECODER(VQA, vqa); REGISTER_DECODER(BITPACKED, bitpacked); diff --git a/libavcodec/v4l2-buffers.c b/libavcodec/v4l2-buffers.c new file mode 100644 index 0000000..bc6ef28 --- /dev/null +++ b/libavcodec/v4l2-buffers.c @@ -0,0 +1,633 @@ +/* + * V4L2 buffer{,pool} helper functions. + * + * Copyright (C) 2017 Alexis Ballier + * Copyright (C) 2017 Jorge Ramirez + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include +#include +#include +#include "avcodec.h" +#include "internal.h" +#include "v4l2-buffers.h" +#include "v4l2-common.h" + +#if 0 +#define V4L_BUFFER_DEBUG */ +#endif + +#define IS_BP_SUPPORTED(bp) ((bp->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) || \ + (bp->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) || \ + (bp->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) || \ + (bp->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)) + +enum V4LBuffer_status { + V4LBUF_AVAILABLE, + V4LBUF_IN_DRIVER, + V4LBUF_RET_USER, +}; + +struct V4LBuffer { + AVBufferRef *bufrefs[VIDEO_MAX_PLANES]; + struct V4LBufferPool *pool; + struct v4l2_plane planes[VIDEO_MAX_PLANES]; + struct v4l2_buffer buf; + + void * mm_addr[VIDEO_MAX_PLANES]; + size_t lengths[VIDEO_MAX_PLANES]; + enum V4LBuffer_status status; + int bytesperline[4]; + int num_planes; + int num_lines; + int index; + int flags; + struct timeval timestamp; + int ref_cnt; +}; + +static int enqueue_v4lbuf(V4LBuffer* avbuf) +{ + int ret; + + memset(&avbuf->buf, 0, sizeof(avbuf->buf)); + avbuf->buf.memory = avbuf->pool->memory; + avbuf->buf.type = avbuf->pool->type; + avbuf->buf.index = avbuf->index; + + if (V4L2_TYPE_IS_MULTIPLANAR(avbuf->pool->type)) { + avbuf->buf.length = avbuf->num_planes; + avbuf->buf.m.planes = avbuf->planes; + } else { + avbuf->buf.bytesused = avbuf->planes[avbuf->index].bytesused; + avbuf->buf.m.userptr = avbuf->planes[avbuf->index].m.userptr; + avbuf->buf.length = avbuf->planes[avbuf->index].length; + } + + avbuf->buf.flags = avbuf->pool->default_flags | avbuf->flags; + avbuf->buf.timestamp = avbuf->timestamp; + + ret = ioctl(avbuf->pool->fd, VIDIOC_QBUF, &avbuf->buf); + if (ret < 0) + return AVERROR(errno); + + avbuf->status = V4LBUF_IN_DRIVER; + avbuf->pool->num_queued++; + +#ifdef V4L_BUFFER_DEBUG + av_log(avbuf->pool->log_ctx, AV_LOG_DEBUG, " buffer enqueued on %s\n", avbuf->pool->name); +#endif + + return 0; +} + +static V4LBuffer* dequeue_v4lbuf(V4LBufferPool *bp) +{ + struct v4l2_plane planes[VIDEO_MAX_PLANES]; + struct v4l2_buffer buf = { 0 }; + V4LBuffer* avbuf = NULL; + struct pollfd pfd; + int ret; + int i; + + if (bp->num_queued < bp->min_queued_buffers) { + return NULL; + } + + if (bp->blocking_dequeue) { + pfd.fd = bp->fd; + switch (bp->type) { + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + pfd.events = POLLIN | POLLERR; + break; + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + pfd.events = POLLOUT | POLLERR | POLLWRNORM; + break; + default: + pfd.events = POLLIN | POLLERR | POLLRDNORM; + } + + ret = poll(&pfd, 1, bp->blocking_dequeue); + if (ret<= 0) { + av_log(bp->log_ctx, AV_LOG_WARNING, "%s: timeout (%d ms)\n", bp->name, bp->blocking_dequeue); + return NULL; + } + } + + memset(&buf, 0, sizeof(buf)); + buf.memory = bp->memory; + buf.type = bp->type; + if (V4L2_TYPE_IS_MULTIPLANAR(bp->type)) { + memset(planes, 0, sizeof(planes)); + buf.length = VIDEO_MAX_PLANES; + buf.m.planes = planes; + } + + ret = ioctl(bp->fd, VIDIOC_DQBUF, &buf); + if (ret) { + if (errno != EAGAIN) { + av_log(bp->log_ctx, AV_LOG_DEBUG, "%s: VIDIOC_DQBUF, errno (%d)\n", bp->name, errno); + bp->broken = errno; + } + return NULL; + } + + avbuf = &(bp->buffers[buf.index]); + if (V4L2_TYPE_IS_MULTIPLANAR(bp->type)) { + memcpy(avbuf->planes, planes, sizeof(planes)); + avbuf->buf.m.planes = avbuf->planes; + } + avbuf->status = V4LBUF_AVAILABLE; + avbuf->pool->num_queued--; + avbuf->buf = buf; + + if (V4L2_TYPE_IS_OUTPUT(avbuf->pool->type)) { + for (i = 0; i < avbuf->num_planes; i++) { + if (avbuf->bufrefs[i]) { + av_buffer_unref(&avbuf->bufrefs[i]); + } + } + } + +#ifdef V4L_BUFFER_DEBUG + av_log(bp->log_ctx, AV_LOG_DEBUG, "dequeued buffer on %s\n", bp->name); +#endif + + return avbuf; +} + +static void buffer_callback(void *opaque, uint8_t *unused) +{ + V4LBuffer* avbuf = opaque; + + if (--avbuf->ref_cnt <= 0) { + if (V4LBUF_IN_DRIVER != avbuf->status) { + if (!V4L2_TYPE_IS_OUTPUT(avbuf->pool->type)) { + enqueue_v4lbuf(avbuf); + } else { + avbuf->status = V4LBUF_AVAILABLE; + } + } + } +} + +static inline int init_buffer(V4LBuffer* avbuf) +{ + int ret, i; + + avbuf->buf.memory = avbuf->pool->memory; + avbuf->buf.type = avbuf->pool->type; + avbuf->buf.index = avbuf->index; + + if (V4L2_TYPE_IS_MULTIPLANAR(avbuf->pool->type)) { + avbuf->buf.length = VIDEO_MAX_PLANES; + avbuf->buf.m.planes = avbuf->planes; + } + + ret = ioctl(avbuf->pool->fd, VIDIOC_QUERYBUF, &avbuf->buf); + if (ret < 0) + return AVERROR(errno); + + if (V4L2_TYPE_IS_MULTIPLANAR(avbuf->pool->type)) { + avbuf->num_planes = 0; + for (;;) { + if (avbuf->num_planes < avbuf->buf.length) { + if (avbuf->buf.m.planes[avbuf->num_planes].length) { + avbuf->num_planes++; + continue; + } + } + break; + } + } else { + avbuf->num_planes = 1; + } + + avbuf->num_lines = avbuf->pool->format.fmt.pix_mp.height; + for (i = 0; i < avbuf->num_planes; i++) { + if (V4L2_TYPE_IS_MULTIPLANAR(avbuf->pool->type)) { + avbuf->bytesperline[i] = avbuf->pool->format.fmt.pix_mp.plane_fmt[i].bytesperline; + } else { + avbuf->bytesperline[i] = avbuf->pool->format.fmt.pix.bytesperline; + } + + switch (avbuf->pool->memory) { + case V4L2_MEMORY_MMAP: + if (V4L2_TYPE_IS_MULTIPLANAR(avbuf->pool->type)) { + avbuf->lengths[i] = avbuf->buf.m.planes[i].length; + avbuf->mm_addr[i] = mmap(NULL, avbuf->buf.m.planes[i].length, + PROT_READ | PROT_WRITE, MAP_SHARED, + avbuf->pool->fd, avbuf->buf.m.planes[i].m.mem_offset); + } else { + avbuf->lengths[i] = avbuf->buf.length; + avbuf->mm_addr[i] = mmap(NULL, avbuf->buf.length, + PROT_READ | PROT_WRITE, MAP_SHARED, + avbuf->pool->fd, avbuf->buf.m.offset); + } + if (avbuf->mm_addr[i] == MAP_FAILED) { + return AVERROR(ENOMEM); + } + break; + case V4L2_MEMORY_USERPTR: + /* Nothing to do */ + break; + default: + av_log(avbuf->pool->log_ctx, AV_LOG_ERROR, "memory type %i not supported\n", avbuf->pool->memory); + return AVERROR_PATCHWELCOME; + } + } + avbuf->status = V4LBUF_AVAILABLE; + + if (!V4L2_TYPE_IS_OUTPUT(avbuf->pool->type)) { + if (avbuf->pool->memory != V4L2_MEMORY_USERPTR) { + return enqueue_v4lbuf(avbuf); + } + } + + return 0; +} + +int avpriv_init_v4lbufpool(V4LBufferPool* bufs) +{ + struct v4l2_requestbuffers req; + int ret, i; + + if (!IS_BP_SUPPORTED(bufs)) { + av_log(bufs->log_ctx, AV_LOG_ERROR, "%type %i not supported\n", bufs->type); + return AVERROR_PATCHWELCOME; + } + + memset(&req, 0, sizeof(req)); + req.count = bufs->num_buffers + bufs->min_queued_buffers; + req.memory = bufs->memory; + req.type = bufs->type; + + ret = ioctl(bufs->fd, VIDIOC_REQBUFS, &req); + if (ret< 0) + return AVERROR(errno); + + bufs->num_buffers = req.count; + bufs->num_queued = 0; + bufs->buffers = av_mallocz(bufs->num_buffers * sizeof(V4LBuffer)); + + for (i = 0; i < req.count; i++) { + V4LBuffer *avbuf = &bufs->buffers[i]; + + avbuf->pool = bufs; + avbuf->index = i; + ret = init_buffer(avbuf); + if (ret < 0) { + av_log(bufs->log_ctx, AV_LOG_ERROR, "%s buffer initialization (%s)\n", bufs->name, av_err2str(ret)); + return ret; + } + } + + return 0; +} + +static void release_buf(V4LBuffer* b) +{ + int i; + + for (i = 0; i < b->num_planes; i++) { + if (b->mm_addr[i] && b->lengths[i]) { + munmap(b->mm_addr[i], b->lengths[i]); + } + } +} + +void avpriv_release_buffer_pool(V4LBufferPool* bp) +{ + if (bp->buffers) { + int i; + + for (i = 0; i < bp->num_buffers; i++) + release_buf(&bp->buffers[i]); + + av_free(bp->buffers); + } +} + +static inline void set_pts(V4LBuffer *out, int64_t pts) +{ + out->timestamp.tv_sec = pts / INT64_C(1000000); + out->timestamp.tv_usec = pts % INT64_C(1000000); +} + +static inline uint64_t get_pts(V4LBuffer *avbuf) +{ + if (avbuf->buf.timestamp.tv_sec || avbuf->buf.timestamp.tv_usec) + return (avbuf->buf.timestamp.tv_sec * INT64_C(1000000) + avbuf->buf.timestamp.tv_usec); + + return AV_NOPTS_VALUE; +} + +static int buf2v4l(V4LBuffer *out, int plane, const uint8_t* data, int size, AVBufferRef* bref) +{ + if (plane >= out->num_planes) + return AVERROR(EINVAL); + + switch (out->pool->memory) { + case V4L2_MEMORY_MMAP: + memcpy(out->mm_addr[plane], data, FFMIN(size, out->lengths[plane])); + break; + case V4L2_MEMORY_USERPTR: + if (!bref) { + av_log(out->pool->log_ctx, AV_LOG_ERROR, + "needs to be set with an AVBufferRef for USERPTR memory type\n"); + return AVERROR_PATCHWELCOME; + } + + if (out->bufrefs[plane]) { + av_log(out->pool->log_ctx, AV_LOG_WARNING, + "V4L buffer already had a buffer referenced\n"); + av_buffer_unref(&out->bufrefs[plane]); + } + + out->bufrefs[plane] = av_buffer_ref(bref); + if (!out->bufrefs[plane]) + return AVERROR(ENOMEM); + + out->planes[plane].m.userptr = (unsigned long)out->bufrefs[plane]->data; + out->lengths[plane] = out->bufrefs[plane]->size; + + break; + default: + av_log(out->pool->log_ctx, AV_LOG_ERROR, + "memory type %i not supported", out->pool->memory); + return AVERROR_PATCHWELCOME; + } + + out->planes[plane].bytesused = FFMIN(size ? size : 1, out->lengths[plane]); + out->planes[plane].length = out->lengths[plane]; + + return 0; +} + +static int avframe_to_v4lbuf(const AVFrame *pict, V4LBuffer* out) { + int i, ret; + + for (i = 0; i < out->num_planes; i++) { + ret = buf2v4l(out, i, pict->buf[i]->data, pict->buf[i]->size, pict->buf[i]); + if (ret) + return ret; + } + set_pts(out, pict->pts); + + return 0; +} + +static int avpkt_to_v4lbuf(const AVPacket *pkt, V4LBuffer *out) { + int ret; + + ret = buf2v4l(out, 0, pkt->data, pkt->size, pkt->buf); + if (ret) + return ret; + + if (pkt->pts != AV_NOPTS_VALUE) + set_pts(out, pkt->pts); + + if (pkt->flags & AV_PKT_FLAG_KEY) + out->flags = V4L2_BUF_FLAG_KEYFRAME; + + if (!pkt->size) + out->flags = V4L2_BUF_FLAG_LAST; + + return 0; +} + +static inline int v4l2bufref(V4LBuffer *in, int plane, AVBufferRef **buf) +{ +#ifdef V4L_BUFFER_DEBUG + av_log(in->pool->log_ctx, AV_LOG_DEBUG, + "Making an avbuffer from V4L buffer %i[%i] on %s (%i,%i)\n", + in->index, plane, in->pool->name, in->pool->type, in->pool->memory); +#endif + + if (plane >= in->num_planes) { + return AVERROR(EINVAL); + } + + switch (in->pool->memory) { + case V4L2_MEMORY_MMAP: + *buf = av_buffer_create(in->mm_addr[plane], in->lengths[plane], buffer_callback, in, 0); + if (!*buf) + return AVERROR(ENOMEM); + + in->status = V4LBUF_RET_USER; + in->ref_cnt++; + break; + case V4L2_MEMORY_USERPTR: + if (!in->bufrefs[plane]) { + av_log(in->pool->log_ctx, AV_LOG_ERROR, "AVBufferRef not found\n"); + return AVERROR(EINVAL); + } + + *buf = av_buffer_ref(in->bufrefs[plane]); + if (!*buf) + return AVERROR(ENOMEM); + + av_buffer_unref(&in->bufrefs[plane]); + in->status = V4LBUF_AVAILABLE; + break; + default: + av_log(in->pool->log_ctx, AV_LOG_ERROR, "memory type %i not supported", in->pool->memory); + return AVERROR_PATCHWELCOME; + } + return 0; +} + +static int v4lbuf_to_avpkt(AVPacket *pkt, V4LBuffer *avbuf) +{ + int ret; + + av_free_packet(pkt); + if (ret = v4l2bufref(avbuf, 0, &pkt->buf)) + return ret; + + pkt->data = pkt->buf->data; + if (V4L2_TYPE_IS_MULTIPLANAR(avbuf->pool->type)) { + pkt->size = avbuf->buf.m.planes[0].bytesused; + } else { + pkt->size = avbuf->buf.bytesused; + } + + if (avbuf->buf.flags & V4L2_BUF_FLAG_KEYFRAME) { + pkt->flags |= AV_PKT_FLAG_KEY; + } + + pkt->pts = get_pts(avbuf); + + return 0; +} + +static int v4lbuf_to_avframe(AVFrame *frame, V4LBuffer *avbuf) +{ + int i, ret; + + av_frame_unref(frame); + + for (i = 0; i < avbuf->num_planes; i++) { + ret = v4l2bufref(avbuf, i, &frame->buf[i]); + if (ret) + return ret; + + frame->linesize[i] = avbuf->bytesperline[i]; + frame->data[i] = frame->buf[i]->data; + + if (avbuf->num_planes == 1) { + if (avbuf->pool->av_pix_fmt == AV_PIX_FMT_NV12) { + frame->linesize[1] = avbuf->bytesperline[0]; + frame->data[1] = frame->buf[0]->data + avbuf->bytesperline[0] * avbuf->num_lines; + } + } + } + + frame->key_frame = !!(avbuf->buf.flags & V4L2_BUF_FLAG_KEYFRAME); + frame->format = avbuf->pool->av_pix_fmt; + frame->height = avbuf->pool->height; + frame->width = avbuf->pool->width; + frame->pts = get_pts(avbuf); + + return 0; +} + +static V4LBuffer* v4lbufpool_get_from_avframe(const AVFrame* frame, V4LBufferPool *p) +{ + int i; + + for (i = 0; i < p->num_buffers; i++) { + if (V4LBUF_RET_USER == p->buffers[i].status) { + if (p->memory == V4L2_MEMORY_MMAP) { + if (p->buffers[i].mm_addr[0] == frame->buf[0]->data) + return &p->buffers[i]; + continue; + } + av_log(p->log_ctx, AV_LOG_ERROR, "memory type %i not supported\n", p->memory); + return NULL; + } + } + + return NULL; +} + +V4LBuffer* avpriv_v4lbufpool_getfreebuf(V4LBufferPool *p, const AVFrame *f, const AVPacket* pkt) +{ + V4LBuffer* ret; + int i; + +#ifdef V4L_BUFFER_DEBUG + av_log(p->log_ctx, AV_LOG_DEBUG, "Polling for a free buffer on %s\n", p->name); +#endif + + if (V4L2_TYPE_IS_OUTPUT(p->type)) { + for (;;) { + if (!dequeue_v4lbuf(p)) { + break; + } + } + } + + if (f) { + ret = v4lbufpool_get_from_avframe(f, p); + if (ret) + return ret; + } + + for (i = 0; i < p->num_buffers; i++) { + if (p->buffers[i].status == V4LBUF_AVAILABLE) + return &p->buffers[i]; + } + + return NULL; +} + +int avpriv_set_stream_status(V4LBufferPool* bp, int cmd) +{ + int type = bp->type; + int ret; + + ret = ioctl(bp->fd, cmd, &type); + if (ret < 0) + return AVERROR(errno); + + bp->streamon = (cmd == VIDIOC_STREAMON); + + return 0; +} + +int avpriv_v4l_enqueue_frame_or_pkt_or_buf(V4LBufferPool* bp, const AVFrame* f, const AVPacket* pkt, const uint8_t* buf, int buf_size) +{ + V4LBuffer* avbuf; + int ret; + + if (!f && !pkt && !buf) { + av_log(bp->log_ctx, AV_LOG_ERROR, "either AVFrame*, AVPacket* or buf must valid\n"); + return AVERROR_BUG; + } + + avbuf = avpriv_v4lbufpool_getfreebuf(bp, f, pkt); + if (!avbuf) + return AVERROR(ENOMEM); + + if (f && (ret = avframe_to_v4lbuf(f, avbuf))) + return ret; + + if (pkt && (ret = avpkt_to_v4lbuf(pkt, avbuf))) + return ret; + + if (buf && (ret = buf2v4l(avbuf, 0, buf, buf_size, NULL))) + return ret; + + if (ret = enqueue_v4lbuf(avbuf)) + return ret; + + return 0; +} + +int avpriv_v4l_dequeue_frame_or_pkt(V4LBufferPool* bp, AVFrame* f, AVPacket* pkt) +{ + V4LBuffer* avbuf = NULL; + + if ((!f && !pkt) || (f && pkt)) { + av_log(bp->log_ctx, AV_LOG_ERROR, "either AVFrame* or AVPacket* must be valid\n"); + return AVERROR_BUG; + } + + if (!(avbuf = dequeue_v4lbuf(bp))) { + if (bp->broken) { + return AVERROR_EOF; + } + return AVERROR(EAGAIN); + } + + if (f) + return v4lbuf_to_avframe(f, avbuf); + + if (pkt) + return v4lbuf_to_avpkt(pkt, avbuf); + + return AVERROR_BUG; +} + + diff --git a/libavcodec/v4l2-buffers.h b/libavcodec/v4l2-buffers.h new file mode 100644 index 0000000..ed0362e --- /dev/null +++ b/libavcodec/v4l2-buffers.h @@ -0,0 +1,247 @@ +/* + * V4L2 buffer{,pool} helper functions. + * + * Copyright (C) 2017 Alexis Ballier + * Copyright (C) 2017 Jorge Ramirez + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_V4L2_BUFFERS_H +#define AVCODEC_V4L2_BUFFERS_H + +#include "v4l2-common.h" +#include "avcodec.h" +#include "libavutil/pixfmt.h" +#include "libavutil/frame.h" + +struct V4LBuffer; +typedef struct V4LBuffer V4LBuffer; + +struct V4LBufferPool; +typedef int (*format_f)(struct V4LBufferPool *, int set); +typedef int (*init_f)(struct V4LBufferPool *); + +typedef struct V4LBufferPoolCfg { + format_f format; + init_f init; +} V4LBufferPoolCfg; + +typedef struct V4LBufferPool { + /** + * Buffer pool initial configuration. + */ + V4LBufferPoolCfg cfg; + + /** + * Log context (for av_log()). Can be NULL. + */ + void *log_ctx; + + /** + * Pool's name. Must be set before calling avpriv_init_v4lbufpool(). + */ + const char* name; + + /** + * File descriptor obtained from opening the associated device. + * Must be set before calling avpriv_init_v4lbufpool(). + * Readonly after init. + */ + int fd; + + /** + * Type of this buffer pool. + * See V4L2_BUF_TYPE_VIDEO_* in videodev2.h + * Must be set before calling avpriv_init_v4lbufpool(). + * Readonly after init. + */ + enum v4l2_buf_type type; + + /** + * Memory type this buffer pool uses. + * See V4L2_MEMORY_* in videodev2.h + * Must be set before calling avpriv_init_v4lbufpool(). + * Readonly after init. + */ + enum v4l2_memory memory; + + /** + * AVPixelFormat corresponding to this buffer pool. + * AV_PIX_FMT_NONE means this is an encoded stream. + */ + enum AVPixelFormat av_pix_fmt; + + /** + * AVCodecID corresponding to this buffer pool. + * AV_CODEC_ID_RAWVIDEO means this is a raw stream and av_pix_fmt must be set to a valid value. + */ + enum AVCodecID av_codec_id; + + /** + * fourcc (LSB first, so "ABCD" -> ('D'<<24) + ('C'<<16) + ('B'<<8) + 'A'). + * This is used to work around some encoder bugs. + * A demuxer should set this to what is stored in the field used to identify the codec. + * If there are multiple such fields in a container then the demuxer should choose the one + * which maximizes the information about the used codec. + * If the codec tag field in a container is larger than 32 bits then the demuxer should + * remap the longer ID to 32 bits with a table or other structure. Alternatively a new + * extra_codec_tag + size could be added but for this a clear advantage must be demonstrated + * first. + * - encoding: Set by user, if not then the default based on codec_id will be used. + * - decoding: Set by user, will be converted to uppercase by libavcodec during init. + */ + unsigned int av_codec_tag; + + /** + * Format returned by the driver after initializing the buffer pool. + * Must be set before calling avpriv_init_v4lbufpool(). + * avpriv_set_pool_format() can set it. + * Readonly after init. + */ + struct v4l2_format format; + + /** + * Width and height of the frames it produces (in case of a capture pool, e.g. when decoding) + * or accepts (in case of an output pool, e.g. when encoding). + * + * For output pools, this must must be set before calling avpriv_init_v4lbufpool(). + * For capture pools, it will be set after having received the information from the driver. + */ + int width, height; + + /** + * Default flags to set on buffers to enqueue. + * See V4L2_BUF_FLAG_*. + */ + int default_flags; + + /** + * Whether the stream has been started (VIDIOC_STREAMON has been sent). + */ + int streamon; + + /** + * Number of queued buffers. + */ + int num_queued; + + /** + * Time (in ms) we can wait for a buffer before considering it a failure. + */ + int blocking_dequeue; + + /** + * Minimum number of buffers that must be kept queued in this queue. + * + * E.g. for decoders, the drivers might have such requirements to produce proper output. + */ + int min_queued_buffers; + + /** + * The actual number of buffers. + * + * Before calling avpriv_init_v4lbufpool() this is the number of buffers we would like to have available. + * avpriv_init_v4lbufpool() asks for (min_buffers + num_buffers) and sets this value to the actual number + * of buffers the driver gave us. + * Readonly after init. + */ + int num_buffers; + + /** + * Opaque pointers to the actual buffers representations. + * After initialization, it is an array of size num_buffers. + */ + V4LBuffer *buffers; + + /** + * Pool in unrecoverable error notified by the V4L2 kernel api + */ + int broken; + +} V4LBufferPool; + +/** + * Initializes a V4LBufferPool. + * + * @param[in] bp A pointer to a V4LBufferPool. See V4LBufferPool description for required variables. + * @return 0 in case of success, a negative value representing the error otherwise. + */ +int avpriv_init_v4lbufpool(V4LBufferPool* bp); + +/** + * Releases a V4LBufferPool. + * + * @param[in] bp A pointer to a V4LBufferPool. + * The caller is reponsible for freeing it. + * It must not be used after calling this function. + */ +void avpriv_release_buffer_pool(V4LBufferPool* bp); + +/** + * Sets the status of a V4LBufferPool. + * + * @param[in] bp A pointer to a V4LBufferPool. + * @param[in] cmd The status to set (VIDIOC_STREAMON or VIDIOC_STREAMOFF). + * Warning: If VIDIOC_STREAMOFF is sent to a buffer pool that still has some frames buffered, + * those frames will be dropped. + * @return 0 in case of success, a negative value representing the error otherwise. + */ +int avpriv_set_stream_status(V4LBufferPool* bp, int cmd); + +/** + * Dequeues a buffer from a V4LBufferPool to either an AVFrame or an AVPacket. + * + * Exactly one of f or pkt must be non NULL. + * @param[in] bp The V4LBufferPool to dequeue from. + * @param[inout] f The AVFrame to dequeue to. + * @param[inout] pkt The AVPacket to dequeue to. + * @return 0 in case of success, AVERROR(EAGAIN) if no buffer was ready, another negative error in case of error. + */ +int avpriv_v4l_dequeue_frame_or_pkt(V4LBufferPool* bp, AVFrame* f, AVPacket* pkt); + +/** + * Enqueues a buffer to a V4LBufferPool from either an AVFrame, an AVPacket or a raw buffer. + * Exactly one of f, pkt or buf must be non NULL. + * + * @param[in] bp The V4LBufferPool to enqueue to. + * @param[in] f A pointer to an AVFrame to enqueue. + * @param[in] pkt A pointer to an AVPacket to enqueue. + * @param[in] buf A pointer to a buffer to enqueue. + * @param[in] buf_size The size of the buffer pointed by buf. + * @return 0 in case of success, a negative error otherwise. + */ +int avpriv_v4l_enqueue_frame_or_pkt_or_buf(V4LBufferPool* bp, const AVFrame* f, const AVPacket* pkt, const uint8_t* buf, int buf_size); + +/** + * Gets a free V4LBuffer from a V4LBufferPool. + * + * If no matching buffer is found (see below), it tries to dequeue a buffer first + * in order to minimize the size of the V4L queue.e + * + * @param[in] p Pointer to a V4LBufferPool where to get the buffer from. + * @param[in] f A pointer to an existing AVFrame: + * If the AVFrame's buffers match a V4LBuffer, this V4LBuffer will be returned. + * Can be NULL. + * @param[in] pkt A pointer to an existing AVPacket: + * If the AVPacket's buffers match a V4LBuffer, this V4LBuffer will be returned. + * Can be NULL. + * @return A pointer to the V4LBuffer or NULL in case of error. + */ +V4LBuffer* avpriv_v4lbufpool_getfreebuf(V4LBufferPool *p, const AVFrame *f, const AVPacket* pkt); + +#endif // AVCODEC_V4L2_BUFFERS_H diff --git a/libavcodec/v4l2-common.c b/libavcodec/v4l2-common.c index 13744fb..f2f98dc 100644 --- a/libavcodec/v4l2-common.c +++ b/libavcodec/v4l2-common.c @@ -19,15 +19,14 @@ #include "v4l2-common.h" const struct v4l_fmt_map avpriv_v4l_fmt_conversion_table[] = { - //ff_fmt codec_id v4l2_fmt pack_flags - { AV_PIX_FMT_YUV420P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YUV420 , FF_V4L_PACK_AVPACKET }, - { AV_PIX_FMT_YUV420P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YVU420 , FF_V4L_PACK_AVPACKET }, - { AV_PIX_FMT_YUV422P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YUV422P , FF_V4L_PACK_AVPACKET }, + /* ff_fmt codec_id v4l2_fmt pack_flags */ + { AV_PIX_FMT_YUV420P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YUV420 , FF_V4L_PACK_AVPACKET | FF_V4L_PACK_AVFRAME }, + { AV_PIX_FMT_YUV422P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YUV422P , FF_V4L_PACK_AVPACKET }, { AV_PIX_FMT_YUYV422, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YUYV , FF_V4L_PACK_AVPACKET | FF_V4L_PACK_AVFRAME }, { AV_PIX_FMT_UYVY422, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_UYVY , FF_V4L_PACK_AVPACKET | FF_V4L_PACK_AVFRAME }, - { AV_PIX_FMT_YUV411P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YUV411P , FF_V4L_PACK_AVPACKET }, - { AV_PIX_FMT_YUV410P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YUV410 , FF_V4L_PACK_AVPACKET }, - { AV_PIX_FMT_YUV410P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YVU410 , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_YUV411P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YUV411P , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_YUV410P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YUV410 , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_YUV410P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YVU410 , FF_V4L_PACK_AVPACKET }, { AV_PIX_FMT_RGB555LE,AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_RGB555 , FF_V4L_PACK_AVPACKET | FF_V4L_PACK_AVFRAME }, { AV_PIX_FMT_RGB555BE,AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_RGB555X , FF_V4L_PACK_AVPACKET | FF_V4L_PACK_AVFRAME }, { AV_PIX_FMT_RGB565LE,AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_RGB565 , FF_V4L_PACK_AVPACKET | FF_V4L_PACK_AVFRAME }, @@ -40,17 +39,17 @@ const struct v4l_fmt_map avpriv_v4l_fmt_conversion_table[] = { #ifdef V4L2_PIX_FMT_Y16 { AV_PIX_FMT_GRAY16LE,AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_Y16 , FF_V4L_PACK_AVPACKET | FF_V4L_PACK_AVFRAME }, #endif - { AV_PIX_FMT_NV12, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_NV12 , FF_V4L_PACK_AVPACKET }, - { AV_PIX_FMT_NONE, AV_CODEC_ID_MJPEG, V4L2_PIX_FMT_MJPEG , FF_V4L_PACK_AVPACKET }, - { AV_PIX_FMT_NONE, AV_CODEC_ID_MJPEG, V4L2_PIX_FMT_JPEG , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NV12, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_NV12 , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_MJPEG, V4L2_PIX_FMT_MJPEG , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_MJPEG, V4L2_PIX_FMT_JPEG , FF_V4L_PACK_AVPACKET }, #ifdef V4L2_PIX_FMT_H264 - { AV_PIX_FMT_NONE, AV_CODEC_ID_H264, V4L2_PIX_FMT_H264 , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_H264, V4L2_PIX_FMT_H264 , FF_V4L_PACK_AVPACKET }, #endif #ifdef V4L2_PIX_FMT_MPEG4 - { AV_PIX_FMT_NONE, AV_CODEC_ID_MPEG4, V4L2_PIX_FMT_MPEG4 , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_MPEG4, V4L2_PIX_FMT_MPEG4 , FF_V4L_PACK_AVPACKET }, #endif #ifdef V4L2_PIX_FMT_CPIA1 - { AV_PIX_FMT_NONE, AV_CODEC_ID_CPIA, V4L2_PIX_FMT_CPIA1 , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_CPIA, V4L2_PIX_FMT_CPIA1 , FF_V4L_PACK_AVPACKET }, #endif #ifdef V4L2_PIX_FMT_SRGGB8 { AV_PIX_FMT_BAYER_BGGR8, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_SBGGR8 , FF_V4L_PACK_AVPACKET | FF_V4L_PACK_AVFRAME }, @@ -59,36 +58,36 @@ const struct v4l_fmt_map avpriv_v4l_fmt_conversion_table[] = { { AV_PIX_FMT_BAYER_RGGB8, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_SRGGB8 , FF_V4L_PACK_AVPACKET | FF_V4L_PACK_AVFRAME }, #endif #ifdef V4L2_PIX_FMT_NV12M - { AV_PIX_FMT_NV12, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_NV12M , FF_V4L_PACK_AVFRAME }, + { AV_PIX_FMT_NV12, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_NV12M , FF_V4L_PACK_AVFRAME }, #endif #ifdef V4L2_PIX_FMT_NV21M - { AV_PIX_FMT_NV21, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_NV21M , FF_V4L_PACK_AVFRAME }, + { AV_PIX_FMT_NV21, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_NV21M , FF_V4L_PACK_AVFRAME }, #endif #ifdef V4L2_PIX_FMT_YUV420M - { AV_PIX_FMT_YUV420P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YUV420M , FF_V4L_PACK_AVFRAME }, + { AV_PIX_FMT_YUV420P, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_YUV420M , FF_V4L_PACK_AVFRAME }, #endif #ifdef V4L2_PIX_FMT_NV16M - { AV_PIX_FMT_NV16, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_NV16M , FF_V4L_PACK_AVFRAME }, + { AV_PIX_FMT_NV16, AV_CODEC_ID_RAWVIDEO, V4L2_PIX_FMT_NV16M , FF_V4L_PACK_AVFRAME }, #endif #ifdef V4L2_PIX_FMT_DV - { AV_PIX_FMT_NONE, AV_CODEC_ID_DVVIDEO, V4L2_PIX_FMT_DV , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_DVVIDEO, V4L2_PIX_FMT_DV , FF_V4L_PACK_AVPACKET }, #endif #ifdef V4L2_PIX_FMT_H263 - { AV_PIX_FMT_NONE, AV_CODEC_ID_H263, V4L2_PIX_FMT_H263 , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_H263, V4L2_PIX_FMT_H263 , FF_V4L_PACK_AVPACKET }, #endif #ifdef V4L2_PIX_FMT_MPEG1 - { AV_PIX_FMT_NONE, AV_CODEC_ID_MPEG1VIDEO, V4L2_PIX_FMT_MPEG1 , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_MPEG1VIDEO, V4L2_PIX_FMT_MPEG1 , FF_V4L_PACK_AVPACKET }, #endif #ifdef V4L2_PIX_FMT_MPEG2 - { AV_PIX_FMT_NONE, AV_CODEC_ID_MPEG2VIDEO, V4L2_PIX_FMT_MPEG2 , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_MPEG2VIDEO, V4L2_PIX_FMT_MPEG2 , FF_V4L_PACK_AVPACKET }, #endif #ifdef V4L2_PIX_FMT_VC1_ANNEX_G - { AV_PIX_FMT_NONE, AV_CODEC_ID_VC1, V4L2_PIX_FMT_VC1_ANNEX_G, FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_VC1, V4L2_PIX_FMT_VC1_ANNEX_G, FF_V4L_PACK_AVPACKET }, #endif #ifdef V4L2_PIX_FMT_VP8 - { AV_PIX_FMT_NONE, AV_CODEC_ID_VP8, V4L2_PIX_FMT_VP8 , FF_V4L_PACK_AVPACKET }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_VP8, V4L2_PIX_FMT_VP8 , FF_V4L_PACK_AVPACKET }, #endif - { AV_PIX_FMT_NONE, AV_CODEC_ID_NONE, 0 }, + { AV_PIX_FMT_NONE, AV_CODEC_ID_NONE, 0 , 0 }, }; uint32_t avpriv_v4l_fmt_ff2v4l(enum AVPixelFormat pix_fmt, enum AVCodecID codec_id, int pack_flags) @@ -101,6 +100,7 @@ uint32_t avpriv_v4l_fmt_ff2v4l(enum AVPixelFormat pix_fmt, enum AVCodecID codec_ (pix_fmt == AV_PIX_FMT_NONE || avpriv_v4l_fmt_conversion_table[i].ff_fmt == pix_fmt) && (avpriv_v4l_fmt_conversion_table[i].pack_flags & pack_flags)) { + return avpriv_v4l_fmt_conversion_table[i].v4l2_fmt; } } diff --git a/libavcodec/v4l2-common.h b/libavcodec/v4l2-common.h index 1df64f8..346887c 100644 --- a/libavcodec/v4l2-common.h +++ b/libavcodec/v4l2-common.h @@ -33,8 +33,8 @@ #include "libavutil/imgutils.h" #include "libavutil/log.h" #include "libavutil/opt.h" -#include "avdevice.h" -#include "timefilter.h" +#include "libavdevice/avdevice.h" +#include "libavdevice/timefilter.h" #include "libavutil/parseutils.h" #include "libavutil/pixdesc.h" #include "libavutil/time.h" diff --git a/libavcodec/v4l2_m2m.c b/libavcodec/v4l2_m2m.c new file mode 100644 index 0000000..a27c637 --- /dev/null +++ b/libavcodec/v4l2_m2m.c @@ -0,0 +1,358 @@ +/* + * V4L mem2mem wrapper + * + * Copyright (C) 2017 Alexis Ballier + * Copyright (C) 2017 Jorge Ramirez + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include +#include +#include +#include "libavutil/imgutils.h" +#include "libavutil/pixfmt.h" +#include "libavutil/pixdesc.h" +#include "avcodec.h" +#include "v4l2_m2m_avcodec.h" +#include "v4l2-buffers.h" +#include "v4l2-common.h" +#include "v4l2_m2m.h" + +#define V4L_MAX_STREAM_SIZE (3*1024*1024) + +static inline int try_raw_format(V4LBufferPool* bp, enum AVPixelFormat pixfmt) +{ + struct v4l2_format *fmt = &bp->format; + int ret, i, h; + + fmt->type = bp->type; + + if (V4L2_TYPE_IS_MULTIPLANAR(bp->type)) { + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(pixfmt); + + fmt->fmt.pix_mp.pixelformat = avpriv_v4l_fmt_ff2v4l(pixfmt, bp->av_codec_id, FF_V4L_PACK_AVFRAME); + if (!fmt->fmt.pix_mp.pixelformat) + return AVERROR(EINVAL); + + fmt->fmt.pix_mp.num_planes = av_pix_fmt_count_planes(pixfmt); + for (i = 0; i < fmt->fmt.pix_mp.num_planes; i++) { + fmt->fmt.pix_mp.plane_fmt[i].bytesperline = av_image_get_linesize(pixfmt, bp->width, i); + h = (i == 1 || i == 2) ? FF_CEIL_RSHIFT(bp->height, desc->log2_chroma_h) : bp->height; + fmt->fmt.pix_mp.plane_fmt[i].sizeimage = fmt->fmt.pix_mp.plane_fmt[i].bytesperline * h; + } + } else { + + fmt->fmt.pix.pixelformat = avpriv_v4l_fmt_ff2v4l(pixfmt, bp->av_codec_id, FF_V4L_PACK_AVFRAME); + if (!fmt->fmt.pix.pixelformat) + return AVERROR(EINVAL); + + fmt->fmt.pix.bytesperline = av_image_get_linesize(pixfmt, bp->width, 0); + fmt->fmt.pix.sizeimage = fmt->fmt.pix.bytesperline * bp->height; + } + + ret = ioctl(bp->fd, VIDIOC_TRY_FMT, fmt); + if (ret) + return AVERROR(EINVAL); + + if (V4L2_TYPE_IS_MULTIPLANAR(bp->type)) { + fmt->fmt.pix_mp.height = bp->height; + fmt->fmt.pix_mp.width = bp->width; + } else { + fmt->fmt.pix.height = bp->height; + fmt->fmt.pix.width = bp->width; + } + + return 0; +} + +static int set_raw_format(V4LBufferPool* bp, int set) +{ + enum AVPixelFormat pixfmt = bp->av_pix_fmt; + struct v4l2_format *fmt = &bp->format; + struct v4l2_fmtdesc fmtdesc = { 0 }; + int ret; + + fmtdesc.type = bp->type; + if (pixfmt != AV_PIX_FMT_NONE) { + ret = try_raw_format(bp, pixfmt); + if (ret) + pixfmt = AV_PIX_FMT_NONE; + } + + while (AV_PIX_FMT_NONE == pixfmt && !ioctl(bp->fd, VIDIOC_ENUM_FMT, &fmtdesc)) { + pixfmt = avpriv_v4l_fmt_v4l2ff(fmtdesc.pixelformat, AV_CODEC_ID_RAWVIDEO); + + ret = try_raw_format(bp, pixfmt); + if (ret) + pixfmt = AV_PIX_FMT_NONE; + + if (pixfmt != AV_PIX_FMT_NONE && set) { + bp->av_pix_fmt = pixfmt; + } + + fmtdesc.index++; + } + + if (pixfmt == AV_PIX_FMT_NONE) + return AVERROR(EINVAL); + + if (set) + return ioctl(bp->fd, VIDIOC_S_FMT, fmt); + + return 0; +} + +static int set_coded_format(V4LBufferPool* bp, int set) +{ + struct v4l2_format *fmt = &bp->format; + struct v4l2_fmtdesc fdesc; + uint32_t v4l2_fmt; + int found = 0; + + v4l2_fmt = avpriv_v4l_fmt_ff2v4l(bp->av_pix_fmt, bp->av_codec_id, FF_V4L_PACK_AVPACKET); + memset(&fdesc, 0, sizeof(fdesc)); + fdesc.type = bp->type; + + while (!ioctl(bp->fd, VIDIOC_ENUM_FMT, &fdesc)) { + if (v4l2_fmt == fdesc.pixelformat) { + found = 1; + break; + } + fdesc.index++; + } + + if (!found) + return AVERROR(EINVAL); + + fmt->type = bp->type; + + if (V4L2_TYPE_IS_MULTIPLANAR(bp->type)) { + fmt->fmt.pix_mp.num_planes = VIDEO_MAX_PLANES; + fmt->fmt.pix_mp.pixelformat = v4l2_fmt; + if (!fmt->fmt.pix_mp.pixelformat) { + av_log(bp->log_ctx, AV_LOG_ERROR, "no V4L codec for id %i\n", bp->av_codec_id); + return AVERROR(EINVAL); + } + fmt->fmt.pix_mp.plane_fmt[0].sizeimage = V4L_MAX_STREAM_SIZE; + fmt->fmt.pix_mp.height = bp->height; + fmt->fmt.pix_mp.width = bp->width; + + } else { + fmt->fmt.pix.pixelformat = v4l2_fmt; + if (!fmt->fmt.pix.pixelformat) { + av_log(bp->log_ctx, AV_LOG_ERROR, "no V4L codec for id %i\n", bp->av_codec_id); + return AVERROR(EINVAL); + } + fmt->fmt.pix.sizeimage = V4L_MAX_STREAM_SIZE; + fmt->fmt.pix.height = bp->height; + fmt->fmt.pix.width = bp->width; + } + + if (set) + return ioctl(bp->fd, VIDIOC_S_FMT, fmt); + + return ioctl(bp->fd, VIDIOC_TRY_FMT, fmt); +} + +int avpriv_set_pool_format(V4LBufferPool* bp, int set) +{ + if (bp->av_codec_id == AV_CODEC_ID_RAWVIDEO) + return set_raw_format(bp, set); + + return set_coded_format(bp, set); +} + +static inline int splane_video(struct v4l2_capability *cap) +{ + if (cap->capabilities & (V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING)) + return 1; + + if (cap->capabilities & (V4L2_CAP_VIDEO_M2M | V4L2_CAP_STREAMING)) + return 1; + + return 0; +} + +static inline int mplane_video(struct v4l2_capability *cap) +{ + if (cap->capabilities & (V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING)) + return 1; + + if (cap->capabilities & (V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING)) + return 1; + + return 0; +} + +static int prepare_pools(V4Lm2mContext* s, void *log_ctx) +{ + int ret; + + s->capture_pool.log_ctx = s->output_pool.log_ctx = log_ctx; + s->capture_pool.broken = s->output_pool.broken = 0; + s->capture_pool.fd = s->output_pool.fd = s->fd; + s->capture_pool.name = "capture pool"; + s->output_pool.name = "output pool"; + + memset(&s->cap, 0, sizeof(s->cap)); + ret = ioctl(s->fd, VIDIOC_QUERYCAP, &s->cap); + if (ret < 0) { + return ret; + } + + if (mplane_video(&s->cap)) { + s->capture_pool.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + s->output_pool.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + return 0; + } + + if (splane_video(&s->cap)) { + s->capture_pool.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + s->output_pool.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; + return 0; + } + + return AVERROR(EINVAL); +} + +static int probe_and_set(V4Lm2mContext* s, void *log_ctx, int set) +{ + int fail_log_level = ( set ? AV_LOG_ERROR : AV_LOG_DEBUG); + int ret; + + s->fd = open(s->devname, O_RDWR | O_NONBLOCK, 0); + if (s->fd < 0) + return AVERROR(errno); + + ret = prepare_pools(s, log_ctx); + if (ret < 0) + goto error; + + if (s->output_pool.cfg.format) { + ret = s->output_pool.cfg.format(&s->output_pool, set); + if (ret) { + av_log(log_ctx, fail_log_level, "can't set input format\n"); + goto error; + } + } + + if (s->capture_pool.cfg.format) { + ret = s->capture_pool.cfg.format(&s->capture_pool, set); + if (ret) { + av_log(log_ctx, fail_log_level, "can't to set output format\n"); + goto error; + } + } + + if (s->output_pool.cfg.init && set) { + ret = s->output_pool.cfg.init(&s->output_pool); + if (ret) { + av_log(log_ctx, fail_log_level, "no output pool's buffers\n"); + goto error; + } + } + + if (s->capture_pool.cfg.init && set) { + ret = s->capture_pool.cfg.init(&s->capture_pool); + if (ret) { + av_log(log_ctx, fail_log_level, "no capture pool's buffers\n"); + goto error; + } + } + + av_log(log_ctx, AV_LOG_INFO, "using driver '%s' on card '%s'\n", s->cap.driver, s->cap.card); + +error: + if (!set || ret) { + close(s->fd); + s->fd = 0; + } + + return ret; +} + +int avpriv_v4lm2m_init(V4Lm2mContext* s, void* log_ctx) +{ + char *devname_save = s->devname; + int ret = AVERROR(EINVAL); + char tmpbuf[PATH_MAX]; + struct dirent *dp; + DIR *dirp; + + if (s->devname && *s->devname) + return probe_and_set(s, log_ctx, 1); + + if (!(dirp = opendir("/dev"))) + return AVERROR(errno); + + for (dp = readdir(dirp); dp; dp = readdir(dirp)) { + + if (!strncmp(dp->d_name, "video", sizeof("video") - 1)) { + snprintf(tmpbuf, sizeof(tmpbuf) - 1, "/dev/%s", dp->d_name); + av_log(log_ctx, AV_LOG_DEBUG, "probing %s\n", tmpbuf); + + s->devname = tmpbuf; + ret = probe_and_set(s, log_ctx, 0); + if (!ret) + break; + } + } + closedir(dirp); + + if (ret) { + av_log(log_ctx, AV_LOG_ERROR, "Could not find a valid device\n"); + s->devname = devname_save; + + return ret; + } + + av_log(log_ctx, AV_LOG_INFO, "Using device %s\n", tmpbuf); + ret = probe_and_set(s, log_ctx, 1); + s->devname = devname_save; + + return ret; +} + +int ff_v4lm2m_codec_init(AVCodecContext *avctx) +{ + V4Lm2mContext *s = avctx->priv_data; + + return avpriv_v4lm2m_init(s, avctx); +} + +int avpriv_v4lm2m_end(V4Lm2mContext* s) +{ + avpriv_release_buffer_pool(&s->output_pool); + avpriv_release_buffer_pool(&s->capture_pool); + avpriv_set_stream_status(&s->output_pool, VIDIOC_STREAMOFF); + avpriv_set_stream_status(&s->capture_pool, VIDIOC_STREAMOFF); + close(s->fd); + + return 0; +} + +int ff_v4lm2m_codec_end(AVCodecContext *avctx) +{ + V4Lm2mContext *s = avctx->priv_data; + + av_log(avctx, AV_LOG_DEBUG, "Closing context\n"); + + return avpriv_v4lm2m_end(s); +} diff --git a/libavcodec/v4l2_m2m.h b/libavcodec/v4l2_m2m.h new file mode 100644 index 0000000..6bfd4c4 --- /dev/null +++ b/libavcodec/v4l2_m2m.h @@ -0,0 +1,69 @@ +/* + * V4L2 mem2mem helper functions + * + * Copyright (C) 2017 Alexis Ballier + * Copyright (C) 2017 Jorge Ramirez + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_V4L2_M2M_H +#define AVCODEC_V4L2_M2M_H + +#include "v4l2-buffers.h" +#include "v4l2-common.h" + +#define V4L_M2M_DEFAULT_OPTS \ + { "device",\ + "Path to the device to use",\ + OFFSET(devname),\ + AV_OPT_TYPE_STRING,\ + {.str = NULL }, 0, 0, FLAGS },\ + { "input_memory",\ + "Input memory model: See V4L2_MEMORY_* in videodev2.h. This depends on the HW but default should work with most but would imply useless memcpy()'s if used improperly.",\ + OFFSET(output_pool.memory),\ + AV_OPT_TYPE_INT,\ + {.i64 = V4L2_MEMORY_MMAP},\ + 0, INT_MAX, FLAGS },\ + { "output_memory",\ + "Output memory model: See V4L2_MEMORY_* in videodev2.h. This depends on the HW but default should work with most.",\ + OFFSET(capture_pool.memory),\ + AV_OPT_TYPE_INT,\ + {.i64 = V4L2_MEMORY_MMAP},\ + 0, INT_MAX, FLAGS },\ + { "num_output_pool_buffers",\ + "Number of buffers in the output pool",\ + OFFSET(output_pool.num_buffers),\ + AV_OPT_TYPE_INT,\ + { .i64 = 16 },\ + 4, INT_MAX, FLAGS } + +typedef struct V4Lm2mContext +{ + AVClass *class; + int fd; + char *devname; + struct v4l2_capability cap; + V4LBufferPool output_pool; + V4LBufferPool capture_pool; +} V4Lm2mContext; + +int avpriv_v4lm2m_init(V4Lm2mContext* s, void* log_ctx); +int avpriv_set_pool_format(V4LBufferPool* bp, int set); +int avpriv_v4lm2m_end(V4Lm2mContext* ctx); + +#endif /* AVCODEC_V4L2_M2M_H */ diff --git a/libavcodec/v4l2_m2m_avcodec.h b/libavcodec/v4l2_m2m_avcodec.h new file mode 100644 index 0000000..2d0f1b6 --- /dev/null +++ b/libavcodec/v4l2_m2m_avcodec.h @@ -0,0 +1,32 @@ +/* + * V4L2 mem2mem avcodec helper functions + * + * Copyright (C) 2017 Alexis Ballier + * Copyright (C) 2017 Jorge Ramirez + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_V4L2_M2M_AVCODEC_H +#define AVCODEC_V4L2_M2M_AVCODEC_H + +#include "avcodec.h" + +int ff_v4lm2m_codec_init(AVCodecContext *avctx); +int ff_v4lm2m_codec_end(AVCodecContext *avctx); + +#endif diff --git a/libavcodec/v4l2_m2m_dec.c b/libavcodec/v4l2_m2m_dec.c new file mode 100644 index 0000000..c09ce6a --- /dev/null +++ b/libavcodec/v4l2_m2m_dec.c @@ -0,0 +1,244 @@ +/* + * V4L2 mem2mem decoders + * + * Copyright (C) 2017 Alexis Ballier + * Copyright (C) 2017 Jorge Ramirez + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include "libavutil/pixfmt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/opt.h" +#include "v4l2_m2m_avcodec.h" +#include "v4l2-common.h" +#include "v4l2-buffers.h" +#include "v4l2_m2m.h" +#include "decode.h" +#include "avcodec.h" + +static int try_start(AVCodecContext *avctx) +{ + V4Lm2mContext *s = avctx->priv_data; + struct v4l2_selection selection; + struct v4l2_control ctrl; + int ret; + + if (s->output_pool.streamon && s->capture_pool.streamon) + return 0; + + /* this will report the size of the frame back (see a4lbuf_to_avframe) */ + s->capture_pool.height = avctx->coded_height; + s->capture_pool.width = avctx->coded_width; + + /* start the output process */ + if (!s->output_pool.streamon) { + ret = avpriv_set_stream_status(&s->output_pool, VIDIOC_STREAMON); + if (ret < 0) { + av_log(avctx, AV_LOG_DEBUG, "VIDIOC_STREAMON on output pool\n"); + return ret; + } + } + + /* get the capture format */ + s->capture_pool.format.type = s->capture_pool.type; + ret = ioctl(s->fd, VIDIOC_G_FMT, &s->capture_pool.format); + if (ret) { + av_log(avctx, AV_LOG_DEBUG, "VIDIOC_G_FMT ioctl\n"); + return ret; + } + + /* store what the decoder gives */ + avctx->pix_fmt = avpriv_v4l_fmt_v4l2ff(s->capture_pool.format.fmt.pix_mp.pixelformat, AV_CODEC_ID_RAWVIDEO); + s->capture_pool.av_pix_fmt = avctx->pix_fmt; + + /* set the crop parameters */ + selection.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + selection.r.height = avctx->coded_height; + selection.r.width = avctx->coded_width; + ret = ioctl(s->fd, VIDIOC_S_SELECTION, &selection); + if (!ret) { + ret = ioctl(s->fd, VIDIOC_G_SELECTION, &selection); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "VIDIOC_G_SELECTION ioctl\n"); + } else { + av_log(avctx, AV_LOG_DEBUG, "crop output %dx%d\n", selection.r.width, selection.r.height); + /* update the size of the resulting frame */ + s->capture_pool.height = selection.r.height; + s->capture_pool.width = selection.r.width; + } + } + + /* get the minimum number of buffers required by capture */ + memset(&ctrl, 0, sizeof(ctrl)); + ctrl.id = V4L2_CID_MIN_BUFFERS_FOR_CAPTURE; + ret = ioctl(s->fd, VIDIOC_G_CTRL, &ctrl); + if (ret) { + s->capture_pool.min_queued_buffers = 6; + } else { + s->capture_pool.min_queued_buffers = ctrl.value; + } + + /* init the capture pool */ + if (!s->capture_pool.buffers) { + ret = avpriv_init_v4lbufpool(&s->capture_pool); + if (ret) { + av_log(avctx, AV_LOG_DEBUG, "can't request output buffers\n"); + return ret; + } + } + + /* start the capture process */ + ret = avpriv_set_stream_status(&s->capture_pool, VIDIOC_STREAMON); + if (ret) { + av_log(avctx, AV_LOG_DEBUG, "VIDIOC_STREAMON, on capture pool\n"); + return ret; + } + + + return 0; +} + +static av_cold int v4lm2m_decode_init(AVCodecContext *avctx) +{ + V4Lm2mContext *s = avctx->priv_data; + + s->output_pool.cfg.format = avpriv_set_pool_format; + s->output_pool.cfg.init = avpriv_init_v4lbufpool; + + s->output_pool.av_codec_tag = avctx->codec_tag; + s->output_pool.av_codec_id = avctx->codec_id; + s->output_pool.av_pix_fmt = AV_PIX_FMT_NONE; + s->output_pool.height = avctx->coded_height; + s->output_pool.width = avctx->coded_width; + s->output_pool.default_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + + s->capture_pool.cfg.format = avpriv_set_pool_format; + s->capture_pool.cfg.init = NULL; + + s->capture_pool.av_codec_tag = 0; + s->capture_pool.av_codec_id = AV_CODEC_ID_RAWVIDEO; + s->capture_pool.av_pix_fmt = avctx->pix_fmt; + s->capture_pool.height = avctx->coded_height; + s->capture_pool.width = avctx->coded_width; + s->capture_pool.default_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + + return ff_v4lm2m_codec_init(avctx); +} + +static inline void set_dequeue_mode(V4Lm2mContext *s) +{ + s->capture_pool.blocking_dequeue = 0; + + if (s->output_pool.num_queued >= s->output_pool.num_buffers - 2) + s->capture_pool.blocking_dequeue = 1000; +} + +static int v4lm2m_receive_frame(AVCodecContext *avctx, AVFrame *frame) +{ + V4Lm2mContext *s = avctx->priv_data; + AVPacket avpkt = {0}; + int ret; + + ret = ff_decode_get_packet(avctx, &avpkt); + if (ret < 0 && ret != AVERROR_EOF) + return ret; + + if (avctx->extradata && avctx->extradata_size) { + ret = avpriv_v4l_enqueue_frame_or_pkt_or_buf(&s->output_pool, NULL, + NULL, avctx->extradata, avctx->extradata_size); + if (ret) + return ret; + + (void) try_start(avctx); + } + + ret = avpriv_v4l_enqueue_frame_or_pkt_or_buf(&s->output_pool, NULL, &avpkt, NULL, 0); + if (ret < 0) + return ret; + + ret = try_start(avctx); + if (ret) + return 0; + + set_dequeue_mode(s); + + return avpriv_v4l_dequeue_frame_or_pkt(&s->capture_pool, frame, NULL); +} + +#define OFFSET(x) offsetof(V4Lm2mContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + + static const AVOption options[] = { + V4L_M2M_DEFAULT_OPTS,{ "num_capture_pool_extra_buffers", + "Number of extra buffers in the capture pool", + OFFSET(capture_pool.num_buffers), AV_OPT_TYPE_INT,{.i64 = 6}, 4, INT_MAX, FLAGS}, + { NULL}, + }; + +#define M2MDEC(NAME, LONGNAME, CODEC, bsf_name) \ +static const AVClass v4l2_m2m_ ## NAME ## _dec_class = {\ + .class_name = #NAME "_v4l2_m2m_decoder",\ + .item_name = av_default_item_name,\ + .option = options,\ + .version = LIBAVUTIL_VERSION_INT,\ +};\ +\ +AVCodec ff_ ## NAME ## _v4l2m2m_decoder = { \ + .name = #NAME "_v4l2m2m" ,\ + .long_name = NULL_IF_CONFIG_SMALL("V4L2 mem2mem " LONGNAME " decoder wrapper"),\ + .type = AVMEDIA_TYPE_VIDEO,\ + .id = CODEC ,\ + .priv_data_size = sizeof(V4Lm2mContext),\ + .priv_class = &v4l2_m2m_ ## NAME ## _dec_class,\ + .init = v4lm2m_decode_init,\ + .receive_frame = v4lm2m_receive_frame,\ + .close = ff_v4lm2m_codec_end,\ + .capabilities = CODEC_CAP_DELAY,\ + .bsfs = bsf_name, \ +}; + +#if CONFIG_H263_V4L2M2M_DECODER + M2MDEC(h263 , "H.263" , AV_CODEC_ID_H263, NULL); +#endif + +#if CONFIG_H264_V4L2M2M_DECODER + M2MDEC(h264 , "H.264" , AV_CODEC_ID_H264, "h264_mp4toannexb"); +#endif + +#if CONFIG_MPEG1_V4L2M2M_DECODER + M2MDEC(mpeg1, "MPEG1", AV_CODEC_ID_MPEG1VIDEO, NULL); +#endif + +#if CONFIG_MPEG2_V4L2M2M_DECODER + M2MDEC(mpeg2, "MPEG2", AV_CODEC_ID_MPEG2VIDEO, NULL); +#endif + +#if CONFIG_MPEG4_V4L2M2M_DECODER + M2MDEC(mpeg4, "MPEG4", AV_CODEC_ID_MPEG4, NULL); +#endif + +#if CONFIG_VC1_V4L2M2M_DECODER + M2MDEC(vc1 , "VC1" , AV_CODEC_ID_VC1, NULL); +#endif + +#if CONFIG_VP8_V4L2M2M_DECODER + M2MDEC(vp8 , "VP8" , AV_CODEC_ID_VP8, NULL); +#endif + + diff --git a/libavcodec/v4l2_m2m_enc.c b/libavcodec/v4l2_m2m_enc.c new file mode 100644 index 0000000..899a54f --- /dev/null +++ b/libavcodec/v4l2_m2m_enc.c @@ -0,0 +1,251 @@ +/* + * V4L2 mem2mem encoders + * + * Copyright (C) 2017 Alexis Ballier + * Copyright (C) 2017 Jorge Ramirez + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include "libavutil/pixfmt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/opt.h" +#include "v4l2_m2m_avcodec.h" +#include "v4l2-buffers.h" +#include "v4l2-common.h" +#include "v4l2_m2m.h" +#include "avcodec.h" + +#define SET_V4L_EXT_CTRL(TYPE, ID, VALUE, CLASS, NAME) \ +{ \ + struct v4l2_ext_control ctrl = { 0 };\ + struct v4l2_ext_controls ctrls = { 0 };\ + ctrl.id = ID ;\ + ctrl.TYPE = VALUE ;\ + ctrls.ctrl_class = CLASS ;\ + ctrls.count = 1;\ + ctrls.controls = &ctrl;\ +\ + if ((ret = ioctl(s->fd, VIDIOC_S_EXT_CTRLS, &ctrls)) < 0)\ + av_log(avctx, AV_LOG_WARNING, "Failed to set " NAME "\n");\ +} + +static inline int v4l_h264_profile_from_ff(int p) +{ + switch(p) { + case FF_PROFILE_H264_BASELINE: + return V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE; + case FF_PROFILE_H264_CONSTRAINED_BASELINE: + return V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE; + case FF_PROFILE_H264_MAIN: + return V4L2_MPEG_VIDEO_H264_PROFILE_MAIN; + case FF_PROFILE_H264_EXTENDED: + return V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED; + case FF_PROFILE_H264_HIGH: + return V4L2_MPEG_VIDEO_H264_PROFILE_HIGH; + case FF_PROFILE_H264_HIGH_10: + return V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_10; + case FF_PROFILE_H264_HIGH_10_INTRA: + return V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_10_INTRA; + case FF_PROFILE_H264_HIGH_422: + return V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422; + case FF_PROFILE_H264_HIGH_422_INTRA: + return V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422_INTRA; + case FF_PROFILE_H264_HIGH_444_PREDICTIVE: + return V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_444_PREDICTIVE; + case FF_PROFILE_H264_HIGH_444_INTRA: + return V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_444_INTRA; + } + + return -1; +} + +static inline int v4l_mpeg4_profile_from_ff(int p) +{ + switch(p) { + case FF_PROFILE_MPEG4_SIMPLE: + return V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE; + case FF_PROFILE_MPEG4_ADVANCED_SIMPLE: + return V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_SIMPLE; + case FF_PROFILE_MPEG4_CORE: + return V4L2_MPEG_VIDEO_MPEG4_PROFILE_CORE; + case FF_PROFILE_MPEG4_SIMPLE_SCALABLE: + return V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE_SCALABLE; + case FF_PROFILE_MPEG4_ADVANCED_CODING: + return V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_CODING_EFFICIENCY; + } + + return -1; +} + +static av_cold int v4lm2m_encode_init(AVCodecContext *avctx) +{ + V4Lm2mContext *s = avctx->priv_data; + int ret, val; + + s->output_pool.cfg.format = avpriv_set_pool_format; + s->output_pool.cfg.init = avpriv_init_v4lbufpool; + s->output_pool.av_pix_fmt = avctx->pix_fmt; + s->output_pool.width = avctx->width; + s->output_pool.height = avctx->height; + s->output_pool.av_codec_id = AV_CODEC_ID_RAWVIDEO; + s->output_pool.default_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + + s->capture_pool.cfg.format = avpriv_set_pool_format; + s->capture_pool.cfg.init = avpriv_init_v4lbufpool; + s->capture_pool.min_queued_buffers = 1; + s->capture_pool.av_pix_fmt = AV_PIX_FMT_NONE; + s->capture_pool.av_codec_id = avctx->codec_id; + s->capture_pool.default_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + + if (ret = ff_v4lm2m_codec_init(avctx)) + return ret; + + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_GOP_SIZE, avctx->gop_size, V4L2_CTRL_CLASS_MPEG, "gop size"); + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_BITRATE , avctx->bit_rate, V4L2_CTRL_CLASS_MPEG, "bit rate"); + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_HEADER_MODE, V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME, V4L2_CTRL_CLASS_MPEG, "header mode"); + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_B_FRAMES, avctx->max_b_frames, V4L2_CTRL_CLASS_MPEG, "number of B-frames"); + + switch(avctx->codec_id) { + case AV_CODEC_ID_H264: + val = v4l_h264_profile_from_ff(avctx->profile); + if (val >= 0) { + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_H264_PROFILE, val, V4L2_CTRL_CLASS_MPEG, "h264 profile"); + } + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_H264_MIN_QP, avctx->qmin, V4L2_CTRL_CLASS_MPEG, "minimum video quantizer scale"); + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_H264_MAX_QP, avctx->qmax, V4L2_CTRL_CLASS_MPEG, "maximum video quantizer scale"); + break; + case AV_CODEC_ID_MPEG4: + val = v4l_mpeg4_profile_from_ff(avctx->profile); + if (val >= 0) { + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE, val, V4L2_CTRL_CLASS_MPEG, "mpeg4 profile"); + } + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_MPEG4_MIN_QP, avctx->qmin, V4L2_CTRL_CLASS_MPEG, "minimum video quantizer scale"); + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_MPEG4_MAX_QP, avctx->qmax, V4L2_CTRL_CLASS_MPEG, "maximum video quantizer scale"); + if (avctx->flags & CODEC_FLAG_QPEL) { + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_MPEG4_QPEL, 1, V4L2_CTRL_CLASS_MPEG, "qpel"); + } + break; + case AV_CODEC_ID_H263: + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_H263_MIN_QP, avctx->qmin, V4L2_CTRL_CLASS_MPEG, "minimum video quantizer scale"); + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_H263_MAX_QP, avctx->qmax, V4L2_CTRL_CLASS_MPEG, "maximum video quantizer scale"); + break; + case AV_CODEC_ID_VP8: + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_VPX_MIN_QP, avctx->qmin, V4L2_CTRL_CLASS_MPEG, "minimum video quantizer scale"); + SET_V4L_EXT_CTRL(value, V4L2_CID_MPEG_VIDEO_VPX_MAX_QP, avctx->qmax, V4L2_CTRL_CLASS_MPEG, "maximum video quantizer scale"); + break; + } + return 0; +} + +static int v4lm2m_encode_frame(AVCodecContext *avctx, AVPacket *pkt, + const AVFrame *pict, int *got_packet) +{ + V4Lm2mContext *s = avctx->priv_data; + int ret; + + if (pict) { + ret = avpriv_v4l_enqueue_frame_or_pkt_or_buf(&s->output_pool, pict, NULL, NULL, 0); + if (ret < 0) + return ret; + + if (!s->output_pool.streamon && (ret = avpriv_set_stream_status(&s->output_pool, VIDIOC_STREAMON))) { + av_log(avctx, AV_LOG_ERROR, "VIDIOC_STREAMON failed on output pool\n"); + return ret; + } + + if (!s->capture_pool.streamon && (ret = avpriv_set_stream_status(&s->capture_pool, VIDIOC_STREAMON))) { + av_log(avctx, AV_LOG_ERROR, "VIDIOC_STREAMON failed on capture pool\n"); + return ret; + } + + } else if (s->output_pool.streamon) { + /* last frame, stop encoder */ + struct v4l2_encoder_cmd cmd; + memset(&cmd, 0, sizeof(cmd)); + cmd.cmd = V4L2_ENC_CMD_STOP; + + ret = ioctl(s->fd, VIDIOC_ENCODER_CMD, &cmd); + if (ret) + av_log(avctx, AV_LOG_ERROR, "Failed to stop encoder (%s)\n", av_err2str(AVERROR(errno))); + + s->output_pool.streamon = 0; + } + + if (s->output_pool.num_queued >= s->output_pool.num_buffers - 2 || !pict) { + s->capture_pool.blocking_dequeue = 100; + } else { + s->capture_pool.blocking_dequeue = 0; + } + + ret = avpriv_v4l_dequeue_frame_or_pkt(&(s->capture_pool), NULL, pkt); + + if (AVERROR(EAGAIN) == ret ) + return 0; + + if (!ret) + *got_packet = 1; + + return ret; +} + +#define OFFSET(x) offsetof(V4Lm2mContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +static const AVOption options[] = { + V4L_M2M_DEFAULT_OPTS, { "num_capture_pool_buffers", "Number of buffers in the capture pool", + OFFSET(capture_pool.num_buffers), AV_OPT_TYPE_INT, {.i64 = 4 }, 4, INT_MAX, FLAGS }, { NULL }, +}; + +#define M2MENC(NAME, LONGNAME, CODEC) \ +static const AVClass v4l2_m2m_ ## NAME ## _enc_class = {\ + .class_name = #NAME "_v4l2_m2m_encoder",\ + .item_name = av_default_item_name,\ + .option = options,\ + .version = LIBAVUTIL_VERSION_INT,\ +};\ +\ +AVCodec ff_ ## NAME ## _v4l2m2m_encoder = { \ + .name = #NAME "_v4l2m2m" ,\ + .long_name = NULL_IF_CONFIG_SMALL("V4L2 mem2mem " LONGNAME " encoder wrapper"),\ + .type = AVMEDIA_TYPE_VIDEO,\ + .id = CODEC ,\ + .priv_data_size = sizeof(V4Lm2mContext),\ + .priv_class = &v4l2_m2m_ ## NAME ##_enc_class,\ + .init = v4lm2m_encode_init,\ + .encode2 = v4lm2m_encode_frame,\ + .close = ff_v4lm2m_codec_end,\ + .capabilities = CODEC_CAP_DELAY,\ +}; + +#if CONFIG_H263_V4L2M2M_ENCODER +M2MENC(h263 , "H.263" , AV_CODEC_ID_H263 ); +#endif + +#if CONFIG_H264_V4L2M2M_ENCODER +M2MENC(h264 , "H.264" , AV_CODEC_ID_H264 ); +#endif + +#if CONFIG_MPEG4_V4L2M2M_ENCODER +M2MENC(mpeg4, "MPEG4", AV_CODEC_ID_MPEG4); +#endif + +#if CONFIG_VP8_V4L2M2M_ENCODER +M2MENC(vp8 , "VP8" , AV_CODEC_ID_VP8 ); +#endif