[FFmpeg-devel] patch for a new H.264 codec with Matrox m264 card.

Submitted by Yufei He on March 14, 2019, 5:55 p.m.

Details

Message ID f6d69466-86fc-fcb7-b9e0-b06b7356cb88@matrox.com
State New
Headers show

Commit Message

Yufei He March 14, 2019, 5:55 p.m.
Hi

Here is the patch for a new H.264 codec with Matrox m264 card.

Please review.

Thanks.

Yufei.

Comments

Tomas Härdin March 15, 2019, 2:02 p.m.
tor 2019-03-14 klockan 17:55 +0000 skrev Yufei He:
> Hi
> 
> Here is the patch for a new H.264 codec with Matrox m264 card.
> 
> Please review.

> --- a/libavcodec/codec_desc.c
> +++ b/libavcodec/codec_desc.c
> @@ -1705,7 +1705,6 @@ static const AVCodecDescriptor
> codec_descriptors[] = {
>          .long_name = NULL_IF_CONFIG_SMALL("Gryphon's Anim
> Compressor"),
>          .props     = AV_CODEC_PROP_LOSSY,
>      },
> -
>      /* various PCM "codecs" */
>      {
>          .id        = AV_CODEC_ID_PCM_S16LE,
> diff --git a/libavcodec/m264dec.c b/libavcodec/m264dec.c

Stray hunk?

> +    #ifdef _WIN32
> +    lib_handle = dlopen("mvM264Ffmpeg.dll", RTLD_LAZY);
> +    #else   
> +    lib_handle = dlopen("libmvM264Ffmpeg.so", RTLD_LAZY);
> +    #endif

Still dlopen() I see. You should link to these libraries properly, as
others have said, and add appropriate license stuff to configure.

/Tomas
Ronald S. Bultje March 15, 2019, 3:07 p.m.
Hi guys,

On Thu, Mar 14, 2019 at 1:55 PM Yufei He <yhe@matrox.com> wrote:

> Hi
>
> Here is the patch for a new H.264 codec with Matrox m264 card.
>

I want to bring up again that this library is closed-source. I don't think
FFmpeg should link to closed-source software in its public mainline
version. Matrox is obviously free :-) to maintain their own modified
version as long as they comply with all licensing conditions.

On IRC, we just discussed the following:
- --enable-nonfree was initially intended for dealing sort of roughly with
incompatible opensource licenses, for example AAC encoders or GPL +
LGPL-compatible-but-LGPL-incomptible libraries;
- nowadays, --enable-nonfree is also used for NDI (which Kieran sent a
patch to remove already) and now this.

I would like to request a vote amongst frequent developers (and if you
don't count me amongst that, that's fine) to decide whether closed-source
software integration in general, as well as this one in particular, is OK.
I don't think the system library provision applies here, since you'll be
hard-pressed to find a system with Matrox hardware (it's high-$$$
enterprise stuff).

Ronald
Jean-Baptiste Kempf March 15, 2019, 3:23 p.m.
On Fri, 15 Mar 2019, at 15:02, Tomas Härdin wrote:
> > +    #ifdef _WIN32
> > +    lib_handle = dlopen("mvM264Ffmpeg.dll", RTLD_LAZY);
> > +    #else   
> > +    lib_handle = dlopen("libmvM264Ffmpeg.so", RTLD_LAZY);
> > +    #endif
> 
> Still dlopen() I see. You should link to these libraries properly, as
> others have said, and add appropriate license stuff to configure.

What are those libraries? Are they drivers/system libraries?

They seem to be a wrapper done just for the FFMpeg integration. If so, what is the license of those?
Yufei He March 15, 2019, 5:42 p.m.
Hi Jean-Baptiste

Sorry for the complexity and confusion to so many people involved on 
this new patch.

We have been working on this for 4 months because some of our big 
customers on internet business have been asking Matrox to make our M264 
card to support FFmpeg on h.264 transcoding, esp on 4k and HD files.

M264 is very powerful on 4k and HD decoding and encoding.

M264 uses a h.264 encoding chip made by Matrox, and there are a lot of 
modules involved to make it work. Those modules are not public in 
Matrox, which is a company of 40 years.

Yes, mvM264Ffmpeg.dll is a wrapper to those modules, it involves all the 
interfaces talking to the chip.

I did not find a better way to make this work.

Thanks a lot to all people involved.

Yufei.


On 03/15/2019 11:23 AM, Jean-Baptiste Kempf wrote:
> On Fri, 15 Mar 2019, at 15:02, Tomas Härdin wrote:

>>> +    #ifdef _WIN32

>>> +    lib_handle = dlopen("mvM264Ffmpeg.dll", RTLD_LAZY);

>>> +    #else

>>> +    lib_handle = dlopen("libmvM264Ffmpeg.so", RTLD_LAZY);

>>> +    #endif

>> Still dlopen() I see. You should link to these libraries properly, as

>> others have said, and add appropriate license stuff to configure.

> What are those libraries? Are they drivers/system libraries?

>

> They seem to be a wrapper done just for the FFMpeg integration. If so, what is the license of those?

>
Nicolas George March 15, 2019, 5:45 p.m.
Yufei He (12019-03-15):
> I did not find a better way to make this work.

It exists: make your client libraries GPL-compatible.

Regards,
BIGLER Don (Framatome) March 15, 2019, 6:19 p.m.
>Yufei He (12019-03-15):
>> I did not find a better way to make this work.

>It exists: make your client libraries GPL-compatible.

Or alternatively creating open-source headers that allow the m264 drivers to be used similar to NVidia's Video Codec SDK:
https://github.com/FFmpeg/nv-codec-headers
https://github.com/FFmpeg/FFmpeg/blob/master/compat/cuda/dynlink_loader.h

Regards,
Don Bigler
Ronald S. Bultje March 15, 2019, 6:30 p.m.
Hi,

On Fri, Mar 15, 2019 at 2:19 PM BIGLER Don (Framatome) <
don.bigler@framatome.com> wrote:

> >Yufei He (12019-03-15):
> >> I did not find a better way to make this work.
>
> >It exists: make your client libraries GPL-compatible.
>
> Or alternatively creating open-source headers that allow the m264 drivers
> to be used similar to NVidia's Video Codec SDK:
>

No, that doesn't address the issue at all. Release the nvidia SDK as open
source, *that* addresses the issue.

Ronald
Marton Balint March 15, 2019, 10:26 p.m.
On Fri, 15 Mar 2019, Ronald S. Bultje wrote:

> Hi guys,
>
> On Thu, Mar 14, 2019 at 1:55 PM Yufei He <yhe@matrox.com> wrote:
>
>> Hi
>>
>> Here is the patch for a new H.264 codec with Matrox m264 card.
>>
>
> I want to bring up again that this library is closed-source. I don't think
> FFmpeg should link to closed-source software in its public mainline
> version. Matrox is obviously free :-) to maintain their own modified
> version as long as they comply with all licensing conditions.
>
> On IRC, we just discussed the following:
> - --enable-nonfree was initially intended for dealing sort of roughly with
> incompatible opensource licenses, for example AAC encoders or GPL +
> LGPL-compatible-but-LGPL-incomptible libraries;
> - nowadays, --enable-nonfree is also used for NDI (which Kieran sent a
> patch to remove already) and now this.
>
> I would like to request a vote amongst frequent developers (and if you
> don't count me amongst that, that's fine) to decide whether closed-source
> software integration in general, as well as this one in particular, is OK.
> I don't think the system library provision applies here, since you'll be
> hard-pressed to find a system with Matrox hardware (it's high-$$$
> enterprise stuff).

I think it is a mistake to tackle this from a licensing point of view 
because it is debatable what counts as a system library or what counts as 
a closed plugin of an LGPL library. Instead of debates about components we
will get debates about licensing. Totally pointless.

Therefore I don't think a generic vote about closed source is a good idea. 
To allow or not a certain closed source component is mostly subjective so 
doing a vote on a case by case basis seems the most fair to me. We don't 
have _that_ many closed source submissions.

When voting, the following can be considered by each voter:
- submitter reputation (past contributions/license awareness/maintenance capability)
- capability to build the component (headers/SDK price/licensing/NDA)
- capability to use the component (library/driver/HW price)
- component target (consumer/professional/enterprise)
- potentional user base

Regards,
Marton
Michael Niedermayer March 15, 2019, 10:34 p.m.
On Thu, Mar 14, 2019 at 05:55:41PM +0000, Yufei He wrote:
> Hi
> 
> Here is the patch for a new H.264 codec with Matrox m264 card.
> 
> Please review.
> 
> Thanks.
> 
> Yufei.
> 
> 

>  Changelog               |    1 
>  configure               |    2 
>  libavcodec/Makefile     |    2 
>  libavcodec/allcodecs.c  |    2 
>  libavcodec/codec_desc.c |    1 
>  libavcodec/m264dec.c    |  315 ++++++++++++++++++++++++++++++++++++++++++++++++
>  libavcodec/m264dec.h    |   39 +++++
>  libavcodec/m264enc.c    |  296 +++++++++++++++++++++++++++++++++++++++++++++
>  libavcodec/m264enc.h    |   38 +++++
>  9 files changed, 695 insertions(+), 1 deletion(-)
> 0ea83a5b3013cb0658872bee0f4c451ddf2ef473  0001-H.264-encoder-and-decoder-support-with-Matrox-M264.patch
> From b1f7779028378f3e0d5360766a8591d0cf5a0a71 Mon Sep 17 00:00:00 2001
> From: yhe <yhe@matrox.com>
> Date: Thu, 14 Mar 2019 10:10:46 -0400
> Subject: [PATCH] H.264 encoder and decoder support with Matrox M264.

This breaks "make -j12 testprogs"

libavcodec/libavcodec.a(m264enc.o): In function `ff_m264_send_frame':
m264enc.c:(.text+0x88): undefined reference to `sws_scale'
m264enc.c:(.text+0x15e): undefined reference to `sws_getContext'
libavcodec/libavcodec.a(m264enc.o): In function `ff_m264_encode_close':
m264enc.c:(.text.unlikely+0x21c): undefined reference to `sws_freeContext'
collect2: error: ld returned 1 exit status
make: *** [libavcodec/tests/snowenc] Error 1

[...]
Tomas Härdin March 17, 2019, 11:56 a.m.
fre 2019-03-15 klockan 18:45 +0100 skrev Nicolas George:
> Yufei He (12019-03-15):
> > I did not find a better way to make this work.
> 
> It exists: make your client libraries GPL-compatible.

I second this

/Tomas

Patch hide | download patch | download mbox

From b1f7779028378f3e0d5360766a8591d0cf5a0a71 Mon Sep 17 00:00:00 2001
From: yhe <yhe@matrox.com>
Date: Thu, 14 Mar 2019 10:10:46 -0400
Subject: [PATCH] H.264 encoder and decoder support with Matrox M264.

---
 Changelog               |   1 +
 configure               |   2 +
 libavcodec/Makefile     |   2 +
 libavcodec/allcodecs.c  |   2 +
 libavcodec/codec_desc.c |   1 -
 libavcodec/m264dec.c    | 315 ++++++++++++++++++++++++++++++++++++++++++++++++
 libavcodec/m264dec.h    |  39 ++++++
 libavcodec/m264enc.c    | 296 +++++++++++++++++++++++++++++++++++++++++++++
 libavcodec/m264enc.h    |  38 ++++++
 9 files changed, 695 insertions(+), 1 deletion(-)
 create mode 100644 libavcodec/m264dec.c
 create mode 100644 libavcodec/m264dec.h
 create mode 100644 libavcodec/m264enc.c
 create mode 100644 libavcodec/m264enc.h

diff --git a/Changelog b/Changelog
index 4d80e5b..ce0daf8 100644
--- a/Changelog
+++ b/Changelog
@@ -19,6 +19,7 @@  version <next>:
 - ARBC decoder
 - libaribb24 based ARIB STD-B24 caption support (profiles A and C)
 - Support decoding of HEVC 4:4:4 content in nvdec and cuviddec
+- Support h.264 encoding and decoding with Matrox M264 card.
 
 
 version 4.1:
diff --git a/configure b/configure
index dcead3a..5a5d7d5 100755
--- a/configure
+++ b/configure
@@ -2697,6 +2697,8 @@  h263p_decoder_select="h263_decoder"
 h263p_encoder_select="h263_encoder"
 h264_decoder_select="cabac golomb h264chroma h264dsp h264parse h264pred h264qpel videodsp"
 h264_decoder_suggest="error_resilience"
+h264_m264_decoder_deps_any="libdl"
+h264_m264_encoder_deps_any="libdl"
 hap_decoder_select="snappy texturedsp"
 hap_encoder_deps="libsnappy"
 hap_encoder_select="texturedspenc"
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 15c43a8..8e8f9ab 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -348,6 +348,8 @@  OBJS-$(CONFIG_H264_DECODER)            += h264dec.o h264_cabac.o h264_cavlc.o \
                                           h264_slice.o h264data.o
 OBJS-$(CONFIG_H264_AMF_ENCODER)        += amfenc_h264.o
 OBJS-$(CONFIG_H264_CUVID_DECODER)      += cuviddec.o
+OBJS-$(CONFIG_H264_M264_DECODER)       += m264dec.o
+OBJS-$(CONFIG_H264_M264_ENCODER)       += m264enc.o
 OBJS-$(CONFIG_H264_MEDIACODEC_DECODER) += mediacodecdec.o
 OBJS-$(CONFIG_H264_MMAL_DECODER)       += mmaldec.o
 OBJS-$(CONFIG_H264_NVENC_ENCODER)      += nvenc_h264.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index b26aeca..c03a5a3 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -737,6 +737,8 @@  extern AVCodec ff_libopenh264_encoder;
 extern AVCodec ff_libopenh264_decoder;
 extern AVCodec ff_h264_amf_encoder;
 extern AVCodec ff_h264_cuvid_decoder;
+extern AVCodec ff_h264_m264_decoder;
+extern AVCodec ff_h264_m264_encoder;
 extern AVCodec ff_h264_nvenc_encoder;
 extern AVCodec ff_h264_omx_encoder;
 extern AVCodec ff_h264_qsv_encoder;
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index a3de8e1..55c02a8 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -1705,7 +1705,6 @@  static const AVCodecDescriptor codec_descriptors[] = {
         .long_name = NULL_IF_CONFIG_SMALL("Gryphon's Anim Compressor"),
         .props     = AV_CODEC_PROP_LOSSY,
     },
-
     /* various PCM "codecs" */
     {
         .id        = AV_CODEC_ID_PCM_S16LE,
diff --git a/libavcodec/m264dec.c b/libavcodec/m264dec.c
new file mode 100644
index 0000000..f16716e
--- /dev/null
+++ b/libavcodec/m264dec.c
@@ -0,0 +1,315 @@ 
+/*
+ * M264 H.264 video decoder
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * Matrox M264 card supports h.264 encoding and decoding.
+ */
+
+#include "decode.h"
+#include "cabac.h"
+#include "error_resilience.h"
+#include "h264_parse.h"
+#include "h264_ps.h"
+#include "h264_sei.h"
+#include "h2645_parse.h"
+#include "h264chroma.h"
+#include "h264dsp.h"
+#include "h264pred.h"
+#include "h264qpel.h"
+#include "internal.h"
+#include "mpegutils.h"
+#include "parser.h"
+#include "qpeldsp.h"
+#include "rectangle.h"
+#include "videodsp.h"
+#include "config.h"
+#include "libswscale/swscale.h"
+#include "m264dec.h"
+#ifdef _WIN32
+#include "compat/w32dlfcn.h"
+#else
+#include <dlfcn.h>
+#endif
+
+typedef struct {
+   uint16_t width;
+   uint16_t height;
+   uint16_t scale;
+   uint16_t rate;
+   uint16_t gop_size;
+   uint32_t bitrate;
+   uint8_t  field_order;
+} M264DecoderInfo;
+
+typedef struct {
+    int (*init_m264_decoder)(void *decoder_info, void **m264_decoder_object);
+    int (*exit_m264_decoder)(void *m264_decoder_object);
+    int (*send_packet)(void *m264_decoder_object, void *extradata, unsigned long extradata_size, void *buffer, unsigned long buffer_size);
+    int (*receive_frame)(void *m264_decoder_object, void **buffer_context, void **buffer, unsigned long *buffer_size);
+    int (*release_frame_buffer)(void *m264_decoder_object, void *buffer_context);
+    void *lib_handle;
+    void *decoder_context;
+    int64_t frames;
+    int eof;
+    AVBSFContext *ctx;
+} M264Decoder;
+
+int m264_mp4_to_annexb(AVCodecContext *avctx, AVPacket *packet)
+{
+    M264Decoder *m264_decoder = (M264Decoder *)avctx->priv_data;
+    int ret = 0;
+
+    if (m264_decoder->ctx)
+    {
+        ret = av_bsf_send_packet(m264_decoder->ctx, packet);
+        if (ret < 0)
+            return ret;
+
+        ret = av_bsf_receive_packet(m264_decoder->ctx, packet);
+        if (ret < 0)
+            return ret;
+    }
+
+    return ret;
+}
+
+int m264_init_spspps_mp4(AVCodecContext *avctx)
+{
+    M264Decoder *m264_decoder = (M264Decoder *)avctx->priv_data;
+    int ret = 0;
+    const AVBitStreamFilter *filter = av_bsf_get_by_name("h264_mp4toannexb");
+    AVBSFContext *ctx = NULL;
+
+    ret = av_bsf_alloc(filter, &ctx);
+    if (ret < 0)
+        return ret;
+
+    ret = avcodec_parameters_from_context(ctx->par_in, avctx);
+    if (ret < 0)
+        return ret;
+
+    ret = av_bsf_init(ctx);
+    if (ret < 0)
+        return ret;
+
+    m264_decoder->ctx = ctx;
+
+    return ret;
+}
+
+av_cold int ff_m264_decode_init(AVCodecContext *avctx)
+{
+    int ret = 0;
+    void *lib_handle;
+    M264Decoder *m264_decoder;
+    M264DecoderInfo decoder_info;
+
+    m264_decoder = av_mallocz(sizeof(M264Decoder));
+    if (!m264_decoder) 
+        return AVERROR(ENOMEM);
+
+    avctx->priv_data = m264_decoder;
+
+    #ifdef _WIN32
+    lib_handle = dlopen("mvM264Ffmpeg.dll", RTLD_LAZY);
+    #else   
+    lib_handle = dlopen("libmvM264Ffmpeg.so", RTLD_LAZY);
+    #endif
+
+    if (!lib_handle) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_DECODER_NOT_FOUND;
+    }
+
+    m264_decoder->lib_handle = lib_handle;
+
+    m264_decoder->init_m264_decoder = dlsym(lib_handle, "m264_ffmpeg_decoder_init");
+    if (!m264_decoder->init_m264_decoder) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_DECODER_NOT_FOUND;
+    }
+
+    m264_decoder->exit_m264_decoder = dlsym(lib_handle, "m264_ffmpeg_decoder_exit");
+    if (!m264_decoder->exit_m264_decoder) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_DECODER_NOT_FOUND;
+    }
+
+    m264_decoder->send_packet = dlsym(lib_handle, "m264_ffmpeg_decoder_send_packet");
+    if (!m264_decoder->send_packet) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_DECODER_NOT_FOUND;
+    }
+
+    m264_decoder->receive_frame = dlsym(lib_handle, "m264_ffmpeg_decoder_receive_frame");
+    if (!m264_decoder->receive_frame) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_DECODER_NOT_FOUND;
+    }
+
+    m264_decoder->release_frame_buffer = dlsym(lib_handle, "m264_ffmpeg_decoder_release_frame_buffer");
+    if (!m264_decoder->release_frame_buffer) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_DECODER_NOT_FOUND;
+    }
+
+    m264_decoder->eof = 0;
+
+    decoder_info.width = avctx->width;
+    decoder_info.height = avctx->height;
+    decoder_info.scale = avctx->framerate.num;
+    decoder_info.rate = avctx->framerate.den;
+    decoder_info.field_order = (uint8_t)avctx->field_order;
+    av_log(avctx, AV_LOG_DEBUG, "m264_decode_init_h264: avctx->field_order is %d \n", avctx->field_order);
+    av_log(avctx, AV_LOG_DEBUG, "m264_decode_init_h264: avctx->width = %d, avctx->height = %d, avctx->framerate.num = %d, avctx->framerate.den = %d\n",
+                                 avctx->width, avctx->height, avctx->framerate.num, avctx->framerate.den);
+
+    ret = m264_decoder->init_m264_decoder(&decoder_info, &(m264_decoder->decoder_context));
+    if (ret < 0)
+        return ret;
+
+    avctx->pix_fmt = AV_PIX_FMT_YUYV422; // m264 card only supports YUYV422 as input. 
+    avctx->bits_per_raw_sample = 8;     
+
+    if (avctx->extradata_size > 0 && avctx->extradata)
+    {
+        ret = m264_init_spspps_mp4(avctx);
+    }
+
+    return ret;
+}
+
+int ff_m264_receive_frame(AVCodecContext *avctx, AVFrame *frame)
+{
+    int ret = 0;
+    AVPacket pkt = {0};
+    void *buffer_context = NULL;
+    void *buffer = NULL;
+    unsigned long buffer_size = 0;
+    unsigned long line_size;
+    int line;
+    int actual_line_size;
+    unsigned char *srcmem,*dstmem;
+
+    M264Decoder *m264_decoder = (M264Decoder *)avctx->priv_data;
+    ret = m264_decoder->receive_frame(m264_decoder->decoder_context, &buffer_context, &buffer, &buffer_size);
+    if (ret == -1) {
+        av_log(avctx, AV_LOG_DEBUG, "ff_m264_receive_frame: m264_decoder->receive_packet get eos, return AVERROR_EOF \n");
+        return AVERROR_EOF;
+    }
+
+    if (buffer_size > 0) {
+        if ((ret = ff_get_buffer(avctx, frame, 0)) < 0)
+            return ret;
+
+        line_size = buffer_size / avctx->height;
+        srcmem = (unsigned char *)buffer;
+        dstmem = (unsigned char *)frame->data[0];
+        actual_line_size = avctx->width * 2;
+
+        for (line = 0; line < avctx->height; line++) {
+            memcpy(dstmem, srcmem, actual_line_size);
+            dstmem += frame->linesize[0];
+            srcmem += line_size;
+        }
+
+        frame->pts = m264_decoder->frames;
+        m264_decoder->frames++;
+        av_log(avctx, AV_LOG_DEBUG, "ff_m264_receive_frame: m264_decoder->frames = %d\n", (int)m264_decoder->frames);
+        ret = m264_decoder->release_frame_buffer(m264_decoder->decoder_context, buffer_context);
+    }
+    else
+    {
+        if (m264_decoder->eof != 1) {
+            ret = ff_decode_get_packet(avctx, &pkt);
+            av_log(avctx, AV_LOG_DEBUG, "decoder: ff_m264_receive_frame: ff_decode_get_packet  pkt.size = %d\n", pkt.size);
+            if (ret < 0 && ret != AVERROR_EOF) 
+                return ret;
+        }
+
+        if (ret == AVERROR_EOF ) {
+            if (m264_decoder->eof != 1) {
+                pkt.data = NULL;
+                pkt.size = 0;
+                m264_decoder->eof = 1;
+                ret = m264_decoder->send_packet(m264_decoder->decoder_context, NULL, 0, pkt.data, pkt.size);
+            }
+        }
+        else 
+        {
+            if (m264_decoder->eof != 1) {
+                av_log(avctx, AV_LOG_DEBUG, "decoder: ff_m264_receive_frame: pkt.side_data_elems = %d\n", pkt.side_data_elems);
+
+                if (pkt.data != NULL)
+                {
+                    ret = m264_mp4_to_annexb(avctx, &pkt);
+                    if (ret < 0)
+                        return ret;
+                }
+
+                if (pkt.flags)
+                    ret = m264_decoder->send_packet(m264_decoder->decoder_context, m264_decoder->ctx->par_out->extradata, m264_decoder->ctx->par_out->extradata_size, pkt.data, pkt.size);
+                else
+                    ret = m264_decoder->send_packet(m264_decoder->decoder_context, NULL , 0, pkt.data, pkt.size);
+            }
+        }
+        av_packet_unref(&pkt);
+        ret = AVERROR(EAGAIN);
+    }
+    return ret;
+}
+
+int av_cold ff_m264_decode_close(AVCodecContext *avctx)
+{
+    M264Decoder *m264_decoder = (M264Decoder *)avctx->priv_data;
+
+    if (m264_decoder)
+    {
+        if (m264_decoder->exit_m264_decoder)
+            m264_decoder->exit_m264_decoder(m264_decoder->decoder_context);
+            
+        if (m264_decoder->lib_handle)
+            dlclose(m264_decoder->lib_handle);   
+
+        if (m264_decoder->ctx)
+        {
+            av_bsf_free(&m264_decoder->ctx);
+        }
+
+        av_free(m264_decoder);
+        avctx->priv_data = NULL;
+    }
+
+    return 0;
+}
+
+AVCodec ff_h264_m264_decoder = {
+    .name           = "m264",
+    .long_name      = NULL_IF_CONFIG_SMALL("Matrox M264 Codec"),
+    .type           = AVMEDIA_TYPE_VIDEO,
+    .id             = AV_CODEC_ID_H264,
+    .init           = ff_m264_decode_init,
+    .receive_frame  = ff_m264_receive_frame,
+    .close          = ff_m264_decode_close,
+    .capabilities   = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE,
+    .caps_internal  = FF_CODEC_CAP_INIT_CLEANUP,
+    .wrapper_name   = "m264",
+};
diff --git a/libavcodec/m264dec.h b/libavcodec/m264dec.h
new file mode 100644
index 0000000..abcedb0
--- /dev/null
+++ b/libavcodec/m264dec.h
@@ -0,0 +1,39 @@ 
+/*
+ * M264 H.264 video decoder
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * Matrox M264 card supports h.264 encoding and decoding.
+ */
+
+#ifndef AVCODEC_M264DEC_H
+#define AVCODEC_M264DEC_H
+
+#include "libavutil/fifo.h"
+
+#include "avcodec.h"
+
+int ff_m264_decode_init(AVCodecContext *avctx);
+int ff_m264_decode_close(AVCodecContext *avctx);
+int ff_m264_receive_frame(AVCodecContext *avctx, AVFrame *frame);
+int m264_init_spspps_mp4(AVCodecContext *avctx);
+int m264_mp4_to_annexb(AVCodecContext *avctx, AVPacket *packet);
+
+#endif //AVCODEC_M264DEC_H
diff --git a/libavcodec/m264enc.c b/libavcodec/m264enc.c
new file mode 100644
index 0000000..14cd570
--- /dev/null
+++ b/libavcodec/m264enc.c
@@ -0,0 +1,296 @@ 
+/*
+ * M264 H.264 video encoder
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * Matrox M264 card supports h.264 encoding and decoding.
+ */
+
+#include "libswscale/swscale.h"
+#include "m264enc.h"
+#include "internal.h"
+
+#ifdef _WIN32
+#include "compat/w32dlfcn.h"
+#else
+#include <dlfcn.h>
+#endif
+
+typedef struct {
+    uint16_t width;
+    uint16_t height;
+    uint16_t scale;
+    uint16_t rate;
+    uint16_t gop_size;
+    uint32_t bitrate;
+} M264EncoderInfo;
+
+typedef struct {
+    int (*init_m264_encoder)(void *encoder_info, void **m264_encoder_object);
+    int (*exit_m264_encoder)(void *m264_encoder_object);
+    int (*get_extradata)(void *m264_encoder_object, void **extradata, unsigned long *extradata_size);
+    int (*get_uncompressed_buffer)(void *m264_encoder_object, void **buffer_context, void **buffer, unsigned long *buffer_size);
+    int (*send_frame)(void *m264_encoder_object, void *buffer_context, int polarity);
+    int (*receive_packet)(void *m264_encoder_object, void *buffer_context, void **buffer, unsigned long *buffer_size, int32_t *pts, int32_t *dts, int8_t *key);
+    int (*release_packet_buffer)(void *m264_encoder_object, void *buffer_context);
+    void *lib_handle;
+    void *encoder_context;
+    struct SwsContext *sw_context;
+} M264Encoder;
+
+av_cold int ff_m264_encode_init(AVCodecContext *avctx)
+{
+    int ret = 0;
+    void *lib_handle;
+    M264Encoder *m264_encoder;
+    M264EncoderInfo encoder_info;
+    void *extradata = NULL;
+    unsigned long extradata_size = 0;
+
+    m264_encoder =  av_mallocz(sizeof(M264Encoder));
+    if (!m264_encoder) 
+        return AVERROR(ENOMEM);
+
+    avctx->priv_data = m264_encoder;
+
+    #ifdef _WIN32
+    lib_handle = dlopen("mvM264Ffmpeg.dll", RTLD_LAZY);
+    #else   
+    lib_handle = dlopen("libmvM264Ffmpeg.so", RTLD_LAZY);
+    #endif
+
+    if (!lib_handle) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_ENCODER_NOT_FOUND;
+    }
+
+    m264_encoder->lib_handle = lib_handle;
+
+    m264_encoder->init_m264_encoder = dlsym(lib_handle, "m264_ffmpeg_encoder_init");
+    if (!m264_encoder->init_m264_encoder) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_ENCODER_NOT_FOUND;
+    }
+
+    m264_encoder->exit_m264_encoder = dlsym(lib_handle, "m264_ffmpeg_encoder_exit");
+    if (!m264_encoder->exit_m264_encoder) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_ENCODER_NOT_FOUND;
+    }
+
+    m264_encoder->get_extradata = dlsym(lib_handle, "m264_ffmpeg_encoder_get_extradata");
+    if (!m264_encoder->get_extradata) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_ENCODER_NOT_FOUND;
+    }
+    
+    m264_encoder->get_uncompressed_buffer = dlsym(lib_handle, "m264_ffmpeg_encoder_get_uncompressed_buffer");
+    if (!m264_encoder->get_uncompressed_buffer) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_ENCODER_NOT_FOUND;
+    }
+
+    m264_encoder->send_frame = dlsym(lib_handle, "m264_ffmpeg_encoder_send_frame");
+    if (!m264_encoder->send_frame) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_ENCODER_NOT_FOUND;
+    }
+
+    m264_encoder->receive_packet = dlsym(lib_handle, "m264_ffmpeg_encoder_receive_packet");
+    if (!m264_encoder->receive_packet) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_ENCODER_NOT_FOUND;
+    }
+
+    m264_encoder->release_packet_buffer = dlsym(lib_handle, "m264_ffmpeg_encoder_release_packet_buffer");
+    if (!m264_encoder->release_packet_buffer) {
+        av_log(avctx, AV_LOG_ERROR, "failed to load mvM264ffmpeg\n");
+        return AVERROR_ENCODER_NOT_FOUND;
+    }
+
+    encoder_info.width = avctx->width;
+    encoder_info.height = avctx->height;
+    encoder_info.rate = avctx->framerate.num;
+    encoder_info.scale = avctx->framerate.den;
+    encoder_info.gop_size = avctx->gop_size;
+    encoder_info.bitrate = avctx->bit_rate;
+    av_log(avctx, AV_LOG_DEBUG, "m264_encode_init_h264: avctx->width = %d, avctx->height = %d, avctx->framerate.num = %d, avctx->framerate.den = %d, avctx->gop_size = %d, avctx->bit_rate = %" PRId64 "\n",
+                                 avctx->width, avctx->height, avctx->framerate.num, avctx->framerate.den, avctx->gop_size, avctx->bit_rate);
+
+    ret = m264_encoder->init_m264_encoder(&encoder_info, &(m264_encoder->encoder_context));
+    if (ret < 0)
+        return ret;
+
+    if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) {
+        ret = m264_encoder->get_extradata(m264_encoder->encoder_context, &extradata, &extradata_size);
+        if (ret < 0)
+            return ret;
+
+        if (extradata_size > 0)
+        {
+            avctx->extradata = av_mallocz(extradata_size + AV_INPUT_BUFFER_PADDING_SIZE);
+            if (!avctx->extradata)
+                return AVERROR(ENOMEM);
+            memcpy(avctx->extradata, extradata, extradata_size);
+            avctx->extradata_size = extradata_size;
+        }
+    }
+
+    return ret;
+}
+
+int ff_m264_send_frame(AVCodecContext *avctx, const AVFrame *frame)
+{
+    int ret = 0;
+    enum AVPixelFormat src_pixel_format;
+    enum AVPixelFormat dest_pixel_format;
+    uint8_t *dst[1];
+    int dst_stride[1];
+    void *buffer_context;
+    unsigned long buffer_size;
+    unsigned long line_size;
+    int line;
+    int actual_line_size;
+    unsigned char *srcmem,*dstmem;
+
+    M264Encoder *m264_encoder = (M264Encoder *)avctx->priv_data;
+
+    if (frame) {
+        src_pixel_format = (enum AVPixelFormat)frame->format;
+        dest_pixel_format = AV_PIX_FMT_YUYV422;
+        if (m264_encoder->sw_context == NULL) {
+            m264_encoder->sw_context = sws_getContext(frame->width, frame->height, src_pixel_format, frame->width, frame->height,
+                                                      dest_pixel_format, SWS_BILINEAR, NULL, NULL, NULL);
+            if (!m264_encoder->sw_context) {
+                av_log(avctx, AV_LOG_ERROR, "failed to load mvm264ffmpeg.so \n");
+                ret = -1;
+                return ret;
+            }
+        }
+        ret = m264_encoder->get_uncompressed_buffer(m264_encoder->encoder_context, &buffer_context, (void **)&dst[0], &buffer_size);
+        if (ret < 0)
+            return ret;
+
+        dst_stride[0] = buffer_size / avctx->height;
+        if (src_pixel_format != AV_PIX_FMT_YUYV422) { 
+            ret = sws_scale(m264_encoder->sw_context, (const uint8_t * const*)frame->data, frame->linesize, 0, frame->height, dst, dst_stride);
+            if (ret < 0)
+                return ret;
+        }
+        else
+        {
+            line_size = buffer_size / avctx->height;
+            dstmem = (unsigned char *)dst[0];
+            srcmem = (unsigned char *)frame->data[0];
+            actual_line_size = avctx->width * 2;
+
+            for (line = 0; line < avctx->height; line++) {
+                memcpy(dstmem, srcmem, actual_line_size);
+                srcmem += frame->linesize[0];
+                dstmem += line_size;
+            }
+        }
+        av_log(avctx, AV_LOG_DEBUG, "ff_m264_send_frame: frame->width = %d, frame->height = %d\n", frame->width, frame->height);
+        ret = m264_encoder->send_frame(m264_encoder->encoder_context, buffer_context, 0);
+        if (ret < 0)
+            return ret;
+    }
+    else
+    {
+        av_log(avctx, AV_LOG_DEBUG, "ff_m264_send_frame: frame is null. eof\n");
+        ret = m264_encoder->send_frame(m264_encoder->encoder_context, NULL, 0);
+        if (ret < 0)
+            return ret;
+    }
+    return ret;
+}
+
+int ff_m264_receive_packet(AVCodecContext *avctx, AVPacket *avpkt)
+{
+    int ret;
+    M264Encoder *m264_encoder = (M264Encoder *)avctx->priv_data;
+    void *buffer_context = NULL;
+    void *buffer = NULL;
+    unsigned long buffer_size = 0;
+    int32_t pts, dts;
+    int8_t key;
+
+    ret = m264_encoder->receive_packet(m264_encoder->encoder_context, &buffer_context, &buffer, &buffer_size, &pts, &dts, &key);
+    if (ret == -1) {
+        av_log(avctx, AV_LOG_DEBUG, "ff_m264_receive_packet: m264_encoder->receive_packet get eos, return AVERROR_EOF\n");
+        return AVERROR_EOF;
+    }
+    else if (buffer_size == 0) {
+        return AVERROR(EAGAIN);
+    }
+
+    av_log(avctx, AV_LOG_DEBUG, "ff_m264_receive_packet  m264_encoder->receive_packet buffer_size = %d\n", (int)buffer_size); 
+
+    ret = ff_alloc_packet2(avctx, avpkt, buffer_size, 0);
+    if (ret < 0)
+        return ret;
+
+    memcpy(avpkt->data, buffer, buffer_size);
+
+    avpkt->dts = dts;
+    avpkt->pts = pts + 2;
+    avpkt->flags = key ? AV_PKT_FLAG_KEY : 0;
+
+    ret = m264_encoder->release_packet_buffer(m264_encoder->encoder_context, buffer_context);
+    if (ret < 0)
+        return ret;
+
+    return ret;
+}
+
+int av_cold ff_m264_encode_close(AVCodecContext *avctx)
+{
+    M264Encoder *m264_encoder = (M264Encoder *)avctx->priv_data;
+
+    if (m264_encoder)
+    {
+        if (m264_encoder->exit_m264_encoder)
+            m264_encoder->exit_m264_encoder(m264_encoder->encoder_context);
+
+        sws_freeContext(m264_encoder->sw_context);
+
+        if (m264_encoder->lib_handle)
+            dlclose(m264_encoder->lib_handle);   
+            
+        av_free(m264_encoder);
+        avctx->priv_data = NULL;
+    }
+    
+    return 0;
+}
+
+AVCodec ff_h264_m264_encoder = {
+    .name           = "m264",
+    .long_name      = NULL_IF_CONFIG_SMALL("Matrox M264 Codec"),
+    .type           = AVMEDIA_TYPE_VIDEO,
+    .id             = AV_CODEC_ID_H264,
+    .init           = ff_m264_encode_init,
+    .send_frame     = ff_m264_send_frame,
+    .receive_packet = ff_m264_receive_packet,
+    .close          = ff_m264_encode_close,
+    .capabilities   = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE,
+    .caps_internal  = FF_CODEC_CAP_INIT_CLEANUP,
+    .wrapper_name   = "m264",
+};
diff --git a/libavcodec/m264enc.h b/libavcodec/m264enc.h
new file mode 100644
index 0000000..0c44b32
--- /dev/null
+++ b/libavcodec/m264enc.h
@@ -0,0 +1,38 @@ 
+/*
+ * M264 H.264 video encoder
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * Matrox M264 card supports h.264 encoding and decoding.
+ */
+
+#ifndef AVCODEC_M264ENC_H
+#define AVCODEC_M264ENC_H
+
+#include "libavutil/fifo.h"
+
+#include "avcodec.h"
+
+int ff_m264_encode_init(AVCodecContext *avctx);
+int ff_m264_encode_close(AVCodecContext *avctx);
+int ff_m264_send_frame(AVCodecContext *avctx, const AVFrame *frame);
+int ff_m264_receive_packet(AVCodecContext *avctx, AVPacket *avpkt);
+
+#endif //AVCODEC_AMAVCODEC_M264ENC_H
-- 
2.7.4