| Message ID | 20161011135233.GA99438@putsch.kolbu.ws |
|---|---|
| State | Superseded |
| Headers | show |
On Tue, Oct 11, 2016 at 15:52:33 +0200, Ståle kristoffersen wrote: > + * VC-1 demuxer > + * Copyright (c) 2016 Ståle Kristoffersen I don't think it's a VC-1 demuxer. ;-) Moritz
On Tue, Oct 11, 2016 at 03:52:33PM +0200, Ståle kristoffersen wrote: > Hi, > I have some transport streams with only one pid, containing MJPEG 2000 > video. Since the PMT/PAT is missing ffmpeg is unable to decode it. > > The attached patch makes ffmpeg able to guess that it does contain a > stream of jpeg 2000, but I am not sure if this is the correct approach. > > Also, should LIBAVFORMAT_VERSION_MICRO and the Changelog be updated? if a new demuxer needs to be added then, yes > > -- > Ståle Kristoffersen > Makefile | 1 > allformats.c | 1 > mj2kdec.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > utils.c | 1 > 4 files changed, 65 insertions(+) > 26d9e6512a4c62e17df622cb4cb0dac7eadfa790 0001-libavf-Auto-detect-mjpeg-2000-in-mpeg-ts.patch > From 0d5594d4dfd1a9608ef55e90fb6a770b0f54cdaa Mon Sep 17 00:00:00 2001 > From: =?UTF-8?q?St=C3=A5le=20Kristoffersen?= <staalebk@ifi.uio.no> > Date: Tue, 11 Oct 2016 15:36:40 +0200 > Subject: [PATCH] libavf: Auto-detect mjpeg 2000 in mpeg-ts > MIME-Version: 1.0 > Content-Type: text/plain; charset=UTF-8 > Content-Transfer-Encoding: 8bit > > This makes it possible to decode motion jpeg 2000 > encoded in a transport stream without a correct PMT/PAT. > > Signed-off-by: Ståle Kristoffersen <staalebk@ifi.uio.no> > --- > libavformat/Makefile | 1 + > libavformat/allformats.c | 1 + > libavformat/mj2kdec.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++ > libavformat/utils.c | 1 + > 4 files changed, 65 insertions(+) > create mode 100644 libavformat/mj2kdec.c > > diff --git a/libavformat/Makefile b/libavformat/Makefile > index 5d827d31..4020b8d 100644 > --- a/libavformat/Makefile > +++ b/libavformat/Makefile > @@ -273,6 +273,7 @@ OBJS-$(CONFIG_MD5_MUXER) += hashenc.o > OBJS-$(CONFIG_MGSTS_DEMUXER) += mgsts.o > OBJS-$(CONFIG_MICRODVD_DEMUXER) += microdvddec.o subtitles.o > OBJS-$(CONFIG_MICRODVD_MUXER) += microdvdenc.o > +OBJS-$(CONFIG_MJPEG_2000_DEMUXER) += rawdec.o mj2kdec.o > OBJS-$(CONFIG_MJPEG_DEMUXER) += rawdec.o > OBJS-$(CONFIG_MJPEG_MUXER) += rawenc.o > OBJS-$(CONFIG_MLP_DEMUXER) += rawdec.o mlpdec.o > diff --git a/libavformat/allformats.c b/libavformat/allformats.c > index 6a216ef..7db6c52 100644 > --- a/libavformat/allformats.c > +++ b/libavformat/allformats.c > @@ -183,6 +183,7 @@ void av_register_all(void) > REGISTER_DEMUXER (MGSTS, mgsts); > REGISTER_MUXDEMUX(MICRODVD, microdvd); > REGISTER_MUXDEMUX(MJPEG, mjpeg); > + REGISTER_DEMUXER (MJPEG_2000, mjpeg_2000); > REGISTER_MUXDEMUX(MLP, mlp); > REGISTER_DEMUXER (MLV, mlv); > REGISTER_DEMUXER (MM, mm); > diff --git a/libavformat/mj2kdec.c b/libavformat/mj2kdec.c > new file mode 100644 > index 0000000..cf9e575 > --- /dev/null > +++ b/libavformat/mj2kdec.c > @@ -0,0 +1,62 @@ > +/* > + * VC-1 demuxer > + * Copyright (c) 2016 Ståle Kristoffersen > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include "libavutil/intreadwrite.h" > +#include "avformat.h" > +#include "rawdec.h" > +#include "libavcodec/jpeg2000.h" > + > + > +#if CONFIG_MJPEG_2000_DEMUXER > +static int mjpeg2000_probe(AVProbeData *p) > +{ > + int i; > + int invalid = 0; > + int frames = 0; > + uint16_t soc, marker, marker_size; > + uint8_t marker_marker; > + > + for (i=0; i<p->buf_size-5; i++) { > + soc = AV_RB16(p->buf + i); > + if (soc == JPEG2000_SOC ) { > + marker = AV_RB16(p->buf + i + 2); > + marker_size = AV_RB16(p->buf + i + 4); > + if (marker == JPEG2000_SIZ) { > + i += marker_size + 2; > + marker_marker = AV_RB8(p->buf + i + 2); missing overflow and out of array checks also the img2 demuxer for mjpeg2000 from img2dec.c does not work for this ? [...]
On 2016-10-11 at 16:23, Moritz Barsnick wrote: > On Tue, Oct 11, 2016 at 15:52:33 +0200, Ståle kristoffersen wrote: > > + * VC-1 demuxer > > + * Copyright (c) 2016 Ståle Kristoffersen > > I don't think it's a VC-1 demuxer. ;-) I guess I took the line "Do not copy and paste it from a random place, use an existing file as template." from the developement policy a bit to litteraly. :)
On 2016-10-11 at 16:27, Michael Niedermayer wrote: <snip> > > + for (i=0; i<p->buf_size-5; i++) { > > + soc = AV_RB16(p->buf + i); > > + if (soc == JPEG2000_SOC ) { > > + marker = AV_RB16(p->buf + i + 2); > > + marker_size = AV_RB16(p->buf + i + 4); > > + if (marker == JPEG2000_SIZ) { > > + i += marker_size + 2; > > + marker_marker = AV_RB8(p->buf + i + 2); > > missing overflow and out of array checks Ah, when I cleaned up the patch I removed the code, my bad. > also the img2 demuxer for mjpeg2000 from img2dec.c does not work > for this ? It does work if I: 1) modify j2k_probe to not only check the first 4 bytes 2) hange the score return to be less than that of mpegts. 3) Add j2k_pipe to fmt_id_type[] in libavf/utils.c I'm a bit worried about number 2. Increasing the score of mpegts might be a better idea, but I am very afraid of touching any of that. Is there a better way?
2016-10-11 15:52 GMT+02:00 Ståle kristoffersen <staalebk@ifi.uio.no>: > I have some transport streams with only one pid, containing MJPEG 2000 > video. Since the PMT/PAT is missing ffmpeg is unable to decode it. Do you also have samples that can be decoded by FFmpeg? (We don't) How are your samples produced? Do you have a way to playback the samples now without FFmpeg? I ask because I would love to see this patch tested: https://ffmpeg.org/pipermail/ffmpeg-devel/2015-August/177303.html > The attached patch makes ffmpeg able to guess that it does contain a > stream of jpeg 2000, but I am not sure if this is the correct approach. The approach is correct but if you want to improve the patch, it should look more like the mjpeg probe function and return a higher score if more valid frames can be found. > Also, should LIBAVFORMAT_VERSION_MICRO It should be MINOR since you add a new symbol to the library. > and the Changelog be updated? I don't Changelog should be updated for this patch. Thank you, Carl Eugen
2016-10-11 16:27 GMT+02:00 Michael Niedermayer <michael@niedermayer.cc>: > also the img2 demuxer for mjpeg2000 from img2dec.c does not work > for this ? We separated the jpg from the mjpeg probe, jpg makes sure that the sample starts like a jpg image and contains an end marker and can be decoded, mjpeg probe is less strict but requires more than one frame. Same could be argued for j2k or was our reasoning bad? Carl Eugen
On 2016-10-12 at 13:54, Carl Eugen Hoyos wrote: > 2016-10-11 15:52 GMT+02:00 Ståle kristoffersen <staalebk@ifi.uio.no>: > > > I have some transport streams with only one pid, containing MJPEG 2000 > > video. Since the PMT/PAT is missing ffmpeg is unable to decode it. > > Do you also have samples that can be decoded by FFmpeg? (We don't) I do (they are decodable by ffmpeg without the patch if I take the whole stream with PMT, or with only the j2k-pid using my patch), but I am unsure if I can share them. > How are your samples produced? They were given to me, so I do not know what produced them. > Do you have a way to playback the samples now without FFmpeg? I have not been able to play them back using any other tool. > I ask because I would love to see this > patch tested: > https://ffmpeg.org/pipermail/ffmpeg-devel/2015-August/177303.html I can confirm that the stream I have does indeed have stream_type set to 21 in the PMT, so the patch matches that. > > The attached patch makes ffmpeg able to guess that it does contain a > > stream of jpeg 2000, but I am not sure if this is the correct approach. > > The approach is correct but if you want to improve the patch, it should > look more like the mjpeg probe function and return a higher score if > more valid frames can be found. That code will have to be reworked, so I'll look into doing that as well. Is there any guidelines for how high score a probe should return? > > Also, should LIBAVFORMAT_VERSION_MICRO > > It should be MINOR since you add a new symbol to the library. Ok. > > and the Changelog be updated? > > I don't Changelog should be updated for this patch. Ok.
2016-10-12 16:57 GMT+02:00 Ståle Kristoffersen <staalebk@ifi.uio.no>: > On 2016-10-12 at 13:54, Carl Eugen Hoyos wrote: >> 2016-10-11 15:52 GMT+02:00 Ståle kristoffersen <staalebk@ifi.uio.no>: >> >> > I have some transport streams with only one pid, containing MJPEG 2000 >> > video. Since the PMT/PAT is missing ffmpeg is unable to decode it. >> >> Do you also have samples that can be decoded by FFmpeg? (We don't) > > I do (they are decodable by ffmpeg without the patch if I take the whole > stream with PMT, or with only the j2k-pid using my patch), but I am unsure > if I can share them. Maybe you could ask for samples that can be shared? >> How are your samples produced? > > They were given to me, so I do not know what produced them. > >> Do you have a way to playback the samples now without FFmpeg? > > I have not been able to play them back using any other tool. Neither with nor without PMT/PAT? Thank you, Carl Eugen
On 2016-10-12 at 17:32, Carl Eugen Hoyos wrote: > 2016-10-12 16:57 GMT+02:00 Ståle Kristoffersen <staalebk@ifi.uio.no>: <snip> > > I do (they are decodable by ffmpeg without the patch if I take the whole > > stream with PMT, or with only the j2k-pid using my patch), but I am unsure > > if I can share them. > > Maybe you could ask for samples that can be shared? I'll look into it, but don't count on it. I can try to 'recreate' something similar with ffmpeg if that is useful. > >> How are your samples produced? > > > > They were given to me, so I do not know what produced them. > > > >> Do you have a way to playback the samples now without FFmpeg? > > > > I have not been able to play them back using any other tool. > > Neither with nor without PMT/PAT? Yes, but I have not really tried very hard.
2016-10-12 17:40 GMT+02:00 Ståle Kristoffersen <staalebk@ifi.uio.no>: > On 2016-10-12 at 17:32, Carl Eugen Hoyos wrote: >> 2016-10-12 16:57 GMT+02:00 Ståle Kristoffersen <staalebk@ifi.uio.no>: >> > I do (they are decodable by ffmpeg without the patch if I take the >> > whole stream with PMT, or with only the j2k-pid using my patch), >> > but I am unsure if I can share them. >> >> Maybe you could ask for samples that can be shared? > > I'll look into it, but don't count on it. I can try to 'recreate' something > similar with ffmpeg if that is useful. Perhaps you can provide ffmpeg -i output for a file with PMT and PAT if you cannot share a sample? Thank you, Carl Eugen
On 2016-10-12 at 23:05, Carl Eugen Hoyos wrote: <snip> > Perhaps you can provide ffmpeg -i output for a file with PMT and PAT > if you cannot share a sample? Okay, here are three samples i have: $ ffmpeg -i sample1.ts ffmpeg version N-80172-g79b6c9a Copyright (c) 2000-2016 the FFmpeg developers built with gcc 4.9.2 (Debian 4.9.2-10) configuration: libavutil 55. 24.100 / 55. 24.100 libavcodec 57. 44.100 / 57. 44.100 libavformat 57. 37.101 / 57. 37.101 libavdevice 57. 0.101 / 57. 0.101 libavfilter 6. 46.100 / 6. 46.100 libswscale 4. 1.100 / 4. 1.100 libswresample 2. 0.101 / 2. 0.101 [mpegts @ 0x306c2a0] PES packet size mismatch Last message repeated 1 times Input #0, mpegts, from 'sample1.ts': Duration: 00:00:05.21, start: 16724.490189, bitrate: 153721 kb/s Program 1 Stream #0:0[0x40]: Video: jpeg2000 ([33][0][0][0] / 0x0021), yuv422p10le, 1920x540, 29.97 fps, 29.97 tbr, 90k tbn Stream #0:1[0x41]: Audio: s302m (BSSD / 0x44535342), 48000 Hz, stereo, s32 (20 bit), 2304 kb/s Stream #0:2[0x42]: Audio: s302m (BSSD / 0x44535342), 48000 Hz, stereo, s32 (20 bit), 2304 kb/s At least one output file must be specified $ ffmpeg -i sample2.ts ffmpeg version N-80172-g79b6c9a Copyright (c) 2000-2016 the FFmpeg developers built with gcc 4.9.2 (Debian 4.9.2-10) configuration: libavutil 55. 24.100 / 55. 24.100 libavcodec 57. 44.100 / 57. 44.100 libavformat 57. 37.101 / 57. 37.101 libavdevice 57. 0.101 / 57. 0.101 libavfilter 6. 46.100 / 6. 46.100 libswscale 4. 1.100 / 4. 1.100 libswresample 2. 0.101 / 2. 0.101 [mpegts @ 0x29d72a0] Stream #0: not enough frames to estimate rate; consider increasing probesize [mpegts @ 0x29d72a0] start time for stream 0 is not set in estimate_timings_from_pts [mpegts @ 0x29d72a0] PES packet size mismatch Last message repeated 7 times [mpegts @ 0x29d72a0] Could not find codec parameters for stream 0 (Video: jpeg2000 ([33][0][0][0] / 0x0021), none): unspecified size Consider increasing the value for the 'analyzeduration' and 'probesize' options Input #0, mpegts, from 'sample2.ts': Duration: 00:00:08.00, start: 71738.433044, bitrate: 100054 kb/s Program 1 Stream #0:0[0x5f]: Video: jpeg2000 ([33][0][0][0] / 0x0021), none, 90k tbr, 90k tbn Stream #0:1[0xc8](ENG): Audio: s302m (BSSD / 0x44535342), 48000 Hz, stereo, s32 (24 bit), 2696 kb/s Stream #0:2[0xc9](ENG): Audio: s302m (BSSD / 0x44535342), 48000 Hz, stereo, s32 (24 bit), 2696 kb/s Stream #0:3[0xca](ENG): Audio: s302m (BSSD / 0x44535342), 48000 Hz, stereo, s32 (24 bit), 2696 kb/s Stream #0:4[0xcb](ENG): Audio: s302m (BSSD / 0x44535342), 48000 Hz, stereo, s32 (24 bit), 2696 kb/s At least one output file must be specified $ ffmpeg -i sample3.ts ffmpeg version N-80172-g79b6c9a Copyright (c) 2000-2016 the FFmpeg developers built with gcc 4.9.2 (Debian 4.9.2-10) configuration: libavutil 55. 24.100 / 55. 24.100 libavcodec 57. 44.100 / 57. 44.100 libavformat 57. 37.101 / 57. 37.101 libavdevice 57. 0.101 / 57. 0.101 libavfilter 6. 46.100 / 6. 46.100 libswscale 4. 1.100 / 4. 1.100 libswresample 2. 0.101 / 2. 0.101 [jpeg2000 @ 0x2ce3dc0] Psot 285327 too big [jpeg2000 @ 0x2ce3dc0] error during processing marker segment ff90 [s302m @ 0x2ce4f80] frame has invalid header Input #0, mpegts, from 'sample3.ts': Duration: 00:00:03.64, start: 13435.189700, bitrate: 219998 kb/s Program 1 Stream #0:0[0x40]: Video: jpeg2000 (JPEG 2000 codestream restriction 0) ([33][0][0][0] / 0x0021), yuv422p10le, 1920x540, 29.97 fps, 29.97 tbr, 90k tbn Stream #0:1[0x41]: Audio: s302m (BSSD / 0x44535342), 48000 Hz, quad, s32 (24 bit), 5376 kb/s Stream #0:2[0x48]: Data: bin_data (VANC / 0x434E4156) At least one output file must be specified
From 0d5594d4dfd1a9608ef55e90fb6a770b0f54cdaa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A5le=20Kristoffersen?= <staalebk@ifi.uio.no> Date: Tue, 11 Oct 2016 15:36:40 +0200 Subject: [PATCH] libavf: Auto-detect mjpeg 2000 in mpeg-ts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This makes it possible to decode motion jpeg 2000 encoded in a transport stream without a correct PMT/PAT. Signed-off-by: Ståle Kristoffersen <staalebk@ifi.uio.no> --- libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/mj2kdec.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++ libavformat/utils.c | 1 + 4 files changed, 65 insertions(+) create mode 100644 libavformat/mj2kdec.c diff --git a/libavformat/Makefile b/libavformat/Makefile index 5d827d31..4020b8d 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -273,6 +273,7 @@ OBJS-$(CONFIG_MD5_MUXER) += hashenc.o OBJS-$(CONFIG_MGSTS_DEMUXER) += mgsts.o OBJS-$(CONFIG_MICRODVD_DEMUXER) += microdvddec.o subtitles.o OBJS-$(CONFIG_MICRODVD_MUXER) += microdvdenc.o +OBJS-$(CONFIG_MJPEG_2000_DEMUXER) += rawdec.o mj2kdec.o OBJS-$(CONFIG_MJPEG_DEMUXER) += rawdec.o OBJS-$(CONFIG_MJPEG_MUXER) += rawenc.o OBJS-$(CONFIG_MLP_DEMUXER) += rawdec.o mlpdec.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 6a216ef..7db6c52 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -183,6 +183,7 @@ void av_register_all(void) REGISTER_DEMUXER (MGSTS, mgsts); REGISTER_MUXDEMUX(MICRODVD, microdvd); REGISTER_MUXDEMUX(MJPEG, mjpeg); + REGISTER_DEMUXER (MJPEG_2000, mjpeg_2000); REGISTER_MUXDEMUX(MLP, mlp); REGISTER_DEMUXER (MLV, mlv); REGISTER_DEMUXER (MM, mm); diff --git a/libavformat/mj2kdec.c b/libavformat/mj2kdec.c new file mode 100644 index 0000000..cf9e575 --- /dev/null +++ b/libavformat/mj2kdec.c @@ -0,0 +1,62 @@ +/* + * VC-1 demuxer + * Copyright (c) 2016 Ståle Kristoffersen + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/intreadwrite.h" +#include "avformat.h" +#include "rawdec.h" +#include "libavcodec/jpeg2000.h" + + +#if CONFIG_MJPEG_2000_DEMUXER +static int mjpeg2000_probe(AVProbeData *p) +{ + int i; + int invalid = 0; + int frames = 0; + uint16_t soc, marker, marker_size; + uint8_t marker_marker; + + for (i=0; i<p->buf_size-5; i++) { + soc = AV_RB16(p->buf + i); + if (soc == JPEG2000_SOC ) { + marker = AV_RB16(p->buf + i + 2); + marker_size = AV_RB16(p->buf + i + 4); + if (marker == JPEG2000_SIZ) { + i += marker_size + 2; + marker_marker = AV_RB8(p->buf + i + 2); + if (marker_marker == 0xFF) { + frames++; + } else { + invalid++; + } + } + } + } + + if (invalid*4 + 1 < frames) { + return 1; + } + + return 0; +} +FF_DEF_RAWVIDEO_DEMUXER2(mjpeg_2000, "raw MJPEG 2000 video", mjpeg2000_probe, "j2k", AV_CODEC_ID_JPEG2000, AVFMT_GENERIC_INDEX|AVFMT_NOTIMESTAMPS) +#endif + diff --git a/libavformat/utils.c b/libavformat/utils.c index 8a51aea..9e9a052 100644 --- a/libavformat/utils.c +++ b/libavformat/utils.c @@ -328,6 +328,7 @@ static int set_codec_from_probe_data(AVFormatContext *s, AVStream *st, { "hevc", AV_CODEC_ID_HEVC, AVMEDIA_TYPE_VIDEO }, { "loas", AV_CODEC_ID_AAC_LATM, AVMEDIA_TYPE_AUDIO }, { "m4v", AV_CODEC_ID_MPEG4, AVMEDIA_TYPE_VIDEO }, + { "mjpeg_2000",AV_CODEC_ID_JPEG2000, AVMEDIA_TYPE_VIDEO }, { "mp3", AV_CODEC_ID_MP3, AVMEDIA_TYPE_AUDIO }, { "mpegvideo", AV_CODEC_ID_MPEG2VIDEO, AVMEDIA_TYPE_VIDEO }, { "truehd", AV_CODEC_ID_TRUEHD, AVMEDIA_TYPE_AUDIO }, -- 2.1.4