[FFmpeg-devel,v2] avformat/mpegts: add merge_pmt_versions option

Submitted by Aman Gupta on May 15, 2018, 11:12 p.m.

Details

Message ID CAK=uwuzd5wVW1CS9pauxT4Y-=3puJc6jDH13POSGT35Citjs4Q@mail.gmail.com
State New
Headers show

Commit Message

Aman Gupta May 15, 2018, 11:12 p.m.
On Tue, May 15, 2018 at 12:23 PM, Aman Gupta <ffmpeg@tmm1.net> wrote:

> From: Aman Gupta <aman@tmm1.net>
>
> This new optional flag makes it easier to deal with mpegts
> samples where the PMT is updated and elementary streams move
> to different PIDs in the middle of playback.
>
> Previously, new AVStreams were created per PID, and it was up
> to the user to figure out which streams had migrated to a new PID
> (by iterating over the list of AVProgram and making guesses), and
> switch seamlessly to the new AVStream during playback.
>
> Transcoding or remuxing these streams with ffmpeg on the CLI was
> also quite painful, and the user would need to extract each set of
> PIDs into a separate file and then stitch them back together.
>
> With this new option, the mpegts demuxer will automatically detect
> PMT changes and feed data from the new PID to the original AVStream
> that was created for the orignal PID. For mpegts samples with
> stream_identifier_descriptor available, the unique ID is used to merge
> PIDs together. If the stream id is not available, the demuxer attempts
> to map PIDs based on their order and relation to the PCR pid.
>
> With this change, I am able to playback and transcode/remux these
> two samples which previously caused issues:
>
>     https://tmm1.s3.amazonaws.com/pmt-version-change.ts
>     https://kuroko.fushizen.eu/videos/pid_switch_sample.ts
>
> I also have another longer sample which contains multiple PMT
> changes, some of which change the ES pids and others which do not:
>
>     https://tmm1.s3.amazonaws.com/multiple-pmt-change.ts
>
> Demuxing this sample with the new option shows several new log
> messages as the PMT changes are handled:
>
>     [mpegts @ 0x7ffe18801200] detected PMT change (version=3/4,
> pcr_pid=0xf98/0xf9b)
>     [mpegts @ 0x7ffe18801200] re-using existing video stream 0 (pid=0xf98)
> for new pid=0xf9b
>     [mpegts @ 0x7ffe18801200] re-using existing audio stream 1 (pid=0xf99)
> for new pid=0xf9c
>     [mpegts @ 0x7ffe18801200] re-using existing audio stream 2 (pid=0xf9a)
> for new pid=0xf9d
>     [mpegts @ 0x7ffe18801200] detected PMT change (version=4/5,
> pcr_pid=0xf9b/0xfa9)
>     [mpegts @ 0x7ffe18801200] re-using existing video stream 0 (pid=0xf98)
> for new pid=0xfa9
>     [mpegts @ 0x7ffe18801200] re-using existing audio stream 1 (pid=0xf99)
> for new pid=0xfaa
>     [mpegts @ 0x7ffe18801200] re-using existing audio stream 2 (pid=0xf9a)
> for new pid=0xfab
>

One issue I found with this patch: if you seek around aggressively
sometimes you may encounter a
new ES before a PMT, which causes it not to get merged in as expected. I'm
adding the following change
locally to fix the issue.

                             pes->st = avformat_new_stream(ts->stream,
NULL);

This has the side-effect that any pids not advertised in a PMT will be
ignored, but I think that's acceptable behavior
if you're opting-in to this new feature.

Aman


> Signed-off-by: Aman Gupta <aman@tmm1.net>
> ---
>  doc/demuxers.texi                     |   4 ++
>  libavformat/mpegts.c                  | 106 ++++++++++++++++++++++++++++++
> ++--
>  tests/fate/mpegts.mak                 |   5 ++
>  tests/ref/fate/mpegts-probe-pmt-merge |  32 ++++++++++
>  4 files changed, 143 insertions(+), 4 deletions(-)
>  create mode 100644 tests/ref/fate/mpegts-probe-pmt-merge
>
> diff --git a/doc/demuxers.texi b/doc/demuxers.texi
> index e7c2abce57..2f7d7e0f3a 100644
> --- a/doc/demuxers.texi
> +++ b/doc/demuxers.texi
> @@ -552,6 +552,10 @@ Show the detected raw packet size, cannot be set by
> the user.
>  Scan and combine all PMTs. The value is an integer with value from -1
>  to 1 (-1 means automatic setting, 1 means enabled, 0 means
>  disabled). Default value is -1.
> +
> +@item merge_pmt_versions
> +Re-use existing streams when a PMT's version is updated and elementary
> +streams move to different PIDs. Default value is 0.
>  @end table
>
>  @section mpjpeg
> diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c
> index 27b1c30a44..f16eae3235 100644
> --- a/libavformat/mpegts.c
> +++ b/libavformat/mpegts.c
> @@ -84,6 +84,8 @@ typedef struct MpegTSSectionFilter {
>      unsigned int end_of_section_reached : 1;
>      SectionCallback *section_cb;
>      void *opaque;
> +    int orig_pcr_pid; /* pmt specific */
> +    int last_pcr_pid;
>  } MpegTSSectionFilter;
>
>  struct MpegTSFilter {
> @@ -147,6 +149,7 @@ struct MpegTSContext {
>      int scan_all_pmts;
>
>      int resync_size;
> +    int merge_pmt_versions;
>
>      /******************************************/
>      /* private mpegts data */
> @@ -172,6 +175,8 @@ static const AVOption options[] = {
>       {.i64 = 0}, 0, 0, AV_OPT_FLAG_DECODING_PARAM | AV_OPT_FLAG_EXPORT |
> AV_OPT_FLAG_READONLY },
>      {"scan_all_pmts", "scan and combine all PMTs",
> offsetof(MpegTSContext, scan_all_pmts), AV_OPT_TYPE_BOOL,
>       {.i64 = -1}, -1, 1, AV_OPT_FLAG_DECODING_PARAM },
> +    {"merge_pmt_versions", "re-use streams when PMT's version/pids
> change", offsetof(MpegTSContext, merge_pmt_versions), AV_OPT_TYPE_BOOL,
> +     {.i64 = 0}, 0, 1,  AV_OPT_FLAG_DECODING_PARAM },
>      {"skip_changes", "skip changing / adding streams / programs",
> offsetof(MpegTSContext, skip_changes), AV_OPT_TYPE_BOOL,
>       {.i64 = 0}, 0, 1, 0 },
>      {"skip_clear", "skip clearing programs", offsetof(MpegTSContext,
> skip_clear), AV_OPT_TYPE_BOOL,
> @@ -1073,7 +1078,8 @@ static int mpegts_push_data(MpegTSFilter *filter,
>                          if (ts->skip_changes)
>                              goto skip;
>
> -                        pes->st = avformat_new_stream(ts->stream, NULL);
> +                        if (!pes->st)
> +                            pes->st = avformat_new_stream(ts->stream,
> NULL);
>                          if (!pes->st)
>                              return AVERROR(ENOMEM);
>                          pes->st->id = pes->pid;
> @@ -1983,6 +1989,69 @@ int ff_parse_mpeg2_descriptor(AVFormatContext *fc,
> AVStream *st, int stream_type
>      return 0;
>  }
>
> +static AVStream *find_matching_stream(MpegTSContext *ts, int pid,
> +                                      int stream_id,
> +                                      int pcr_pid, int orig_pcr_pid)
> +{
> +    AVFormatContext *s = ts->stream;
> +    int i, orig_pid = orig_pcr_pid + (pid - pcr_pid);
> +    AVStream *found = NULL;
> +
> +    for (i = 0; i < s->nb_streams; i++) {
> +        AVStream *st = s->streams[i];
> +        if (stream_id != -1) {
> +            if (st->stream_identifier == stream_id+1) {
> +                found = st;
> +                break;
> +            }
> +        } else if (pcr_pid != orig_pcr_pid && st->id == orig_pid) {
> +            found = st;
> +            break;
> +        }
> +    }
> +
> +    if (found) {
> +        av_log(ts->stream, AV_LOG_DEBUG, "re-using existing %s stream %d
> (pid=0x%x) for new pid=0x%x\n",
> +               av_get_media_type_string(found->codecpar->codec_type), i,
> found->id, pid);
> +    }
> +
> +    return found;
> +}
> +
> +static int parse_stream_identifier_desc(const uint8_t *p, const uint8_t
> *p_end)
> +{
> +    const uint8_t **pp = &p;
> +    const uint8_t *desc_list_end;
> +    const uint8_t *desc_end;
> +    int desc_list_len;
> +    int desc_len, desc_tag;
> +
> +    desc_list_len = get16(pp, p_end);
> +    if (desc_list_len < 0)
> +        return -1;
> +    desc_list_len &= 0xfff;
> +    desc_list_end  = p + desc_list_len;
> +    if (desc_list_end > p_end)
> +        return -1;
> +
> +    while (1) {
> +        desc_tag = get8(pp, desc_list_end);
> +        if (desc_tag < 0)
> +            return -1;
> +        desc_len = get8(pp, desc_list_end);
> +        if (desc_len < 0)
> +            return -1;
> +        desc_end = *pp + desc_len;
> +        if (desc_end > desc_list_end)
> +            return -1;
> +
> +        if (desc_tag == 0x52) {
> +            return get8(pp, desc_end);
> +        }
> +        *pp = desc_end;
> +    }
> +}
> +
>  static int is_pes_stream(int stream_type, uint32_t prog_reg_desc)
>  {
>      return !(stream_type == 0x13 ||
> @@ -2000,6 +2069,8 @@ static void pmt_cb(MpegTSFilter *filter, const
> uint8_t *section, int section_len
>      int program_info_length, pcr_pid, pid, stream_type;
>      int desc_list_len;
>      uint32_t prog_reg_desc = 0; /* registration descriptor */
> +    int last_ver = tssf->last_ver;
> +    int stream_id = -1;
>
>      int mp4_descr_count = 0;
>      Mp4Descr mp4_descr[MAX_MP4_DESCR_COUNT] = { { 0 } };
> @@ -2032,6 +2103,14 @@ static void pmt_cb(MpegTSFilter *filter, const
> uint8_t *section, int section_len
>      pcr_pid &= 0x1fff;
>      add_pid_to_pmt(ts, h->id, pcr_pid);
>      set_pcr_pid(ts->stream, h->id, pcr_pid);
> +    if (!tssf->orig_pcr_pid)
> +        tssf->orig_pcr_pid = pcr_pid;
> +
> +    if (last_ver != -1 && h->version != last_ver) {
> +        av_log(ts->stream, AV_LOG_DEBUG, "detected PMT change
> (version=%d/%d, pcr_pid=0x%x/0x%x)\n",
> +               last_ver, h->version, tssf->last_pcr_pid, pcr_pid);
> +    }
> +    tssf->last_pcr_pid = pcr_pid;
>
>      av_log(ts->stream, AV_LOG_TRACE, "pcr_pid=0x%x\n", pcr_pid);
>
> @@ -2083,14 +2162,22 @@ static void pmt_cb(MpegTSFilter *filter, const
> uint8_t *section, int section_len
>          if (pid < 0)
>              goto out;
>          pid &= 0x1fff;
> +        stream_id = parse_stream_identifier_desc(p, p_end);
>          if (pid == ts->current_pid)
>              goto out;
>
>          /* now create stream */
>          if (ts->pids[pid] && ts->pids[pid]->type == MPEGTS_PES) {
>              pes = ts->pids[pid]->u.pes_filter.opaque;
> +            if (ts->merge_pmt_versions && !pes->st) {
> +                st = find_matching_stream(ts, pid, stream_id, pcr_pid,
> tssf->orig_pcr_pid);
> +                if (st) {
> +                    pes->st = st;
> +                    pes->stream_type = stream_type;
> +                }
> +            }
>              if (!pes->st) {
> -                pes->st     = avformat_new_stream(pes->stream, NULL);
> +                pes->st = avformat_new_stream(pes->stream, NULL);
>                  if (!pes->st)
>                      goto out;
>                  pes->st->id = pes->pid;
> @@ -2100,7 +2187,14 @@ static void pmt_cb(MpegTSFilter *filter, const
> uint8_t *section, int section_len
>              if (ts->pids[pid])
>                  mpegts_close_filter(ts, ts->pids[pid]); // wrongly added
> sdt filter probably
>              pes = add_pes_stream(ts, pid, pcr_pid);
> -            if (pes) {
> +            if (ts->merge_pmt_versions && pes && !pes->st) {
> +                st = find_matching_stream(ts, pid, stream_id, pcr_pid,
> tssf->orig_pcr_pid);
> +                if (st) {
> +                    pes->st = st;
> +                    pes->stream_type = stream_type;
> +                }
> +            }
> +            if (pes && !pes->st) {
>                  st = avformat_new_stream(pes->stream, NULL);
>                  if (!st)
>                      goto out;
> @@ -2110,7 +2204,11 @@ static void pmt_cb(MpegTSFilter *filter, const
> uint8_t *section, int section_len
>              int idx = ff_find_stream_index(ts->stream, pid);
>              if (idx >= 0) {
>                  st = ts->stream->streams[idx];
> -            } else {
> +            }
> +            if (ts->merge_pmt_versions && !st) {
> +                st = find_matching_stream(ts, pid, stream_id, pcr_pid,
> tssf->orig_pcr_pid);
> +            }
> +            if (!st) {
>                  st = avformat_new_stream(ts->stream, NULL);
>                  if (!st)
>                      goto out;
> diff --git a/tests/fate/mpegts.mak b/tests/fate/mpegts.mak
> index 2b128492e0..bbcbfc47b2 100644
> --- a/tests/fate/mpegts.mak
> +++ b/tests/fate/mpegts.mak
> @@ -15,6 +15,11 @@ fate-mpegts-probe-program: SRC =
> $(TARGET_SAMPLES)/mpegts/loewe.ts
>  fate-mpegts-probe-program: CMD = run $(PROBE_CODEC_NAME_COMMAND)
> -select_streams p:769:v:0 -i "$(SRC)"
>
>
> +FATE_MPEGTS_PROBE-$(call DEMDEC, MPEGTS) += fate-mpegts-probe-pmt-merge
> +fate-mpegts-probe-pmt-merge: SRC = $(TARGET_SAMPLES)/mpegts/pmtchange.ts
> +fate-mpegts-probe-pmt-merge: CMD = run $(PROBE_CODEC_NAME_COMMAND)
> -merge_pmt_versions 1 -i "$(SRC)"
> +
> +
>  FATE_SAMPLES_FFPROBE += $(FATE_MPEGTS_PROBE-yes)
>
>  fate-mpegts: $(FATE_MPEGTS_PROBE-yes)
> diff --git a/tests/ref/fate/mpegts-probe-pmt-merge
> b/tests/ref/fate/mpegts-probe-pmt-merge
> new file mode 100644
> index 0000000000..6e424af5d2
> --- /dev/null
> +++ b/tests/ref/fate/mpegts-probe-pmt-merge
> @@ -0,0 +1,32 @@
> +[PROGRAM]
> +[STREAM]
> +codec_name=ac3
> +[/STREAM]
> +[STREAM]
> +codec_name=ac3
> +[/STREAM]
> +[STREAM]
> +codec_name=ac3
> +[/STREAM]
> +[STREAM]
> +codec_name=mpeg2video
> +[/STREAM]
> +[STREAM]
> +codec_name=scte_35
> +[/STREAM]
> +[/PROGRAM]
> +[STREAM]
> +codec_name=ac3
> +[/STREAM]
> +[STREAM]
> +codec_name=ac3
> +[/STREAM]
> +[STREAM]
> +codec_name=ac3
> +[/STREAM]
> +[STREAM]
> +codec_name=mpeg2video
> +[/STREAM]
> +[STREAM]
> +codec_name=scte_35
> +[/STREAM]
> --
> 2.14.2
>
>

Patch hide | download patch | download mbox

diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c
index f16eae3235..3d8e9da66f 100644
--- a/libavformat/mpegts.c
+++ b/libavformat/mpegts.c
@@ -1077,6 +1077,8 @@  static int mpegts_push_data(MpegTSFilter *filter,
                     if (!pes->st) {
                         if (ts->skip_changes)
                             goto skip;
+                        if (ts->merge_pmt_versions)
+                            goto skip; /* wait for PMT to merge new stream
*/

                         if (!pes->st)