Message ID | CAF7m-44Jp5H8FyOERzoth6uwz4BKVg7eFiktU+pWR5tDy1t4Ww@mail.gmail.com |
---|---|
State | Superseded |
Headers | show |
On Thu, 22 Aug 2019, Anthony Delannoy wrote: > Hi > >>> fails to build on MIPS >> >> Seems to be because of these two structs within EPGTable and EPGSubTable: >> + struct { >> + int nb_descriptors; >> + void **descriptors; >> + }; > > I made modifications to avoid issues you encountered and put fate > modifications in the right commit(s). > >> Afaict, some patches change files that were added in earlier patches. This may >> not be ideal. > > I committed changements to files (dvb.{h,c}) I added earlier because I > need descriptors files (dvbdescriptors.{h,c}) > before those changes and descriptors files need the first version of > the dvb file (basic get functions). > >> Why are the big api changes necessary at all? > > I made those big changements to ease the implementation of all others > DVB tables (i'd like to add support for NIT, > AIT, TDT,... after EPG) by using the same helper functions > (avpriv_dvb_get{8,16,32}()) and especially the same > DVB descriptors code because each descriptor is supported by multiple > table in general. I think we should only merge the part of this patchset which makes the EIT available as a data stream. Parsing the whole EIT or dumping the data as ASCII is not libavcodec's or libavutil's job. Also there is no such concept in libavcodec as a data decoder, if something happens to work with avcodec_send_packet/avcodec_receive_frame that is mostly luck I believe. I am also not sure if we should add the EIT PID to all programs, that would mess up the direct relation between a PMT and an AVProgram, and we probably don't want that. So I'd rather see the EIT data stream as a standalone PID separate from the programs. Regards, Marton
> I think we should only merge the part of this patchset which makes the EIT > available as a data stream. Parsing the whole EIT or dumping the data as > ASCII is not libavcodec's or libavutil's job. The EPG decoder does not change the table's data, it just store them and it happens to contains text sometimes. Some utilites functions I made in libavutil/dvb can convert those raw data in text description(s) for certain descriptors. > Also there is no such concept in libavcodec as a data decoder, if something happens to > work with avcodec_send_packet/avcodec_receive_frame that is mostly luck I > believe. avcodec_send_packet and avcodec_receive_frame both call AVCodec::receive_frame if it is implemented. That's why my implementation of the EPG decoder does contain this function. For now my test scripts consists to: ``` 99 if (st->codecpar->codec_id != AV_CODEC_ID_EPG) 100 goto free_pkt; 101 102 ret = avcodec_send_packet(dec_ctx, &pkt); ... 112 while (1) { 113 ret = avcodec_receive_frame(dec_ctx, frame); 114 if (ret < 0) 115 break; 116 117 for (int i = 0; i < frame->nb_side_data; i++) { 118 AVFrameSideData *sd = frame->side_data[i]; 119 if (sd && sd->type == AV_FRAME_DATA_EPG_TABLE) { 120 EPGTable *table = sd->data; 121 av_epg_show_table(table, AV_LOG_WARNING); 122 } 123 } 124 av_frame_unref(frame); 125 } 126 127 free_pkt: 128 av_packet_unref(&pkt); ``` It works as intended and permits to decode EPGTable without issues, I tried on multiple channels. I wanted to permit the table decoding and not just an EPG data stream to permit easy reading and in the future easy modification before encoding EPG back. > I am also not sure if we should add the EIT PID to all programs, that > would mess up the direct relation between a PMT and an AVProgram, and we > probably don't want that. So I'd rather see the EIT data stream as a > standalone PID separate from the programs. I'm not an expert but I think each service/program contains a PMT table and all others. So one EPG stream (if available) for each service. That's what I understood from the ETSI EN 300 468 V1.16.1
On Fri, 23 Aug 2019, Anthony Delannoy wrote: >> I think we should only merge the part of this patchset which makes the EIT >> available as a data stream. Parsing the whole EIT or dumping the data as >> ASCII is not libavcodec's or libavutil's job. > > The EPG decoder does not change the table's data, it just store them > and it happens to > contains text sometimes. > Some utilites functions I made in libavutil/dvb can convert those raw > data in text > description(s) for certain descriptors. > >> Also there is no such concept in libavcodec as a data decoder, if something happens to >> work with avcodec_send_packet/avcodec_receive_frame that is mostly luck I >> believe. > > avcodec_send_packet and avcodec_receive_frame both call > AVCodec::receive_frame if it is > implemented. That's why my implementation of the EPG decoder does > contain this function. > > For now my test scripts consists to: > ``` > 99 if (st->codecpar->codec_id != AV_CODEC_ID_EPG) > 100 goto free_pkt; > 101 > 102 ret = avcodec_send_packet(dec_ctx, &pkt); > ... > 112 while (1) { > 113 ret = avcodec_receive_frame(dec_ctx, frame); > 114 if (ret < 0) > 115 break; > 116 > 117 for (int i = 0; i < frame->nb_side_data; i++) { > 118 AVFrameSideData *sd = frame->side_data[i]; > 119 if (sd && sd->type == AV_FRAME_DATA_EPG_TABLE) { > 120 EPGTable *table = sd->data; > 121 av_epg_show_table(table, AV_LOG_WARNING); > 122 } > 123 } > 124 av_frame_unref(frame); > 125 } > 126 > 127 free_pkt: > 128 av_packet_unref(&pkt); > ``` > It works as intended and permits to decode EPGTable without issues, I > tried on multiple channels. > > I wanted to permit the table decoding and not just an EPG data stream > to permit easy reading > and in the future easy modification before encoding EPG back. > >> I am also not sure if we should add the EIT PID to all programs, that >> would mess up the direct relation between a PMT and an AVProgram, and we >> probably don't want that. So I'd rather see the EIT data stream as a >> standalone PID separate from the programs. > > I'm not an expert but I think each service/program contains a PMT > table and all others. So one > EPG stream (if available) for each service. > That's what I understood from the ETSI EN 300 468 V1.16.1 The EPG stream is a single stream in the whole TS, it is on a single PID. The PMTs do not reference the EIT PID, therefore we should not make the EPG stream part of the programs, even if the EPG stream can contain the schedule of the programs. Regards, Marton
Okay, thanks I will patch that Le sam. 24 août 2019 à 20:09, Marton Balint <cus@passwd.hu> a écrit : > > > On Fri, 23 Aug 2019, Anthony Delannoy wrote: > > >> I think we should only merge the part of this patchset which makes the EIT > >> available as a data stream. Parsing the whole EIT or dumping the data as > >> ASCII is not libavcodec's or libavutil's job. > > > > The EPG decoder does not change the table's data, it just store them > > and it happens to > > contains text sometimes. > > Some utilites functions I made in libavutil/dvb can convert those raw > > data in text > > description(s) for certain descriptors. > > > >> Also there is no such concept in libavcodec as a data decoder, if something happens to > >> work with avcodec_send_packet/avcodec_receive_frame that is mostly luck I > >> believe. > > > > avcodec_send_packet and avcodec_receive_frame both call > > AVCodec::receive_frame if it is > > implemented. That's why my implementation of the EPG decoder does > > contain this function. > > > > For now my test scripts consists to: > > ``` > > 99 if (st->codecpar->codec_id != AV_CODEC_ID_EPG) > > 100 goto free_pkt; > > 101 > > 102 ret = avcodec_send_packet(dec_ctx, &pkt); > > ... > > 112 while (1) { > > 113 ret = avcodec_receive_frame(dec_ctx, frame); > > 114 if (ret < 0) > > 115 break; > > 116 > > 117 for (int i = 0; i < frame->nb_side_data; i++) { > > 118 AVFrameSideData *sd = frame->side_data[i]; > > 119 if (sd && sd->type == AV_FRAME_DATA_EPG_TABLE) { > > 120 EPGTable *table = sd->data; > > 121 av_epg_show_table(table, AV_LOG_WARNING); > > 122 } > > 123 } > > 124 av_frame_unref(frame); > > 125 } > > 126 > > 127 free_pkt: > > 128 av_packet_unref(&pkt); > > ``` > > It works as intended and permits to decode EPGTable without issues, I > > tried on multiple channels. > > > > I wanted to permit the table decoding and not just an EPG data stream > > to permit easy reading > > and in the future easy modification before encoding EPG back. > > > >> I am also not sure if we should add the EIT PID to all programs, that > >> would mess up the direct relation between a PMT and an AVProgram, and we > >> probably don't want that. So I'd rather see the EIT data stream as a > >> standalone PID separate from the programs. > > > > I'm not an expert but I think each service/program contains a PMT > > table and all others. So one > > EPG stream (if available) for each service. > > That's what I understood from the ETSI EN 300 468 V1.16.1 > > The EPG stream is a single stream in the whole TS, it is on a single PID. > The PMTs do not reference the EIT PID, therefore we should not make the > EPG stream part of the programs, even if the EPG stream can contain the > schedule of the programs. > > Regards, > Marton > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
From 1cfd16650bb91ce25c03a289cf6723b02587a892 Mon Sep 17 00:00:00 2001 From: Anthony Delannoy <anthony.2lannoy@gmail.com> Date: Wed, 21 Aug 2019 16:07:59 +0200 Subject: [PATCH 10/10] lavc/epgdec: add EPG data decoder New EPG table decoder which store an EPGTable in AVFrame::side_data as a AV_FRAME_DATA_EPG_TABLE type for each AVPacket decoded. --- Changelog | 1 + configure | 1 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/epgdec.c | 278 +++++++++++++++++++++++++++++++++++++++++ 5 files changed, 282 insertions(+) create mode 100644 libavcodec/epgdec.c diff --git a/Changelog b/Changelog index 52096eed0e..7a369330fe 100644 --- a/Changelog +++ b/Changelog @@ -2,6 +2,7 @@ Entries are sorted chronologically from oldest to youngest within each release, releases are sorted from youngest to oldest. version <next>: +- EPG data decoder - v360 filter - Intel QSV-accelerated MJPEG decoding - Intel QSV-accelerated VP9 decoding diff --git a/configure b/configure index 13d9726606..4dad16d73c 100755 --- a/configure +++ b/configure @@ -2687,6 +2687,7 @@ eac3_encoder_select="ac3_encoder" eamad_decoder_select="aandcttables blockdsp bswapdsp idctdsp mpegvideo" eatgq_decoder_select="aandcttables" eatqi_decoder_select="aandcttables blockdsp bswapdsp idctdsp" +epg_decoder_select="dvb" exr_decoder_deps="zlib" ffv1_decoder_select="rangecoder" ffv1_encoder_select="rangecoder" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index e49188357b..b9c42bdb14 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -300,6 +300,7 @@ OBJS-$(CONFIG_EATQI_DECODER) += eatqi.o eaidct.o mpeg12.o mpeg12data.o OBJS-$(CONFIG_EIGHTBPS_DECODER) += 8bps.o OBJS-$(CONFIG_EIGHTSVX_EXP_DECODER) += 8svx.o OBJS-$(CONFIG_EIGHTSVX_FIB_DECODER) += 8svx.o +OBJS-$(CONFIG_EPG_DECODER) += epgdec.o OBJS-$(CONFIG_ESCAPE124_DECODER) += escape124.o OBJS-$(CONFIG_ESCAPE130_DECODER) += escape130.o OBJS-$(CONFIG_EVRC_DECODER) += evrcdec.o acelp_vectors.o lsp.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 22985325e0..b669c80bca 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -419,6 +419,7 @@ extern AVCodec ff_dss_sp_decoder; extern AVCodec ff_dst_decoder; extern AVCodec ff_eac3_encoder; extern AVCodec ff_eac3_decoder; +extern AVCodec ff_epg_decoder; extern AVCodec ff_evrc_decoder; extern AVCodec ff_ffwavesynth_decoder; extern AVCodec ff_flac_encoder; diff --git a/libavcodec/epgdec.c b/libavcodec/epgdec.c new file mode 100644 index 0000000000..2003bb1905 --- /dev/null +++ b/libavcodec/epgdec.c @@ -0,0 +1,278 @@ +/* + * epg data decoder + * Copyright (c) 2019 Anthony Delannoy + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file epg data decoder + */ + +#include "avcodec.h" +#include "internal.h" +#include "decode.h" +#include "libavutil/common.h" +#include "libavutil/dvb.h" +#include "libavutil/dvbdescriptors.h" +#include "libavformat/mpegts.h" + +typedef struct EpgTidInfo { + int last_version; + int last_section_num; + int last_segment_section_num; +} EpgTidInfo; + +typedef struct EPGContext { + const AVClass *class; + struct EpgTidInfo infos[4]; + int last_sched_table_id; + int o_last_sched_table_id; + AVPacket *pkt; +} EPGContext; + +static int epg_handle_descriptor(DvbDescriptorHeader *h, DvbDescriptor *desc, + EPGSubTable *subtable, const uint8_t **pp, + const uint8_t *p_end) +{ + void *data = desc->parse(desc, pp, p_end); + if (!data) + return AVERROR_INVALIDDATA; + memcpy(data, h, sizeof(DvbDescriptorHeader)); + + if (av_reallocp_array(&subtable->descriptors, (subtable->nb_descriptors + 1), sizeof(void*)) < 0) { + desc->free(data); + return AVERROR(ENOMEM); + } + subtable->descriptors[subtable->nb_descriptors++] = data; + + return 0; +} + +static int epg_receive_frame(AVCodecContext *avctx, AVFrame *frame) +{ + EPGContext *epg_ctx = avctx->priv_data; + EpgTidInfo *epg_info; + DvbSectionHeader h1, *h = &h1; + const uint8_t *p, *p_end; + int val, ret, max_last_table_id; + uint8_t next_version; + AVFrameSideData *sd; + AVBufferRef *buf_ref; + EPGTable *table; + + if (!epg_ctx->pkt->data) { + ret = ff_decode_get_packet(avctx, epg_ctx->pkt); + if (ret < 0) + return ret; + } + + p = epg_ctx->pkt->data; + p_end = p + epg_ctx->pkt->size - 4; + + if (avpriv_dvb_parse_section_header(h, &p, p_end) < 0) + goto fail; + + table = av_epg_table_alloc(); + if (!table) + goto fail; + table->h = h1; + + val = avpriv_dvb_get16(&p, p_end); + if (val < 0) + goto fail; + table->ts_id = val; + + val = avpriv_dvb_get16(&p, p_end); + if (val < 0) + goto fail; + table->network_id = val; + + val = avpriv_dvb_get8(&p, p_end); + if (val < 0) + goto fail; + table->segment_last_section_num = val; + + val = avpriv_dvb_get8(&p, p_end); + if (val < 0) + goto fail; + table->last_tid = val; + + // Check eit data + switch (h->tid) { + case EIT_TID: + epg_info = &epg_ctx->infos[0]; + if (table->last_tid != EIT_TID || h->last_sec_num != table->segment_last_section_num) + goto fail; + break; + case OEIT_TID: + epg_info = &epg_ctx->infos[1]; + if (table->last_tid != OEIT_TID || h->last_sec_num != table->segment_last_section_num) + goto fail; + break; + case EITS_START_TID ... EITS_END_TID: + epg_info = &epg_ctx->infos[2]; + if (table->segment_last_section_num != h->sec_num) + goto fail; + max_last_table_id = FFMAX(table->last_tid, epg_ctx->last_sched_table_id); + if (table->last_tid != max_last_table_id) + goto fail; + epg_ctx->last_sched_table_id = max_last_table_id; + break; + case OEITS_START_TID ... OEITS_END_TID: + epg_info = &epg_ctx->infos[3]; + if (table->segment_last_section_num != h->sec_num) + goto fail; + max_last_table_id = FFMAX(table->last_tid, epg_ctx->o_last_sched_table_id); + if (table->last_tid != max_last_table_id) + goto fail; + epg_ctx->o_last_sched_table_id = max_last_table_id; + break; + default: + goto fail; + } + + next_version = (epg_info->last_version == 31) ? 0 : epg_info->last_version + 1; + if (epg_info->last_version != (-1) && + (h->version == epg_info->last_version || h->version == next_version)) + goto fail; + + // Subtables + while (p < p_end) { + const uint8_t *desc_list_end; + EPGSubTable *subtable; + + if (av_reallocp_array(&table->subtables, (table->nb_subtables + 1), sizeof(EPGSubTable*)) < 0) + goto fail; + + table->subtables[table->nb_subtables] = av_epg_subtable_alloc(); + if (!table->subtables[table->nb_subtables]) + goto fail; + subtable = table->subtables[table->nb_subtables++]; + + // Get event_id + val = avpriv_dvb_get16(&p, p_end); + if (val < 0) + break; + subtable->event_id = val; + + memcpy(subtable->start_time, p, 5); + p += 5; + + memcpy(subtable->duration, p, 3); + p += 3; + + val = avpriv_dvb_get16(&p, p_end); + if (val < 0) + break; + subtable->running_status = (val >> 13); + subtable->free_ca_mode = (val >> 12) & 0x1; + subtable->desc_loop_len = val & 0xfff; + + desc_list_end = p + subtable->desc_loop_len; + + // Descriptors + for (;;) { + DvbDescriptor *desc; + DvbDescriptorHeader h; + const uint8_t *desc_end; + + if (av_dvb_parse_descriptor_header(&h, &p, p_end) < 0) + break; + + if (!(desc = (DvbDescriptor*)av_dvb_get_descriptor(&h))) { + p += h.len; + continue; + } + + desc_end = p + h.len; + if (desc_end > desc_list_end) + break; + + if (epg_handle_descriptor(&h, desc, subtable, &p, desc_end) < 0) + break; + + p = desc_end; + } + p = desc_list_end; + } + + // CRC32 + val = avpriv_dvb_get32(&p_end, (p_end + 4)); + if (val < 0) + goto fail; + table->crc = val; + + buf_ref = av_buffer_allocz(sizeof(*table) + AV_INPUT_BUFFER_PADDING_SIZE); + if (!buf_ref) + goto fail; + buf_ref->data = (uint8_t*)table; + frame->buf[0] = buf_ref; + + sd = av_frame_new_side_data(frame, AV_FRAME_DATA_EPG_TABLE, sizeof(*table)); + sd->data = (uint8_t*)table; + + frame->pts = epg_ctx->pkt->pts; + frame->pkt_dts = epg_ctx->pkt->dts; + frame->pkt_size = epg_ctx->pkt->size; + + av_packet_unref(epg_ctx->pkt); + return 0; + +fail: + av_packet_unref(epg_ctx->pkt); + av_epg_table_free(&table); + return AVERROR_INVALIDDATA; +} + +static av_cold int decode_init(AVCodecContext *avctx) +{ + EPGContext *epg_ctx = avctx->priv_data; + + for (int i = 0; i < 4; i++) + epg_ctx->infos[i].last_version = (-1); + epg_ctx->last_sched_table_id = EITS_START_TID; + epg_ctx->o_last_sched_table_id = OEITS_START_TID; + + epg_ctx->pkt = av_packet_alloc(); + return epg_ctx->pkt ? 0 : AVERROR(ENOMEM); +} + +static av_cold int decode_end(AVCodecContext *avctx) +{ + EPGContext *epg_ctx = avctx->priv_data; + av_packet_free(&epg_ctx->pkt); + return 0; +} + +static const AVClass epg_class = { + .class_name = "EPG Decoder", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +AVCodec ff_epg_decoder = { + .name = "epg", + .long_name = NULL_IF_CONFIG_SMALL("EPG (Electronic Program Guide)"), + .type = AVMEDIA_TYPE_DATA, + .id = AV_CODEC_ID_EPG, + .priv_data_size = sizeof(EPGContext), + .init = decode_init, + .close = decode_end, + .receive_frame = epg_receive_frame, + .priv_class = &epg_class, +}; -- 2.23.0