diff mbox

[FFmpeg-devel,OPW] OPW Project Proposal

Message ID CAC7Kmr2SE_AmQgXdbkqDsFUqjsT9O6+pTT1Z0LhfMsFutSbEKw@mail.gmail.com
State Superseded
Headers show

Commit Message

Pallavi Kumari Oct. 27, 2016, 6:08 p.m. UTC
Hi Michael,

I have attached a patch with the mail.

With `avcodec_get_frame_defaults (&frame)` in the function readAudio
program was working fine with old version in my system. But this function
is not in the latest git repo so I used `av_frame_unref(&frame)` instead
and the program segfaults. I am not sure why or how to fix this.

Also, I haven't properly understood the input and output flow of
libavfilters (how to write input and output links? and overall flow). Could
you explain it to me?



On Thu, Oct 27, 2016 at 3:24 AM, Michael Niedermayer <michael@niedermayer.cc
> wrote:

> On Thu, Oct 27, 2016 at 12:40:53AM +0530, Pallavi Kumari wrote:
> > I mean deciding a timeline for the opw project. Which is to be mentioned
> in
> > application
>
> its your application, you can choose whatever timeline you feel makes
> sense.
> As far as iam concered whats important is to finish the qualification
> task well before we have to choose the applicant(s) we accept this
> year.
>
> The timeline for the main project in dec-mar can be segmented as you
> like but leave plenty of time toward the end, problems always occur
> and things get delayed.
> Also both for qualification and the main project submit code early
> if you have questions ask early,
> if you dont get satisfactory reply from me on something ask again and
> louder/clearer ... i do sometimes also forget to reply until i then
> remember again days later ...
>
> also i think the architecture wasnt discussed much yet
> do you have some plans about it ?
>
> I mean for example there could be a avfilter that passes audio through
> and adds fingerint(s) as metadata into thf AVFrame
> (as with avpriv_frame_get_metadatap in vf_idet.c) and also print the
> fingerprints via av_log()
>
> a 2nd filter could then look these fingerprints from the metadata
> up in some file/database
>
> a 3rd one could add the fingerprints to the file/database
>
> or these also could all be in one filter of course
>
> this can be extended in the future to for example have a filter
> delay the audio until lookup succeeds and then once identified
> lookup lyrics online and add them or such (this of course would be a
> future thing after outreachy/opw)
>
> but maybe you have a different plan on how the components would
> interact ?
>
> thx
>
> >
> > On Wed, Oct 26, 2016 at 6:01 PM, Michael Niedermayer
> <michael@niedermayer.cc
> > > wrote:
> >
> > > Hi
> > >
> > > On Mon, Oct 17, 2016 at 02:06:26PM +0530, Pallavi Kumari wrote:
> > > > Hi Michael,
> > > >
> > > > I figured out the use of fft. Help me with the time line setting.
> Thanks
> > >
> > > I dont understand the question,
> > > if its about AVFILTER_FLAG_SUPPORT_TIMELINE*, please ignore this for
> > > now, its not needed
> > >
> > > [...]
> > > --
> > > Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC7
> 87040B0FAB
> > >
> > > Rewriting code that is poorly written but fully understood is good.
> > > Rewriting code that one doesnt understand is a sign that one is less
> smart
> > > then the original author, trying to rewrite it will not make it better.
> > >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> He who knows, does not speak. He who speaks, does not know. -- Lao Tsu
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>

Comments

Michael Niedermayer Oct. 27, 2016, 11:08 p.m. UTC | #1
On Thu, Oct 27, 2016 at 11:38:27PM +0530, Pallavi Kumari wrote:
> Hi Michael,
> 
> I have attached a patch with the mail.
> 
> With `avcodec_get_frame_defaults (&frame)` in the function readAudio
> program was working fine with old version in my system. But this function
> is not in the latest git repo so I used `av_frame_unref(&frame)` instead
> and the program segfaults. I am not sure why or how to fix this.

theres "AVFrame frame;"

thats bad, as it depends on sizef(AVFrame) to allocat frame on the
stack.
you should always allocate/clone/... one if one is not already
available, each of these would be setup correctly and not need
avcodec_get_frame_defaults()


> 
> Also, I haven't properly understood the input and output flow of
> libavfilters (how to write input and output links? and overall flow). Could
> you explain it to me?

see existing filters, af_*.c
maybe libavfilter/af_astats.c has a similar structure to the filter
here

theres also
doc/filter_design.txt


> 
> 
> 
> On Thu, Oct 27, 2016 at 3:24 AM, Michael Niedermayer <michael@niedermayer.cc
> > wrote:
> 
> > On Thu, Oct 27, 2016 at 12:40:53AM +0530, Pallavi Kumari wrote:
> > > I mean deciding a timeline for the opw project. Which is to be mentioned
> > in
> > > application
> >
> > its your application, you can choose whatever timeline you feel makes
> > sense.
> > As far as iam concered whats important is to finish the qualification
> > task well before we have to choose the applicant(s) we accept this
> > year.
> >
> > The timeline for the main project in dec-mar can be segmented as you
> > like but leave plenty of time toward the end, problems always occur
> > and things get delayed.
> > Also both for qualification and the main project submit code early
> > if you have questions ask early,
> > if you dont get satisfactory reply from me on something ask again and
> > louder/clearer ... i do sometimes also forget to reply until i then
> > remember again days later ...
> >
> > also i think the architecture wasnt discussed much yet
> > do you have some plans about it ?
> >
> > I mean for example there could be a avfilter that passes audio through
> > and adds fingerint(s) as metadata into thf AVFrame
> > (as with avpriv_frame_get_metadatap in vf_idet.c) and also print the
> > fingerprints via av_log()
> >
> > a 2nd filter could then look these fingerprints from the metadata
> > up in some file/database
> >
> > a 3rd one could add the fingerprints to the file/database
> >
> > or these also could all be in one filter of course
> >
> > this can be extended in the future to for example have a filter
> > delay the audio until lookup succeeds and then once identified
> > lookup lyrics online and add them or such (this of course would be a
> > future thing after outreachy/opw)
> >
> > but maybe you have a different plan on how the components would
> > interact ?
> >
> > thx
> >
> > >
> > > On Wed, Oct 26, 2016 at 6:01 PM, Michael Niedermayer
> > <michael@niedermayer.cc
> > > > wrote:
> > >
> > > > Hi
> > > >
> > > > On Mon, Oct 17, 2016 at 02:06:26PM +0530, Pallavi Kumari wrote:
> > > > > Hi Michael,
> > > > >
> > > > > I figured out the use of fft. Help me with the time line setting.
> > Thanks
> > > >
> > > > I dont understand the question,
> > > > if its about AVFILTER_FLAG_SUPPORT_TIMELINE*, please ignore this for
> > > > now, its not needed
> > > >
> > > > [...]
> > > > --
> > > > Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC7
> > 87040B0FAB
> > > >
> > > > Rewriting code that is poorly written but fully understood is good.
> > > > Rewriting code that one doesnt understand is a sign that one is less
> > smart
> > > > then the original author, trying to rewrite it will not make it better.
> > > >
> > > _______________________________________________
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > --
> > Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> >
> > He who knows, does not speak. He who speaks, does not know. -- Lao Tsu
> >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> >

>  Makefile        |    1 
>  af_peakpoints.c |  263 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  allfilters.c    |    1 
>  version.h       |    2 
>  4 files changed, 266 insertions(+), 1 deletion(-)
> 5cc989fc1ba9466043f833059ee82efcaa3cf898  0001-avfilter-add-peakpoints-filter.patch
> From 60429fb7d83d11d6dd733c3477950a940bd0d84a Mon Sep 17 00:00:00 2001
> From: Atana <atana@openmailbox.org>
> Date: Mon, 24 Oct 2016 17:16:09 +0530
> Subject: [PATCH] avfilter: add peakpoints filter
> 
> ---
>  libavfilter/Makefile        |   1 +
>  libavfilter/af_peakpoints.c | 263 ++++++++++++++++++++++++++++++++++++++++++++
>  libavfilter/allfilters.c    |   1 +
>  libavfilter/version.h       |   2 +-
>  4 files changed, 266 insertions(+), 1 deletion(-)
>  create mode 100644 libavfilter/af_peakpoints.c
> 
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index 7ed4696..1a18902 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -96,6 +96,7 @@ OBJS-$(CONFIG_LADSPA_FILTER)                 += af_ladspa.o
>  OBJS-$(CONFIG_LOUDNORM_FILTER)               += af_loudnorm.o
>  OBJS-$(CONFIG_LOWPASS_FILTER)                += af_biquads.o
>  OBJS-$(CONFIG_PAN_FILTER)                    += af_pan.o
> +OBJS-$(CONFIG_PEAKPOINTS_FILTER)             += af_peakpoints.o
>  OBJS-$(CONFIG_REPLAYGAIN_FILTER)             += af_replaygain.o
>  OBJS-$(CONFIG_RESAMPLE_FILTER)               += af_resample.o
>  OBJS-$(CONFIG_RUBBERBAND_FILTER)             += af_rubberband.o
> diff --git a/libavfilter/af_peakpoints.c b/libavfilter/af_peakpoints.c
> new file mode 100644
> index 0000000..84b51da
> --- /dev/null
> +++ b/libavfilter/af_peakpoints.c
> @@ -0,0 +1,263 @@
> +/*
> + * Copyright (c) 2016 Pallavi Kumari
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#include "libavcodec/avcodec.h"
> +#include "libavcodec/avfft.h"
> +#include "libavformat/avformat.h"
> +#include "libswscale/swscale.h"
> +#include "avfilter.h"
> +#include "libavutil/opt.h"
> +
> +
> +/* Structure to contain peak points context */
> +typedef struct {
> +    const AVClass *class;
> +    double *data;
> +    int nsamples;
> +    double *peaks;
> +    int size; // number of peaks
> +    int windowSize;
> +    char *inputFile;
> +} PeakPointsContext;
> +
> +/* returns maximum value from an array conditioned on start and end index */
> +static double getMax(double *res_arr, int startIndex, int endIndex) {
> +    int i;
> +    double max = res_arr[startIndex];
> +    for (i = startIndex; i <= endIndex; i++) {
> +	    if (res_arr[i] > max) {
> +	        max = res_arr[i];
> +	    }
> +    }
> +    return max;
> +}
> +

> +/* Stores peak frequency for each window(of chunkSize) in peaks array */
> +static void getPeakPointInChunk(int chunkSize, double *res_arr, int size, double *peaks) {
> +    int endIndex, i = 0, peakIndex = 0;
> +    int startIndex = 0;
> +    double max;
> +    // get a chunk and find max value in it
> +    while (i < size) {

> +	    if (i % chunkSize-1 == 0) {

% is slow (compared to &)
also doing a i++ loop instead of i+= chunkSize-1 is slow too
does lots of unneeed operations, which the compiler might or might
not optimize out


> +            max = getMax(res_arr, startIndex, i);
> +	        peaks[peakIndex++] = max;
> +	        startIndex = startIndex + chunkSize;
> +	    }

tabs are also forbidden in ffmpeg git


> +        i += 1;
> +    }
> +}
> +

> +/* read audio stream from input file */
> +static void readAudio (int *sampleRate, PeakPointsContext *ppc) {   
> +    // Initialize FFmpeg
> +    av_register_all ();
> +
> +    AVFormatContext *formatContext = NULL;
> +    AVCodecContext *codecContext = NULL;
> +    AVCodec *codec = NULL;
> +    AVPacket packet;
> +    AVFrame frame;
> +
> +    ssize_t nSamples = 0;
> +
> +    // open the file and autodetect the format
> +    if (avformat_open_input (&formatContext, ppc->inputFile, NULL, NULL) < 0)
> +        printf("Can not open input file %s", ppc->inputFile);
> +
> +    if (avformat_find_stream_info (formatContext, NULL) < 0)
> +        printf("Can not find stream information");
> +
> +    // find audio stream (between video/audio/subtitles/.. streams)
> +        int audioStreamId = av_find_best_stream (formatContext, AVMEDIA_TYPE_AUDIO,
> +                                            -1, -1, &codec, 0);
> +    if (audioStreamId < 0)
> +        printf("Can not find audio stream in the input file");
> +
> +    codecContext = formatContext->streams[audioStreamId]->codec;
> +
> +    // init the audio decoder
> +    if (avcodec_open2 (codecContext, codec, NULL) < 0)
> +        printf("Can not open audio decoder");
> +
> +    // read all packets
> +    int i = 0;
> +    while (av_read_frame (formatContext, &packet) == 0)  {

A libavfilter would generally analyze its audio input stream instead
of opening a audio file on its own



> +        if (packet.stream_index == audioStreamId)  {
> +            //avcodec_get_frame_defaults (&frame);
> +            av_frame_unref (&frame);
> +            int gotFrame = 0;
> +            if (avcodec_decode_audio4 (codecContext,&frame,&gotFrame,&packet) < 0){
> +                printf("Error decoding audio");
> +            }
> +            if (gotFrame)  {
> +                // audio frame has been decoded
> +                int size = av_samples_get_buffer_size (NULL,
> +                                                   codecContext->channels,
> +                                                   frame.nb_samples,
> +                                                   codecContext->sample_fmt,
> +                                                   1);
> +                if (size < 0)  {
> +                    printf("av_samples_get_buffer_size invalid value");
> +                }
> +
> +	            // printf("%d\n", *frame.data[0]);
> +                ppc->data[i] = (double)*frame.data[0];
> +                i += 1;
> +                nSamples += frame.nb_samples;
> +            }
> +        }
> +        av_free_packet (&packet);
> +    }
> +    ppc->nsamples = i;
> +
> +    if (nSamples < 1)
> +        printf("Decoded audio data is empty");
> +
> +    int sampleSize = av_get_bytes_per_sample (codecContext->sample_fmt);
> +    if (sampleSize < 1)
> +        printf("Invalid sample format");
> +
> +    // optional, return value with sample rate
> +    if (sampleRate) *sampleRate = codecContext->sample_rate;
> +
> +    if (codecContext) avcodec_close (codecContext);
> +
> +    avformat_close_input (&formatContext);
> +}
> +
> +/* Get peaks points from windowed frequency domain data*/
> +static void getPeakPoints(PeakPointsContext *ppc) {
> +    int i, k, size, chunkSize, chunkSampleSize, resSize, sample_rate;
> +    double *fft_res;
> +    int m;
> +
> +    ppc->data = malloc(sizeof(double)*10000);
> +    readAudio (&sample_rate, ppc);
> +
> +    size = ppc->nsamples;
> +    m = log2(ppc->windowSize);
> +    chunkSize = ppc->windowSize;
> +    chunkSampleSize = size/chunkSize;
> +    resSize = chunkSize * chunkSampleSize;
> +    fft_res = malloc(sizeof(double) * resSize);
> +
> +    RDFTContext *rdftC = av_rdft_init(m, DFT_R2C);
> +    FFTSample *data;

> +    data = malloc(sizeof(FFTSample)*chunkSize);
> +    // FFT transform for windowed time domain data
> +    // window is of size chunkSize
> +    k = 0;
> +    while (k < resSize) {
> +        //copy data
> +        for (i = 0; i < chunkSize; i++) {
> +            data[i] = ppc->data[i+k];

missing malloc failure check


> +        }
> +        //calculate FFT
> +        av_rdft_calc(rdftC, data);
> +        for (i = 0; i < chunkSize; i++) {
> +	    fft_res[i+k] = data[i];
> +        }
> +        k = k + chunkSize;
> +    }
> +
> +    av_rdft_end(rdftC);
> +
> +    int pSize = resSize/chunkSize;
> +    ppc->size = pSize;
> +    ppc->peaks = malloc(sizeof(double)*pSize);
> +    getPeakPointInChunk(chunkSize, fft_res, resSize, ppc->peaks);
> +}
> +
> +
> +#define OFFSET(x) offsetof(PeakPointsContext, x)
> +
> +static const AVOption peakpoints_options[] = {
> +    { "input",  "set input audio file", OFFSET(inputFile), AV_OPT_TYPE_STRING, {.str="sample.mp3"}, CHAR_MIN, CHAR_MAX },
> +    { "wsize",  "set window size", OFFSET(windowSize),  AV_OPT_TYPE_INT,    {.i64=16},    0, INT_MAX},
> +    { NULL },
> +};
> +
> +static const char *peakpoints_get_name(void *ctx) {
> +    return "peakpoints";
> +}
> +
> +static const AVClass peakpoints_class = {
> +    .class_name = "PeakPointsContext",
> +    .item_name  = peakpoints_get_name,
> +    .option     = peakpoints_options,
> +};
> +
> +
> +AVFILTER_DEFINE_CLASS(peakpoints);
> +
> +static av_cold int init(AVFilterContext *ctx)
> +{
> +    PeakPointsContext *p = ctx->priv;
> +
> +    if (!(p->inputFile)) {
> +        av_log(ctx, AV_LOG_ERROR, "Input audio file must be passed\n");
> +        return AVERROR(EINVAL);
> +    }
> +
> +    if (p->windowSize < 16) {
> +	     av_log(ctx, AV_LOG_ERROR, "window size must be greater than or equal to 16");
> +        return AVERROR(EINVAL);
> +    }
> +    //printf("Getting peak points..");
> +    // get peaks
> +    getPeakPoints(p);
> +    //printf("Peak points collected.");
> +    // print peaks
> +    int i;
> +    for (i = 0; i < p->size; i++) {
> +        av_log(ctx, AV_LOG_DEBUG, "%f", p->peaks[i]);
> +    }
> +    return 0;
> +}
> +

> +static av_cold void uninit(AVFilterContext *ctx)
> +{
> +    PeakPointsContext *p = ctx->priv;
> +    int i;
> +

> +    // free memory allocated for audio data
> +    for (i = 0; p->nsamples; i++) {
> +	free(&p->data[i]);
> +    }
> +
> +    // free memory allocated for peaks
> +    for (i = 0; p->size; i++) {
> +	free(&p->data[i]);
> +    }

looks like a double free

also please use av_freep() and av_malloc*()

[...]
diff mbox

Patch

From 60429fb7d83d11d6dd733c3477950a940bd0d84a Mon Sep 17 00:00:00 2001
From: Atana <atana@openmailbox.org>
Date: Mon, 24 Oct 2016 17:16:09 +0530
Subject: [PATCH] avfilter: add peakpoints filter

---
 libavfilter/Makefile        |   1 +
 libavfilter/af_peakpoints.c | 263 ++++++++++++++++++++++++++++++++++++++++++++
 libavfilter/allfilters.c    |   1 +
 libavfilter/version.h       |   2 +-
 4 files changed, 266 insertions(+), 1 deletion(-)
 create mode 100644 libavfilter/af_peakpoints.c

diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 7ed4696..1a18902 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -96,6 +96,7 @@  OBJS-$(CONFIG_LADSPA_FILTER)                 += af_ladspa.o
 OBJS-$(CONFIG_LOUDNORM_FILTER)               += af_loudnorm.o
 OBJS-$(CONFIG_LOWPASS_FILTER)                += af_biquads.o
 OBJS-$(CONFIG_PAN_FILTER)                    += af_pan.o
+OBJS-$(CONFIG_PEAKPOINTS_FILTER)             += af_peakpoints.o
 OBJS-$(CONFIG_REPLAYGAIN_FILTER)             += af_replaygain.o
 OBJS-$(CONFIG_RESAMPLE_FILTER)               += af_resample.o
 OBJS-$(CONFIG_RUBBERBAND_FILTER)             += af_rubberband.o
diff --git a/libavfilter/af_peakpoints.c b/libavfilter/af_peakpoints.c
new file mode 100644
index 0000000..84b51da
--- /dev/null
+++ b/libavfilter/af_peakpoints.c
@@ -0,0 +1,263 @@ 
+/*
+ * Copyright (c) 2016 Pallavi Kumari
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavcodec/avcodec.h"
+#include "libavcodec/avfft.h"
+#include "libavformat/avformat.h"
+#include "libswscale/swscale.h"
+#include "avfilter.h"
+#include "libavutil/opt.h"
+
+
+/* Structure to contain peak points context */
+typedef struct {
+    const AVClass *class;
+    double *data;
+    int nsamples;
+    double *peaks;
+    int size; // number of peaks
+    int windowSize;
+    char *inputFile;
+} PeakPointsContext;
+
+/* returns maximum value from an array conditioned on start and end index */
+static double getMax(double *res_arr, int startIndex, int endIndex) {
+    int i;
+    double max = res_arr[startIndex];
+    for (i = startIndex; i <= endIndex; i++) {
+	    if (res_arr[i] > max) {
+	        max = res_arr[i];
+	    }
+    }
+    return max;
+}
+
+/* Stores peak frequency for each window(of chunkSize) in peaks array */
+static void getPeakPointInChunk(int chunkSize, double *res_arr, int size, double *peaks) {
+    int endIndex, i = 0, peakIndex = 0;
+    int startIndex = 0;
+    double max;
+    // get a chunk and find max value in it
+    while (i < size) {
+	    if (i % chunkSize-1 == 0) {
+            max = getMax(res_arr, startIndex, i);
+	        peaks[peakIndex++] = max;
+	        startIndex = startIndex + chunkSize;
+	    }
+        i += 1;
+    }
+}
+
+/* read audio stream from input file */
+static void readAudio (int *sampleRate, PeakPointsContext *ppc) {   
+    // Initialize FFmpeg
+    av_register_all ();
+
+    AVFormatContext *formatContext = NULL;
+    AVCodecContext *codecContext = NULL;
+    AVCodec *codec = NULL;
+    AVPacket packet;
+    AVFrame frame;
+
+    ssize_t nSamples = 0;
+
+    // open the file and autodetect the format
+    if (avformat_open_input (&formatContext, ppc->inputFile, NULL, NULL) < 0)
+        printf("Can not open input file %s", ppc->inputFile);
+
+    if (avformat_find_stream_info (formatContext, NULL) < 0)
+        printf("Can not find stream information");
+
+    // find audio stream (between video/audio/subtitles/.. streams)
+        int audioStreamId = av_find_best_stream (formatContext, AVMEDIA_TYPE_AUDIO,
+                                            -1, -1, &codec, 0);
+    if (audioStreamId < 0)
+        printf("Can not find audio stream in the input file");
+
+    codecContext = formatContext->streams[audioStreamId]->codec;
+
+    // init the audio decoder
+    if (avcodec_open2 (codecContext, codec, NULL) < 0)
+        printf("Can not open audio decoder");
+
+    // read all packets
+    int i = 0;
+    while (av_read_frame (formatContext, &packet) == 0)  {
+        if (packet.stream_index == audioStreamId)  {
+            //avcodec_get_frame_defaults (&frame);
+            av_frame_unref (&frame);
+            int gotFrame = 0;
+            if (avcodec_decode_audio4 (codecContext,&frame,&gotFrame,&packet) < 0){
+                printf("Error decoding audio");
+            }
+            if (gotFrame)  {
+                // audio frame has been decoded
+                int size = av_samples_get_buffer_size (NULL,
+                                                   codecContext->channels,
+                                                   frame.nb_samples,
+                                                   codecContext->sample_fmt,
+                                                   1);
+                if (size < 0)  {
+                    printf("av_samples_get_buffer_size invalid value");
+                }
+
+	            // printf("%d\n", *frame.data[0]);
+                ppc->data[i] = (double)*frame.data[0];
+                i += 1;
+                nSamples += frame.nb_samples;
+            }
+        }
+        av_free_packet (&packet);
+    }
+    ppc->nsamples = i;
+
+    if (nSamples < 1)
+        printf("Decoded audio data is empty");
+
+    int sampleSize = av_get_bytes_per_sample (codecContext->sample_fmt);
+    if (sampleSize < 1)
+        printf("Invalid sample format");
+
+    // optional, return value with sample rate
+    if (sampleRate) *sampleRate = codecContext->sample_rate;
+
+    if (codecContext) avcodec_close (codecContext);
+
+    avformat_close_input (&formatContext);
+}
+
+/* Get peaks points from windowed frequency domain data*/
+static void getPeakPoints(PeakPointsContext *ppc) {
+    int i, k, size, chunkSize, chunkSampleSize, resSize, sample_rate;
+    double *fft_res;
+    int m;
+
+    ppc->data = malloc(sizeof(double)*10000);
+    readAudio (&sample_rate, ppc);
+
+    size = ppc->nsamples;
+    m = log2(ppc->windowSize);
+    chunkSize = ppc->windowSize;
+    chunkSampleSize = size/chunkSize;
+    resSize = chunkSize * chunkSampleSize;
+    fft_res = malloc(sizeof(double) * resSize);
+
+    RDFTContext *rdftC = av_rdft_init(m, DFT_R2C);
+    FFTSample *data;
+    data = malloc(sizeof(FFTSample)*chunkSize);
+    // FFT transform for windowed time domain data
+    // window is of size chunkSize
+    k = 0;
+    while (k < resSize) {
+        //copy data
+        for (i = 0; i < chunkSize; i++) {
+            data[i] = ppc->data[i+k];
+        }
+        //calculate FFT
+        av_rdft_calc(rdftC, data);
+        for (i = 0; i < chunkSize; i++) {
+	    fft_res[i+k] = data[i];
+        }
+        k = k + chunkSize;
+    }
+
+    av_rdft_end(rdftC);
+
+    int pSize = resSize/chunkSize;
+    ppc->size = pSize;
+    ppc->peaks = malloc(sizeof(double)*pSize);
+    getPeakPointInChunk(chunkSize, fft_res, resSize, ppc->peaks);
+}
+
+
+#define OFFSET(x) offsetof(PeakPointsContext, x)
+
+static const AVOption peakpoints_options[] = {
+    { "input",  "set input audio file", OFFSET(inputFile), AV_OPT_TYPE_STRING, {.str="sample.mp3"}, CHAR_MIN, CHAR_MAX },
+    { "wsize",  "set window size", OFFSET(windowSize),  AV_OPT_TYPE_INT,    {.i64=16},    0, INT_MAX},
+    { NULL },
+};
+
+static const char *peakpoints_get_name(void *ctx) {
+    return "peakpoints";
+}
+
+static const AVClass peakpoints_class = {
+    .class_name = "PeakPointsContext",
+    .item_name  = peakpoints_get_name,
+    .option     = peakpoints_options,
+};
+
+
+AVFILTER_DEFINE_CLASS(peakpoints);
+
+static av_cold int init(AVFilterContext *ctx)
+{
+    PeakPointsContext *p = ctx->priv;
+
+    if (!(p->inputFile)) {
+        av_log(ctx, AV_LOG_ERROR, "Input audio file must be passed\n");
+        return AVERROR(EINVAL);
+    }
+
+    if (p->windowSize < 16) {
+	     av_log(ctx, AV_LOG_ERROR, "window size must be greater than or equal to 16");
+        return AVERROR(EINVAL);
+    }
+    //printf("Getting peak points..");
+    // get peaks
+    getPeakPoints(p);
+    //printf("Peak points collected.");
+    // print peaks
+    int i;
+    for (i = 0; i < p->size; i++) {
+        av_log(ctx, AV_LOG_DEBUG, "%f", p->peaks[i]);
+    }
+    return 0;
+}
+
+static av_cold void uninit(AVFilterContext *ctx)
+{
+    PeakPointsContext *p = ctx->priv;
+    int i;
+
+    // free memory allocated for audio data
+    for (i = 0; p->nsamples; i++) {
+	free(&p->data[i]);
+    }
+
+    // free memory allocated for peaks
+    for (i = 0; p->size; i++) {
+	free(&p->data[i]);
+    }
+}
+
+
+AVFilter ff_af_peakpoints = {
+    .name          = "peakpoints",
+    .description   = NULL_IF_CONFIG_SMALL("peak points from frequency domain windowed data."),
+    .init          = init,
+    .uninit        = uninit,
+    //.query_formats = query_formats,
+    .priv_size     = sizeof(PeakPointsContext),
+    //.inputs        = inputs,
+    //.outputs       = outputs,
+    .priv_class    = &peakpoints_class,
+};
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 82a65ee..759e63a 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -114,6 +114,7 @@  void avfilter_register_all(void)
     REGISTER_FILTER(LOUDNORM,       loudnorm,       af);
     REGISTER_FILTER(LOWPASS,        lowpass,        af);
     REGISTER_FILTER(PAN,            pan,            af);
+    REGISTER_FILTER(PEAKPOINTS,     peakpoints,     af);
     REGISTER_FILTER(REPLAYGAIN,     replaygain,     af);
     REGISTER_FILTER(RESAMPLE,       resample,       af);
     REGISTER_FILTER(RUBBERBAND,     rubberband,     af);
diff --git a/libavfilter/version.h b/libavfilter/version.h
index 8c6cf06..adf8c71 100644
--- a/libavfilter/version.h
+++ b/libavfilter/version.h
@@ -30,7 +30,7 @@ 
 #include "libavutil/version.h"
 
 #define LIBAVFILTER_VERSION_MAJOR   6
-#define LIBAVFILTER_VERSION_MINOR  64
+#define LIBAVFILTER_VERSION_MINOR  65
 #define LIBAVFILTER_VERSION_MICRO 100
 
 #define LIBAVFILTER_VERSION_INT AV_VERSION_INT(LIBAVFILTER_VERSION_MAJOR, \
-- 
1.9.1