From patchwork Wed May 29 04:59:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jun Li X-Patchwork-Id: 13320 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 884F1449879 for ; Wed, 29 May 2019 08:06:39 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 673E26803B3; Wed, 29 May 2019 08:06:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EB101680258 for ; Wed, 29 May 2019 08:06:32 +0300 (EEST) Received: by mail-pf1-f196.google.com with SMTP id a23so813487pff.4 for ; Tue, 28 May 2019 22:06:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=I8OxTbqNAk67YQ6U9ygqWoo++dS+IBYQhOaygq37Z4o=; b=T1wfyCq6Zk3OctHQx2pITpF3XnoUcQlM1Xg7v5C3GgZcmL8mEy4qsCdX2UbRhhkc+o /4J5WMx7+A94hMj467kEwr4HNqiI+SXxzVKK7zH7yyootQ8xDKIA2V6e78Y6SSq2x4Pm yeCKw1+A0KIFQNeIW/9/v7U3m16FGXSMJ7sguFvRhiNbtBzuQ068JUq08q2gqvJeoVzE 5QCES1atokTE4dqYe8mQxNgzg2Clpz7lJNcD7sCeiZWC1GSCoPYTtENmhLwxbusB0af1 GeHL6fe7a1TWUTtLmu3uWCd9wZZJlPJdJHbQ5fK6w3wiK+y5/q6ghb0zI3DVE1NHva8z JuPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=I8OxTbqNAk67YQ6U9ygqWoo++dS+IBYQhOaygq37Z4o=; b=HSr1giXu3x5NZ9bvLcfxJAV+YKVwoDK1xYPZ9l4OBbttoUQxwfqnBJIrrneraFO+5c jszAO6dwOm7pDFiWLgV6dnhzAOwdXSsEt7M/5LK/gz0goWMLEdkq6ABASNGu2nlJfhzD NHmAa1JQg7q7D0qU93bZpvPDyb3ghd4ToEb0/Mgm/csJ4H1ntO6lPGJVVYerIR+XWFXi 9GvrplOJo9ygUnv9nQ/+iSUnOUXjkXru9oAOTS2xuYLrnT15nF/YoCTY4g+LImz/oYjE Et7jSnIM669wqGJNaPKxAxdNcwQ0io7g9Q9U5TE2geBygwhjn8bKYrEIENS3GBew2A82 dSvA== X-Gm-Message-State: APjAAAVJkqrAgctjBqjXXXMZfMRtZrMVsKY85MP1eA23FISdC9sHVBlU ad2eeaf9P8MOCTjcE02RYXI5A3LS X-Google-Smtp-Source: APXvYqwkImsAd1/M7rzjf8IJSCx167I9bj0RZzQPOLzbAb8cP6e8V7iBpkxjQpla7e556IRv/PUSNg== X-Received: by 2002:a63:470e:: with SMTP id u14mr25041404pga.135.1559105987766; Tue, 28 May 2019 21:59:47 -0700 (PDT) Received: from jun-ubuntu.corp.microsoft.com ([2001:4898:80e8:7:f7ca:c45e:a7ce:5d49]) by smtp.googlemail.com with ESMTPSA id i9sm4645468pjd.29.2019.05.28.21.59.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 May 2019 21:59:46 -0700 (PDT) From: Jun Li To: ffmpeg-devel@ffmpeg.org Date: Tue, 28 May 2019 21:59:39 -0700 Message-Id: <20190529045940.10198-1-junli1026@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH v4 1/2] lavf/vf_transpose: add exif orientation support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Add exif orientation support and expose an option. --- libavfilter/hflip.h | 2 + libavfilter/transpose.h | 14 ++++ libavfilter/vf_hflip.c | 41 ++++++--- libavfilter/vf_transpose.c | 166 ++++++++++++++++++++++++++++++++----- 4 files changed, 189 insertions(+), 34 deletions(-) diff --git a/libavfilter/hflip.h b/libavfilter/hflip.h index 204090dbb4..4e89bae3fc 100644 --- a/libavfilter/hflip.h +++ b/libavfilter/hflip.h @@ -35,5 +35,7 @@ typedef struct FlipContext { int ff_hflip_init(FlipContext *s, int step[4], int nb_planes); void ff_hflip_init_x86(FlipContext *s, int step[4], int nb_planes); +int ff_hflip_config_props(FlipContext* s, AVFilterLink *inlink); +int ff_hflip_filter_slice(FlipContext *s, AVFrame *in, AVFrame *out, int job, int nb_jobs, int vlifp); #endif /* AVFILTER_HFLIP_H */ diff --git a/libavfilter/transpose.h b/libavfilter/transpose.h index aa262b9487..5da08bddc0 100644 --- a/libavfilter/transpose.h +++ b/libavfilter/transpose.h @@ -34,4 +34,18 @@ enum TransposeDir { TRANSPOSE_VFLIP, }; +enum OrientationType { + ORIENTATION_AUTO_TRANSPOSE = -2, + ORIENTATION_AUTO_FLIP = -1, + ORIENTATION_NONE = 0, + ORIENTATION_NORMAL, + ORIENTATION_HFLIP, + ORIENTATION_ROTATE180, + ORIENTATION_VFLIP, + ORIENTATION_HFLIP_ROTATE270CW, + ORIENTATION_ROTATE90CW, + ORIENTATION_HFLIP_ROTATE90CW, + ORIENTATION_ROTATE270CW +}; + #endif diff --git a/libavfilter/vf_hflip.c b/libavfilter/vf_hflip.c index b77afc77fc..97161000fd 100644 --- a/libavfilter/vf_hflip.c +++ b/libavfilter/vf_hflip.c @@ -37,6 +37,7 @@ #include "libavutil/intreadwrite.h" #include "libavutil/imgutils.h" + static const AVOption hflip_options[] = { { NULL } }; @@ -125,9 +126,8 @@ static void hflip_qword_c(const uint8_t *ssrc, uint8_t *ddst, int w) dst[j] = src[-j]; } -static int config_props(AVFilterLink *inlink) +int ff_hflip_config_props(FlipContext* s, AVFilterLink *inlink) { - FlipContext *s = inlink->dst->priv; const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(inlink->format); const int hsub = pix_desc->log2_chroma_w; const int vsub = pix_desc->log2_chroma_h; @@ -144,6 +144,12 @@ static int config_props(AVFilterLink *inlink) return ff_hflip_init(s, s->max_step, nb_planes); } +static int config_props(AVFilterLink *inlink) +{ + FlipContext *s = inlink->dst->priv; + return ff_hflip_config_props(s, inlink); +} + int ff_hflip_init(FlipContext *s, int step[4], int nb_planes) { int i; @@ -170,14 +176,10 @@ typedef struct ThreadData { AVFrame *in, *out; } ThreadData; -static int filter_slice(AVFilterContext *ctx, void *arg, int job, int nb_jobs) +int ff_hflip_filter_slice(FlipContext *s, AVFrame *in, AVFrame *out, int job, int nb_jobs, int vflip) { - FlipContext *s = ctx->priv; - ThreadData *td = arg; - AVFrame *in = td->in; - AVFrame *out = td->out; uint8_t *inrow, *outrow; - int i, plane, step; + int i, plane, step, outlinesize; for (plane = 0; plane < 4 && in->data[plane] && in->linesize[plane]; plane++) { const int width = s->planewidth[plane]; @@ -187,19 +189,36 @@ static int filter_slice(AVFilterContext *ctx, void *arg, int job, int nb_jobs) step = s->max_step[plane]; - outrow = out->data[plane] + start * out->linesize[plane]; - inrow = in ->data[plane] + start * in->linesize[plane] + (width - 1) * step; + if (vflip) { + outrow = out->data[plane] + (height - start - 1)* out->linesize[plane]; + outlinesize = -out->linesize[plane]; + } else { + outrow = out->data[plane] + start * out->linesize[plane]; + outlinesize = out->linesize[plane]; + } + + inrow = in->data[plane] + start * in->linesize[plane] + (width - 1) * step; + for (i = start; i < end; i++) { s->flip_line[plane](inrow, outrow, width); inrow += in ->linesize[plane]; - outrow += out->linesize[plane]; + outrow += outlinesize; } } return 0; } +static int filter_slice(AVFilterContext *ctx, void *arg, int job, int nb_jobs) +{ + FlipContext *s = ctx->priv; + ThreadData *td = arg; + AVFrame *in = td->in; + AVFrame *out = td->out; + return ff_hflip_filter_slice(s, in, out, job, nb_jobs, 0); +} + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; diff --git a/libavfilter/vf_transpose.c b/libavfilter/vf_transpose.c index dd54947bd9..d3e6a8eea1 100644 --- a/libavfilter/vf_transpose.c +++ b/libavfilter/vf_transpose.c @@ -39,6 +39,7 @@ #include "internal.h" #include "video.h" #include "transpose.h" +#include "hflip.h" typedef struct TransVtable { void (*transpose_8x8)(uint8_t *src, ptrdiff_t src_linesize, @@ -48,16 +49,24 @@ typedef struct TransVtable { int w, int h); } TransVtable; -typedef struct TransContext { - const AVClass *class; +typedef struct TransContextData { int hsub, vsub; int planes; int pixsteps[4]; + TransVtable vtables[4]; +} TransContextData; +typedef struct TransContext { + const AVClass *class; int passthrough; ///< PassthroughType, landscape passthrough mode enabled int dir; ///< TransposeDir - - TransVtable vtables[4]; + int orientation; ///< OrientationType + int autoflip; + int autotranspose; + union { + TransContextData transpose; + FlipContext flip; + } ctx; } TransContext; static int query_formats(AVFilterContext *ctx) @@ -207,14 +216,19 @@ static int config_props_output(AVFilterLink *outlink) s->passthrough = TRANSPOSE_PT_TYPE_NONE; } - s->hsub = desc_in->log2_chroma_w; - s->vsub = desc_in->log2_chroma_h; - s->planes = av_pix_fmt_count_planes(outlink->format); + s->autoflip = s->orientation == ORIENTATION_AUTO_FLIP; + s->autotranspose = s->orientation == ORIENTATION_AUTO_TRANSPOSE; + if (s->autoflip || (s->orientation > 0 && s->orientation <= 4)) + return ff_hflip_config_props(&(s->ctx.flip), inlink); + + s->ctx.transpose.hsub = desc_in->log2_chroma_w; + s->ctx.transpose.vsub = desc_in->log2_chroma_h; + s->ctx.transpose.planes = av_pix_fmt_count_planes(outlink->format); av_assert0(desc_in->nb_components == desc_out->nb_components); - av_image_fill_max_pixsteps(s->pixsteps, NULL, desc_out); + av_image_fill_max_pixsteps(s->ctx.transpose.pixsteps, NULL, desc_out); outlink->w = inlink->h; outlink->h = inlink->w; @@ -226,10 +240,10 @@ static int config_props_output(AVFilterLink *outlink) outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; for (int i = 0; i < 4; i++) { - TransVtable *v = &s->vtables[i]; - switch (s->pixsteps[i]) { + TransVtable *v = &s->ctx.transpose.vtables[i]; + switch (s->ctx.transpose.pixsteps[i]) { case 1: v->transpose_block = transpose_block_8_c; - v->transpose_8x8 = transpose_8x8_8_c; break; + v->transpose_8x8 = transpose_8x8_8_c; break; case 2: v->transpose_block = transpose_block_16_c; v->transpose_8x8 = transpose_8x8_16_c; break; case 3: v->transpose_block = transpose_block_24_c; @@ -264,19 +278,60 @@ typedef struct ThreadData { AVFrame *in, *out; } ThreadData; -static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, - int nb_jobs) +static int get_frame_orientation(AVFilterContext *ctx, const AVFrame *frame) +{ + TransContext *s = ctx->priv; + AVDictionaryEntry *entry = NULL; + int orientation = 0; + + if (s->orientation > 0) + return s->orientation; + + if (!s->autoflip && !s->autotranspose) + return orientation; + + // read exif orientation data + entry = av_dict_get(frame->metadata, "Orientation", NULL, 0); + if (entry && entry->value) + orientation = atoi(entry->value); + + if (s->autoflip) { + if (orientation > 0 && orientation <= 4) { + return orientation; + } else { + av_log(ctx, AV_LOG_WARNING, + "found orientation value: %i, but auto-flip only cover range [1, 4]. Set to default value 1.\n", + orientation); + return 1; // return the min value of orientation range, which has the same effect of passthrough + } + } + if (s->autotranspose) { + if (orientation > 4) { + return orientation; + } else { + av_log(ctx, AV_LOG_WARNING, + "found orientation value: %i, but auto-transpose only cover range [5, 8]. Set to default value 5.\n", + orientation); + return 5; // return the min value of orientation range, which has the same effect of dir=0 + } + } + return orientation; +} + +static int transpose_filter_slice(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) { TransContext *s = ctx->priv; ThreadData *td = arg; AVFrame *out = td->out; AVFrame *in = td->in; + TransContextData *c = &(s->ctx.transpose); int plane; + int orientation = get_frame_orientation(ctx, in); - for (plane = 0; plane < s->planes; plane++) { - int hsub = plane == 1 || plane == 2 ? s->hsub : 0; - int vsub = plane == 1 || plane == 2 ? s->vsub : 0; - int pixstep = s->pixsteps[plane]; + for (plane = 0; plane < c->planes; plane++) { + int hsub = plane == 1 || plane == 2 ? c->hsub : 0; + int vsub = plane == 1 || plane == 2 ? c->vsub : 0; + int pixstep = c->pixsteps[plane]; int inh = AV_CEIL_RSHIFT(in->height, vsub); int outw = AV_CEIL_RSHIFT(out->width, hsub); int outh = AV_CEIL_RSHIFT(out->height, vsub); @@ -285,19 +340,36 @@ static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, uint8_t *dst, *src; int dstlinesize, srclinesize; int x, y; - TransVtable *v = &s->vtables[plane]; + int dir = s->dir; + TransVtable *v = &c->vtables[plane]; dstlinesize = out->linesize[plane]; dst = out->data[plane] + start * dstlinesize; src = in->data[plane]; srclinesize = in->linesize[plane]; + switch (orientation) { + case ORIENTATION_HFLIP_ROTATE270CW: + dir = 0; // mirror horizontal and rotate 270 CW, same as dir=cclock_flip + break; + case ORIENTATION_ROTATE90CW: + dir = 1; // rotate 90 CW, same as dir=clock + break; + case ORIENTATION_HFLIP_ROTATE90CW: + dir = 3; // mirror horizontal and rotate 90 CW, same as dir=clock_flip + break; + case ORIENTATION_ROTATE270CW: + dir = 2; // rotate 270 CW, same as dir=cclock + break; + default: + break; + } - if (s->dir & 1) { + if (dir & 1) { src += in->linesize[plane] * (inh - 1); srclinesize *= -1; } - if (s->dir & 2) { + if (dir & 2) { dst = out->data[plane] + dstlinesize * (outh - start - 1); dstlinesize *= -1; } @@ -321,11 +393,27 @@ static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, srclinesize, dst + (y - start) * dstlinesize + 0 * pixstep, dstlinesize, outw, end - y); + } return 0; } +static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, + int nb_jobs) +{ + TransContext *s = ctx->priv; + ThreadData *td = arg; + AVFrame *out = td->out; + AVFrame *in = td->in; + int orientation = get_frame_orientation(ctx, in); + + if (orientation && orientation < 5) { + return ff_hflip_filter_slice(&s->ctx.flip, in, out, jobnr, nb_jobs, orientation == 3); + } + return transpose_filter_slice(ctx, arg, jobnr, nb_jobs); +} + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; @@ -333,11 +421,31 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFilterLink *outlink = ctx->outputs[0]; ThreadData td; AVFrame *out; + int orientation; if (s->passthrough) return ff_filter_frame(outlink, in); + if (s->orientation) { + orientation = get_frame_orientation(ctx, in); + if (orientation == ORIENTATION_NORMAL) // orientation 'Normal', do nothing + return ff_filter_frame(outlink, in); + + if (orientation == ORIENTATION_VFLIP) { + int i; + for (i = 0; i < 4; i ++) { + int height = s->ctx.flip.planeheight[i]; + if (in->data[i]) { + in->data[i] += (height - 1) * in->linesize[i]; + in->linesize[i] = -in->linesize[i]; + } + } + return ff_filter_frame(outlink, in); + } + } + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) { av_frame_free(&in); return AVERROR(ENOMEM); @@ -346,7 +454,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) if (in->sample_aspect_ratio.num == 0) { out->sample_aspect_ratio = in->sample_aspect_ratio; - } else { + } else if (s->autotranspose || orientation > 4) { // transpose out->sample_aspect_ratio.num = in->sample_aspect_ratio.den; out->sample_aspect_ratio.den = in->sample_aspect_ratio.num; } @@ -366,13 +474,25 @@ static const AVOption transpose_options[] = { { "clock", "rotate clockwise", 0, AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CLOCK }, .flags=FLAGS, .unit = "dir" }, { "cclock", "rotate counter-clockwise", 0, AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CCLOCK }, .flags=FLAGS, .unit = "dir" }, { "clock_flip", "rotate clockwise with vertical flip", 0, AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CLOCK_FLIP }, .flags=FLAGS, .unit = "dir" }, - + { "passthrough", "do not apply transposition if the input matches the specified geometry", OFFSET(passthrough), AV_OPT_TYPE_INT, {.i64=TRANSPOSE_PT_TYPE_NONE}, 0, INT_MAX, FLAGS, "passthrough" }, { "none", "always apply transposition", 0, AV_OPT_TYPE_CONST, {.i64=TRANSPOSE_PT_TYPE_NONE}, INT_MIN, INT_MAX, FLAGS, "passthrough" }, { "portrait", "preserve portrait geometry", 0, AV_OPT_TYPE_CONST, {.i64=TRANSPOSE_PT_TYPE_PORTRAIT}, INT_MIN, INT_MAX, FLAGS, "passthrough" }, { "landscape", "preserve landscape geometry", 0, AV_OPT_TYPE_CONST, {.i64=TRANSPOSE_PT_TYPE_LANDSCAPE}, INT_MIN, INT_MAX, FLAGS, "passthrough" }, - + { "orientation", "apply frame exif orientation", + OFFSET(orientation), AV_OPT_TYPE_INT, {.i64 = 0 }, -2, 8, FLAGS, "orientation" }, + { "auto_transpose", "apply tranpose if frame's orietaion value is in range [5,8]", 0, AV_OPT_TYPE_CONST, {.i64=ORIENTATION_AUTO_TRANSPOSE}, INT_MIN, INT_MAX, FLAGS, "orientation" }, + { "auto_flip", "apply flip if frame's orietaion value is in range [1,4]", 0, AV_OPT_TYPE_CONST, {.i64=ORIENTATION_AUTO_FLIP}, INT_MIN, INT_MAX, FLAGS, "orientation" }, + { "none", "apply default transposition, same as dir=cclock_flip", 0, AV_OPT_TYPE_CONST, {.i64=ORIENTATION_NONE}, INT_MIN, INT_MAX, FLAGS, "orientation" }, + { "normal", "exif orientation 'Normal', do not apply any flip or transposition", 0, AV_OPT_TYPE_CONST, {.i64=ORIENTATION_NORMAL}, INT_MIN, INT_MAX, FLAGS, "orientation" }, + { "hflip", "exif orientation 'Mirror horizontal', apply horizontal flip", 0, AV_OPT_TYPE_CONST, {.i64=ORIENTATION_HFLIP}, INT_MIN, INT_MAX, FLAGS, "orientation" }, + { "rotate180", "exif orientation 'Rotate 180', apply rotation 180", 0, AV_OPT_TYPE_CONST, {.i64=ORIENTATION_ROTATE180}, INT_MIN, INT_MAX, FLAGS, "orientation" }, + { "vflip", "exif orientation 'Mirror vertical', apply vertical flip", 0, AV_OPT_TYPE_CONST, {.i64=ORIENTATION_VFLIP}, INT_MIN, INT_MAX, FLAGS, "orientation" }, + { "hflip_rotate270cw", "exif orientation 'Mirror horizontal and rotate 270 CW', same effect as dir=cclock_flip", 0, AV_OPT_TYPE_CONST, {.i64=ORIENTATION_HFLIP_ROTATE270CW}, INT_MIN, INT_MAX, FLAGS, "orientation" }, + { "rotate90cw", "exif orientation 'Rotate 90 CW', same as dir=clock", 0, AV_OPT_TYPE_CONST, {.i64=ORIENTATION_ROTATE90CW}, INT_MIN, INT_MAX, FLAGS, "orientation" }, + { "hflip_rotate90cw", "exif orientation 'Mirror horizontal and rotate 90 CW', same effect as dir=clock_flip", 0, AV_OPT_TYPE_CONST, {.i64=ORIENTATION_HFLIP_ROTATE90CW}, INT_MIN, INT_MAX, FLAGS, "orientation" }, + { "rotate270", "exif orientation 'Rotate 270 CW', same as dir=cclock", 0, AV_OPT_TYPE_CONST, {.i64=ORIENTATION_ROTATE270CW}, INT_MIN, INT_MAX, FLAGS, "orientation" }, { NULL } };