From patchwork Tue May 28 06:18:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jun Li X-Patchwork-Id: 13311 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 07532447DCD for ; Tue, 28 May 2019 09:18:43 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DCC2368978D; Tue, 28 May 2019 09:18:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 721D8680A1D for ; Tue, 28 May 2019 09:18:36 +0300 (EEST) Received: by mail-pf1-f196.google.com with SMTP id a23so10822904pff.4 for ; Mon, 27 May 2019 23:18:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=yjYyhkmVzCV0KwqG7d4bTkww98VbQTZHfTVWs8pyUqo=; b=U491yfdhL7jyuj3SrlaEpV9DNFHA4SEom+KKKWnJqWFnn2LVlGhww2Jz3lcj4eIxnE bujCVqFOyJKJOcM/o+ITvOcFZIYItQMimu7c3rEd6XYFMvH1ejLQHlNhy1ame05TdRJw fEbmMrKfRrqQi+i6VMrGc4yylqc9gpx6QAKhJh+dKDzaUHnClGh9GJZelomEbZmqBQ0Y vzZXHk6YzcSk82UwyPlnDTQuQiyUcMD7If/VhpMU61X0z6+iUe8gwLpIVQbpS2Ae1kdb oqJaLU4wZ6DotQCHqMp2bZNXYybyJ6Ogqf0T16AM84MG2QoBu9EfcH3UHq1e7Tu39Xj2 3/4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=yjYyhkmVzCV0KwqG7d4bTkww98VbQTZHfTVWs8pyUqo=; b=Qpm72AcOP9Mm2ICbLJOS7o008X9YrtwAAcOSgcF3/dm3fU5wZikvy/X/U4Lckj6ayF QwItPFpSRy8WRbWaHi2Agd0nVeXobDGyt0mT2wIckG1JnSHMIBT8uSycFVgOvOPIaJqI cRKpWs92WFDSpb3ek/xTQ1Ij5PQVeyfLn6M4H7iXCxXK6EItPFEqXAmbmv6f5qB3tuml X6aExcy3+gEXOp8gIjClKmBbnhMq+nzQO7GuL+tFFmMrjsZg2Z7KpZL/198BUVr/5qQ5 BvdG396tzZlVYvu2FC125jBCh+KwGnWLqbPFgZGbDZNu5sE73ONV32gQfWKiI7D2IWwB iUVA== X-Gm-Message-State: APjAAAVb8I21+QqL3QlK89zhAEmuW1OrIeS/Zo+IDaBecOMggLAlh8qH PH0UIBNxUW/8ScNiLXVLMv4g5hB9 X-Google-Smtp-Source: APXvYqwIz7MLMFv6gxqztl+OgN1gbpZVcjiqmTXWLUncUaCYNhC6SSo+wao16JJXf9/9noyybr54gA== X-Received: by 2002:a63:c509:: with SMTP id f9mr2145999pgd.143.1559024313934; Mon, 27 May 2019 23:18:33 -0700 (PDT) Received: from jun-ubuntu.corp.microsoft.com ([2001:4898:80e8:f:e013:c96a:c2c8:7f91]) by smtp.googlemail.com with ESMTPSA id k6sm13385320pfi.86.2019.05.27.23.18.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 May 2019 23:18:32 -0700 (PDT) From: Jun Li To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 May 2019 23:18:25 -0700 Message-Id: <20190528061826.22537-1-junli1026@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH v3 1/2] lavf/vf_transpose: add exif orientation support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Add exif orientation support and expose an option. --- libavfilter/hflip.h | 3 + libavfilter/vf_hflip.c | 43 ++++++--- libavfilter/vf_transpose.c | 173 +++++++++++++++++++++++++++++-------- 3 files changed, 171 insertions(+), 48 deletions(-) diff --git a/libavfilter/hflip.h b/libavfilter/hflip.h index 204090dbb4..3d837f01b0 100644 --- a/libavfilter/hflip.h +++ b/libavfilter/hflip.h @@ -35,5 +35,8 @@ typedef struct FlipContext { int ff_hflip_init(FlipContext *s, int step[4], int nb_planes); void ff_hflip_init_x86(FlipContext *s, int step[4], int nb_planes); +int ff_hflip_config_props(FlipContext* s, AVFilterLink *inlink); +int ff_hflip_filter_slice(FlipContext *s, AVFrame *in, AVFrame *out, int job, int nb_jobs, int vlifp); + #endif /* AVFILTER_HFLIP_H */ diff --git a/libavfilter/vf_hflip.c b/libavfilter/vf_hflip.c index b77afc77fc..84fd8975b1 100644 --- a/libavfilter/vf_hflip.c +++ b/libavfilter/vf_hflip.c @@ -37,6 +37,7 @@ #include "libavutil/intreadwrite.h" #include "libavutil/imgutils.h" + static const AVOption hflip_options[] = { { NULL } }; @@ -125,9 +126,8 @@ static void hflip_qword_c(const uint8_t *ssrc, uint8_t *ddst, int w) dst[j] = src[-j]; } -static int config_props(AVFilterLink *inlink) +int ff_hflip_config_props(FlipContext* s, AVFilterLink *inlink) { - FlipContext *s = inlink->dst->priv; const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(inlink->format); const int hsub = pix_desc->log2_chroma_w; const int vsub = pix_desc->log2_chroma_h; @@ -140,10 +140,15 @@ static int config_props(AVFilterLink *inlink) s->planeheight[1] = s->planeheight[2] = AV_CEIL_RSHIFT(inlink->h, vsub); nb_planes = av_pix_fmt_count_planes(inlink->format); - return ff_hflip_init(s, s->max_step, nb_planes); } +static int config_props(AVFilterLink *inlink) +{ + FlipContext *s = inlink->dst->priv; + return ff_hflip_config_props(s, inlink); +} + int ff_hflip_init(FlipContext *s, int step[4], int nb_planes) { int i; @@ -170,14 +175,10 @@ typedef struct ThreadData { AVFrame *in, *out; } ThreadData; -static int filter_slice(AVFilterContext *ctx, void *arg, int job, int nb_jobs) +int ff_hflip_filter_slice(FlipContext *s, AVFrame *in, AVFrame *out, int job, int nb_jobs, int vflip) { - FlipContext *s = ctx->priv; - ThreadData *td = arg; - AVFrame *in = td->in; - AVFrame *out = td->out; uint8_t *inrow, *outrow; - int i, plane, step; + int i, plane, step, outlinesize; for (plane = 0; plane < 4 && in->data[plane] && in->linesize[plane]; plane++) { const int width = s->planewidth[plane]; @@ -187,19 +188,35 @@ static int filter_slice(AVFilterContext *ctx, void *arg, int job, int nb_jobs) step = s->max_step[plane]; - outrow = out->data[plane] + start * out->linesize[plane]; - inrow = in ->data[plane] + start * in->linesize[plane] + (width - 1) * step; + if (vflip) { + outrow = out->data[plane] + (height - start - 1)* out->linesize[plane]; + outlinesize = -out->linesize[plane]; + } else { + outrow = out->data[plane] + start * out->linesize[plane]; + outlinesize = out->linesize[plane]; + } + + inrow = in->data[plane] + start * in->linesize[plane] + (width - 1) * step; + for (i = start; i < end; i++) { s->flip_line[plane](inrow, outrow, width); inrow += in ->linesize[plane]; - outrow += out->linesize[plane]; + outrow += outlinesize; } } - return 0; } +static int filter_slice(AVFilterContext *ctx, void *arg, int job, int nb_jobs) +{ + FlipContext *s = ctx->priv; + ThreadData *td = arg; + AVFrame *in = td->in; + AVFrame *out = td->out; + return ff_hflip_filter_slice(s, in, out, job, nb_jobs, 0); +} + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; diff --git a/libavfilter/vf_transpose.c b/libavfilter/vf_transpose.c index dd54947bd9..9cc07a1e6f 100644 --- a/libavfilter/vf_transpose.c +++ b/libavfilter/vf_transpose.c @@ -39,6 +39,7 @@ #include "internal.h" #include "video.h" #include "transpose.h" +#include "hflip.h" typedef struct TransVtable { void (*transpose_8x8)(uint8_t *src, ptrdiff_t src_linesize, @@ -48,16 +49,24 @@ typedef struct TransVtable { int w, int h); } TransVtable; -typedef struct TransContext { - const AVClass *class; +typedef struct TransContextData { int hsub, vsub; int planes; int pixsteps[4]; + TransVtable vtables[4]; +} TransContextData; +typedef struct TransContext { + const AVClass *class; int passthrough; ///< PassthroughType, landscape passthrough mode enabled int dir; ///< TransposeDir - - TransVtable vtables[4]; + int orientation; ///< Orientation + int autoflip; + int autotranspose; + union { + TransContextData transpose; + FlipContext flip; + } ctx; } TransContext; static int query_formats(AVFilterContext *ctx) @@ -207,15 +216,19 @@ static int config_props_output(AVFilterLink *outlink) s->passthrough = TRANSPOSE_PT_TYPE_NONE; } - s->hsub = desc_in->log2_chroma_w; - s->vsub = desc_in->log2_chroma_h; - s->planes = av_pix_fmt_count_planes(outlink->format); + s->autoflip = s->orientation == -1; + s->autotranspose = s->orientation == -2; + if (s->autoflip || (s->orientation > 0 && s->orientation <= 4)) + return ff_hflip_config_props(&(s->ctx.flip), inlink); + + s->ctx.transpose.hsub = desc_in->log2_chroma_w; + s->ctx.transpose.vsub = desc_in->log2_chroma_h; + s->ctx.transpose.planes = av_pix_fmt_count_planes(outlink->format); av_assert0(desc_in->nb_components == desc_out->nb_components); - - av_image_fill_max_pixsteps(s->pixsteps, NULL, desc_out); - + av_image_fill_max_pixsteps(s->ctx.transpose.pixsteps, NULL, desc_out); + outlink->w = inlink->h; outlink->h = inlink->w; @@ -226,10 +239,10 @@ static int config_props_output(AVFilterLink *outlink) outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; for (int i = 0; i < 4; i++) { - TransVtable *v = &s->vtables[i]; - switch (s->pixsteps[i]) { + TransVtable *v = &s->ctx.transpose.vtables[i]; + switch (s->ctx.transpose.pixsteps[i]) { case 1: v->transpose_block = transpose_block_8_c; - v->transpose_8x8 = transpose_8x8_8_c; break; + v->transpose_8x8 = transpose_8x8_8_c; break; case 2: v->transpose_block = transpose_block_16_c; v->transpose_8x8 = transpose_8x8_16_c; break; case 3: v->transpose_block = transpose_block_24_c; @@ -264,19 +277,61 @@ typedef struct ThreadData { AVFrame *in, *out; } ThreadData; -static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, - int nb_jobs) + +static int get_frame_orientation(AVFilterContext *ctx, const AVFrame *frame) +{ + TransContext *s = ctx->priv; + AVDictionaryEntry *entry = NULL; + int orientation = 0; + + if (s->orientation > 0) + return s->orientation; + + if (!s->autoflip && !s->autotranspose) + return orientation; + + // read exif orientation data + entry = av_dict_get(frame->metadata, "Orientation", NULL, 0); + if (entry) + orientation = atoi(entry->value); + + if (s->autoflip) { + if (orientation > 0 && orientation <= 4) { + return orientation; + } else { + av_log(ctx, AV_LOG_WARNING, + "found orientation value: %i, but auto-flip only cover range [1, 4]. Set to default value 1.\n", + orientation); + return 1; // return the min value of orientation range, which has the same effect of passthrough + } + } + if (s->autotranspose) { + if (orientation > 4) { + return orientation; + } else { + av_log(ctx, AV_LOG_WARNING, + "found orientation value: %i, but auto-transpose only cover range [5, 8]. Set to default value 5.\n", + orientation); + return 5; // return the min value of orientation range, which has the same effect of dir=0 + } + } + return orientation; +} + +static int transpose_filter_slice(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) { TransContext *s = ctx->priv; ThreadData *td = arg; AVFrame *out = td->out; AVFrame *in = td->in; + TransContextData *c = &(s->ctx.transpose); int plane; + int orientation = get_frame_orientation(ctx, in); - for (plane = 0; plane < s->planes; plane++) { - int hsub = plane == 1 || plane == 2 ? s->hsub : 0; - int vsub = plane == 1 || plane == 2 ? s->vsub : 0; - int pixstep = s->pixsteps[plane]; + for (plane = 0; plane < c->planes; plane++) { + int hsub = plane == 1 || plane == 2 ? c->hsub : 0; + int vsub = plane == 1 || plane == 2 ? c->vsub : 0; + int pixstep = c->pixsteps[plane]; int inh = AV_CEIL_RSHIFT(in->height, vsub); int outw = AV_CEIL_RSHIFT(out->width, hsub); int outh = AV_CEIL_RSHIFT(out->height, vsub); @@ -285,19 +340,31 @@ static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, uint8_t *dst, *src; int dstlinesize, srclinesize; int x, y; - TransVtable *v = &s->vtables[plane]; + int dir = s->dir; + TransVtable *v = &c->vtables[plane]; dstlinesize = out->linesize[plane]; dst = out->data[plane] + start * dstlinesize; src = in->data[plane]; srclinesize = in->linesize[plane]; + switch (orientation) { + case 5: // mirror horizontal and rotate 270 CW + dir = 0; break; + case 6: // rotate 90 CW + dir = 1; break; + case 7: // mirror horizontal and rotate 90 CW + dir = 3; break; + case 8: // rotate 270 CW + dir = 2; break; + default: break; + } - if (s->dir & 1) { + if (dir & 1) { src += in->linesize[plane] * (inh - 1); srclinesize *= -1; } - if (s->dir & 2) { + if (dir & 2) { dst = out->data[plane] + dstlinesize * (outh - start - 1); dstlinesize *= -1; } @@ -305,27 +372,42 @@ static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, for (y = start; y < end - 7; y += 8) { for (x = 0; x < outw - 7; x += 8) { v->transpose_8x8(src + x * srclinesize + y * pixstep, - srclinesize, - dst + (y - start) * dstlinesize + x * pixstep, - dstlinesize); + srclinesize, + dst + (y - start) * dstlinesize + x * pixstep, + dstlinesize); } if (outw - x > 0 && end - y > 0) v->transpose_block(src + x * srclinesize + y * pixstep, - srclinesize, - dst + (y - start) * dstlinesize + x * pixstep, - dstlinesize, outw - x, end - y); + srclinesize, + dst + (y - start) * dstlinesize + x * pixstep, + dstlinesize, outw - x, end - y); } if (end - y > 0) v->transpose_block(src + 0 * srclinesize + y * pixstep, - srclinesize, - dst + (y - start) * dstlinesize + 0 * pixstep, - dstlinesize, outw, end - y); + srclinesize, + dst + (y - start) * dstlinesize + 0 * pixstep, + dstlinesize, outw, end - y); + } - return 0; } +static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, + int nb_jobs) +{ + TransContext *s = ctx->priv; + ThreadData *td = arg; + AVFrame *out = td->out; + AVFrame *in = td->in; + int orientation = get_frame_orientation(ctx, in); + + if (orientation && orientation < 5) { + return ff_hflip_filter_slice(&s->ctx.flip, in, out, jobnr, nb_jobs, orientation == 3); + } + return transpose_filter_slice(ctx, arg, jobnr, nb_jobs); +} + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; @@ -333,11 +415,31 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFilterLink *outlink = ctx->outputs[0]; ThreadData td; AVFrame *out; + int orientation; if (s->passthrough) return ff_filter_frame(outlink, in); + if (s->orientation) { + orientation = get_frame_orientation(ctx, in); + if (orientation == 1) // passthrough, do nothing + return ff_filter_frame(outlink, in); + + if (orientation == 4) { // vflip + int i; + for (i = 0; i < 4; i ++) { + int height = s->ctx.flip.planeheight[i]; + if (in->data[i]) { + in->data[i] += (height - 1) * in->linesize[i]; + in->linesize[i] = -in->linesize[i]; + } + } + return ff_filter_frame(outlink, in); + } + } + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) { av_frame_free(&in); return AVERROR(ENOMEM); @@ -346,7 +448,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) if (in->sample_aspect_ratio.num == 0) { out->sample_aspect_ratio = in->sample_aspect_ratio; - } else { + } else if (s->autotranspose || orientation > 4) { // transpose out->sample_aspect_ratio.num = in->sample_aspect_ratio.den; out->sample_aspect_ratio.den = in->sample_aspect_ratio.num; } @@ -366,13 +468,14 @@ static const AVOption transpose_options[] = { { "clock", "rotate clockwise", 0, AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CLOCK }, .flags=FLAGS, .unit = "dir" }, { "cclock", "rotate counter-clockwise", 0, AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CCLOCK }, .flags=FLAGS, .unit = "dir" }, { "clock_flip", "rotate clockwise with vertical flip", 0, AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CLOCK_FLIP }, .flags=FLAGS, .unit = "dir" }, - + { "passthrough", "do not apply transposition if the input matches the specified geometry", OFFSET(passthrough), AV_OPT_TYPE_INT, {.i64=TRANSPOSE_PT_TYPE_NONE}, 0, INT_MAX, FLAGS, "passthrough" }, { "none", "always apply transposition", 0, AV_OPT_TYPE_CONST, {.i64=TRANSPOSE_PT_TYPE_NONE}, INT_MIN, INT_MAX, FLAGS, "passthrough" }, { "portrait", "preserve portrait geometry", 0, AV_OPT_TYPE_CONST, {.i64=TRANSPOSE_PT_TYPE_PORTRAIT}, INT_MIN, INT_MAX, FLAGS, "passthrough" }, { "landscape", "preserve landscape geometry", 0, AV_OPT_TYPE_CONST, {.i64=TRANSPOSE_PT_TYPE_LANDSCAPE}, INT_MIN, INT_MAX, FLAGS, "passthrough" }, - + { "orientation", "apply frame exif orientation. Value -1 will auto flip, cover orientation from 1 to 4; value -2 will auto transpose, cover from 5 to 8", + OFFSET(orientation), AV_OPT_TYPE_INT, {.i64 = 0 }, -2, 8, FLAGS, "orientation" }, { NULL } };