From patchwork Sun Nov 26 04:33:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Geoffrey McRae X-Patchwork-Id: 44804 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bca6:b0:181:818d:5e7f with SMTP id fx38csp2502148pzb; Sat, 25 Nov 2023 20:33:35 -0800 (PST) X-Google-Smtp-Source: AGHT+IGXePSbT9JuujUdh+gxI54BShm1vUCn0LnBZY0uIAP6eAuvtBDo5q0arzvbFpPaLO+Urd7U X-Received: by 2002:a17:906:5299:b0:a00:8706:c82e with SMTP id c25-20020a170906529900b00a008706c82emr6209570ejm.18.1700973215502; Sat, 25 Nov 2023 20:33:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700973215; cv=none; d=google.com; s=arc-20160816; b=uMCMlAXi1D+xscDGUmlCviMlYEtQ4UzW6xMxxfwXlvQmyZBGKFzgeogLNyCZRMK9mB B1DtEmvgXmYCh4rMQ7eTq8emmzDNfGt1E1QVOh8kdjoQNRduX1hEtV2txjvct3vAZoLH /3rNaHkiG+JU4ah4tw4pCSndFiNllpw95lMSokl2plHbiRF9mYlLpwMeVwFJ8izZBsld bq8yOAffqfNKFP7sCHBQpWbaY6+2jgMXh7EqWkm6K45tPhfWMB3rKLUP/v9SDuGBUh/i h+u5j2iCPd4aOGL15XzcdPladbnVpCN5puLxgDMVsu+PH1DtO3jBWMAatITwUpmucmKX 3bdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to :delivered-to; bh=36va9m8ovVsRz9QG5+haB29O0k4fraRutbsKiq4q+wE=; fh=hNE7h42qM2XxuRTcO+7OnYPFIYww24ZCpnHAyUPDybE=; b=WrHQ7JyihmPDd7NTYctsQQcDJOZPxZ4aHiJv0lhkxDvmmiuj5HemT8CdzDUhfj0Yqu RmIbByuVhkz0YFlErlsCXUPQ1stwyV1dFO4uEJu9aLg6niV343aSUfgDKZ01wqGubf5/ Cc4jOBo9qkZ+7jBGQwrJvXUEJhebsvyyGiRJN5Dl7j+hhsevfQIUYnt84R7+445HHYJr iehILxg17s5J6c6yc5I99jnvDKyQ5cQD4DrCXkXF8dLc6crdpWX/jrtlK8ISaILahO8I lAsYNm7cRBizhCIh/5YJoQNrO5HQizw28q39DPyVp6hytqvAgQT3m8KYfOZObPMeGUa1 DVjA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v12-20020a170906380c00b00a02f8f44c14si3427914ejc.1024.2023.11.25.20.33.34; Sat, 25 Nov 2023 20:33:35 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0175268CF5B; Sun, 26 Nov 2023 06:33:31 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.hostfission.com (mail1.hostfission.com [118.127.8.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7854E68CC8F for ; Sun, 26 Nov 2023 06:33:24 +0200 (EET) Received: from aeryn.lan.ktmba (office.hostfission.com [220.233.29.71]) by mail1.hostfission.com (Postfix) with ESMTPS id 7106C1E1A28; Sun, 26 Nov 2023 15:33:23 +1100 (AEDT) Received: by aeryn.lan.ktmba (Postfix, from userid 1000) id 41CFE2E064F; Sun, 26 Nov 2023 15:33:23 +1100 (AEDT) To: ffmpeg-devel@ffmpeg.org Date: Sun, 26 Nov 2023 15:33:22 +1100 Message-Id: <20231126043322.1681663-1-geoff@hostfission.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] libavcodec/dcadec: adjust the `ch_layout` when downmix is active X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Geoffrey McRae via ffmpeg-devel From: Geoffrey McRae Reply-To: FFmpeg development discussions and patches Cc: Geoffrey McRae Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: BGGtSGIe2bFl Applications making use of this codec with the `downmix` option are segfaulting unless the `ch_layout` is overridden after `avcodec_open2` as can be seen in projects like MythTV[1] This patch fixes this by overriding the ch_layout as done in other decoders such as AC3. 1: https://github.com/MythTV/mythtv/blob/af6f362a140cd59b9ed784a8c639fd456b5f6967/mythtv/libs/libmythtv/decoders/avformatdecoder.cpp#L4607 Signed-off-by: Geoffrey McRae --- libavcodec/dcadec.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/libavcodec/dcadec.c b/libavcodec/dcadec.c index 1fee49cf4d..cc23731b90 100644 --- a/libavcodec/dcadec.c +++ b/libavcodec/dcadec.c @@ -366,12 +366,19 @@ FF_ENABLE_DEPRECATION_WARNINGS if (s->downmix_layout.nb_channels) { if (!av_channel_layout_compare(&s->downmix_layout, &(AVChannelLayout)AV_CHANNEL_LAYOUT_STEREO) || - !av_channel_layout_compare(&s->downmix_layout, &(AVChannelLayout)AV_CHANNEL_LAYOUT_STEREO_DOWNMIX)) + !av_channel_layout_compare(&s->downmix_layout, &(AVChannelLayout)AV_CHANNEL_LAYOUT_STEREO_DOWNMIX)) { s->request_channel_layout = DCA_SPEAKER_LAYOUT_STEREO; - else if (!av_channel_layout_compare(&s->downmix_layout, &(AVChannelLayout)AV_CHANNEL_LAYOUT_5POINT0)) + av_channel_layout_uninit(&avctx->ch_layout); + avctx->ch_layout = (AVChannelLayout)AV_CHANNEL_LAYOUT_STEREO; + } else if (!av_channel_layout_compare(&s->downmix_layout, &(AVChannelLayout)AV_CHANNEL_LAYOUT_5POINT0)) { s->request_channel_layout = DCA_SPEAKER_LAYOUT_5POINT0; - else if (!av_channel_layout_compare(&s->downmix_layout, &(AVChannelLayout)AV_CHANNEL_LAYOUT_5POINT1)) + av_channel_layout_uninit(&avctx->ch_layout); + avctx->ch_layout = (AVChannelLayout)AV_CHANNEL_LAYOUT_5POINT0; + } else if (!av_channel_layout_compare(&s->downmix_layout, &(AVChannelLayout)AV_CHANNEL_LAYOUT_5POINT1)) { s->request_channel_layout = DCA_SPEAKER_LAYOUT_5POINT1; + av_channel_layout_uninit(&avctx->ch_layout); + avctx->ch_layout = (AVChannelLayout)AV_CHANNEL_LAYOUT_5POINT1; + } else av_log(avctx, AV_LOG_WARNING, "Invalid downmix layout\n"); }