From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 6A5C44AFFE for ; Sun, 26 May 2024 00:39:30 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0F8F268D4C1; Sun, 26 May 2024 03:39:27 +0300 (EEST) Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F00A068D306 for ; Sun, 26 May 2024 03:39:20 +0300 (EEST) Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-1f44b42e9a6so13034275ad.0 for ; Sat, 25 May 2024 17:39:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1716683958; x=1717288758; darn=ffmpeg.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id:from :to:cc:subject:date:message-id:reply-to; bh=dLIhIGqH7Ing3kxaFfD/HREERyh/75xxgu5IvLl0zNE=; b=MX6/EHITyrDjj53n1Nby3p9lmluni3+PIeGgt5GMVAfdkq+SpX/njtY+5empGL2+ao i1Bm7orFatGr7KfQs3WQ7H563d04frCNlqj2qBqWAqfLHZJJZbSPX/KTy+rTpsck1m/t jyo1NhAFWtZnpl+PJdXE/dVFfNs21OlsHi6iZU6k8Q6mcoLYngD4tybizszquJdQW6W0 WwPW2QoZuAkWp1YYvOiK81fxuPoA0E3HbimqJSQZNt5ju9vaAuj5tbQqwptXy3YYH+Qa 6BUN+sMxAPLqD096wDZcdIVGwLbUjgaMkVZRRrJallEyf2Xx/+yF7lC4mQ9LdSaOny9z XGAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716683958; x=1717288758; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dLIhIGqH7Ing3kxaFfD/HREERyh/75xxgu5IvLl0zNE=; b=aWTwYj437v2PQnwxQd1kuuLIhv02GcPaensK/2e/tHZfTqBpIb29SMYPmN42ydN7LB PjCw2105+3m/BbcZbj5ozEWrzFUaM/LoZcLBgm3hjtFh7WYruujpCxrNhPrfkK5RbgI5 6ofLkxwthCDDOf/CKmuhFzjYaBmyEzIfYjDxnTYfDhZvkJoJ8JstB67PjP+iYZ8ZEbGv a8ywRC5pgiss9amYh4zKbb+G2MlArrKtOie4IjIEbc7Z2Dyca4Ta6h8yHCJ/AiLB9qal YpWeVeR+vdxDRK9THZu1RNBK7inumIIby1InNGa/GDLD6D+6WSrx/XCRMeUum1pZeyLD VlRg== X-Gm-Message-State: AOJu0YzTyRcep0gBT4iLdv+hhVT/lrBYetr/NgKfcF2doU7D2luXs07s bZhsf/E1io9RsQcMjwWiPVdVbkVydM6NwV+QuDkVycNCnd4+9mmNKwHlVQ== X-Google-Smtp-Source: AGHT+IEOBQsEV/N4Dk/iR42w9ReZzDwTfNvdwtDOOS0ERXVbP236CxFjTfPwgqvqvQWKaD/zhBiMHQ== X-Received: by 2002:a17:903:2345:b0:1f4:5477:4be6 with SMTP id d9443c01a7336-1f454774e94mr49632055ad.41.1716683957592; Sat, 25 May 2024 17:39:17 -0700 (PDT) Received: from [192.168.0.10] ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f44c558ccdsm36635505ad.0.2024.05.25.17.39.16 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 25 May 2024 17:39:17 -0700 (PDT) Message-ID: <7c7a2d54-126c-4a53-aeb6-9f27bf316574@gmail.com> Date: Sat, 25 May 2024 21:39:24 -0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: ffmpeg-devel@ffmpeg.org References: <20240525205731.2578146-1-dev@lynne.ee> Content-Language: en-US From: James Almer In-Reply-To: <20240525205731.2578146-1-dev@lynne.ee> Subject: Re: [FFmpeg-devel] [PATCH] lpc: rewrite lpc_compute_autocorr in external asm X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 5/25/2024 5:57 PM, Lynne via ffmpeg-devel wrote: > The inline asm function had issues running under checkasm. > So I came to finish what I started, and wrote the last part > of LPC computation in assembly. > > autocorr_10_c: 135525.8 > autocorr_10_sse2: 50729.8 > autocorr_10_fma3: 19007.8 > autocorr_30_c: 390100.8 > autocorr_30_sse2: 142478.8 > autocorr_30_fma3: 50559.8 > autocorr_32_c: 407058.3 > autocorr_32_sse2: 151633.3 > autocorr_32_fma3: 50517.3 > --- > libavcodec/x86/lpc.asm | 91 +++++++++++++++++++++++++++++++++++++++ > libavcodec/x86/lpc_init.c | 87 ++++--------------------------------- > 2 files changed, 100 insertions(+), 78 deletions(-) > > diff --git a/libavcodec/x86/lpc.asm b/libavcodec/x86/lpc.asm > index a585c17ef5..790841b7f4 100644 > --- a/libavcodec/x86/lpc.asm > +++ b/libavcodec/x86/lpc.asm > @@ -32,6 +32,8 @@ dec_tab_sse2: times 2 dq -2.0 > dec_tab_scalar: times 2 dq -1.0 > seq_tab_sse2: dq 1.0, 0.0 > > +autoc_init_tab: times 4 dq 1.0 There's one_tab already, so no need to add this. > + > SECTION .text > > %macro APPLY_WELCH_FN 0 > @@ -261,3 +263,92 @@ APPLY_WELCH_FN > INIT_YMM avx2 > APPLY_WELCH_FN > %endif > + > +%macro COMPUTE_AUTOCORR_FN 0 > +cglobal lpc_compute_autocorr, 4, 7, 8, data, len, lag, autoc, lag_p, data_l, len_p > + > + shl lagd, 3 > + shl lenq, 3 > + xor lag_pq, lag_pq > + > +.lag_l: > + movaps m8, [autoc_init_tab] > + > + mov len_pq, lag_pq > + > + lea data_lq, [lag_pq + mmsize - 8] > + neg data_lq ; -j - mmsize > + add data_lq, dataq ; data[-j - mmsize] > +.len_l: > + ; We waste the upper value here on SSE2, > + ; but we use it on AVX. > + movupd xm0, [dataq + len_pq] ; data[i] > + movupd m1, [data_lq + len_pq] ; data[i - j] > + > +%if cpuflag(avx) > + vbroadcastsd m0, xm0 > + vperm2f128 m1, m1, m1, 0x01 You can do vpermpd m1, [data_lq + len_pq], q0123 Which saves you the movupd + vperm2f128 + shufpd for fma3. > +%endif > + > + shufpd m0, m0, m0, 1100b > + shufpd m1, m1, m1, 0101b > + > +%if cpuflag(fma3) > + fmaddpd m8, m0, m1, m8 ; sum += data[i]*data[i-j] So i found out this instruction is what's killing performance for me. If it remove it and use mulpd + addpd like in sse2, i see the same boost you got. Why? I have no idea. Other functions using this same instruction, like ff_vector_dmac_scalar_fma3, don't see any performance hit. > +%else > + mulpd m0, m1 > + addpd m8, m0 ; sum += data[i]*data[i-j] > +%endif > + > + add len_pq, 8 > + cmp len_pq, lenq > + jl .len_l > + > + movups [autocq + lag_pq], m8 ; autoc[j] = sum > + add lag_pq, mmsize > + cmp lag_pq, lagq > + jl .lag_l > + > + ; The tail computation is guaranteed never to happen > + ; as long as we're doing multiples of 4, rather than 2. > + ; It is trivial to convert this to avx if ever needed. > +%if !cpuflag(avx) > + jg .end > + ; If lag_p == lag fallthrough > + > +.tail: > + movaps xm2, [autoc_init_tab] > + > + mov len_pq, lag_pq > + sub len_pq, mmsize > + > + lea data_lq, [lag_pq] > + neg data_lq ; -j > + add data_lq, dataq ; data[-j] > + > +.tail_l: > + movupd xm0, [dataq + len_pq] > + movupd xm1, [data_lq + len_pq] > + > + mulpd xm0, xm1 > + addpd xm2, xm0 ; sum += data[i]*data[i-j] > + > + add len_pq, mmsize > + cmp len_pq, lenq > + jl .tail_l > + > + shufpd xm1, xm2, xm2, 01b > + addpd xm2, xm1 > + > + movhpd [autocq + lag_pq], xm2 > +%endif > + > +.end: > + RET > + > +%endmacro > + > +INIT_XMM sse2 > +COMPUTE_AUTOCORR_FN > +INIT_YMM fma3 > +COMPUTE_AUTOCORR_FN > diff --git a/libavcodec/x86/lpc_init.c b/libavcodec/x86/lpc_init.c > index f2fca53799..96469fae40 100644 > --- a/libavcodec/x86/lpc_init.c > +++ b/libavcodec/x86/lpc_init.c > @@ -28,89 +28,20 @@ void ff_lpc_apply_welch_window_sse2(const int32_t *data, ptrdiff_t len, > double *w_data); > void ff_lpc_apply_welch_window_avx2(const int32_t *data, ptrdiff_t len, > double *w_data); > - > -DECLARE_ASM_CONST(16, double, pd_1)[2] = { 1.0, 1.0 }; > - > -#if HAVE_SSE2_INLINE > - > -static void lpc_compute_autocorr_sse2(const double *data, ptrdiff_t len, int lag, > - double *autoc) > -{ > - int j; > - > - if((x86_reg)data & 15) > - data++; > - > - for(j=0; j - x86_reg i = -len*sizeof(double); > - if(j == lag-2) { > - __asm__ volatile( > - "movsd "MANGLE(pd_1)", %%xmm0 \n\t" > - "movsd "MANGLE(pd_1)", %%xmm1 \n\t" > - "movsd "MANGLE(pd_1)", %%xmm2 \n\t" > - "1: \n\t" > - "movapd (%2,%0), %%xmm3 \n\t" > - "movupd -8(%3,%0), %%xmm4 \n\t" > - "movapd (%3,%0), %%xmm5 \n\t" > - "mulpd %%xmm3, %%xmm4 \n\t" > - "mulpd %%xmm3, %%xmm5 \n\t" > - "mulpd -16(%3,%0), %%xmm3 \n\t" > - "addpd %%xmm4, %%xmm1 \n\t" > - "addpd %%xmm5, %%xmm0 \n\t" > - "addpd %%xmm3, %%xmm2 \n\t" > - "add $16, %0 \n\t" > - "jl 1b \n\t" > - "movhlps %%xmm0, %%xmm3 \n\t" > - "movhlps %%xmm1, %%xmm4 \n\t" > - "movhlps %%xmm2, %%xmm5 \n\t" > - "addsd %%xmm3, %%xmm0 \n\t" > - "addsd %%xmm4, %%xmm1 \n\t" > - "addsd %%xmm5, %%xmm2 \n\t" > - "movsd %%xmm0, (%1) \n\t" > - "movsd %%xmm1, 8(%1) \n\t" > - "movsd %%xmm2, 16(%1) \n\t" > - :"+&r"(i) > - :"r"(autoc+j), "r"(data+len), "r"(data+len-j) > - NAMED_CONSTRAINTS_ARRAY_ADD(pd_1) > - :"memory" > - ); > - } else { > - __asm__ volatile( > - "movsd "MANGLE(pd_1)", %%xmm0 \n\t" > - "movsd "MANGLE(pd_1)", %%xmm1 \n\t" > - "1: \n\t" > - "movapd (%3,%0), %%xmm3 \n\t" > - "movupd -8(%4,%0), %%xmm4 \n\t" > - "mulpd %%xmm3, %%xmm4 \n\t" > - "mulpd (%4,%0), %%xmm3 \n\t" > - "addpd %%xmm4, %%xmm1 \n\t" > - "addpd %%xmm3, %%xmm0 \n\t" > - "add $16, %0 \n\t" > - "jl 1b \n\t" > - "movhlps %%xmm0, %%xmm3 \n\t" > - "movhlps %%xmm1, %%xmm4 \n\t" > - "addsd %%xmm3, %%xmm0 \n\t" > - "addsd %%xmm4, %%xmm1 \n\t" > - "movsd %%xmm0, %1 \n\t" > - "movsd %%xmm1, %2 \n\t" > - :"+&r"(i), "=m"(autoc[j]), "=m"(autoc[j+1]) > - :"r"(data+len), "r"(data+len-j) > - NAMED_CONSTRAINTS_ARRAY_ADD(pd_1) > - ); > - } > - } > -} > - > -#endif /* HAVE_SSE2_INLINE */ > +void ff_lpc_compute_autocorr_sse2(const double *data, ptrdiff_t len, int lag, > + double *autoc); > +void ff_lpc_compute_autocorr_fma3(const double *data, ptrdiff_t len, int lag, > + double *autoc); > > av_cold void ff_lpc_init_x86(LPCContext *c) > { > int cpu_flags = av_get_cpu_flags(); > > -#if HAVE_SSE2_INLINE > - if (INLINE_SSE2_SLOW(cpu_flags)) > - c->lpc_compute_autocorr = lpc_compute_autocorr_sse2; > -#endif > + if (EXTERNAL_SSE2(cpu_flags)) > + c->lpc_compute_autocorr = ff_lpc_compute_autocorr_sse2; > + > + if (EXTERNAL_FMA3(cpu_flags)) > + c->lpc_compute_autocorr = ff_lpc_compute_autocorr_fma3; > > if (EXTERNAL_SSE2(cpu_flags)) > c->lpc_apply_welch_window = ff_lpc_apply_welch_window_sse2; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".