From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id AEC884A219 for ; Mon, 25 Mar 2024 15:04:39 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8DB6A68D5E8; Mon, 25 Mar 2024 17:03:05 +0200 (EET) Received: from mail-lf1-f54.google.com (mail-lf1-f54.google.com [209.85.167.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D720C68D547 for ; Mon, 25 Mar 2024 17:02:49 +0200 (EET) Received: by mail-lf1-f54.google.com with SMTP id 2adb3069b0e04-512ed314881so3221639e87.2 for ; Mon, 25 Mar 2024 08:02:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=martin-st.20230601.gappssmtp.com; s=20230601; t=1711378969; x=1711983769; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=akZCDgMV6o1M5bUOl1PcuMv5rzM9ss2pHWr5Rfd69x0=; b=O7vGY77rASDtF9rDSRR/bSKHfNJ+gYK/gS7b2uZ2AGAsiu1iwP+vMvHmQaHKiJdtou 5Xa8t8uFpTfKTsY6zEg2Ny8cIz4K2WEXhyAJg+htzZ7PP4BvubKvjieNtjAlmD59DviD CMEIZgf9k7tq1KcbKsJk40i+W1n3NOAzmnaUAPggcprSL8vDVrOQ3W+TJZ88yh7IY7Sc 4XyIZCWqDLlpvipqousSJBtWFSI4GQVi+K4xK3qnNJ8nKOLnHEA24f+JqdEPT1ucKSCj +IJOxNz7hLC+2ix82dTwGyVsa/zGNzBvHwltKSvruJ2MBwky78FeYLwYTFREoRhZivx5 775Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711378969; x=1711983769; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=akZCDgMV6o1M5bUOl1PcuMv5rzM9ss2pHWr5Rfd69x0=; b=KGhIWC9mcG8WpoVj7C+8JfHlhEugipCh942ygqyW79Cd/YO4QU9Sy5Jo2tdYcTCdnm P6GZnHDchtD5yylOqYj5xx1Z3JQAhxGYbV/Bmr4MTj1ZhDI0bUU8wd+P697x2l6b/f58 SLI/bOMUxk/Q6LsFJuy2jBuib+JWbqChBs+SIzxdKkG+n4Nlesh9UBPRwFcmT4Pqwx5v kpSFRplTdT+gUpwZF+iYkzQA9juDJqOWNJ7zdAIhwMKxVcToqe9SufQmSadg/qXxV9zs nr6rsOen7gy+Egou60F//R5SBxaWdv27W66kX14yc5Gdj5jZT0TmWg5LNePObuT1CbW2 h+Vw== X-Gm-Message-State: AOJu0Yxzx4KM5SneksCDzxvI/VDRSFV5TZ+3uF42suB+gSyAtmIQPLpd Vphb7VtR0qOQf8cUKdFyNVDUJqCqfyOrAj2msBIvDZ7k6+x0e+6U1CzexDoC6FJT+nKAC+h8ziC BvBl0 X-Google-Smtp-Source: AGHT+IELSOWBAFFnf/ksYoXpfxZd5POPWm4axJrIVUMrbay78oPT34r0tIDKBVM4G/f+Oh5KSChuiQ== X-Received: by 2002:a19:5e4e:0:b0:515:a975:2e6b with SMTP id z14-20020a195e4e000000b00515a9752e6bmr3329839lfi.38.1711378968820; Mon, 25 Mar 2024 08:02:48 -0700 (PDT) Received: from localhost (host-114-191.parnet.fi. [77.234.114.191]) by smtp.gmail.com with ESMTPSA id w8-20020ac254a8000000b00513b324ccedsm1106009lfk.286.2024.03.25.08.02.48 (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 25 Mar 2024 08:02:48 -0700 (PDT) From: =?UTF-8?q?Martin=20Storsj=C3=B6?= To: ffmpeg-devel@ffmpeg.org Date: Mon, 25 Mar 2024 17:02:28 +0200 Message-Id: <20240325150243.59058-7-martin@martin.st> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240325150243.59058-1-martin@martin.st> References: <20240325150243.59058-1-martin@martin.st> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/21] aarch64: hevc: Implement a neon version of put_hevc_epel_h*_8 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Logan Lyu , "J . Dekker" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: AWS Graviton 3: put_hevc_epel_h4_8_c: 64.7 put_hevc_epel_h4_8_neon: 25.0 put_hevc_epel_h4_8_i8mm: 21.2 put_hevc_epel_h6_8_c: 130.0 put_hevc_epel_h6_8_neon: 40.7 put_hevc_epel_h6_8_i8mm: 36.5 put_hevc_epel_h8_8_c: 209.0 put_hevc_epel_h8_8_neon: 45.2 put_hevc_epel_h8_8_i8mm: 41.2 put_hevc_epel_h12_8_c: 465.5 put_hevc_epel_h12_8_neon: 104.5 put_hevc_epel_h12_8_i8mm: 86.5 put_hevc_epel_h16_8_c: 830.7 put_hevc_epel_h16_8_neon: 134.2 put_hevc_epel_h16_8_i8mm: 114.0 put_hevc_epel_h24_8_c: 1844.7 put_hevc_epel_h24_8_neon: 282.2 put_hevc_epel_h24_8_i8mm: 277.2 put_hevc_epel_h32_8_c: 3227.5 put_hevc_epel_h32_8_neon: 501.5 put_hevc_epel_h32_8_i8mm: 396.0 put_hevc_epel_h48_8_c: 7229.2 put_hevc_epel_h48_8_neon: 1120.2 put_hevc_epel_h48_8_i8mm: 901.2 put_hevc_epel_h64_8_c: 12869.0 put_hevc_epel_h64_8_neon: 1999.2 put_hevc_epel_h64_8_i8mm: 1610.5 --- libavcodec/aarch64/hevcdsp_epel_neon.S | 194 +++++++++++++++++++++- libavcodec/aarch64/hevcdsp_init_aarch64.c | 17 ++ 2 files changed, 209 insertions(+), 2 deletions(-) diff --git a/libavcodec/aarch64/hevcdsp_epel_neon.S b/libavcodec/aarch64/hevcdsp_epel_neon.S index d3f0a26f79..419e83529a 100644 --- a/libavcodec/aarch64/hevcdsp_epel_neon.S +++ b/libavcodec/aarch64/hevcdsp_epel_neon.S @@ -1321,8 +1321,6 @@ function ff_hevc_put_hevc_epel_uni_v64_8_neon, export=1 ret endfunc -#if HAVE_I8MM -ENABLE_I8MM .macro EPEL_H_HEADER movrel x5, epel_filters @@ -1332,6 +1330,198 @@ ENABLE_I8MM mov x10, #(MAX_PB_SIZE * 2) .endm +function ff_hevc_put_hevc_epel_h4_8_neon, export=1 + EPEL_H_HEADER + sxtl v0.8h, v30.8b +1: ld1 {v4.8b}, [x1], x2 + subs w3, w3, #1 // height + uxtl v4.8h, v4.8b + ext v5.16b, v4.16b, v4.16b, #2 + ext v6.16b, v4.16b, v4.16b, #4 + ext v7.16b, v4.16b, v4.16b, #6 + mul v16.4h, v4.4h, v0.h[0] + mla v16.4h, v5.4h, v0.h[1] + mla v16.4h, v6.4h, v0.h[2] + mla v16.4h, v7.4h, v0.h[3] + st1 {v16.4h}, [x0], x10 + b.ne 1b + ret +endfunc + +function ff_hevc_put_hevc_epel_h6_8_neon, export=1 + EPEL_H_HEADER + sxtl v0.8h, v30.8b + add x6, x0, #8 +1: ld1 {v3.16b}, [x1], x2 + subs w3, w3, #1 // height + uxtl2 v4.8h, v3.16b + uxtl v3.8h, v3.8b + ext v5.16b, v3.16b, v4.16b, #2 + ext v6.16b, v3.16b, v4.16b, #4 + ext v7.16b, v3.16b, v4.16b, #6 + mul v16.8h, v3.8h, v0.h[0] + mla v16.8h, v5.8h, v0.h[1] + mla v16.8h, v6.8h, v0.h[2] + mla v16.8h, v7.8h, v0.h[3] + st1 {v16.4h}, [x0], x10 + st1 {v16.s}[2], [x6], x10 + b.ne 1b + ret +endfunc + +function ff_hevc_put_hevc_epel_h8_8_neon, export=1 + EPEL_H_HEADER + sxtl v0.8h, v30.8b +1: ld1 {v3.16b}, [x1], x2 + subs w3, w3, #1 // height + uxtl2 v4.8h, v3.16b + uxtl v3.8h, v3.8b + ext v5.16b, v3.16b, v4.16b, #2 + ext v6.16b, v3.16b, v4.16b, #4 + ext v7.16b, v3.16b, v4.16b, #6 + mul v16.8h, v3.8h, v0.h[0] + mla v16.8h, v5.8h, v0.h[1] + mla v16.8h, v6.8h, v0.h[2] + mla v16.8h, v7.8h, v0.h[3] + st1 {v16.8h}, [x0], x10 + b.ne 1b + ret +endfunc + +function ff_hevc_put_hevc_epel_h12_8_neon, export=1 + EPEL_H_HEADER + add x6, x0, #16 + sxtl v0.8h, v30.8b +1: ld1 {v3.16b}, [x1], x2 + subs w3, w3, #1 // height + uxtl2 v4.8h, v3.16b + uxtl v3.8h, v3.8b + ext v5.16b, v3.16b, v4.16b, #2 + ext v6.16b, v3.16b, v4.16b, #4 + ext v7.16b, v3.16b, v4.16b, #6 + ext v20.16b, v4.16b, v4.16b, #2 + ext v21.16b, v4.16b, v4.16b, #4 + ext v22.16b, v4.16b, v4.16b, #6 + mul v16.8h, v3.8h, v0.h[0] + mla v16.8h, v5.8h, v0.h[1] + mla v16.8h, v6.8h, v0.h[2] + mla v16.8h, v7.8h, v0.h[3] + mul v17.4h, v4.4h, v0.h[0] + mla v17.4h, v20.4h, v0.h[1] + mla v17.4h, v21.4h, v0.h[2] + mla v17.4h, v22.4h, v0.h[3] + st1 {v16.8h}, [x0], x10 + st1 {v17.4h}, [x6], x10 + b.ne 1b + ret +endfunc + +function ff_hevc_put_hevc_epel_h16_8_neon, export=1 + EPEL_H_HEADER + sxtl v0.8h, v30.8b +1: ld1 {v1.8b, v2.8b, v3.8b}, [x1], x2 + subs w3, w3, #1 // height + uxtl v1.8h, v1.8b + uxtl v2.8h, v2.8b + uxtl v3.8h, v3.8b + ext v5.16b, v1.16b, v2.16b, #2 + ext v6.16b, v1.16b, v2.16b, #4 + ext v7.16b, v1.16b, v2.16b, #6 + ext v20.16b, v2.16b, v3.16b, #2 + ext v21.16b, v2.16b, v3.16b, #4 + ext v22.16b, v2.16b, v3.16b, #6 + mul v16.8h, v1.8h, v0.h[0] + mla v16.8h, v5.8h, v0.h[1] + mla v16.8h, v6.8h, v0.h[2] + mla v16.8h, v7.8h, v0.h[3] + mul v17.8h, v2.8h, v0.h[0] + mla v17.8h, v20.8h, v0.h[1] + mla v17.8h, v21.8h, v0.h[2] + mla v17.8h, v22.8h, v0.h[3] + st1 {v16.8h, v17.8h}, [x0], x10 + b.ne 1b + ret +endfunc + +function ff_hevc_put_hevc_epel_h24_8_neon, export=1 + EPEL_H_HEADER + sxtl v0.8h, v30.8b +1: ld1 {v1.8b, v2.8b, v3.8b, v4.8b}, [x1], x2 + subs w3, w3, #1 // height + uxtl v1.8h, v1.8b + uxtl v2.8h, v2.8b + uxtl v3.8h, v3.8b + uxtl v4.8h, v4.8b + ext v5.16b, v1.16b, v2.16b, #2 + ext v6.16b, v1.16b, v2.16b, #4 + ext v7.16b, v1.16b, v2.16b, #6 + ext v20.16b, v2.16b, v3.16b, #2 + ext v21.16b, v2.16b, v3.16b, #4 + ext v22.16b, v2.16b, v3.16b, #6 + ext v23.16b, v3.16b, v4.16b, #2 + ext v24.16b, v3.16b, v4.16b, #4 + ext v25.16b, v3.16b, v4.16b, #6 + mul v16.8h, v1.8h, v0.h[0] + mla v16.8h, v5.8h, v0.h[1] + mla v16.8h, v6.8h, v0.h[2] + mla v16.8h, v7.8h, v0.h[3] + mul v17.8h, v2.8h, v0.h[0] + mla v17.8h, v20.8h, v0.h[1] + mla v17.8h, v21.8h, v0.h[2] + mla v17.8h, v22.8h, v0.h[3] + mul v18.8h, v3.8h, v0.h[0] + mla v18.8h, v23.8h, v0.h[1] + mla v18.8h, v24.8h, v0.h[2] + mla v18.8h, v25.8h, v0.h[3] + st1 {v16.8h, v17.8h, v18.8h}, [x0], x10 + b.ne 1b + ret +endfunc + +function ff_hevc_put_hevc_epel_h32_8_neon, export=1 + EPEL_H_HEADER + ld1 {v1.8b}, [x1], #8 + sub x2, x2, w6, uxtw // decrement src stride + mov w7, w6 // original width + sub x2, x2, #8 // decrement src stride + sub x10, x10, w6, uxtw #1 // decrement dst stride + sxtl v0.8h, v30.8b + uxtl v1.8h, v1.8b +1: ld1 {v2.8b, v3.8b}, [x1], #16 + subs w6, w6, #16 // width + uxtl v2.8h, v2.8b + uxtl v3.8h, v3.8b + ext v5.16b, v1.16b, v2.16b, #2 + ext v6.16b, v1.16b, v2.16b, #4 + ext v7.16b, v1.16b, v2.16b, #6 + ext v20.16b, v2.16b, v3.16b, #2 + ext v21.16b, v2.16b, v3.16b, #4 + ext v22.16b, v2.16b, v3.16b, #6 + mul v16.8h, v1.8h, v0.h[0] + mla v16.8h, v5.8h, v0.h[1] + mla v16.8h, v6.8h, v0.h[2] + mla v16.8h, v7.8h, v0.h[3] + mul v17.8h, v2.8h, v0.h[0] + mla v17.8h, v20.8h, v0.h[1] + mla v17.8h, v21.8h, v0.h[2] + mla v17.8h, v22.8h, v0.h[3] + st1 {v16.8h, v17.8h}, [x0], #32 + mov v1.16b, v3.16b + b.gt 1b + subs w3, w3, #1 // height + add x1, x1, x2 + b.le 9f + ld1 {v1.8b}, [x1], #8 + mov w6, w7 + add x0, x0, x10 + uxtl v1.8h, v1.8b + b 1b +9: + ret +endfunc + +#if HAVE_I8MM +ENABLE_I8MM function ff_hevc_put_hevc_epel_h4_8_neon_i8mm, export=1 EPEL_H_HEADER 1: ld1 {v4.8b}, [x1], x2 diff --git a/libavcodec/aarch64/hevcdsp_init_aarch64.c b/libavcodec/aarch64/hevcdsp_init_aarch64.c index 1e9f5e32db..ece911b8d4 100644 --- a/libavcodec/aarch64/hevcdsp_init_aarch64.c +++ b/libavcodec/aarch64/hevcdsp_init_aarch64.c @@ -223,6 +223,10 @@ NEON8_FNPROTO_PARTIAL_4(qpel_uni_w_v, (uint8_t *_dst, ptrdiff_t _dststride, int height, int denom, int wx, int ox, intptr_t mx, intptr_t my, int width),); +NEON8_FNPROTO(epel_h, (int16_t *dst, + const uint8_t *_src, ptrdiff_t _srcstride, + int height, intptr_t mx, intptr_t my, int width),); + NEON8_FNPROTO(epel_h, (int16_t *dst, const uint8_t *_src, ptrdiff_t _srcstride, int height, intptr_t mx, intptr_t my, int width), _i8mm); @@ -290,6 +294,17 @@ NEON8_FNPROTO(qpel_bi_hv, (uint8_t *dst, ptrdiff_t dststride, member[8][v][h] = ff_hevc_put_hevc_##fn##48_8_neon##ext; \ member[9][v][h] = ff_hevc_put_hevc_##fn##64_8_neon##ext; +#define NEON8_FNASSIGN_SHARED_32(member, v, h, fn, ext) \ + member[1][v][h] = ff_hevc_put_hevc_##fn##4_8_neon##ext; \ + member[2][v][h] = ff_hevc_put_hevc_##fn##6_8_neon##ext; \ + member[3][v][h] = ff_hevc_put_hevc_##fn##8_8_neon##ext; \ + member[4][v][h] = ff_hevc_put_hevc_##fn##12_8_neon##ext; \ + member[5][v][h] = ff_hevc_put_hevc_##fn##16_8_neon##ext; \ + member[6][v][h] = ff_hevc_put_hevc_##fn##24_8_neon##ext; \ + member[7][v][h] = \ + member[8][v][h] = \ + member[9][v][h] = ff_hevc_put_hevc_##fn##32_8_neon##ext; + #define NEON8_FNASSIGN_PARTIAL_4(member, v, h, fn, ext) \ member[1][v][h] = ff_hevc_put_hevc_##fn##4_8_neon##ext; \ member[3][v][h] = ff_hevc_put_hevc_##fn##8_8_neon##ext; \ @@ -384,6 +399,8 @@ av_cold void ff_hevc_dsp_init_aarch64(HEVCDSPContext *c, const int bit_depth) NEON8_FNASSIGN(c->put_hevc_epel_uni_w, 1, 0, epel_uni_w_v,); NEON8_FNASSIGN_PARTIAL_4(c->put_hevc_qpel_uni_w, 1, 0, qpel_uni_w_v,); + NEON8_FNASSIGN_SHARED_32(c->put_hevc_epel, 0, 1, epel_h,); + if (have_i8mm(cpu_flags)) { NEON8_FNASSIGN(c->put_hevc_epel, 0, 1, epel_h, _i8mm); NEON8_FNASSIGN(c->put_hevc_epel, 1, 1, epel_hv, _i8mm); -- 2.39.3 (Apple Git-146) _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".