[FFmpeg-devel] [PATCH v2 FFmpeg 1/20] configure: add tokenizers-cpp support

[FFmpeg-devel] [PATCH v2 FFmpeg 1/20] configure: add tokenizers-cpp support

[m.kaindl0208]

Signed-off-by: MaximilianKaindl <m.kaindl0208@gmail.com>
---
 configure | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/configure b/configure
index 04b83a8868..7219faeaf2 100755
--- a/configure
+++ b/configure
@@ -285,6 +285,7 @@ External library support:
   --enable-libtls          enable LibreSSL (via libtls), needed for https support
                            if openssl, gnutls or mbedtls is not used [no]
   --enable-libtorch        enable Torch as one DNN backend [no]
+  --enable-libtokenizers   enable tokenizers-cpp library [no]
   --enable-libtwolame      enable MP2 encoding via libtwolame [no]
   --enable-libuavs3d       enable AVS3 decoding via libuavs3d [no]
   --enable-libv4l2         enable libv4l2/v4l-utils [no]
@@ -1967,6 +1968,7 @@ EXTERNAL_LIBRARY_LIST="
     libtesseract
     libtheora
     libtorch
+    libtokenizers
     libtwolame
     libuavs3d
     libv4l2
@@ -2873,7 +2875,7 @@ dirac_parse_select="golomb"
 dovi_rpudec_select="golomb"
 dovi_rpuenc_select="golomb"
 dnn_deps="avformat swscale"
-dnn_deps_any="libtensorflow libopenvino libtorch"
+dnn_deps_any="libtensorflow libopenvino libtorch libtokenizers"
 error_resilience_select="me_cmp"
 evcparse_select="golomb"
 faandct_deps="faan"
@@ -7049,7 +7051,7 @@ enabled libtensorflow     && require libtensorflow tensorflow/c/c_api.h TF_Versi
 enabled libtesseract      && require_pkg_config libtesseract tesseract tesseract/capi.h TessBaseAPICreate
 enabled libtheora         && require libtheora theora/theoraenc.h th_info_init -ltheoraenc -ltheoradec -logg
 enabled libtls            && require_pkg_config libtls libtls tls.h tls_configure
-enabled libtorch          && check_cxxflags -std=c++17 && require_cpp libtorch torch/torch.h "torch::Tensor" -ltorch -lc10 -ltorch_cpu -lstdc++ -lpthread
+enabled libtokenizers     && check_cxxflags -std=c++17 && require_cpp libtokenizers "tokenizers_cpp.h tokenizers_c.h" "" -ltokenizers_cpp -ltokenizers_c -lstdc++
 enabled libtwolame        && require libtwolame twolame.h twolame_init -ltwolame &&
                              { check_lib libtwolame twolame.h twolame_encode_buffer_float32_interleaved -ltwolame ||
                                die "ERROR: libtwolame must be installed and version must be >= 0.3.10"; }
--
2.34.1


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 FFmpeg 1/20] configure: add tokenizers-cpp support

[Guo, Yejun]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> m.kaindl0208@gmail.com
> Sent: Tuesday, March 11, 2025 3:50 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH v2 FFmpeg 1/20] configure: add tokenizers-
> cpp support
> 
> Signed-off-by: MaximilianKaindl <m.kaindl0208@gmail.com>
> ---
>  configure | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/configure b/configure
> index 04b83a8868..7219faeaf2 100755
> --- a/configure
> +++ b/configure
> @@ -285,6 +285,7 @@ External library support:
>    --enable-libtls          enable LibreSSL (via libtls), needed for https support
>                             if openssl, gnutls or mbedtls is not used [no]
>    --enable-libtorch        enable Torch as one DNN backend [no]
> +  --enable-libtokenizers   enable tokenizers-cpp library [no]

Not finished all the review yet, I have several comments in some of the commits.

Is libtokenizers a popular library? How to install? Please also add in commit comments, thanks.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".