From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id E9F5743B34 for ; Sun, 11 Sep 2022 19:00:24 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AA3A268BB06; Sun, 11 Sep 2022 22:00:21 +0300 (EEST) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0669E68B97C for ; Sun, 11 Sep 2022 22:00:14 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1662922814; bh=loQbLp2/e/W2YlK4rwiTAGIK7Ye7Z2Vv6dZKRYahWZc=; h=X-UI-Sender-Class:From:To:Subject:Date:In-Reply-To:References; b=N2a8g1fsKvXYD5/fKFNb1aK5MkvW7vhf7XHNtn31s1Ca6qdgFv/0JnIYjf2j2cDia UseE08Z3WM17tee3Yhr9NszzeExhbc9tNqFQHijsPYQ5M3fMOucgfmVDILpN/Xyh8y JitXA+0/H+Dl5YnhYmKOIYLabxyO0bcpkZHF0tkk= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [94.134.107.176] ([94.134.107.176]) by web-mail.gmx.net (3c-app-gmx-bs18.server.lan [172.19.170.70]) (via HTTP); Sun, 11 Sep 2022 21:00:14 +0200 MIME-Version: 1.0 Message-ID: From: Lukas Fellechner To: ffmpeg-devel@ffmpeg.org Date: Sun, 11 Sep 2022 21:00:14 +0200 Importance: normal Sensitivity: Normal In-Reply-To: <4d712553-76b1-3bfb-9c91-1061d87365f2@martin.st> References: <20220905122020.4103129-1-martin@martin.st> <6bfb9a2c-6db5-a888-d13e-e6c5ef14c23a@martin.st> <4d712553-76b1-3bfb-9c91-1061d87365f2@martin.st> X-UI-Message-Type: mail X-Priority: 3 X-Provags-ID: V03:K1:6gWhVV05Mqgaf+U1vHn/5zjygW3rP42oRktfET9M54u65yR8+SfKm19dsuhiA37zeaxFm GCPI2S8Z33dy3BdYYgrFJx9ZCVVviCYuu3mTNcBPVlI9by9CxTSOjPENmtARq8Te4CvOrEdwX0VF AgzCuiAKvsJS+p+bwrKWSw69Akwzs0dS3oYFxG5I1ia4i0dPZo/vgm4uPNi0O+A1wb3jLdlqtzLb LfhYv5guatQSLhJca/yEQg0ZtyCMYy1L0D32otpzIQ3tJaymSpJ1cK064at+VI+nJKpp6aoKmYjM hA= X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:RqeXiMzaTkc=:jDpiwWLFTkqttJtGS6kiYa tN0vKMnZdBOWLcT+B2WAjM61FJbpuW/DR9qB0p/4omF+Xrxq1sm8iPp81IZSF2XXpRz2/0Y+9 9y+UYX31viOuWEFpAlsVeB7DXRpDxPcQBCcJTG98Z+j9/9hKQXHYdsyDmU0sxske7F7JG2CCa v1Jsyv6GRA6hgg8SqfmvcsXAZIb2oN7vLZo8wCc/9HXMAuEwiEY4pbj9EUNnkhC5Q7K5T+YS2 Lso8FwKvk3Owm/iVfwUiXekoaGSjelp2MrIMskATjRGbAKL7wh/hDZuzSsrFJiEmoBrUdIhQc yf7rr4CSofT7C1Wz+PLLi6araf20zI3P1ud4cShPb+Tnb17XYKtglXBMOevBK/BSEjVX8OiET 59IRmI6rVVgyVmCVq0couF/7qtL0GV0MD1gyI5jzsQCsxBI4P+FHySEJHFU+dQlBTC6YMa68E VwQszEN9lF+v8SAYp1K8eN9xAQcbP/ZiQzeapkeZ6flvzXj+Vta5kPxXttnu0Pctr/A8kMpAP 5HvJOxCu53e0/uOdocvnwNFVEfTzWV5knn3paoI4j9SUjMBetmGBE2QsaIQVnOAJ41Id0u+ed /hu7ur+YOMUM5voLn/eRYIYgf0ysH3RrauA6WsVwCaykSKsaCrXT+pSHpKmjzlGnC8ZvmIGVD 6CqK0zHXBj7JFWNAsHlWXclCW2AUOYj3NkOV9LCofpwmNL6Gso7HPEYQw36zAxzh0QW5oWEGQ KB5xXXCJ5rUYcAT+PO/OvM8S2dCVdFH4jqyym7QJjlYgYJpHj//W7ZCvBa/s0rCzVxPqjyp0L YSWrM4u Subject: Re: [FFmpeg-devel] [PATCH] slicethread: Limit the automatic number of threads to 16 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: >> 2. Spawning too many threads when "auto" is used in multiple places >> >> This can indeed be an efficiency problem, although probably not major. >> Since usually only one part of the pipeline is active at any time, >> many of the threads will be sleeping, consuming very little resources. > > For 32 bit processes running out of address space, yes, the issue is with > "auto" being used in many places at once. > > But in general, allowing arbitrarily high numbers of auto threads isn't > beneficial - the optimal cap of threads depends a lot on the content at > hand. > > The system I'm testing on has 160 cores - and it's quite certain that > doing slice threading with 160 slices doesn't make sense. Maybe the cap of > 16 is indeed too low - I don't mind raising it to 32 or something like > that. Ideally, the auto mechanism would factor in the resolution of the > content. > > Just for arguments sake - here's the output from 'time ffmpeg ...' for a > fairly straightforward transcode (decode, transpose, scale, encode), 1080p > input 10bit, 720p output 8bit, with explicitly setting the number of > threads ("ffmpeg -threads N -i input -threads N -filter_threads N > output"). > > 12: > real 0m25.079s > user 5m22.318s > sys 0m5.047s > > 16: > real 0m19.967s > user 6m3.607s > sys 0m9.112s > > 20: > real 0m20.853s > user 6m21.841s > sys 0m28.829s > > 24: > real 0m20.642s > user 6m28.022s > sys 1m1.262s > > 32: > real 0m29.785s > user 6m8.442s > sys 4m45.290s > > 64: > real 1m0.808s > user 6m31.065s > sys 40m44.598s > > I'm not testing this with 160 threads for each stage, since 64 already was > painfully slow - while you suggest that using threads==cores always should > be preferred, regardless of the number of cores. The optimum here seems to > be somewhere between 16 and 20. These are interesting scores. I would not have expected such a dramatic effect of having too many threads. You are probably right that always using the core count as auto threads is not such a good idea. But the encoding part works on 720p, so there each of the 64 threads only has 11 lines and 14.000 pixels to process, which is really not much. I do not have a CPU with so many cores, but when doing 4K -> 4K transcode, I sure see a benefit of using 32 vs 16 cores. Maybe the best approach would really be to decide auto thread count on the amount of pixels to process (I would not use line count because when line count doubles, the pixel count usually goes up by factor 4). This would probably need some more test data. I will also try to do some testing on my side. - Lukas _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".