[FFmpeg-devel] ffmpeg.org issue today

[FFmpeg-devel] ffmpeg.org issue today

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1277 bytes --]

Hi all

ffmpeg.org was down today, heres what I know & what happened

i woke up, looked at IRC saw someone saying ffmpeg.org is down, tried loging
in with ssh and that timed out (i kept trying to login while trying other things)
i mailed the other root admins and started to look around
what i found was the host and VM where running, no signs of any anomalies on the host
loging into the VM was not successfull, i decided to wait for raz or others to
take a look before attempting to restart it as raz has vastly more knowledge and
experience and i didnt want to do more harm or loose information about what
happened. raz also failed to gather usefull information about the vm so he
restarted ffmpeg.org
the box came up as if nothing ever happened, we both looked over logs in search
of the cause, but came up empty handed. logs just ended Jul 26 22:47:14 which is
probably when the box did hang
all relevant packages where uptodate, the running kernel also seems to have
been the latest avalable for the ubuntu distrubution

thx

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Dictatorship naturally arises out of democracy, and the most aggravated
form of tyranny and slavery out of the most extreme liberty. -- Plato

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] ffmpeg.org issue today

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 3528 bytes --]

Hi everyone

as some may have noticed there was a 2nd hang happening yesterday 5th/6th august
so heres a update about what i/we know/guess

the 2nd hang seems to have looked like the first.
ive spend some time yesterday looking over logs, there are some showers of
slightly odd entries in the apache webserver logs, similar entries occured 
3 times, 2 of which preceeded the server hangs by 1-2 minutes
these entries come from 2 IP addresses 143.198.181.190 and 159.89.231.156
first preceeded the first hang and 2nd the 2nd hang
https://www.malwareurl.com/listing.php?domain=143.198.181.190
says "Vulnerability Scanner"
https://www.malwareurl.com/listing.php?domain=159.89.231.156
says "Brute-force attacker IP"

it seemed these IPs spidered over random bits of the webpage, lots of gitweb
a little bit of old trac links and other tiny bits of random bits, some
nonsensical urls too
with randomly faked browser ids

all of this is alot of guessing but if we assume this was some sort of
Vulnerability scanner seaching for some interresting bits on public web
(where it maybe did not find much)
a subsequent step of the scanner may have triggered a unfixed bug in
something that caused the server to hang. 
thats alot speculation, and other things are possible including some
hw issue that got triggered from higher load
no signs of any successfull intrusion, not even anything crashing, its
just the whole box hanging

raz has updated the kernel now beyond what is teh default for our ubuntu

also i recommand everyone to always check signatures of releases
that obviosuly is not just for ffmpeg but for anything you download from
the internet.

thx


On Wed, Jul 27, 2022 at 01:16:45PM +0200, Michael Niedermayer wrote:
> Hi all
> 
> ffmpeg.org was down today, heres what I know & what happened
> 
> i woke up, looked at IRC saw someone saying ffmpeg.org is down, tried loging
> in with ssh and that timed out (i kept trying to login while trying other things)
> i mailed the other root admins and started to look around
> what i found was the host and VM where running, no signs of any anomalies on the host
> loging into the VM was not successfull, i decided to wait for raz or others to
> take a look before attempting to restart it as raz has vastly more knowledge and
> experience and i didnt want to do more harm or loose information about what
> happened. raz also failed to gather usefull information about the vm so he
> restarted ffmpeg.org
> the box came up as if nothing ever happened, we both looked over logs in search
> of the cause, but came up empty handed. logs just ended Jul 26 22:47:14 which is
> probably when the box did hang
> all relevant packages where uptodate, the running kernel also seems to have
> been the latest avalable for the ubuntu distrubution
> 
> thx
> 
> -- 
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> Dictatorship naturally arises out of democracy, and the most aggravated
> form of tyranny and slavery out of the most extreme liberty. -- Plato



> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".


-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Observe your enemies, for they first find out your faults. -- Antisthenes

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] ffmpeg.org issue today

[Timo Rothenpieler]

I had similar issues with Apache on my own server, and never found a 
concise reason as to what was going on.
It seemed like Apache sometimes felt like stopping operation until 
restarting it.
Stopped happening after some Apache updates.

Most likely candidate in my case is the DoS attack that was fixed in 2.4.38.
It only manifests if Apache is used with openssl 1.1.1+ (CVE-2019-0190).
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".