Bug 553410 - Investigate possible DoS attack from 2:00am ET to 4:00am
Summary: Investigate possible DoS attack from 2:00am ET to 4:00am
Status: RESOLVED WORKSFORME
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: Servers (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 blocker (vote)
Target Milestone: ---   Edit
Assignee: Eclipse Webmaster CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-11-25 02:49 EST by Alexander Fedorov CLA
Modified: 2019-11-25 12:29 EST (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Fedorov CLA 2019-11-25 02:49:30 EST
[INFO] Fetching p2.index from https://download.eclipse.org/releases/2019-12/

[INFO] Adding repository https://download.eclipse.org/releases/2019-12

Nov 25, 2019 7:30:17 AM org.apache.http.impl.execchain.RetryExec execute

INFO: I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {s}->https://download.eclipse.org:443: The target server failed to respond

Nov 25, 2019 7:30:17 AM org.apache.http.impl.execchain.RetryExec execute
Comment 1 Thomas Wolf CLA 2019-11-25 02:55:09 EST
It's not just download.eclipse.org. Access to _all_ *.eclipse.org sites is extremely slow. Gerrit (web & ssh), Bugzilla, Wiki, ... all take several minutes to load, if they load at all.
Comment 2 Frederic Gurr CLA 2019-11-25 06:44:20 EST
Seems like this was an intermittent hick up. All services look much better again now. Can you confirm?
Comment 3 Thomas Wolf CLA 2019-11-25 06:48:51 EST
Yes, appears to be back. But look at https://status.eclipse.org/ . There was definitely something strange going on from around 07:00 to 10:00.

Web page access seems to be OK again, but download speed? Perhaps the graph is not up to date; don't notice any unusual slowness in running CI builds, which also download from downloads.eclipse.org.
Comment 4 Frederic Gurr CLA 2019-11-25 07:09:17 EST
(In reply to Thomas Wolf from comment #3)
> Yes, appears to be back. But look at https://status.eclipse.org/ . There was
> definitely something strange going on from around 07:00 to 10:00.
> 
> Web page access seems to be OK again, but download speed? Perhaps the graph
> is not up to date; don't notice any unusual slowness in running CI builds,
> which also download from downloads.eclipse.org.
I'm not denying that there was an issue. We will look into it.
Comment 5 Denis Roy CLA 2019-11-25 12:29:42 EST
A DoS attack was triggered at 1:51am localtime (UTC-5), originating from South Korea. We'll be filing an abuse claim to the netblock owners.

Timeline of events:
01:51: Eclipse firewall blocks one attacking IP address in South Korea
02:13: Eclipse firewall dropping >27,000 attack packets per second. CPU usage maxes out and legitimate traffic flow is affected

02:30: Attack persists, legitimate traffic flow is severely degraded. Timeouts begin to occur.

04:03: Attack ceases. Traffic flows return to normal.

Our Firewall, which is about 10 years old, does not provide adequate protection for this type of DoS (malicious or not; see bug 515596). We are tracking a replacement since 2017-09. 

For now, worksforme is all I can do.