On Wednesday, GitHub survived the largest DDoS attack to date, with the traffic at about 1.3 Tbps (Terabits per second). The previous largest recorded attack took place in 2016 when the Mirai Botnet launched a 1.2 Tbps DDoS against DYN DNS, bringing down their site, and much of the internet along with it.
What caused these most recent attacks? Was this another botnet attack?
A day before the attack, Akamai (Intelligent DDoS and Cloud Mitigation service) discovered a new DDoS reflection attack vector involving a network tool called memcached. Reflection attacks use spoofed UDP traffic from a server that is hosting the exploited service, to force sending data to an unsuspecting victim(s). Amplification attacks take the reflection attack a step further. The attacker will trigger a UDP service to send a large amount of data to the victim, with only a small packet being sent by the attacker. This specific reflection attack uses memcached to amplify packet sizes in order to flood the target site with data. The attack is like other known DDoS reflection and amplification attacks, such as DNS, TFTP, LDAP, SNMP, BitTorrent, and others. The key difference is the amplification power that memcached provides.
In a typical DNS-based amplification attack, you would see amplification factors for around 100, whereas with reflection from a memcache server, you would see a factor of over 50,000. Per Akamai‘s blog post on the recently discovered attack, it is difficult to determine the exact amplification factor:
“When a system receives a memcached get request, it forms a response by collecting the requested values from memory, sending them over the wire in an uninterrupted stream. This response is sent to the target in multiple UDP packets, each with a length of up to 1400 bytes. It is difficult to determine the exact amplification factor of memcached, but the attacks Akamai saw generated nearly 1 Gbps per reflector. Other organizations have reported attacks in excess of 500 Gbps using memcached reflection.”
How can you prevent this?
The best and most obvious way to prevent against this type of attack is to make sure that potential reflectors (DNS, MemCache, TFTP, etc), are not exposed to the internet. Every system administrator is HIGHLY encouraged to disable the memcache protocol on any internet-exposed server, or at the very least block UDP port 11211.
This attack against GitHub shows we need to be prepared for more multi-gigabit attacks, just as we have seen with memcache protocol and the Mirai botnet before it. IT administrators should plan accordingly to mitigate these risks.
DDoS Prevention and Mitigation:
- Utilize on-premise firewalls or content filters
- Specialized equipment / load balancers
- ISP mitigation
- 3rd Party mitigation
- On WatchGuard Firewalls, we can block the sender’s IP address via default packet handling on the device. The Firebox has default thresholds set for both client and servers. Once a threshold has been reached for the destination IP address, the Firebox drops incoming connection requests from any host.
You are also encouraged to block port 11211 to prevent your servers from becoming a reflector.
- You can implement a load balancing solution for your ISP connections so that the traffic is handled in a round-robin or overflow scenario. You would usually utilize some sort of equipment for handling the load on the Boarder of the network. Be aware that these devices cannot handle volumetric attacks (attacks with a large amount of traffic, such as these amplification attacks). It will become a bottleneck on the network.
Steps 3 and 4 are usually combined. For large scale attacks, you will need to depend on the coordination of your ISP and a 3rd party cloud mitigation service. As Corey stated in his Secpliticy post:
“Cloud or hybrid DDoS solutions handle much of the attack up-stream, distributing some of the load through a large, distributed network, and blocking much of the traffic before it even reaches your gates. “
These Cloud and Hybrid DDoS solutions have the infrastructure, bandwidth and resources available to mitigate these attacks. –Tanner Harrison