The FlowTraq team was recently at the RSA 2017 conference. While there, we enjoyed a great event learning, networking, and…helping you defeat zombies.
In this case, “zombies” are computers or Internet devices that get taken over by hackers turn against you by launching DDoS attacks. To help you protect yourself, we have compiled a list of “10 Tips for Surviving the Zombie Apocalypse”.
In the movie Zombieland, a virus turns most of the population into zombies, and the world’s surviving humans remain locked in an ongoing battle against the hungry undead. Four survivors band together and follow a list of proven survival rules and zombie-killing strategies as they make their way to a rumored safe haven.
Here is an updated version of Zombieland’s 10 rules of keeping you and your network optimally prepared for the ongoing zombie botnet apocalypse.
Preparation is everything. When your cyber apocalypse comes, you have to be in great physical shape. This means you need to deploy today the defenses you may need tomorrow. And when it comes to zombie botnets launching DDoS attacks, our reference to good cardio means three things:
• Visibility: You must have complete visibility to everything in your network, not sample data, and not just at the border. You also need full-fidelity where every record counts. Why? Because in the hunt for zombie devices, the control-channel communications are extremely small, and tend to go missing when using sampled flow.
• Defenses: The means egress filtering on all your devices. If the source is not your net block, drop it. Most DDoS attacks rely on spoofing the source address. Often the spoofed source address is that of the attack (for reflection/amplification attacks). You are simply a good Internet neighbor by dropping any packet that has a spoofed source address.
• Offenses: On-prem bad-packet scrubbing. Discarding any packet that is a protocol anomaly goes a long way in mitigating inbound and outbound DDoS. Look for same source and destination IP, protocol zero, no TCP flags, etc.
Once you catch a zombie, you will need to put it down. But re-imaging of the host is not enough. The hacker got in once – do you understand how? Can it be prevented from happening again? Understanding the attack vector, and closing it off permanently, is critical.
Preventing re-infection is a delicate game. You need to ensure you get all of them, because shutting zombies down will get noticed by the controlling hacker. This means he or she might change tack, and you lose your advantage. So when you identify the attack vector, make sure you close it down for all systems in your network!
The very best technique to ensure you understand the vector is simply inspecting your network traffic for the zombie device for the last couple of weeks or months. Full-fidelity flow is ideal for this effort since it is lightweight, and can give months of history for relatively little storage.
Full packet capture can also be helpful here. Ideally, you should attempt to understand the mechanisms of the zombie with insight into the hacker’s IP address/es, ports used, and the timing of the communication. This way you can quickly infer if ports need to be closed, IPs need to be blocked, passwords need to be reset, or applications need to be patched.
Think quick: What is your most vulnerable position? In Zombieland, it is bathrooms, but it is a little more difficult to figure out on your network.
Remember that hackers don’t care what a particular box is, and what function it serves in your network. They search for any way in, which usually leads to the forgotten machine getting hacked first. And once in, the zombie infection spreads quickly. Weak credentials, bad internal security defenses, and bad patching practices allow one zombie to quickly become many zombie computers in your network.
So, to stick with the metaphor, beware of bathrooms and take a disciplined approach to applying security best practices. This will help you:
• Educate: Educate your co-workers to recognize phishing email attempts, be suspicious of downloaded applications, and err on the side of caution.
• Update: Update your operating systems and software. Once inside your network, zombie infections propagate through common exploits and weak credentials. Religiously apply patches to both operating systems and their software programs.
• Separate: Separate your systems and resources. This is perhaps the least obvious rule, but it is much important. Eventually an infection happens, a vulnerability is exploited, or someone accesses a hacked device or connects a VPN from their compromised home computer. A hacker will get in. At that point, you want to make sure it is as difficult as possible for the hacker to move laterally. Deploy internal firewalls, enforce strong access controls, and keep key resources separated.
Always be safe, which in this case means systems back up. This may seem obvious, but it’s often overlooked.
When the going gets tough, and you realize that half of your network is under control of an evil hacker, you may not have much of a choice but to throw a tactical nuke. Re-image all systems, restore your data, and get your organization operational again as soon as possible. But make sure you back up data separately from operating environments. Your OS backup may contain the “puppet master’s” evil control code.
Another common trick that zombies will perform is to encrypt all your data, and then ask you to pay some bitcoin to get the password to decrypt it (aka ransomware). So be extremely smart in your backup strategy – keep physically separate copies that go back many revisions. If you do daily backups, make sure you also have weekly and monthly backups. If the price of the ransom is too expensive, you can also considering restoring from backup.
Finally, to make sure you understand how the ransomware zombies got on your network in the first place, refer back to Tip #2: “Always Double-Tap.”
This one should be obvious. The more stuff on your network, the bigger the chances that our network could become a fertile breeding ground for large numbers of zombie devices.
However, this is not easy to avoid. As organizations move through projects, products, and various deliverables, new equipment, servers, and services are added. Similarly, holes are poked in firewalls, accountants may be shared, and credentials may be compromised.
But do you take them away when they are no longer needed? Most firewalls use old rules that are no longer needed. Networks are stacked with older services and services. And with the proliferation of virtualization, the number of servers quickly spun up (“just so we can try it”) has exploded. Especially those that use weak or frequently re-used passwords for easier access.
The problem is that few are ever shut down, and all of them run on powerful hardware. The strategy to overcome this challenge is a two-step process:
1. Put policies in place that track servers, applications, and accounts and enforce a planned expiration on all of them.
2. Conduct regular searches of network activity through skillful cyber hunting. This is important because all connected systems wind up communicating something on the network from time to time, even when they’re not in use.
So make sure you clean it up, clean it out, and travel light to give yourself a better chance of outrunning the zombie apocalypse.
Everyone likes the idea that some flashy new tool will do the job for you. No one likes the grind – the effort required to educate, update, and separate.
But it is the grind that is your best defense. After all, the groundwork in keeping your network resilient is extremely valuable when the zombie computers start popping up. Spend time each and every day hunting for bad guys in your network. Cyber hunting may seem like busy work, but the effort helps you learn much more about your network as you do, and you will soon start to recognize the good from the bad.
There’s little glory in prevention. When you catch that botnet and shut it down – successfully preventing massive downtime – few will cheer your victory. But cyber hunting prevents the agony of dealing with the aftermath: late nights cleaning up as zombies relentlessly blast their traffic while executives are forced to make nerve-wracking moves to limit brand damage.
Hunt for zombies daily
Back up frequently
Don’t be a hero. Do your legwork and stay prepared!
In this tip, the idea of limbering up means to use overcapacity and plenty of reserves.
Your ability to weather a storm is directly proportional to the reserves you have in your environment. Although this may seem counter-intuitive, having overcapacity actually helps you in the fight against zombie botnets attempting a DDoS attack. Lots of capacity typically means that zombies can whip up a bigger DDoS attack. But when CPU capacity is plentiful, networks have plenty of overcapacity and firewalls and scrubbers are over-provisioned, giving you the tools you need to keep your organization operating under duress.
Typically, network capacity is plentiful as well as system CPU resources. But what is often overlook is the capacity of routers, switches, and firewalls to handle bandwidth. Their CPU power is a bit restrained, and although their links have plenty of capacity, the devices are unable to move packets fast enough. One more tip here: To determine your breaking point, run a small DDoS attack from one segment of your network to another. (But just make sure you do this during off-hours!)
Have a backup plan. Or, even better, have three!
When the going gets tough, and you feel you might be losing the battle, make sure you have a plan to evade. This can include alternate internet paths, offsite data storage, and well-documented playbooks that can easily be followed by less-skilled helpers.
Here are a few more details on each of these topics:
• Create alternate internet paths: This could be as simple as the wifi network at the local coffee shop, or as sophisticated as load-balanced redundant internet paths.
• Take advantage of offsite data storage: Any operational data that can be stored off-site, including redundant email systems (think “cloud”), database-driven applications, or even backup file storage. It is a good practice to architect a business so that it may operate after a loss of access to the premises.
• Write well-documented playbooks: This one is very important. An ounce of preparation here can make a huge difference. You should create playbooks for a variety of processes and procedures, including using off-site or redundant resources as well as cleaning up, re-imaging, and remaining operational. Make sure they contain concise instructions that can be followed by people with less technical skills. Practice your playbooks, and don’t be afraid to test your plans ahead of time with scheduled fire drills.
When attempting to keep your network safe, don’t go it alone. Even when resources are constrained, try to pull in someone whose job is not necessarily a full-time network defender. By having a second set of eyes, you may pick up on patterns that you previously missed. Have each other’s back at all times. Also, remember that computers are not inherently evil, and they don’t hack each other by themselves. You are always hunting down a bad guy somewhere, and he/she is the one orchestrating this attack!
As you work with a partner, try to explain what a botnet is, how zombie computers work, and what you can do to catch them and shut them down. This can be very helpful in getting the two of you on the same page and even help find new ways to hunt. Sharing access to visibility tools allows others to pick up on patterns you may miss and accelerates investigations when you do catch something.
In Zombieland, this meant always checking the back seat of any car, just so you were never surprised when you were most vulnerable.
The same rule applies to cyber security. Once you catch a zombie on your network, stay cool, and study it carefully. Find out how it is communicating with the hacker. All of this represents your network’s “back seat” so make sure you check everything you can.
Then, using what you have learned by observing the zombie, details such as the IP addresses it talks to, the ports it uses, the timing of the communications, and any other information you can find, you can quickly start searching for other systems in your network with the exact same behavior. For this, you use full-fidelity flow data.
When you are going to take the fight to your adversaries, make sure you know where they all are. Then take them down all at once. It usually doesn’t take the hacker long to notice that his zombie devices are dropping, and he may switch tactics, or even stop completely.
We realize this was list of 10 tips, but we wanted to give you one more. With the constant stress of staying alive in today’s world of zombies, it is important to take a moment to smell the roses. Pausing should also serve as a grave reminder that the people behind the zombies may adapt, so what you did to protect yourself yesterday may not be enough for tomorrow. There are no silver bullets, and just as Rule #3 advises (“Don’t be a Hero”), you need to prepare yourself to do the work.
We hope these tips will truly help you avoid the zombie apocalypse. To keep a reminder of these tips, please download our Top 10 Zombie Tips infographic now.