Our mission as a company is to make the Internet a safer place. We have a masterplan on how to achieve this goal, which I would like to share with you right now.
There have always been security threats and corresponding technologies and procedures that reacted to them – out there in the real world as well as within our field of information technologies. The Chinese built the Great Wall and it was effective. Until it was not. The wall is now a piece of cake, you can fly over it with a motored paraglide, airplane, or just destroy a part of it. It no longer serves its purpose and is more or less a historical landmark for tourists.
In the early 1990s, the main battleground in IT security was viruses vs. antivirus software. Viruses were written by hobbyists and were pretty sophisticated, using polymorphism to avoid signature-based detection and so on. The main motivation of the virus authors was to create something that would spread. Financial gain was rarely involved and it was mainly on the part of antivirus companies.
The Internet of 1990s was based on an open-door policy. There were passwords, but you could either guess them or bypass them using months-old exploits. The administrators of the systems were so happy that everything worked as they intended, they didn’t want to touch their systems as long as there were no problems. Encryption was virtually non-existent; logins went through plaintext password-based protocols.
Then came the mighty firewall, with companies selling firewall software. First came stateless (packet-based) firewall, then stateful firewall that could connect the packets into TCP connections, then deep packet inspection firewalls that understood relations between the connections and the content of the protocols itself. Application firewalls like HTTP proxies became popular. This approach was based on a similar premise as Chinese wall – the attackers are out there and we need to keep them there.
This approach is called perimeter security. And it no longer works. I am not saying you should throw your firewall out of the window, you shouldn’t. But don’t count on it as the only thing either. It is nice to think that you have “done something for security”, but it is like having a door in the house – you should have a door, but it does not mean you have done everything and you don’t need a lock.
There are many reasons that this approach no longer works. First of all, the perimeter is very fuzzy. Users bring their phones and other devices to work, so where is the perimeter? They are connected to their home network in the morning, to the mobile network on their way to work, to a Starbucks wifi at coffee time, to an airport network in the evening. So where is the perimeter? There’s an obvious solution to this problem – let’s set up a perimeter on the device, make sure it has the firewall and the security policy applied. But this is not the only problem.
And of course, internal users cannot be trusted either. A large portion of attacks come from inside of the network. Are you sure that your last temporary hire is not Elliot from Mr. Robot? I would not be.
Then there is the physical perimeter. We have seen so many companies that have bought expensive IPS and firewall hardware only to give the wifi password of the internal network to visitors, allowing unknown devices to be connected to the ethernet network. Please look at your printer and make sure that it is not connected to a Raspberry Pi and then to your internal network. Go now, we’ll wait…
So perimeter security does not work, but it helps. There are IPS solutions, single sign on, security tokens, port security – both wired and wireless. You buy penetration testing for all your applications and infrastructure. Are you safe?
No, not really. We have to explain over and over again why we don’t give our customers a stamp or a certificate saying that they are now secure. It would be an outright lie.
All these technologies help, but they help against one type of attack only. It’s another type of Chinese wall waiting for an invention of a bomb-carrying drone. So do we sell snake oil? Definitely not. Let me explain.
All these technologies, processes, audits help against something called known unknowns. We all know that if your web application has an SQL injection or cross site scripting vulnerability, or if it runs vulnerable software, it is bad. Known unknowns are things that we know could happen, we just need to make sure that the impact is minimal, preferably none. We test our applications (ideally before putting them out there on the wild Internet) to make sure SQL injection never happens.
We have redundant datacenters, because a possibility of fire is a real threat that we understand and we want to make sure that if it happens, the second datacenter will take over. While an impact will be there, it won’t be catastrophic.
For some risks, like the possibility of an employee taking a picture of a sensitive document with their phone’s camera and sending it out of the company, we accept the risk and make sure the employees know they will be held responsible for an action like this.
We build these walls, rules and fix holes. But what about the bomb-carrying drone that we don’t know about? What about giant low-earth orbit space lasers? What about aliens with weapons we don’t know about – at all?
Let us postpone this question and first look at the asymmetries of our systems’ security. One obvious asymmetry is that there are many attackers out there, while our security staff and our security budget is limited. There’s literally an army behind our firewall, shooting various bullets at our systems – most of them get stopped, some get past the firewall, but get noticed and stopped.
Of course there are also many targets out there, so the attackers point and shoot, mostly randomly at different targets. Some targets have higher value and they get focused attention. Hacking systems like SWIFT (international interbanking transfer system) or international flight reservation systems is probably more lucrative than hacking your home PC with photos from your tropical vacation (we hope it was a great one and you had lots of fun!).
There are automated scans that target almost everyone online, there are directed attacks from competition or from people who want to sell your data or to blackmail you. And you have no idea how many of them are out there.
This asymmetry translates into economical terms – your security budget, however large, is limited. You cannot afford to have an SQL injection or XSS on your website, because you will be hacked. It’s always just a question of time; these bugs will get exploited by automated bots. So no, we are not selling snake oil, you cannot afford bugs in your web application or outdated software on your infrastructure. And no, the IPS will not protect you against all of these problems.
Is there a way to turn this economic equation around? For the attackers there is always a prize at the end – different for different attackers.
Your competition wants to render you unable to continue doing your business. Retailers on Black Friday face huge distributed denial of service attacks and I suspect that if they just formed a non-attacking pact, they would be much better off. Peace is always a cheaper solution.
A random attacker could sell your database on the open market. Thanks to Bitcoin, Tor and cryptomarkets, this is safe for them and even if you find the listing online, you won’t be able to stop it or find out who did it.
Some attackers may even just need your computer resources to launch DDoS attacks or sending spam.
So let’s make a prize and motivate a huge of group people to help you. This approach is called a bug bounty program and they are used by all the big companies, from Facebook, Google and Uber, even Pentagon. You promise to pay out a certain sum of money for certain type of vulnerabilities, if they are reported to you and not abused. You attract maybe tens or hundreds of skilled ethical hackers to help you and you only pay for the vulnerabilities they find. Citadelo is a founding partner of a new bug bounty platform called HackTrophy. Check it out, your company almost certainly needs it. Today.
I hear you asking – if I sign up for a bug bounty program, am I safe? No, remember the space lasers? But you need a bug bounty program just like you need a firewall. More on that later.
If you are not a web portal with zero confidential information, you also need a penetration test. If there are “known unknowns” that are difficult to find and you offer the finder a small reward, it might not be worth it for the ethical hackers to even follow through and find it. Think about it like this – if someone finds a vulnerability that you missed and it is out there, someone from the internet can easily find and abuse it, is it worth to pay for it? I definitely would, if the rules are set upfront by my company and there is no blackmailing involved, just a voluntary mutually beneficial transaction.
As I mentioned in the beginning, security of the Internet in the 1990s was basically nonexistent. We believe this situation is still here in areas outside of traditional IT. We have learnt how to protect credit card numbers, medical records and other sensitive information. At least in theory. But what about the controller that opens the dam? What about the power plant, or the chemical factory? What about the trains and airplanes? All of these are run by intelligent machines these days. And the story is almost always the same:
A company buys this great controller that allows better and more effective automation. When shaking their hands, the company’s CISO asks curiously: “Just by the way, what about security?” It is not his job, the CISO is taking care of the billing system and e-mail server and all the other traditional IT. He is asking just because he wants to make sure that it is not his job to make this part safe. “Don’t worry, it’s air-gapped”. An air gap means that the device is not connected to Internet, either directly or indirectly (through the corporate network).
The system is sucessfully implemented, but the board wants to know about their investment. The system needs to be online in order to generate reports for the board. They buy a GSM or wifi module and connect it to the network. There goes the air-gap and all the security. It still has access only to read-only reporting features, but you know how these limits work (or do not work).
Industrial automation has its own challenges. Standard protocols are rarely used and if they are, it’s with vendor extensions or as a vendor’s interpretation of a standard protocol. It is quite difficult to use standard penetration testing tools. Forget about buying “SCADA auditing software” (SCADA being one of the “standards”). We need to roll up our sleeves and write custom software. The other challenge is that we can’t just try to pentest the application and see what happens. You don’t want to open the dam by accident, because you just found a buffer overflow. Or causing a DDoS on a power plant. There is seldom a real testing environment – of course they test the software somewhere, but the environment is not the same. Sometimes, industrial automation uses hardware with slow processors or low memory, so you can’t implement encryption and message signing. It really is like the Internet of 1990s, only the impact of the hack can be catastrophic in comparison to merely unpleasant.
Most of the industry is just looking the other way. They are moving from old controllers to web-based applications and they are really happy that these new toys work, they get reports for the board, they can finally show the savings and this is all great. We just need to make it secure as well. And this is what we’ve been doing for the past few months, writing custom code for weird protocols, cleaning the dust of our old TCP sequence number prediction tools, man-in-the-middle arsenal from the 1990s. It has been an interesting experience. The world needs us now more than ever.
Unknown Unknowns Back to space lasers. The unknown unknowns are events that can happen and we cannot even name them. If you audit or test using a methodology like OWASP Testing Guide, you are testing for various types of vulnerabilities. These are known unknowns – you don’t know if your application has a vulnerability, but you know that there are SQL injection, buffer overflows, TCP sequence number attacks or any of the described attacks. You need to find them and it is not always easy. But every once in a while a brand new attack vector is discovered. Your IPS has no idea about it, it is not included in any methodology, you don’t know what to look for in your logs. So what can we do about these unknown unknowns – like a space laser or an armed drone to an ancient Chinese civilization?
You won’t find them and no technology will protect you against them. The next best thing is knowing about an attack as soon as possible and to make life really difficult for the attacker once it passes through the wall. Imagine Indiana Jones with current technology entering an ancient pyramid. Poisoned arrows are aimed at any trespassers, there are trapdoors, there’s acid. There is a fake prize and unless you built the pyramid, it is very difficult to go in, take what you want and get out.
We need to build these traps inside of our network. We need to create fake targets, plant fake documents, create a fake credit card database, let them hack into systems of no real value that will let us know that something is happening. We will create another type of asymmetry, one that counts on the possibility that someone will break in, that will let us know of the attack and will waste the attacker’s time and resources, maybe even help us learn who is attacking and how they got in.
“Security by obscurity does not work,” I hear you whispering while reading these lines. But this is something different. We are not creating obscure protocols or moving our services to different ports in order not to get attacked. We are creating these trapdoors that should lure in any attacker. This approach is not obscuring something in hope that we will not get attacked; it is achieving a goal in case someone breaks into the network.
One of the technologies that we can use is a honeypot network. We can create vulnerable targets inside our network with good alerting and make sure that the attacker cannot recognize them from real production systems. Poisoned arrows and trapdoors.
Honeypots have been traditionally used outside of perimeter, in order to collect information about the jungle out there that is the plain unfiltered Internet. They were hoping to collect new attack vectors, with full network logging and good reporting. They were mapping botnets, finding sources of attacks and filtering these addresses on production networks.
We don’t want to do any of that. We already know that the internet is jungle, we know that there are scripts and botnets attacking us all the time and this information is mainly noise.
But if there is a honeypot inside of our network, in the perimeter, and if someone attacks a system there, we already know for sure that something is wrong. It’s the kind of alert that should wake your security guys up in the middle of the night and while you try to find out what happened, you want to make sure that the attacker is busy with tens of vulnerable systems of no real value. The space laser is destroying decoy buildings while the hospital and the vault are hidden in plain sight.
Conclusion The security landscape gets complicated. Our mission is a safe Internet and that includes industrial automation. Our security team is working with various suppliers of controllers and users of industrial control systems to make sure they are not vulnerable.
For security in general, we are looking for known unknowns like everyone else and we are really good at it. We are helping businesses find security vulnerabilities like everyone else.
We are trying to turn the asymmetry of “everyone vs us” around with our bug bounty program HackTrophy.
And we are helping businesses to set up traps inside of their network in case someone breaks in through a hole that no one even knows about – from a rogue employee to a new type of vulnerability, a zero day or just a conceptually different kind of vulnerability.
And this is how we make a safe Internet a reality. If you care about this important mission, make sure to talk to us. This mission takes more than a single company and it is an important one.