The water treatment system of the city of Oldsmar, FL has been in the news lately after attackers breached its control systems and increased the levels of sodium hydroxide. The targeted treatment plant provides water to businesses and about 15,000 residents in the Tampa, FL suburb. Was the attacker’s intention to harm these residents? Could the hacker have been an expert in chemicals and known higher level of sodium hydroxide could deteriorate human health causing irritation, burns, temporary hair loss, damage human cells and other complications? Could the invader have known of potential adverse effects if they instead decreased the levels? This is a developing story with forensic units, FBI and secret service all in the race to uncover the mystery attacker towards region’s public drinking water.
The Story
An outdated version of Windows and a weak cybersecurity practices allowed hackers to break into the Florida town’s water treatment plant’s computer system and momentarily tamper with the water supply. The person with remote access briefly increased the amount of sodium hydroxide, also known as lye, by a factor of more than 100, from 100 parts per million to 11,100 parts per million. It was a botched attempt at what could have amounted to a mass poisoning. The breach of Oldsmar’s computer systems is an awakening call for poor security practices. The hack used a dormant remote access software that could have been to a sewer system or variety of other systems in many other towns – thus, it is vital to spread awareness that we need to have network security everywhere.
Safeguards and Security measures
1. Monitoring, Alerting and Logging
The first and foremost consideration is to have alerts and threshold monitors set up in such context when higher or lower levels are a concern. For general security, you should ensure to leverage logs for debugging and make sure to store meaningful log messages. Also have audit logs which note every user action carried out on an application. The addition of dashboard monitoring which includes event and metric aggregation would be useful in noticing trends daily/weekly. The good news is that monitoring protocols seemed to be put in place in this specific scenario and the city confirmed that the alarms in the system would have caught the change in pH levels even if the boost in chemical to a toxic level went unnoticed.
2. Secure and monitor remote access
Even for company heads, remote access should require 2FA or approval by users logged in to accept the remote session. This should only be done by setting a schedule for the week for access to supervisors or in case of emergency. The bad news is that a plant employee actually witnessed the event while they were viewing the shared desktop, but apparently the operator didn’t think much of it because it’s normal for his supervisors to use the remote access feature to monitor his computer screen at times. You must also disable remote connections immediately upon detection of unauthorized usage.
3. Awareness of software in use
Even minutes and seconds can matter in an attack. Be sure to audit every third-party application you use to identify any weaknesses and update them to latest firmware version. In this attack, the threat actor used TeamViewer to access the system remotely for about three to five minutes. The obscurity of TeamViewer technical details such as how the cyber criminals potentially obtained login credentials, which are set and encrypted solely on the device makes the scene indefinite.
4. Have dedicated IT and security staff, or work with a service provider
In most cities, the municipal water utilities are extremely underfunded and under-resourced, and that makes them a soft target for cyber-attacks. With the pandemic hitting us hard last year and forcing many to work remotely, organizations began to expose vital infrastructure computer networks to the internet. With remote access multiplying, inexperienced administrators often sacrifice security measures or take time putting them in place. All industrial control systems should be updated and able to detect malware for maximum protection. The Oldsmar hack highlights the need for more training and basic security protocols.
It is unclear where this attack came from, whether foreign or domestic or a warning to other cities and their security protocols. It would have taken between 24 and 36 hours for that water to hit the water supply system, so definitely not a vicious intrusion. Investigators have yet to determine the motive behind the remote access method of conducting an attack. A week ago chemistryworld published details on why minute amounts of sodium hydroxide are commonly added to drinking water. Considering this, many cybersecurity experts have declared that the culprit was merely unaware of the consequences of increase in chemical levels and must have just stumbled upon into a vulnerability – further trying to figure out what can be done with such access, maybe a nation state attack or next global crisis cyberattack?