The emergency sirens were activated in Dallas County last Friday night at 11:42pm. This is not an unusual event in Dallas and the surrounding areas, in fact this is kind of a common occurrence during the springtime. That is when we get the most severe storms here in Texas (well, pretty much this entire region of the country). Colloquially known as the “tornado sirens”, they provide an early warning system in the event of a tornado warning being issued by the National Weather Service.
Additionally, they are activated whenever a storm is capable of producing straight-line winds in excess of 70 miles per hour or hail larger than one inch in diameter - basically any conditions that will endanger lives (usually limited to weather conditions). While intended to alert people who are outdoors to encourage them to move inside, they can be heard indoors fairly easily unless you are in the middle of listening to loud music, a VERY sound sleeper, or you live on the very outer edge of the nearest siren’s coverage area.
The Emergency Operations Center (EOC), run by the Dallas Office of Emergency Management (OEM). Photo courtesy Dallas OEM.
However, in this case, the sky was clear and there was no storm in sight, in fact, there wasn’t even a storm remotely close to the Dallas area at all. First reported as a malfunction, it was later discovered to be a hack.
Here is what we know so far:
- A physical location where the computer or computers used to control what the sirens do (the length and duration of a siren sound) was compromised.
- The system was then activated, probably through the use of Dual-Tone Multi-Frequency (DTMF) signaling via radio.
- All 156 sirens were activated and went through 15 cycles of 90 seconds and were triggered approximately 60 times before they were shut down.
- The sirens sounded repeatedly from 11:42pm until 1:17am.
- The system was down until Monday afternoon at approximately 2:00pm for a total outage of roughly 52 hours.
There are a few things to explore around this, such as what technical details we can surmise about the hack and why someone might do it.
What Happened
One of the first things I did when I heard it was a hack was to search for information on siren systems. The usual setup involves a number of sirens which are triggered/controlled by a series of DTMF tones via radio, typically via UHF 450mHz. Repeaters are used to allow for a central location to send out the commands to the various sirens. This technology dates back decades. Newer systems allow programming of pre-determined siren sounds and their durations, and can be triggered via a central location. Some vendors of these systems allow for complete automation - if a National Weather Service alert comes in that would warrant the sirens going off, the sirens are activated automatically without human intervention.
The Dallas County system is like most systems - it is a hybrid mix of old and new, and most vendors selling the new systems realize that the system they will be administering will be this odd mix. Additionally, due simply to the legacy nature of these systems, most sirens themselves are still air-gapped, so remote activation is usually done via a central system that uses the legacy radio and repeaters from decades ago to perform the DTMF triggering of the sirens.
Per the Dallas Office of Emergency Management (OEM), they could not turn off the system remotely or at Dallas OEM headquarters, and decided to turn off the entire radio system, including the repeaters (at multiple locations), to silence the sirens. Frighteningly, the Dallas city officials and Dallas OEM personnel that have the authority to activate and deactivate the sirens can do so not just from their desktop computer systems at work, but also via an app on their iPhones. Obviously this central control system is not air-gapped.
During a press conference, they stated that they had eliminated any of their “control systems” or “remote logins” as being used in the hack. They traced it down to one area where they believed the hack took place. Apparently it was put into a mode where it repeatedly triggered the sirens. The fix was to simply disconnect everything, and since they typically never take the system down, they had to follow a special series of steps to gracefully shut it down so they can get it back up again later.
So What Really Happened?
This obviously sounds like a central computer was accessed and settings were adjusted to not only make the sirens repeatedly go off, but to prevent Dallas OEM from being able to shut it off.
This implies two things - a computer was accessed most likely via a compromised password (or worse, a default or even no password) and an attacker-supplied password was set to prevent the attacker’s changes from being overwritten.
Dallas OEM also stated that it had not contacted the local police, but had contacted the FCC for help in determining where the hack came from. Since the FCC regulates communications via a number of mediums including radio, and Dallas OEM stated they shut down their radios and repeaters, this suggests that while one system was accessed to set up a mode of repeating siren activations, there was also some type of use of radio during the event as well.
The FCC may be able to interpret such things as logs or other data gathered to help pinpoint where this illicit radio communication came from. For instance, if multiple devices are set up to receive radio transmissions, by examining signal strength, one could triangulate the position of the source of the transmission. Even by doing live experiments with a single receiver, they could potentially get an idea of roughly where, and, if they get lucky, it will be a parking lot that has security cameras aimed at it.
The FCC does more than just radio, and it sounds like they are looking over all associated logs across all related systems, but since Dallas OEM shut down the radio system as well as the central computers, it does imply radio usage. Who Did It?
This is where things get kind of interesting. Since this wasn’t something that could be done just spur of the moment, it suggested some level of planning. Now it could be just a goof hack done as a prank, maybe performed by some clever teen wearing a hoodie and getting their Mr. Robot on. However, a few things about this suggest it might be something else.
The timing was interesting, there was plenty of uneasiness in the US after last week’s response to Syria's chemical attacks. While quite possibly a coincidence, a number of people responded to the sirens by going onto social media and opening speculating that we were under attack by Russia and WW III had started. Already understaffed, Dallas’ E911 call center was flooded with calls, leading to call wait times of up to six minutes during its peak. And while Dallas OEM responded via social media not to call 911, many people on Facebook were convinced that the local government was lying and that something bad was happening. Rumors started of similar sirens in other cities in Texas and Oklahoma (which turned out to be false).
So panic - or at least confusion - set in fairly quickly. Considering it happened at night and the sirens repeatedly went off over and over, it seems that bare minimum it was planned to irritate, but it doesn’t take much of a leap to realize that using a system designed to warn people about danger is going to cause concern if it keeps going off.
The Siren Hacker Did Their Homework
While reading up on Outdoor Warning Siren (OWS) systems, I read that many older models could only run for a limited amount of time before they would overheat and need to recover before being activated again. The attacker apparently knew this as well, as the pattern of the sirens being turned on and back off in cycles would prevent the overheating. I doubt all of the 156 sirens have been upgraded to modern sirens that do not have this same issue, so the attacker compensated as they intended the sirens to not only go off and on repeatedly, but to do so over a long period of time. And because of this, the attacker made adjustments to lock out Dallas OEM from their own system and maximize the length of the event.
So the attacker had some knowledge of how things were put together. Most of this knowledge could be gleaned from Google searches, you can download manuals for a lot of different sirens and systems, and most of the software being sold to control these systems can be downloaded for free (demo versions only) allowing for a crash course in OWS management. But a few things such as the use of radios, knowing exactly what systems were in place, and what radio frequencies might be involved suggest a bit more planning. So I am thinking potentially three scenarios - a disgruntled insider, an attacker wanting to see at scale how a city would react and kind of measure the results, or a clever movie plot device to distract us while George Clooney and Brad Pitt pull off some robbery.
The latter isn’t as insane as it might sound - on December 31st, 1982 in Tulsa, Oklahoma, a group of burglars cut through 50,000 phone lines in a very small room inside a telephone substation. They used a chainsaw to cut the lines. This triggered hundreds of alarm systems at area businesses with police trying to respond, and in all the confusions, the burglars broke into a drug company warehouse and stole $1.3 million dollars worth of narcotics (roughly worth $3.3 million in 2017 dollars). Yes, this actually happened, however I kind of doubt it for our Dallas scenario. As a movie fan, of course I am hoping for some jewelry store robbery or museum art heist, but odds are it was either the insider theory or the “measure the results” theory. A disgruntled insider is a simple enough scenario, but the “measure the results” theory might warrant some thoughts.
Testing Real-Life Incident Response
One thing an attacker might want to do, particularly if they’re part of a terrorist group or a state-sponsored attack, would be to perform a “dry run” and see how a panic-inducing scenario might affect not just the local population, but measure response times and how other services (911, police, fire department, etc.) were impacted.
For example, it took a bit before 911 was overwhelmed and wait times became life-threatening for those with a real emergency. Knowing that could allow an attacker with a future plan to prepare accordingly. The same would go for the response from social media. A lot of people were more annoyed than anything else, but for some, they were legitimately freaked out.
For a scenario to freak out more people, the attacker now knows a bit better how this scenario worked and might think of additional activities that could further induce panic. This doesn’t mean Dallas might be targeted again, it could mean another city might be targeted with Dallas being used as a testbed. We, of course, have no evidence of this scenario being the case, but it’s not outright impossible that this was the motivation.
So who do I think did it? My money is on a disgruntled insider. However I don’t want to dismiss the other scenarios outright, as there is a non-zero chance it is one of those.
Lessons Learned
From the looks of innovation and the advent of IoT moving into industrial control systems such as OWS, there are probably going to be more events like this that occur.
I think the best lesson we can learn is one of planning for emergencies. What is the worse thing that could happen to your organization? Does your incident response and disaster planning cover all kinds of threats and all kinds of strange scenarios? Sitting down and having regular tabletop exercises for the entire incident response and disaster recovery team are important when preparing for the real things. Throwing in some movie plot or spy ring scenarios are kind of fun, but these extreme scenarios will also show you some of the cracks in your response and recovery plans that ordinary or mundane scenarios might not. A part of the those tabletop exercises should include everything you can think of, from minor to major. Will it affect money? Your customer’s private data? Be a PR nightmare? Upper management at an off-site planning retreat on the ski slopes and unreachable when the unthinkable happens? Cover it all.
Dallas OEM said in the press conference and in press interviews that they had not planned for this at all. Remarkably they did an excellent job of at least getting the sirens off, and now have a plan to get things back online with methods in place to prevent the hack from happening again. And even if they never catch who did this or learn why, it is still a valuable lesson for all of us.
Have a disaster plan, test it regularly, and don’t forget that George Clooney is just wandering around out there planning something awful. Be prepared!