How bad was the crowdstrike outage really?
Around 2 A.M. on July 19th, 2024, I recieved multiple calls and had no clue what was going on. I hopped on a group chat that was already going and started reading the history. My heart started to race a bit as I saw that every server had the infamous blue screen of death (BSOD)! I am in charge of security so I thought we had been hit and thats why they called me. I went straight into investigation mode! I tried going to our SIEM, our EDR, etc. None of these worked! They werent running. Now I was starting to get worried. Luckily the group chat showed an article by the BBC that this was a global outage. That atleast calmed me for a bit while we looked for a solution.
This was the same immediate reaction and finding that millions across the globe woke up to. I began settling in as I knew this was going to be a long night! It turns out Crowdstrike the well known EDR solution that is used globally across the world, pushed out a .sys file that updates the signatures of what the solution is looking for on systems. This one measley file began crashing systems across the globe. In total the number was around 8.5 million depending on the sources. Flights began being grounded as there were no systems to run the airports, hospitals began doing paper charting, companies had to revamp their entire workflows to get ready for the day. This was truly a disaster.
The disaster came as time progressed. The solution was found quite early but required manually restarting each machine into safe mode and deleting the file. This was a very tedious and slow process for most companies. Especially when you have thousands of systems to restart and fix. Crowdstrike themselves were in full crisis mode. The tickets people opened with them went unanswered, the solution was not provided by them, and they didnt call some customers until the next day! This was a PR nightmare for crowdstrike and the losses we have not even seen the full scope of yet.
A week later companies are still trying to recover. Some hospitals are not fully functional, flights are still not on schedule, and companies are working on contingency plans. This was an outage that we had never seen the likes of before. The 8.5 million systems is a drop in the bucket of how many systems could have been effected but the 8.5 systems were mainly servers that manage critical infrastructure across the globe. This showed many companies just how much crowdstrike can damage the world. This is making many rethink their future security strategies and reevaluate what EDR tools they use.
Crowdstrike’s stock has fallen from a high of $377/share on july 15th to $254 as of writing this article. This is a drop of over 33%. This is just the beginning for crowdstrike as far as financial and legal issues they will have from this. The long term lasting effects are uknown at this time but will crowdstrike ever recover, that I’m not sure of. One simple mistake can destroy one of the largest companies in this cyberspace. Be careful and make sure your always prepared for when that outage does occur!
