Tag: Azure

Todays we are seeing major disruptions to IT infrastructure worldwide, afecting Airlines, Banks, Hospitals, Emergency services, Telecom companies, Media outlets, Payments processing, among others. [1] [2]

The root cause seems to be a faulty update released by CrowdStrike, a Cybersecurity company, for Falcon Sensor, their Endpoint Protection solution, which caused computer to lock-up and not turn-on properly, showing a Blue-screen error.

In an unrelated event, Microsoft Azure Cloud services also had major issues around the time of the “CrowdStrike problem” a Central US Azure outage (Tracking Id: 1K80-N_8) – those issues with Azure seem to be already mostly resolved. [3]

Solution for IT admins

As the affected computers and not running properly, unfortunately it seems they will need to be fixed one-by-one.

The solution seems to be:

1. Boot into Safe mode or the Windows Recovery Environment
2. Run the command:
 del "C:\Windows\System32\drivers\CrowdStrike\C-00000291*.sys"
3. Reboot

Key takeaways

• Software and Hardware Vendors have a high responsibility and should extensively test their products – both with automated tools and manually
• Vendors should do gradual releases of software updates, with live monitoring for issues and clients reported issues
• You don’t release significant updates on a Friday!
• Vendors should provide easy options or tools to control updates: delay updates for X amount of days, completely disable updates should always be available for IT admins
• For companies at large, they should have mechanisms to delay or manually approve updates for their entire IT infrastrucutre – unfortunately this is not always easy and is largely dependant of options provided (or not provided) by Vendors

Remember the 3 basic principles of Cybersecurity are Confidentiality, Integrity and Availability, in the quest to secure the systems we should pay attention to issues like this, which end-up causing as much disruptions as a major CyberAttack.

We should also be aware that Cybersecurity is, in large part, a risk-management and a balancing act between those 3 pillars.

Statement from CrowdStrike

Advice from Microsoft to solve this issue