Business Lessons from Rackspace Massive Server Outage
Key Points:
- Cloud and email hosting giant, Rackspace, experienced an extensive outage that kept customers out of their email inboxes.
- The company couldn’t stand up for its email environment for two days, which shocked many because it’s a big player in the industry.
- Rackspace had been on the cloud long before Microsoft and Amazon, and everyone expected them to have backups of customer data after the security incident.
- However, the company proved otherwise with its unclear data restoration timeline after the Dec 2 ransomware attack.
- Even though you can’t predict outage events, how Rackspace handled its security incident is a wake-up call for businesses worldwide to prepare for better threat response.
Rackspace Technology — one of the most significant email and cloud hosting providers in the U.S., suffered a catastrophic outage on Dec 2, 2022. Initially, the company described the event as a simple login and connectivity issue but later updated it to a ransomware attack.
The attackers potentially accessed data on Rackspace servers, locking out thousands of small-to-medium-sized businesses out of their emails. In an SEC filing, the company said it has hired a leading cyber defense firm to investigate the incident but couldn’t provide a timeline for client data restoration.
The Rackspace Outage in Details
In the early morning of Dec 2, 2022, the Rackspace email system comprising hosted Exchange Servers went down. Users started reporting issues accessing services or emails in their online mailboxes.
The problem escalated when the Rackspace system started to bounce back the incoming emails and sent NDR (Non-Delivery Report) messages to the recipients after several hours. Thousands of customers — from healthcare firms to financial institutions, started sharing on LinkedIn that their emails were still down because of the outage.
At first, Rackspace referred to the situation as connectivity and login issues. Later, the company confirmed the case as a ransomware attack.
The Workaround Rackspace Offered Its Customers
After nearly 24 hours of downtime, Rackspace officially announced that its Hosted Exchange service suffered a ransomware attack and hired a leading cyber defense firm to investigate the attack.
Since the company had no estimated time to fix the incident, Rackspace recommended its customers to migrate to Microsoft 365 for free as a workaround before bringing the system back online.
Customers Rage on Social Platforms
While Rackspace offered recommendations and support to shift to Microsoft 365, their clients started flooding social platforms complaining about a lack of communication from the company. The number of people who reached out for migration procedures was overwhelmingly huge. The existing support team couldn’t bear it.
Many businesses were still concerned about the safety of their emails, contact lists, and calendars, and some started switching to other platforms. However, a Rackspace spokesperson said the company had added more staff to their support team to help customers with the long wait time.
What Businesses Worldwide Can Learn From the Rackspace Outage
The Rackspace crisis was a wake-up call for businesses worldwide, showing the importance of having adequate resources and an effective communication plan when under attack.
Although outage issues are unexpected, businesses can manage such incidents with the following better approaches.
1. Set Up Automatic Cloud-to-Cloud Data Backup
As much as you have your company data on the cloud, you need a backup elsewhere to minimize downtime should disaster strike your cloud provider.
Most businesses protect their data by using cloud storage vendors. Even though cloud storage technology prevents you from losing data, your backup protection is only on the vendor’s side. You risk compromising your data and daily operations if your vendor faces an issue. Having an extra backup will present an additional layer of protection to your business.
Automate the backup to eliminate possible human error from inconsistent backup. If you can’t automate — hire someone to execute regular backups. This will enable you to restore client’s data in the event of a security incident and continue with business operations while addressing the security issue.
2. Understand Who’s Responsible for the Security of Your Live Data
Security of systems in the cloud is a joint responsibility between you and your cloud provider. If you understand who’s responsible for what aspect of security, you’ll be in the right position to handle security effectively.
The cloud provider is always responsible for the security of the data center, network, and devices. Your data, users, access privileges, and the end-user device’s security is always your responsibility.
Depending on your agreement with the service provider, ascertain that your provider backs up your data on other platforms. That will ensure your business operations won’t be compromised even if your cloud vendor is under attack.
The Rackspace outage is just one of the astronomical examples that things could go sideways. You need to have a way to back up your business to keep it elsewhere just in case anything happens to your vendor.
3. Gather Enough Resources
After the massive outage, Rackspace promised to help customers to set up and configure Microsoft 365 accounts as a temporary solution to the existing issue. Since thousands of businesses were affected, customers started creating many support tickets that overwhelmed the existing support team.
With work overload, the support team couldn’t address all the customer issues in time. The delayed Rackspace customers took to social media platforms and started complaining about the lack of communication from the company and unanswered support tickets.
The best approach Rackspace would have taken was to immediately partner with others in the industry to meet the surge in support demand. While the company eventually outsourced support, it was a bit late. The damage had already been done.
Businesses need all the necessary resources to overcome such a disaster.
A Better Approach Can Protect Your Reputation After a Data Breach
The outage wouldn’t have been severe if Rackspace had the right ransomware mitigation processes. While the company was transparent, it would have done better by proactively issuing statements, helping customers resolve any concerns, and scaling up the customer support team.
Additionally, if the company had good data backup practices, it would have restored the customer’s data quickly before proceeding with the investigation. Instead, customers couldn’t access their emails, compromising their operations for several days. While disasters are unpredictable, businesses can prepare better response measures.
