How To Avoid Costly IT & Telecom Downtime

Posted in: InsiderIT SecurityNewsPBXTelecommunicationsTelephone SystemsTips & Tricks

From database and hardware issues to software malfunctions, IT and Telecom failure or downtime is frustrating and can be incredibly costly. It is an unneeded interruption for your team and, if it happens too often, it can have a negative effect on the trust your customers have in your company.

 

In recent years, we’ve seen what downtime and IT failure can do to even very big brands.

Look at the example Virgin Blue’s outage in 2010, which compromised their check-in and online booking system, interrupted business for 11 days, affected 50,000 passengers and 400 flights, and ended up costing the company $20 million dollars.

Or how in 2015, a “major technical glitch” ground the New York stock exchange to a halt for several hours in an unprecedented scale. The Department of Homeland Security and the FBI reported that “there was no indication of malicious actors” involved, which meant that a technical malfunction was the sole reason why $28 trillion of transactions were halted for several hours.

We can also look at what happened to Sony’s PlayStation Network as another great example of the costs of IT failure. Sony’s multi-day outage (In which hackers compromised the personal records for more than 77 million users) cost the company about $171 million to clean up. Even though Sony was the victim in this situation, their vulnerability was still seen as a failure, on the part of the company, to protect its customers personal information and that greatly affected consumer trust in Sony.

(see 5 Best Practices to Decrease Risk of Information Breach)

 

Whether your company is big or small, system downtime, technical problems, or unnecessary failures always equal extra costs to your business.

Did you know that the average business suffers from 14 hours of downtime per year? And that the cost of that downtime across industries was pegged at $5,600 per minute?

A survey of 200 companies across North America and Europe (looking at the financial losses incurred from IT outage) found that smaller enterprises reported an average of $55,000 in lost revenue due to IT failures while mid-sized companies reported their average losses at $91,000 per year.

Surprisingly, although 87% of businesses surveyed said that failure to recover data would be “damaging” and 23% said it would be “disastrous,” over half (56%) of businesses surveyed had no disaster recovery plan in place.

Informationweek.com reported that, “One of the primary obstacles preventing organizations from investing in a disaster recovery policy is cost…” the other issue (according to the article) is the “daunting complexity of implementing a comprehensive IT preparedness and recovery plan.”

They continued to say, “A lot of times companies don’t fully understand the cost of not preparing, so as a result they are not willing to spend dollars to ensure disaster doesn’t occur or they can recover quickly from a disaster.”

System unavailability, whether from a failure in communications ability or in access to data, is a costly disruption to the regular order of business and can affect an organization in a variety of ways.

There are the costs to restore the (IT/Telecommunications) system, penalties and violations that have to be rectified, ripple effects in the supply chain, potential litigation, missed deadlines, employees that must be diverted from original tasks, and the most dangerous of all – a loss of trust from your customers.

 

“A lot of times companies don’t fully understand the cost of not preparing, so as a result they are not willing to spend dollars to ensure disaster doesn’t occur or they can recover quickly from a disaster.”

 

 

Now that we’ve seen how costly technological failure or downtime can be, what can we do to drastically reduce or even avoid this kind of situation completely.

The majority of unplanned outages are a result of ill-timed changes by administrators or developers; even the most-trained people make mistakes when they’re in a rush, tired, or take shortcuts.

In fact the recent data breach at Equifax, the largest in American history, was apparently caused by miscommunication and the failure of one person to deploy a patch for that particular vulnerability, one that had been available prior to the breach.

Former Equifax CEO, Richard Smith told the subcommittee for Digital Commerce and Consumer Protection that, ” The human error was that the individual who’s responsible for communicating in the organization to apply the patch, did not.”

An article by techcrunch.com spelled out the issue quite well,

“According to Smith’s written testimony, Equifax sent out an internal email on March 9th to deploy the Apache Struts update within 48 hours. However, Smith said, the system failed to identify any vulnerabilities. A few days later, the IT department also ran scans but failed to recognize the vulnerability. Then it was apparently all up to one person to communicate that there was a patch available for a discovered vulnerability.

Hackers, who quickly recognized the vulnerability, despite a team of 225 cybersecurity experts at one of the largest credit reporting agencies failing to do so, started to access the sensitive information on March 13th and continued to do so over a period of months.”

 

How can we avoid IT and Telecom breaches and Downtime?

  1.  One way to address this is to ensure that proper processes and good communication are in place. Often what is needed is communicating in a way that may feel like over communication, but there is no better way to keep a team on the same page and functioning properly than regular – great – communication.
  2. A second step to take is to begin a business impact assessment to reveal the true cost of downtime for your business and take steps to look at disaster recovery planning and testing so you have a planned response plan.
    You can use this free online calculator to help gain a better understanding of what the cost of downtime would be for your business and then weigh that cost with the cost of preparing a quality disaster recovery plan.

 

Building a disaster recovery plan
IT is more than just your data centre, computers, and software; those are a part of your overall ability to communicate but are still only one part. Remember, the priority is to protect all of your “mission critical” data and communications systems. These may be your data storage and backup capabilities and likely will also include your communications systems (telephones, internet, etc.) as well as any and all data connected to those systems.

A good telecommunication provider will be able help you put together a robust disaster recovery plan. It’s likely that they’ve already had a discussion around disaster recovery with you during your buying process; regardless, it should be a part of your conversation before you commit to buying any phone system.

There are different levels of disaster recovery. In our world (Telecom) it can be as simple as adding a secondary internet connection as a fail-over (for VoIP and Hosted systems) or as complex as designing multi-site, and multiple point-of-failure, solutions. Each solution is as simple or as complex as is needed for its own unique set of circumstances.

The factors that weigh in are the necessary levels of fail-over and recovery as well as the budget you have available – but don’t let that dissuade you, often you will find that your solution is much less costly than you think.

Disaster prevention and recovery doesn’t have to be a complex issue. Speak to your IT and Telecom providers; they have the expertise available to help you design a perfect solution for your business, one that can save you a lot of time, frustration, and money in the long run.

If you’re not sure who to turn to give us a call, we would be happy to help you find out how best to protect your business from the high and unnecessary costs of IT and Telecom downtime.

 

 

business phone systems

Subscribe to our Blog