SHARE



ABOUT THE AUTHOR


Andrew Prokop
Andrew Prokop has been heavily involved in the world of communications since the early 1980s. He holds five United States...
Read Full Bio >>
SHARE



Andrew Prokop | February 22, 2016 |

 
   

Planning for Disaster is Planning to Succeed

Planning for Disaster is Planning to Succeed Business continuity in the face of disaster is achievable through small, medium, and large steps.

Business continuity in the face of disaster is achievable through small, medium, and large steps.

"There's nowhere you can be that isn't where you're meant to be."
― John Lennon

As big of a Beatles fan that I am, I really don't believe that. Yes, we can influence where we "are meant to be" by our actions and attitudes, but I am a firm believer that some things just happen. Bad things. Good things. Accidents are just that. Accidents.

However, that doesn't excuse us from being prepared for whatever the world might throw our way. If you drive a car, you take out insurance. If you ride a bike, you wear a helmet. You don't necessarily plan for the worst, but you protect yourself in order to survive the worst while you are out there minding your own business.

The same holds true here in the world of communications. Despite the best intentions of the manufacturers, designers, installers, service providers, IT managers, and technicians; the best laid plans sometimes go awry. Equipment fails, lines are cut, supposedly innocuous changes take down entire systems, and Mother Nature has a nasty habit of reminding us who's the boss. Tiny disruptions can be annoying while major interruptions in service result in lost revenue, disgruntled customers, lost productivity, and in the most extreme cases, injury or death.

I recently assisted a major contact center in conducting a thorough business risk assessment and the results were sobering. The company knew that its contact center was a significant source of revenue, but what it didn't realize was just how valuable it was. After analyzing months of contact center and sales records it was determined that it would lose upwards of $36,000 an hour for every hour the contact center was down. Since it runs an international, 24/7 business, that comes to approximately $700,000 a day. Clearly, a catastrophic outage of days or weeks would wreak havoc on its bottom line and could even jeopardize its ability to stay in business.

Problems, Problems, Problems

The reasons for lost communications are vast and varied. I recently came across an article from Level 3 that presented The 10 Most Bizarre and Annoying Causes of Fiber Cuts. While some are quite laughable, the damage they caused was not.

While I wasn't surprised at what human mistakes and Mother Nature can do to underground and above ground cables, who would have thought that 17% of Level 3's damage was caused by squirrels? Yes, you read that right. Those cute and furry critters that romp through trees and drive dogs wild, are responsible for a significant number of lost telephone calls. I will never look at those buck teeth in the same way again.

The next most surprising statistic was that 7% of Level 3's annual outages were caused by people using fiber cables for gun practice. The article went on to say that many of these shot up cables were in tough parts of town, and repair technicians required bodyguards on their service calls. You would have thought that shooting up stop signs would be enough, but there appears to be an attraction to fiber that I just don't get.

In addition to cut, gnawed, and shot up cables, problems can occur in network and on-premises equipment. These failures come in many different flavors -- software, hardware, and configuration.

Planning for the Inevitable

The building blocks of a fully resilient solution are many, and the costs and complexities to add additional 9's depends on how much risk you are willing to accept.

While this is not an exhaustive list, I consider the following as table stakes:

  • Eliminate as many single points of failure as possible. This includes duplicated and high availability instances of routers, call servers, networks, session border controllers, session management servers, etc. The goal is to be able to take a sledge hammer to anything without losing your ability to make and receive calls.
  • Consider multiple carriers. It's extremely rare that two carriers stop providing service to a particular area at the same time. Even in huge natural disasters such as Hurricane Sandy, some carriers stayed up while others faltered.
  • Spread your 800 numbers across multiple carriers. I am a big fan of using a third party RespOrg, such as ATL Communications, to manage your 800 numbers. This allows you to develop number distributions that save money during sunny days (least cost routing, etc.) and keep your agents and customers talking during rainy days (carrier failover).
  • Create network failover paths that support both wired and wireless networks. This is most applicable to branch offices that can use wireless technologies such as 4G/LTE in disaster situations.
  • Move to SIP. In addition to saving money, SIP supports robust disaster recovery scenarios not possible with TDM.
  • Look to the cloud. The resiliency possibilities that cloud-based communications offers are myriad. You might use the cloud strictly for business continuity during major outages, or you might choose to move everything off premise. A mix and match approach might include provisioning trunks from traditional carriers such as AT&T and Verizon along with Internet trunks from the likes of Twilio. The cloud comes in many different flavors and it's important to choose the one that is right for your business.
Mischief Managed

I would be overjoyed if the answer to every problem was "all you need is love," but unfortunately, love isn't all you need. To keep your lines of communication open, contact center agents busy, customers happy, and accountants busy counting all the money you are making, you need to plan, prepare, and build for disasters. Your losses may not be in the tens of thousands of dollars an hour, but significant money will be lost when the phones stop ringing and customers stop buying.

The point of this article is that business continuity in the face of disaster is achievable through small, medium, and large steps. The approach that one enterprise takes may be different than that of another, but the end result of survivability is the same.

There will always be squirrels with a taste for fiber and yahoos with itchy trigger fingers, but you get to decide how they will affect your operations – a little, a lot, or not at all. I know what I would choose.

Andrew Prokop writes about all things unified communications on his popular blog, SIP Adventures.

Follow Andrew Prokop on Twitter and LinkedIn!
@ajprokop
Andrew Prokop on LinkedIn





COMMENTS



August 16, 2017

Contact centers have long been at the leading edge of innovation in communications technology, given their promise of measurable ROI and the continual need to optimize customer interactions and sta

July 12, 2017

Enterprises have been migrating Unified Communications & Collaboration applications to datacenters - private clouds - for the past few years. With this move comes the opportunity to leverage da

May 31, 2017

In the days of old, people in suits used to meet at a boardroom table to update each other on their work. Including a remote colleague meant setting a conference phone on the table for in-person pa

August 16, 2017
World Vision U.S. is finding lots of goodness in RingCentral's cloud communications service, but as Randy Boyd, infrastructure architect at the global humanitarian nonprofit, tells us, he and his team....
August 11, 2017
Alicia Gee, director of unified communications at Sutter Physician Services, oversees the technical team supporting a 1,000-agent contact center running on Genesys PureConnect. She catches us up on th....
August 4, 2017
Andrew Prokop, communications evangelist with Arrow Systems Integration, has lately been working on integrating enterprise communications into Internet of Things ecosystems. He shares examples and off....
July 27, 2017
Industry watcher Elka Popova, a Frost & Sullivan program director, shares her perspective on this acquisition, discussing Mitel's market positioning, why the move makes sense, and more.
July 14, 2017
Lantre Barr, founder and CEO of Blacc Spot Media, urges any enterprise that's been on the fence about integrating real-time communications into business workflows to jump off and get started. Tune and....
June 28, 2017
Communications expert Tsahi Levent-Levi, author of the popular BlogGeek.me blog, keeps a running tally and comprehensive overview of communications platform-as-a-service offerings in his "Choosing a W....
June 9, 2017
If you think telecom expense management applies to nothing more than business phone lines, think again. Hyoun Park, founder and principal investigator with technology advisory Amalgam Insights, tells ....
June 2, 2017
Enterprises strategizing on mobility today, including for internal collaboration, don't have the luxury of learning as they go. Tony Rizzo, enterprise mobility specialist with Blue Hill Research, expl....
May 24, 2017
Mark Winther, head of IDC's global telecom consulting practice, gives us his take on how CPaaS providers evolve beyond the basic building blocks and address maturing enterprise needs.
May 18, 2017
Diane Myers, senior research director at IHS Markit, walks us through her 2017 UC-as-a-service report... and shares what might be to come in 2018.
April 28, 2017
Change isn't easy, but it is necessary. Tune in for advice and perspective from Zeus Kerravala, co-author of a "Digital Transformation for Dummies" special edition.
April 20, 2017
Robin Gareiss, president of Nemertes Research, shares insight gleaned from the firm's 12th annual UCC Total Cost of Operations study.
March 23, 2017
Tim Banting, of Current Analysis, gives us a peek into what the next three years will bring in advance of his Enterprise Connect session exploring the question: Will there be a new model for enterpris....
March 15, 2017
Andrew Prokop, communications evangelist with Arrow Systems Integration, discusses the evolving role of the all-important session border controller.
March 9, 2017
Organizer Alan Quayle gives us the lowdown on programmable communications and all you need to know about participating in this pre-Enterprise Connect hackathon.
March 3, 2017
From protecting against new vulnerabilities to keeping security assessments up to date, security consultant Mark Collier shares tips on how best to protect your UC systems.
February 24, 2017
UC analyst Blair Pleasant sorts through the myriad cloud architectural models underlying UCaaS and CCaaS offerings, and explains why knowing the differences matter.
February 17, 2017
From the most basics of basics to the hidden gotchas, UC consultant Melissa Swartz helps demystify the complex world of SIP trunking.
February 7, 2017
UC&C consultant Kevin Kieller, a partner at enableUC, shares pointers for making the right architectural choices for your Skype for Business deployment.
February 1, 2017
Elka Popova, a Frost & Sullivan program director, shares a status report on the UCaaS market today and offers her perspective on what large enterprises need before committing to UC in the cloud.
January 26, 2017
Andrew Davis, co-founder of Wainhouse Research and chair of the Video track at Enterprise Connect 2017, sorts through the myriad cloud video service options and shares how to tell if your choice is en....
January 23, 2017
Sheila McGee-Smith, Contact Center/Customer Experience track chair for Enterprise Connect 2017, tells us what we need to know about the role cloud software is playing in contact centers today.