1 Year, 50 Sites: Lessons Learned in Cloud VoIP Project

As I discussed in my previous post, I’m a senior voice architect for a Fortune 500 enterprise who spent much of 2018 selecting and upgrading 50 facilities to cloud-based VoIP service. I learned a few lessons along the way, as I’ll share here.

 

At any given time during the year, I had between 10 and 17 projects going simultaneously -- all in different stages. Without a strict regimen, I would have drowned in my own chaos.

 

Organization was key to leading these projects, and tracking all activities in a single document proved invaluable. As I noted in the first piece in this series, if something wasn’t noted in this document, it didn’t happen. More than just a project plan, this Google Sheets workbook contained tabs for tracking project components such as scope of work, scope management, project cost management, project resource management, and project stakeholder management.

 

Lest you think that sounds like a few chapters in the Project Management Institute’s PMBOK Guide, I didn’t actually use those names in the workbook. I’m not PMI-certified, but I read a PMI book recently and laughed as I realized that the PMBOK Guide contained many of the strategies that I had employed. So rather than such formal project management lingo, I used regular terms like “contacts,” “pricing,” and “inventory.”

 

Another organizational aid I found quite useful are templates -- what I call a cheater’s tool. I’m a huge believer in templates, and I don’t understand why more people don’t use them. Why manually draft an important email over and over again? If you manually create a message multiple times, no two messages will be identical. That means you’ll have variance. And with variance comes fluctuations in results. No thank you. I prefer consistent quality.

 

The first time I drafted an email to introduce myself and the project, I saved it. The next time I needed it, I tweaked it a bit to make some improvements and then re-saved it. After a few times of doing that, it was perfect for my needs. And I’ve used that template about 45 additional times. If each of those emails would normally take five minutes to draft, I figure having a template saved me almost four hours of work.

 

I have about 15 templates I regularly used to send emails. Some were for my internal customers (end users) and some were specific to field IT. Others were status updates to let management know that we’d completed a particular location’s VoIP migration.

 

The Google workbook I used is a template, too. It contained formulas for counting the quantity of phones for each model. Another formula checked for duplicate information, and turned duplicate cells red. It contained my price of phones and tallied the entire upgrade costs as the business filled out the cut sheet. With a “match” formula, it also pulled in email addresses automatically -- who wants to type those manually?

 

Templates aren’t just for the lazy, they help to ensure accuracy, provide consistency, and support a level of automation.

 

I found that some of my company’s facilities had comparatively low telecom operating costs. Different factors played into this but, in general, the biggest reason was size -- or number of telephones -- at a site. If a facility had an old but reliable and fully depreciated Avaya (or Lucent) Definity or Nortel Option 11 supporting 300 or 400 phones and only using a single telco PRI, then we couldn’t justify migrating to a cloud solution based on telco costs. Even after throwing maintenance costs into the ROI formula, the needle didn’t move much. Here's why:

 

These days, providers price cloud-based VoIP service like subscription-based software. Each “seat,” or telephone, has a cost. So a cloud solution with 400 telephones (400 X $ per seat) is likely to be much more expensive than the cost of one or two PRI circuits. At large sites, I found the cost per phone on a legacy PBX to be much lower than the per-seat cost of cloud-based service.

 

A common argument tossed back at me is “risk.” Legacy systems run the risk of failure, and maintenance support on these dinosaurs is best effort. While there is truth to this argument, a large company such as ours has a warehouse full of spare (complete) systems and parts. I suspect this is true of many Fortune 500 companies based on conversations I’ve had with industry colleagues.

 

On the flip side, we migrated plenty of small to medium-sized offices that had expensive PRI circuits and frequently achieved a return on our investment within one to six months.

 

Imagine my surprise when I discovered people to be predictably unpredictable. Despite my best efforts and clear communication, I found that sometimes people still behaved like... humans. Some were silent no-shows to my meetings. Some didn’t follow directions. Some had poor excuses. Overall, I found people to be consistently inconsistent.

To be completely fair and honest, some people were spectacular. Some were excited to receive a new phone system -- driven, motivated, energetic, respectful, and reliable. It was a pleasure working with this crowd.

 

I say all of this not to be negative, but rather to point out that you need to anticipate delays and pad your project timelines. In other words, hope for the best and plan for the worst.

 

Expect telco issues, such as port rejects, order mistakes, and inconsistent policies between telcos. For instance, a PRI “routing number,” which may or may not be a fully functional telephone number, may be portable with one phone company, but not portable with another phone company.

 

Another common issue that causes delays with orders is that telcos only allow one open order at a time on an account. You can’t have orders queued up. A problem that I sometimes ran into is that a completed order would be held open in the telco’s system due to an unexplainable glitch. Sometimes I had to escalate an issue for a week or two before the telco would close the order and allow me to submit another one.

 

By far the source of my biggest headaches throughout the entire project was the losing carrier. I’ve been doing this for more than 20 years now, and I still run across new excuses from time to time. I’m convinced that carriers intentionally look for excuses to deny port requests because they know that they’re losing revenue in the process. (If you can confirm this, please reach out to me on LinkedIn.)

 

You must handle port requests as carefully as you would a legal contract. If the authorized name on the account is “Tim” and you sign the letter of authorization as “Timothy,” expect it to be rejected.

 

We recently had a port request rejected -- of course without explanation -- because we had multiline hunting on a single-member trunk group. After having to ask why the request had been rejected, the telco told us that if it ported the number, that would leave multiline hunting on a memberless hunt group, and since you can’t have hunting on a hunt group with no members, it couldn’t fill the port request. The telco went on to say that it was incumbent upon me to submit an order to remove hunting before placing a port request.

 

Really. Just a single number -- I don’t know why hunting was on it. Silly, I know. But equally silly is why the telco required me to place an order to move it before it would process my port request. I couldn’t have made this stuff up if I tried!

 

The good, the bad, the ugly. It’s all part of the job. Expect to see it all in any project that extends across multiple sites and affects hundreds or thousands of people. Grind through it. Don’t let emotions control your words or actions. Learn from each experience and streamline your process.